Adding Charging Powercurve Selection for EV Charging

[the-bay-kay]

Context

This PR builds off @shankari and I's work in Issue #64 . This work will be done on the functional DepartureTime demos.

High Level Goal:

... to present the user with multiple charging profiles, which they then can select and use for their current charging session

Subsequent Goals:

Edit: Updated timeline to reflect 2-week sprint goals. Multiple SASchedules is a stretch goal, implementation of power curve selection is higher priority.

EDIT 2: Changed goals to reflect implementation adjustments.

• Test options for including a power-curve visualization, including (i) plotting coefficients, and (ii) Sending an image (S
  • Add MQTT Broadcast of data
  • New NodeRED component giving curb visualization
  • Add "listener"
• Incorporate Power Curve algorithm, including "choose algorithm" flag and inputs
  • Incorporate Algorithm in Node-RED
  • Incorperate in PyJosEv
• Add commands to MQTT server
• Two Stages of visualization (Performed via working method determined above):
  • When setting algorithm, add dynamic view of power curve
  • After initiating a charging session, broadcast the power curve that meets grid constraints
• Allow user to select from a list of power curves / algorithms

[shankari]

@the-bay-kay I believe that the plan is for the "optimized curve" to be computed on the car.

1. The station will return two SASchedules in ChargeParameterDiscoveryRes

• lowest_viable_power and maximum_supported_power

2. The car will then look at those schedules, compute the optimized curve and send it in the PowerDeliveryReq

Note that the important part, per DJ and Jacob, is (2) since the car should have control of the charging schedule for the departure time. We are only implementing (1) because it lets us showcase multiple SASchedules without fully supporting tariffs (and because I want to 😄 ) So you might want to tackle (2) first, since even if we don't finish (1), we will have a clean demo.

[the-bay-kay]

I believe that the plan is for the "optimized curve" to be computed on the car.... So you might want to tackle (2) first, since even if we don't finish (1), we will have a clean demo.

Aaah, right! Cool cool -- sounds like a good plan!

[the-bay-kay]

Updated Timeline and Issue name to reflect the project's immediate goals.

[shankari]

note that node-red has a charts module: https://stevesnoderedguide.com/using-the-node-red-chart-node
https://flows.nodered.org/node/node-red-contrib-graphs

[the-bay-kay]

... including (i) plotting coefficients, and (ii) Sending an image

Let's start by investigating the first option. As currently configured, our Node-RED does have a chart function, though it seems to be geared to presentation-style charts (Bar Charts, Pie Charts, etc.). I don't believe we would be able to plot the anticipated equations.

Node-RED does have a curve plotting module we could add, that seems geared to plotting functions specifically! (link)

note that node-red has a charts module: ...

Likewise, this looks like a good option -- may be more flexible than the curve module, will compare the two options and go forward from there!

Existing Line Chart Screenshot

[the-bay-kay]

Assessing our options:

1. The biggest downside with the contrib-graph module is that it does not plot to the ui (http://127.0.0.1:1880/ui): Rather, it plots to (http://127.0.0.1:1880/dashboard/). While this isn't a dealbreaker, setting two windows side-by-side isn't ideal (especially when we already have a modular UI).
   a. Likewise, I think this module only allows us to feed a single datapoint at a time (e.g., one {XYZ} per JSON object). I may be mistaken, but this would get very clunky very quickly (We'd need to broadcast a mountain of datapoints each time we re-plotted the curve).
2. The curve-plotting algorithm unfortunately only takes a single variable as input (f(x)). This almost certainly disqualifies it from use, unless (a) I fork the plugin and update the module, or (b) get creative with the Node-RED Flow Contexts.

Let's take a shot at (2b) first, before exploring alternatives. My hope is that we can insert the dt and eamount flow nodes into the function as "captured" variables, and use the function input for our "K" variable.

[shankari]

can't you also just plot a line graph and have DJ give you a set of coordinates?

[the-bay-kay]

can't you also just plot a line graph and have DJ give you a set of coordinates?

I was searching for an option to plot via coefficients like we discussed a few days ago, but that is an option. I'll go ahead with that approach!

[shankari]

I was searching for an option to plot via coefficients like we discussed a few days ago, but that is an option. I'll go ahead with that approach!

Right, I think that plotting a standard quadratic curve would be more efficient, but if that is complex, we can fall back to the option that is known to work, at least for now

[the-bay-kay]

Just wasted a solid hour and a half trying to fix this bug... turns out the old curve module I was experimenting with bricked the UI... Anyway, we've got a basic chart running: let's write the function and generate some datapoints...

Chart Screenshot

[the-bay-kay]

When attempting to plot an array of data, we do so for the points, but do not receive a line as we would expect. My intuition says this is either (i) an issue with how we're formatting the inbound data, or (ii) an issue with how the chart is configured to accept the data.

Another minor issue with the line-chart method of plotting is that I have not figured out how to use non-timestamps within the X axis label. We can theoretically define custom labels (setting to "automatic" attempts to format as a time stamp), but it's uncertain if we the custom labels need be tied to a timestamp format like the defaults (HH:MM, for example)

Screenshots / Recordings

Quadratic

Screen.Recording.2024-08-15.at.3.05.05.PM.mov

Linear

Screen.Recording.2024-08-15.at.3.12.14.PM.mov

Linear w / Timestamps as X val

Screen.Recording.2024-08-15.at.3.14.06.PM.mov

[shankari]

The x axis does represent time - the curve represents power drawn during the charge session, right?

[the-bay-kay]

... we do not receive a line as we would expect. My intuition says this is either (i) an issue with how we're formatting the inbound data, or....

Yup, this was my problem. Changing the way the arrays were nested, we get an OK result. Let's add some templates for the powercurve data next

Rudimentary Graph Screenshot

[the-bay-kay]

We're successfully plotting multiple curves -- let's hook up the MQTT messages now...

Curve Plotting Screenshot

[the-bay-kay]

Before making the changes to everest-core/modules/EvseManager/EvseManager.cpp to enable MQTT broadcasting, let's better understand a few things, including

1. The Algebraic Riccati Equation Linear-Quadratic Regulator (LQR) that DJ's code will use to create the optimal curve
2. What "dials" we have available with this function
   a. (As I understand it: We have the departureTime, EAmount, and a 'scale' factor). These will then be clamped by a min value, and PMax.)
3. What the new charging initialization sequence will look like
   a. E.g., there will be an additional step between receiving an SASchedule and initializing charging. Likewise, we may need to adjust the schedule set by the user ("Add the clamps"), and will have to broadcast this to MQTT. Perhaps I should sketch up a sequence diagram for this...

[the-bay-kay]

Effectively, there's two modes for our graph: "Preview", and "Actual". Since we've got the rough preview working, let's see how we'll graph the "Actual" curve, once the optimizer is run on the SASchedule.

To re-render the graph, it seems like we receive ChargeParameterDiscoveryResponse (and likewise, the SASchedule) in evcc/states/din_spec_states.py, specifically in ChargeParameterDiscovery(). My first thought is that we will want to utilize EVEREST_CTX's publish() method, as we are already doing with AC_EVPowerReady. That being said, the definition of this publication seems vague... Let's look for where this publication is received, or try to find the way we publish via MQTT.

Regardless, it's good to find this chunk of code where we find the SASchedule -- once we add DJ's optimizer, it should slot in roughly here.

While tracing these calls, I've been sketching out a rough sequence diagram to highlight the important calls within this process (attached below the cut).

In Progress Seq. Diagram

[the-bay-kay]

An aside -- here is a sample of the powercurve graph preview, changing dynamically with the scale and departureTime -- this is with a dummy linear function for now, but switching to a different (quadratic) function should be as straight forward as swapping the function.

In testing, I into some limitations -- Node-RED freezes when attempting to chart a large number of points with a large range. If necessary, we can limit the "sample rate" of the visualization, to reduce the number of points being charted (e.g., plotting the value every minute instead of every second may increase the speed of rendering more complex functions)

Recording of PowerCurve "dummy demo"

Screen.Recording.2024-08-21.at.12.47.38.PM.mov

[the-bay-kay]

... Let's look for where this publication is received, or try to find the way we publish via MQTT.

We know the answer to the first half, at least. We subscribe to the context publications within JsEvManager's index.js. Let's start by close reading this file to look examples of MQTT publication -- after all, JsEvManager receives the initial commands through an MQTT subscription...

[the-bay-kay]

While we subscribe to MQTT broadcasts in EVManager's car_simulatorImpl.cpp, it appears we do not publish to MQTT as we do in EVSEManager's evse_managerImpl.cpp. Since PowerCurve selection should occur within the EV (context), we'll likely need to add publication to this file. So, follow up question: how will JSEvManager communicate with car_simulatorImpl? Let's investigate...

[shankari]

there will be an additional step between receiving an SASchedule and initializing charging.

This is the PowerDeliveryReq that the EVCC sends to the EVSE. It is already sent as part of the ISO message pair

[the-bay-kay]

Out of the myriad of locations where we find PowerDeliveryReq, it seems that we want to add the optimizer to the async process_message function within the PreCharge Class in ext-switchev-iso15118/evcc/states/din_spec_states.py, confirmed when comparing it to the PowerDeliveryReq() logs (screenshot below). Let's trace from this to see where we can receive this information in Node-RED

Logs

[shankari]

DIN != ISO, it is a precursor to ISO. I would look to see where I had to edit the departure time in my previous set of changes. Maybe it is in the din_states file, but I would expect to find it in the iso_states or similar file

[the-bay-kay]

In din_states, it seems we'll want to modify iso15118_2_states.py's process_message(), specifically around line 807. Knowing where we want to change this, there are a few unanswered questions:

• How can we preview the Optimization Curve before charging starts?
  • Currently, we use a set of coefficients to calculate a series of points that we then plot. As of writing, this we are plotting a few dummy linear functions.
  • I've spent a few hours staring at DJ's code, and I don't think there is a clean way to convert this Python code to JavaScript. We could re-implement the aspects of the control and numpy libraries that the script utilizes, but this seems like re-inventing the wheel. This is within possibility, but will bulk up the Node-RED code substantially. (The Math.js library has some, but not all of our equivalent functions -- matrix(), etc.).
  • I briefly explored calling a copy of the python function from Node-RED (link), but I'm not sure this is an avenue we want to explore. This would require our host machine to have python installed (with numpy and control), which we don't want to ask of the users.
  • Our final option is to have a sub-routine that (i) sends the coefficients to a brand new module, (ii) the module computes an example curve, and (iii) the module sends back the calculated coordinates to plot. This seems like gilding the lily -- out of all of these options, the first seems to be the best.
• Where do we add the power curve optimizer call?
  • We have the answer, as above
• How do we send the new curve points back to the UI for plotting?
  • Tracing where we use the context to publish AC_EVPowerReady, this publication is received in car_simulatorImpl.cpp and our trusty JsEvManager/index.js.
  • Within both of the aforementioned files, I don't see any existing MQTT publications we can use as an example. Compare this to the evse_managerImpl.cpp, where we can see an example broadcast.
  • I'm assuming we'll want to publish from index.js. The question then is how. Do we follow the example of JsYetiSimulator's index.js? Does the mod object perform the same? Or, is this publication in JsEvManager what we should use as a template?
  • Once we know this, receiving the MQTT signal from Node-RED and plotting the points should be relatively trivial!

[the-bay-kay]

This seems like gilding the lily -- out of all of these options, the first seems to be the best.

After some discussion with the team, we've opted to go for "B" : Running the python script from Node-RED for the preview. This will leave us with less overhead in the preview (we won't need to wait for the MQTT messages to bounce around), and saves the hassle of re-inventing the wheel.

Currently, we've got basic files running on Node-RED -- an example script can be launched from the UI, and an output can be read. The kicker is installing additional Python packages. We need both Numpy and Control, and ideally want to upgrade to Python 3.x (Our Node-RED image is currently on 2.7). From what I understand, we'll need to re-build the docker image, editing the dockerfile to install the correct pip packages (relevant thread here). So, we need to (i) find the dockerfile (E.x., the manager dockerfile is here), and (ii) re-build. Ideally, this change should be similar to the recent changes to Everest-Manager here.

The relevant Dockerfile is here: so, let's make the changes...

[the-bay-kay]

Changes have been made to nodered/Dockerfile, now we are (temporarily) building locally. Now that we are doing so, we can use apk to install PIP, and then run pip to install the necessary dependencies. My initial tests failed, but I believe this is because I was installing / running pip incorrectly (we install python-devel OK, but I believe need to call the correct version of pip installed with the devel package...)

File Changes

# More above...
USER root
RUN apk update && \
    apk add --virtual python-dev 
USER node-red
# More below...

docker-compose.ocpp201.yml

# More above...
  nodered:
    # image: ghcr.io/everest/everest-demo/nodered:${TAG}
    build: ./nodered
# More below...

[shankari]

Is there no existing python install on that container? Did you try docker exec and then run python? Most unix distros come with python pre-installed now

[the-bay-kay]

... Most unix distros come with python pre-installed now

Correct! Our docker comes pre-packaged with Python 2.7.18: the issue is, it does not come with pip (we need this to install control and numpy for the optimizer). See:

[shankari]

why not use conda like we use elsewhere? You can download miniconda like we do in e-mission, it can set up python3 and comes with pip

[the-bay-kay]

why not use conda like we use elsewhere?

Not a bad idea -- that seems cleaner than doing the python package management within the Dockerfile (e.g., I've successfully updated Python, still having issues with pip...). Let me try adding miniconda to the container now!

[shankari]

I tried creating the container using docker-compose.yml and ran into a different issue, which is related to installing on Alpine Linux https://superuser.com/questions/1621329/conda-install-in-alpine-image

~ $ curl -o miniconda.sh -L https://repo.anaconda.com/miniconda/Miniconda3-py39_23.5.2-0-Linux-x86_64.sh 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 89.0M  100 89.0M    0     0  8796k      0  0:00:10  0:00:10 --:--:-- 10.2M
~ $ bash miniconda.sh -b -p $HOME
ERROR: File or directory already exists: '/usr/src/node-red'
If you want to update an existing installation, use the -u option.
~ $ bash miniconda.sh -b -p $HOME/miniconda3
PREFIX=/usr/src/node-red/miniconda3
Unpacking payload ...
miniconda.sh: line 353: /usr/src/node-red/miniconda3/conda.exe: No such file or directory

I can retry with the full multi-tier docker compose but let me see if I can figure it out with this first.

[shankari]

Idea from: https://stackoverflow.com/a/62555259, ensurepip docs: https://docs.python.org/3/library/ensurepip.html

Something as simple as `python3 -m ensurepip` seems to work

~ $ apk list -I  | grep python
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/main: No such file or directory
WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/community: No such file or directory
python2-2.7.18-r2 x86_64 {python2} (custom) [installed]
python3-3.9.17-r0 x86_64 {python3} (PSF-2.0) [installed]

~ $ python3
Python 3.9.17 (main, Jun  9 2023, 02:31:24) 
[GCC 10.3.1 20210424] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 

~ $ python3 -m ensurepip
Defaulting to user installation because normal site-packages is not writeable
Looking in links: /tmp/tmpdzwf4nk2
Processing /tmp/tmpdzwf4nk2/setuptools-58.1.0-py3-none-any.whl
Processing /tmp/tmpdzwf4nk2/pip-23.0.1-py3-none-any.whl
Installing collected packages: setuptools, pip
  WARNING: The scripts pip3 and pip3.9 are installed in '/usr/src/node-red/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-23.0.1 setuptools-58.1.0

~ $ export PATH=/usr/src/node-red/.local/bin:$PATH

~ $ pip3

Usage:   
  pip3 <command> [options]

Commands:
  install                     Install packages.
  download                    Download packages.
  uninstall                   Uninstall packages.
  freeze                      Output installed packages in requirements format.

The problem with this is that we shouldn't really be installing `numpy` using `pip`, it doesn't have as many precompiled optimizations, which is why we use conda. But it looks like it works, so let's go with it!

~ $ pip3 install numpy
Defaulting to user installation because normal site-packages is not writeable
Collecting numpy
  Downloading numpy-2.0.2-cp39-cp39-musllinux_1_1_x86_64.whl (19.9 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 19.9/19.9 MB 7.4 MB/s eta 0:00:00
Installing collected packages: numpy
Successfully installed numpy-2.0.2

[notice] A new release of pip is available: 23.0.1 -> 24.2
[notice] To update, run: python3 -m pip install --upgrade pip

~ $ python3
Python 3.9.17 (main, Jun  9 2023, 02:31:24) 
[GCC 10.3.1 20210424] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.zeros(10)
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
>>> 

[the-bay-kay]

Woot! That was the main roadblock -- we're running the script now 🥳 Let me hook up the UI inputs, and I'll push my changes and move on to the simulator portion.

Successful Script Run

[the-bay-kay]

Had to futz with some of the inputs, but we now can use the UI to change these values! There seems to be little change when the ks value is changed -- focusing on getting this working, we can investigate later...

Currently editing the python script to "return" a JSON object, so that the chart can read these values in the correct format!

Screen Recording of Functionality

Runtime.mov

[the-bay-kay]

We're finally plotting the curve!! The output is a bit wonky -- there are some issues plotting the YC component, but it is being plotted. In the interest of time, I plan to push what I have, and move on to adding the powercurve function to the simulator. Below is a recording of the dynamic curve plotting. Once changes are pushed to my branch, I will update with steps to reproduce.

To Reproduce:

1. Run bash demo-iso15118-2-ac-plus-ocpp.sh -r $(pwd) -b test-demo -3
2. Once launched, navigate to the node-red docker container, and navigate to the exec tab (alternatively, perform these commands with docker exec)'
3. Run the following series of commands

python3 -m ensurepip
/usr/src/node-red/.local/bin/pip3 install numpy control

4. You should now be able to select your plug & charge, and set the DT / EA with basic powercurve visualization!

TODO:

• Incorporate pip workflow into Dockerfile
• Build & Push relevant docker image, reverting compose file changes
• Fix flow s.t. visualization is improved (second curve is clear)

Curve Visualization

CurveRuntime.mp4

[the-bay-kay]

Adding a simple logger.info('Hello World!') to the place where I thought we needed to add the powercurve to PowerDeliveryReq (link), we don't see the log within a charging session (plug-in, charging, unplug).... Perhaps I was off the mark? Let's look for where else we could add this

EDIT: It should be noted that this may not necessarily be an issue with iso15118_2_states.py, there are other locations where the PowerDeliveryReq is used. Let's explore those first before weeding through the other 22 files

Perhaps we should cut out the middle man, and edit the shared message directly? Let's try that... Let's trythe PowerDelivery class specifically....

[the-bay-kay]

It seems the existing build_current_demand_data is only for DC, so if we want to modify the AC charging event, we may need to modify this... First, let's explore the EVCC's simulator.py, specifically get_charge_params_v2

[the-bay-kay]

To get a hack-y demo working, our current progress has included:

• Generating the curve within evcc/simulator.py's process_sa_schedules_v2 Using the PMax and Duration_Time from secc_schedule.p_max_schedule.schedule_entries.
  • We will eventually need to figure out how to extract EAmount from these values (e.g., use PMax and DT to re-calculate the Watt-Hours )-- at the moment, the scalar factor is hardcoded (which we have discussed), and we are hard-coding EAmount. We'll need to shift some things around to get these values s.t. the curve can be also passed to PowerDeliveryReq
• We've added paho-mqtt back to ext-switchev-iso15118, in order to broadcast this updated curve to Node-RED. (I say "back", because this repository used to use paho-mqtt once upon a time!) We've successfully connected this to Node-RED, all that is left is to adjust Node-RED for the visual.
  • As with the other pip additions, this will need to be added to the docker image / dockerfile

[the-bay-kay]

Some unexpected behavior: When running a paho-mqtt from a preview script, it behaves ok...

Proof of Concept Call

Screen.Recording.2024-09-04.at.12.52.25.PM.mov

But when adding the exact same call to evcc/simulator.py...

curve_vals = LQRchargecurve(departure_time, eamount, ks)
publish.single("everest_external/nodered/{}/evcc/powercurve", "!dlrow olleH", hostname="mqtt-server")

We fail to start charging, with the following error (We know this is the issue, as removing the publication allows us to charge as expected -- the LQR runs OK)

While we aren't communicating via MQTT in this module, perhaps we're crowding the outgoing messages, and have some overlap elsewhere? Let's explore this further...

EDIT: Publishing within iso15118_2_states.py causes no issues - let's see if we can get that method to work instead

[the-bay-kay]

Finally got the curve data sent to MQTT!! Had to return the curve data with process_sa_schedule_v2, and then send it within iso15118_2_states.py (had to convert to a string) -- just need to finish the Node-RED plotting log, and then we should be good to go!

[the-bay-kay]

We've got a working demo! 🥳

Below is a recording of the demo -- highlights include:

• Adjusting DT, EAmount, and Scale update the Curve Preview
• The PowerCurve updates when we begin charging
• While charging, we cannot change the curve -- we can only set an "ideal" curve in the "Idle" state!

There are definitely some kinks that we need to iron out, but exciting to see it working. Once I get the relevant code committed, I'll go ahead and post steps to re-produce this demo.

FinalCurveScreenCap.mp4

[the-bay-kay]

With the Node-RED visualiations working, and the powercurve correctly added, the next step is to adjust the simulation process in order to reflect the EVCC's powercurve.

I believe the best way to do so is by generating a ChargingProfile for the PowerDeliveryReq, sent directly after the ChargeParameterDiscoveryRes(). I believe this approach:

1. Follows the spirit of the ISO 15118-2, as we are not adding any responses to the PowerDeliveryReq
2. Complies with [V2G2-741] as we are allowed to change the ChargingProfile, but not the SAScheduleTuple. (Because each tuple has a unique ID that we cannot mutate after initialization, I interpret this as "we cannot mutate the tuple)

• Likewise, [V2G2-741] Note 3 states that we may change the Charging Profile.

One drawback to this approach is the limitation of the ChargingProfile type. According to Table 71 of the spec, we are allocated a maximum of 24 ChargingProfileEnteries. As such, we'd only be able to "sample" 24 points of the curve -- E.g., if a charging session occured over 12 hours, we'd adjust the power draw every half hour.

Below is a sequence diagram roughly outlining the flow of PowerCurve Generation & Adoption:

SequenceDiagram

[shankari]

@the-bay-kay what is the timing of these steps? Is there actually potential to wait for the user to select a power curve?
I assume not, but I wonder if the spec only requires strict timing after the PowerDeliveryReq and whether there is room for limited user interaction that is compatible with the spec

[the-bay-kay]

@the-bay-kay what is the timing of these steps? Is there actually potential to wait for the user to select a power curve? I assume not, but I wonder if the spec only requires strict timing after the PowerDeliveryReq and whether there is room for limited user interaction that is compatible with the spec

Good question -- as written, we immediately shoot off the PowerDeliveryReq after receiving the ChargeParameterDsicoveryRequest. Looking at the spec...

• [V2G2-847] specifies that a state transition must occur no later than 250ms after the PowerDeliveryRequest -- not an issue, since we'd set the profile before...
  • Likewise, [V2G2-866] requires an SECC to set the PWM to 100% after the PowerDeliveryReq, not relevant....

Other than these 2 clauses (additionally, V2G2-848 for the "stop" PowerDeliveryReq), it seems there are no restrictions on timings within the spec. I see no reason why we couldn't pause for user input at this point, when we want to add algorithm selection down the road!

EDIT: this diagram suggests there should be a 250 second requirement after the ChargeParameterDiscoveryRequest, but I could not verify this in the spec. Correct me if I'm missing something! Likewise, we'd be making the changes after the Resolution, so even if this were the case, I'm not entirely sure it'd apply

[the-bay-kay]

I've official got the new schedule generating, as described above, but am running into a timeout when attempting to send the entire schedule. Below is a copy of the error, and the two messages being sent (the latter causing the error). Currently investigating what may cause, this, will update accordingly... My first thought is to try a shorter schedule, perhaps this is too much text for a single message?

EDIT: Yup, that's looking like the issue. Paring down the new schedule to 4 items as a test, we start charging O.K.... So what gives? I guess Table 71 of ISO 15118-2 says we can have a maximum of 24, but maybe EVerest doesn't support that yet... Let's see exactly how many we can fit in before we reach the timeout

EDIT 2: Well, when we pass a schedule of length 23, it works fine!! Weird... We can investigate that discrepancy later -- for now, this seems to be working!! I'll include a video and some future steps after this.

EDIT: Working Schedule Msg

OLD: Screenshot of Error, JSON messages

Screenshot of Timeout

Original Message (no Timeout)

{
  "V2G_Message": {
    "Header": {
      "SessionID": "11442E78D7FBFFF3"
    },
    "Body": {
      "PowerDeliveryReq": {
        "ChargeProgress": "Start",
        "SAScheduleTupleID": 1,
        "ChargingProfile": {
          "ProfileEntry": [
            {
              "ChargingProfileEntryStart": 0,
              "ChargingProfileEntryMaxPower": {
                "Value": 15000,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 8400,
              "ChargingProfileEntryMaxPower": {
                "Value": 0,
                "Multiplier": 0,
                "Unit": "W"
              }
            }
          ]
        }
      }
    }
  }
}

New Schedule Msg (Timeout)

{
  "V2G_Message": {
    "Header": {
      "SessionID": "81D3EF0FB732B27D"
    },
    "Body": {
      "PowerDeliveryReq": {
        "ChargeProgress": "Start",
        "SAScheduleTupleID": 1,
        "ChargingProfile": {
          "ProfileEntry": [
            {
              "ChargingProfileEntryStart": 0,
              "ChargingProfileEntryMaxPower": {
                "Value": 15000,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 60,
              "ChargingProfileEntryMaxPower": {
                "Value": 56,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 120,
              "ChargingProfileEntryMaxPower": {
                "Value": 53,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 181,
              "ChargingProfileEntryMaxPower": {
                "Value": 50,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 241,
              "ChargingProfileEntryMaxPower": {
                "Value": 47,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 302,
              "ChargingProfileEntryMaxPower": {
                "Value": 44,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 362,
              "ChargingProfileEntryMaxPower": {
                "Value": 41,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 423,
              "ChargingProfileEntryMaxPower": {
                "Value": 39,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 483,
              "ChargingProfileEntryMaxPower": {
                "Value": 36,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 543,
              "ChargingProfileEntryMaxPower": {
                "Value": 34,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 604,
              "ChargingProfileEntryMaxPower": {
                "Value": 32,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 664,
              "ChargingProfileEntryMaxPower": {
                "Value": 30,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 725,
              "ChargingProfileEntryMaxPower": {
                "Value": 29,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 785,
              "ChargingProfileEntryMaxPower": {
                "Value": 27,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 846,
              "ChargingProfileEntryMaxPower": {
                "Value": 25,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 906,
              "ChargingProfileEntryMaxPower": {
                "Value": 24,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 966,
              "ChargingProfileEntryMaxPower": {
                "Value": 22,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 1027,
              "ChargingProfileEntryMaxPower": {
                "Value": 21,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 1087,
              "ChargingProfileEntryMaxPower": {
                "Value": 20,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 1148,
              "ChargingProfileEntryMaxPower": {
                "Value": 19,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 1208,
              "ChargingProfileEntryMaxPower": {
                "Value": 17,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 1269,
              "ChargingProfileEntryMaxPower": {
                "Value": 16,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 1329,
              "ChargingProfileEntryMaxPower": {
                "Value": 15,
                "Multiplier": 0,
                "Unit": "W"
              }
            },
            {
              "ChargingProfileEntryStart": 8400,
              "ChargingProfileEntryMaxPower": {
                "Value": 0,
                "Multiplier": 0,
                "Unit": "W"
              }
            }
          ]
        }
      }
    }
  }
}

[the-bay-kay]

Good news -- @shankari , like we expected, we're able to interrupt the charging initialization during process_sa_schedule_v2. Below is a video of this working -- just need to send the charging curve data before the interception, and the "resume" function will return the curve that we use!

MqttStartCharging.mp4

[the-bay-kay]

Looking for our options to dynamically display a choice to the user, this appears to be non-trivial. At first I thought nodered-dashboard's ui-notificaiton would be our answer, but this does not allow us to display selections. I was hopeful that Node-RED's API RED.notify would be the final answer, but we receive the error below when attempting to call this.... We're using Node-RED 2.2.3, so our version should be compatible with this API. Am I misunderstanding the context which it's used? I briefly explored using an implementation similar to this, but was unsuccessful (I'm assuming it was a conflict with the dashboard package we use?) Will keep de-bugging the RED.notify method, and update with results.

EDIT: Yup, I was misunderstanding -- it appears RED.notify is used for the implementation of Node-RED modules. If we were designing our own modules, this may be helpful. Unfortunately, we need something a little more "out of the box"...

Much to my chagrin, I think the best option (to get this working, at least) is to have a dropdown component that is dynamically populated via the MQTT in. A notification will then prompt the user to make a selection. Upon doing so, the selection will be sent back, and we will clear the dropdown. This is far from elegant (We'll have any empty dropdown hanging around the UI), but I think it will at least get us to a working state.

RED.notify() TypeError

[the-bay-kay]

I think the best option ...a drop down dynamically populated via the MQTT in. A notification will then prompt the user to make a selection... and we will clear the dropdown.

Done! I need to fix the curve preview again, but that should be straight forward. Good news is, the mid-charge selection process is now working! Below is a video of this in action -- for the sake of our code and my sanity, I'll wrap up tomorrow after some sleep 😀

Mid-Charge.Algorithm.Selection.mp4

[the-bay-kay]

We've got a functioning demo! Below is a video of the demo being run: the user may select from one of two "preset curves" (a stand in for our algorithm selection), and then the charge session will occur accordingly. Users are able to preview curves during the idle phase, and user inputs are used when calculating the curve presets.

While exciting, this isn't necessarily a "camera-ready" demo -- I still want to fix the notification process, for example. After showing this off to the team tomorrow, I'll create a series of clean-up tasks for this project.

FullDemo.mov

[the-bay-kay]

Tasks for Demo Cleanup

Now that we've got a 90% working demo, there are several steps we need to take in order to make this production worthy / "showroom" worthy. Below is a list of the changes that need to take place in order to finish this current iteration (roughly in order):

• Fix schedule send through PowerDeliveryRequest
  • (currently, schedule is generated w/o exchange... I seem to have broke it at some point during curve selection changes)
• Fix powercurve visualization
  • There are two ways of going about this:
    • (i) continue to use the chart node, fixing the user scale / presentation
    • (ii) Use MatPlotLib to visualize the charts as we do in the sample script, and find a way to present images in the dashboard
• Fix LQR algorithm by:
  • Fixing the distribution in schedule splicing
• Add Option to skip mid-charge selection
• Fix Timeout Issues
  • Mid-Process Algorithm Selection times out after ~30 Seconds
• Integrate other optimizer functions beyond LQR
• Clean up code, standardize naming conventions, etc.
• Build & Push relevant docker image, reverting compose file changes
• Update the "one-line" demo script & Dockerfiles to include:
  • The updated Node-RED flows
  • Pip installation / operation for Node-RED manager

Future Work

There are some other ways in which we can improve this functionality. Because these projects have a much larger scale (implementing entirely new portions of ISO15118), I've separated them out into their own list.

• Adoption of multiple SASchedules
• Integration of Tariffs ("pay more to charge fast, pay less to charge slow", etc.)

[the-bay-kay]

... Add Option to skip mid-charge selection

Facing a bizarre issue trying to implement this. Below is a video: TL;DR, the MQTT exchange I've set up works perfect the first time the Node-RED UI is loaded, and then fails subsequent charging sessions.

I've spent a few hours trying fixes -- this issue has never occurred with the algorithm selection process, which made me think it was a timing issue. Delaying the messages to avoid race conditions (e.g., node-red publication prior to simulator subscription), re-arranging messages, and none of it seems to work.... will update accordingly once I find the cause.

MQTT exchange issue

MQTT.Error.mov

[the-bay-kay]

We've made the change to selecting our algorithm prior to the charging session (as opposed to during the PowerDeliveryReq handshake). While this works most of the time, we're still running into the bizarre issue described in my last comment.

EDIT: It seems my intuition was correct, but my method of execution was wrong. Adding a Node-Red "Delay" node fixed the issue, no need to futz with setTimeout(). Keeping the text below for debugging context!

---

I've included a timing chart with my current theory as to the cause of this issue. Because of the behavior when re-publishing the message, I believe the issue may be that we subscribe to the confirm message after it has already been sent -- this "race condition: then causes us to stall out. I'm not sold this is the cause, as adding a delay to the confirmAlgorithm() does not eliminate this bug. Likewise, I'm confused as to why re-deploying Node-RED "fixes" the issue temporarily. I don't have many leads yet, but I will continue to diagnose...

Timing Sequence & Screen Recording

Below is a video of the behavior occurring. First, we "stall out" on waiting for the publication of confirmAlgorithm. By injecting the value -- and thus, re-publishing the MQTT message -- we successfully restart the charging process, and perform as expected. Then, I re-delploy the Node-RED flow, and charge with the same settings - except a different algorithm. This charging session occurs as expected. I have ruled out the algorithm choice as being a confounding variable -- the error has occurred with both the "smooth" and "aggressive" curves.

Message.Timings.1.mp4

[the-bay-kay]

I've cleaned up and uploaded the docker images to my personal fork -- once I figure out someissues with GPG I'm having, I'll push the updated scripts / demo-repo files. Once that is done, we'll close this issue and move our work to #71 !

[the-bay-kay]

@shankari , this demo should be ready for presentation. To reproduce, you should be able to spin up my branch....

git clone https://github.com/the-bay-kay/everest-demo.git; cd everest-demo; git checkout powercurve-addition; bash demo-iso15118-2-ac-plus-ocpp.sh -r $(pwd) -b test-demo -3

... and start the manager as usual with sh /ext/source/build/run-scripts/run-sil-ocpp201-pnc.sh.* Once the manager is successfully spun upshould be able to interact with the demo as in the videos above! Once you've successfully reproduced this demo, feel free to close the issue. and we can continue any polishing in the next one.

*There is a slim chance the manager may need to be re-started after a fresh clone (charging will stall on the initialization sequence)... I haven't been able to reproduce this in 3-4 attempts, but am investigating further. Will update this comment and push the patch once I've got the fix nailed down!

[the-bay-kay]

It seems that we stall out when we process the "ChargeParameterDiscoveryRequest" (PrepareCharging)... It appears to be an issue with how we initialize EAmount or DepartureTime within Node-RED. That is, on a fresh clone, we stall until we set the value. The demo works otherwise (e.g., a fresh clone where you punch in the numbers should work OK). Let me confirm this theory and I'll push an updated image!

EDIT: Changes pushed -- while doing so, I fixed the formatting for the powercurve visualizations. Doing another pass to make sure the numbers are OK and things perform as expected

[the-bay-kay]

@shankari I've pushed the changes that fix the PowerDeliveryRequest -- upon a fresh clone (e.g., deleting old containers & images), it is working as expected for me. LMK if you cannot replicate this on your end!

Screen.Recording.2024-10-01.at.2.19.12.PM.mov

[shankari]

Just confirming that, after deleting all instances and all images, I am now able to see values for the curve.
I think we can close this now, and move the discussion to #71

[the-bay-kay]

Just confirming that, after deleting all instances and all images, I am now able to see values for the curve. I think we can close this now, and move the discussion to #71

Glad it's working on your end! I just tested Car Control + Grid Control, and everything looks A-OK there!

I did notice what I bevel is a regression in the "no DepartureTime" behavior (grid only) -- When setting the 4Hour 10A grid clamp, we're receiving a PMax of 2500W, not the expected 6900W. I think this is because there is a default DT of 86400 somewhere -- once that's patched, I'll push the image and close this issue.

EDIT: Yup, just set the default when I was rewriting the Node-RED flows. Closing once the changes are pushed