Scheduling: which algorithms are planned & next iteration for a more REST-like API

[nhoening]

Scheduling is the central feature of FlexMeasures. The schedule is the actual usable result, where FlexMeasures puts data into actions for optimal operation.

At the February TSC, we had a deeper discussion about the scheduling roadmap: which schedulers to build in and how to expose them. We know we are still not where we need to be:

1. FlexMeasures needs to support many people's use case out of the box
2. Triggering and retrieving schedules needs to be straightforward for developers to implement

We'll now discuss which algorithms we'll support, then how they'll be exposed.

Currently planned algorithms

For heat optimization, we'll need a basic scheduler that can model a state of heat storage and some loss over time. There are more sophisticated algorithms possible to model some heat storages and heat sinks in more detail, but that will probably be custom implementations (or even later additions to the FlexMeasures core).

• The current storage algorithm can be expanded to be able to model simple heat storage. [soon]
• A simple shiftable algorithm can be added. [ETA unclear]
• Sector coupling within the storage scheduler (allowing heat storage as well as batteries in one site) is another extension. [ETA unclear]

Exposing (API)

Exposing schedules will probably get a re-design, as we believe we are not developer-friendly enough, w.r.t. REST design and documentation (for instance, now the sensor resource is being used, which is not straightforward and also not future-proof, as we plan to schedule more than one flexible asset in the future).

In our latest iteration we tried to give only one endpoint for triggering all kinds of schedulers to do work. But in the end, the CLI already went the way of distinct commands per type of scheduler.

We want to focus the API more on jobs (as RESTful resource). For instance, we aim to build this set of endpoints in the next re-design:

• POST /scheduling-job/storage: triggers a job - needs sensor ID, flex-model and flex-context
• GET /scheduling-job/: gets the schedule status
• DELETE /scheduling-job/: kills a non-computed job (otherwise return 404 or fitting code)

This way, we can make the extensions we'll need later, e.g.:

• POST /scheduling-job/shifting
• POST /forecasting-job: trigger a forecasting job

[Flix6x]

In the CLI, we went the way of having distinct commands per type of scheduler because we were reluctant to introduce the flex-model and flex-context options as JSON strings (which, just like in the API, would only be deserialized to a dictionary, and its contents would be further deserialized by the scheduler). We don't have that issue with the API, so I'm wondering whether we would prefer POST /scheduling-job over POST /scheduling-job/<scheduler>, where the former seems more REST-like.

[nhoening]

Good comment. On the other hand, I thought it'd be nice if CLI and API are in lockstep.

The type of scheduler is a path argument in the API I proposed, and you want it to move inside the POST body, correct?

Would you make the argument optional, even, and let FlexMeasures try to guess which scheduler should be used?

[Flix6x]

Yes, I'd move it inside the POST body, and make it optional.

I'd prefer the API and CLI to feel the same, too. But I don't think the current CLI design expands well to multi-sensor scheduling. Right now we're exposing the fields inside the flex-model as separate CLI options (likewise for the flex-context fields). For the multi-sensor case, each sensor will have a flex model, and I see a need to communicate flex-model fields as a nested dictionary or a list of dictionaries, in order to assign the right flex model field to the right sensor. This leads me back again to considering JSON strings for CLI options.

[nhoening]

I consider the case where the whole flex model is sent in (via API or CLI) as a dev case.
Usually, we should aim for setting the flex model on the data model, e.g. as an attribute on the asset.

[Flix6x]

The topic of HTTP status codes isn't as clear-cut as one would want hope it to be, ad 2023. But this article seems like a very decent starting point.

[nhoening]

We worked on reporting in the meantime, and the experiences from that architecture might inform this design question, as well.