Skip to content

Add "Create forecast" button to sensor page#1985

Open
Copilot wants to merge 31 commits intomainfrom
copilot/add-forecasting-button
Open

Add "Create forecast" button to sensor page#1985
Copilot wants to merge 31 commits intomainfrom
copilot/add-forecasting-button

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 24, 2026

image

Adds a "Create forecast" side-panel to the sensor page, letting users trigger a regression forecast with a single click — no CLI or API knowledge required.

Behaviour

  • Panel is visible only to users with permission to record data on sensors
  • Forecast duration defaults to 48 hours (driven by FLEXMEASURES_PLANNING_HORIZON config) and can be adjusted up to 7 days via a duration input field
  • Button is enabled only when the sensor has ≥ 2 days of historical data; otherwise shown disabled with an explanation
  • Info icon tooltip shows the humanized default duration (e.g. "2 days") from the server config
  • Clicking triggers POST /api/v3_0/sensors/{id}/forecasts/trigger (using the selected duration, no start → defaults to now), polls GET /api/v3_0/sensors/{id}/forecasts/{uuid} every 3 s, surfaces progress via Toast messages, and refreshes the chart data on success (without a full page reload)
  • A "More options ↗" link next to the button opens the relevant OpenAPI docs endpoint in a new tab

Changes

View (ui/views/sensors.py)

  • Checks create-children permission via user_can_create_children(sensor) before calling get_timerange (DB short-circuit for users without permission)
  • Reads FLEXMEASURES_PLANNING_HORIZON from app config and passes the ISO string, humanized string, and integer days to the template

Template (ui/templates/sensors/index.html)

  • New sidepanel-container block matching the style of the existing "Upload data" panel
  • Info icon tooltip uses the humanized planning horizon from config
  • Duration (days) number input pre-populated from the server config, capped at 7 days
  • "More options ↗" link to the OpenAPI trigger endpoint (opens in new tab), placed left of the "Trigger job" button
  • JS reads the duration input and sends the ISO duration to the API; on success dispatches sensorsToShowUpdated to reload chart data via the existing embedAndLoad handler in graphs.html
  • JS block conditionally emitted by Jinja2 (only when both permission and data flags are true)

Tests (ui/tests/test_sensor_views.py)

New test file with 8 cases: page access, 404, login redirect, panel visibility per permission level (own-account user / admin / other-account user), button state at the 2-day boundary, and a mock-based guard verifying get_timerange is not called without permission.

Documentation

  • documentation/views/sensors.rst — new "Creating a forecast" section with cross-reference; updated to use plain-language permissions and config-based duration
  • documentation/features/forecasting.rst — new "Forecasting via the UI" subsection alongside the existing CLI and API sections; plain-language permissions, config-based default duration, and a bridge sentence directing users to the API for more control
  • documentation/changelog.rst — entry added
Original prompt

This section details on the original issue you should resolve

<issue_title>Forecasting button</issue_title>
<issue_description>A button should appear on the sensor page, which will create a simple forecast. This can make it easy to demonstrate this feature in demonstrations and quickly build examples.

At first, this is limited to the most basic version (vision: as little effort as just the button click).

  • forecast 24 hours
  • only regressive
  • only active when at least two days of data is given and if the users has permissions to create-children on the sensor

UX:

  • The button calls the endpoint per Ajax, and polls the job every 3 seconds, to give feedback in form of a Toast message.
  • If data is available we could reload the graph.
  • An info icon explains the functionality and limitations.</issue_description>

<agent_instructions>FM v0.31.0 will see new forecasting API endpoints, with all timing parameters getting sensible defaults for ease of use [POST] /sensors/(id)/forecasts/trigger <api/v3_0.html#post--api-v3_0-sensors-id-forecasts-trigger>_ and [GET] /sensors/(id)/forecasts/(uuid) <api/v3_0.html#get--api-v3_0-sensors-id-forecasts-uuid>_ to forecast sensor data [see PR #1813 <https://www.github.com/FlexMeasures/flexmeasures/pull/1813>, PR #1823 <https://www.github.com/FlexMeasures/flexmeasures/pull/1823>, PR #1917 <https://www.github.com/FlexMeasures/flexmeasures/pull/1917>_ and PR #1982 <https://www.github.com/FlexMeasures/flexmeasures/pull/1982>_]

Please add a forecasting button on the UI's sensor page accordingly, maybe in proximity to and in a similar visual fashion as the "Upload data" side-panel, which also deals with recording new time series data on the sensor.

Employ your agent team and make sure to run the test specialist, the docs specialist and the coordinator. I suspect the coordinator might want to start a ui-specialist.md with the lessons learned. Make sure to tell it to consider that.

You must check precommit hooks and the full test suite passes before asking for my review.</agent_instructions>

Comments on the Issue (you are @copilot in this section)

Custom agent used: review-lead
Orchestrates team of specialist agents for comprehensive code reviews and synthesizes their findings into unified recommendations


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

@read-the-docs-community
Copy link
Copy Markdown

read-the-docs-community bot commented Feb 24, 2026

Documentation build overview

📚 flexmeasures | 🛠️ Build #32065254 | 📁 Comparing 97274b7 against latest (30d3057)


🔍 Preview build

Show files changed (8 files in total): 📝 8 modified | ➕ 0 added | ➖ 0 deleted
File Status
changelog.html 📝 modified
genindex.html 📝 modified
_autosummary/flexmeasures.data.models.forecasting.pipelines.train_predict.html 📝 modified
_autosummary/flexmeasures.data.models.forecasting.utils.html 📝 modified
_autosummary/flexmeasures.data.models.generic_assets.html 📝 modified
api/v3_0.html 📝 modified
features/forecasting.html 📝 modified
views/sensors.html 📝 modified

Copilot AI added 2 commits February 24, 2026 18:37
Context:
- PR #1985 added the 'Create Forecast' button to the sensor page UI
- No agent existed to encode Flask/Jinja2/JS UI patterns for FlexMeasures
- Patterns were worked out from scratch during the session

Change:
- Created .github/agents/ui-specialist.md covering:
  - Side-panel pattern (sidepanel-container / left-sidepanel-label hierarchy)
  - Permission-gating idiom: always short-circuit DB calls behind permission check
  - JS fetch → poll loop → Toast → reload pattern with correct status-code handling
  - Toast/spinner setup and teardown discipline
  - Disabled-button vs. hidden-panel distinction
  - Marshmallow data_key awareness for JS POST payloads
  - UI test checklist (access, permissions, data availability, boundary, not-called guard)
  - Known architectural debt (absent CSRF, session-expiry in poll loop, type hint gap)
Context:
- PR #1985 established concrete UI patterns (side panel, JS poll loop,
  permission-gated view logic) that the agent system had no coverage for
- A new UI Specialist agent was created to own those patterns
- Coordinator roster and domain knowledge should reflect this

Change:
- Added UI Specialist to the agent roster (item 9)
- Added 'UI Development Patterns' subsection to Domain Knowledge with:
  - Jinja2 side-panel template snippet
  - Python view short-circuit data-guard pattern
  - JS fetch/poll/Toast/reload skeleton
  - Agent responsibility table for UI-touching PRs
Copilot AI changed the title [WIP] Add forecasting button to sensor page Add one-click "Create forecast" button to sensor page Feb 24, 2026
Copilot AI requested a review from Flix6x February 24, 2026 18:42
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
…nit__() missing 1 required argument: 'connection'`

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x marked this pull request as ready for review February 25, 2026 10:15
…docs updates

Co-authored-by: Flix6x <30658763+Flix6x@users.noreply.github.com>
Copilot AI changed the title Add one-click "Create forecast" button to sensor page Add "Create forecast" button to sensor page Feb 25, 2026
Copilot AI requested a review from Flix6x February 25, 2026 10:48
…d updating

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x added this to the 0.31.0 milestone Feb 25, 2026
@Flix6x Flix6x requested a review from nhoening February 25, 2026 11:30
@Flix6x Flix6x requested a review from joshuaunity March 4, 2026 10:04
@Flix6x
Copy link
Copy Markdown
Contributor

Flix6x commented Mar 4, 2026

I believe this PR also closes #1782.

@Flix6x Flix6x linked an issue Mar 4, 2026 that may be closed by this pull request
Copy link
Copy Markdown
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a review from me testing this out, I see two issues:

First, when there is not enough data, I cannot read the text on the button very well:

Image

Second, maybe the training period needs some tweaking?
In my example, I had uploaded two days of data, which is a week old or so. The forecasting went okay, but it forecast zeroes, maybe due to the traning period being long (Training cycle from 2026-02-09 11:15:00+01:00 to 2026-03-11 11:15:00+01:00 started ....)
The image below can illustrate it (the forecasts start 11 march). I don't think the UX around this is useful enough yet. Happy to discuss.

Image

Finally, the button is not easy to find right now. Maybe we can ask Copilot in a follo-up PR if the sliding sidebar elements can stop taking in vertical space when they are closed.

@Flix6x
Copy link
Copy Markdown
Contributor

Flix6x commented Mar 11, 2026

In my example, I had uploaded two days of data, which is a week old or so. The forecasting went okay, but it forecast zeroes [...]

The forecast doesn't look particularly wrong to me given the training data. Maybe test with a more complete history?

Finally, the button is not easy to find right now. Maybe we can ask Copilot in a follow-up PR if the sliding sidebar elements can stop taking in vertical space when they are closed.

This side panels have grown over time to the point where the sensor page is vertically scrollable. I propose that we get rid of the headers on the side panels, which I find quite redundant. I think this should reduce enough vertical spacing. [update: alas, even without the headers I can still scroll the sensor page on my laptop]

@nhoening
Copy link
Copy Markdown
Contributor

The forecast doesn't look particularly wrong to me given the training data. Maybe test with a more complete history?

I only added two days of training data, which are non-zero. I expect the model to be trained on that, not interpolated zeroes of a one-month period around my actual data.

Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x
Copy link
Copy Markdown
Contributor

Flix6x commented Mar 11, 2026

So your expectation is that the UI automatically finds out the available data range (limited within the default training period of the last 30 days?), and selects that as the training period? I don't think our API even supports setting a training end that is different than the predict start.

Flix6x added 2 commits March 11, 2026 12:24
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x requested a review from nhoening March 11, 2026 11:32
Flix6x added 3 commits March 11, 2026 14:19
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Copy link
Copy Markdown
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making a bit better use of the space. I still believe we should not have all this vertical space between the buttons, and I started a chatbot session to find a better way. Should be a follow-up PR.

I would address the UX / expectation problem in two ways (I know we want to keep this feature as simple as we can for now, but if users have a bad time, we should not ship it at all, or only show it to admins):

  • The help text should more precisely explain what the forecaster is doing, for instance that the training window is fixed and not adjusted to available data.
  • I can imagine that users often have data sets (and often in the phase when this button is interesting, they upload them), but they are not perfectly current, i.e. often there is a gap of a day or more. I suggest that the code forecasts by definition from the end of available data. We can discuss limiting that data to beliefs with horizon <= 0, and/or to add a dropdown for choosing between "forecasts start now" and "forecasts start at the end of available data")

Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x
Copy link
Copy Markdown
Contributor

Flix6x commented Mar 11, 2026

I suggest that the code forecasts by definition from the end of available data.

I'll assume you mean the UI code, which is what I would choose (and not, say the API code). I'd prefer to just expand the form with a few basic customisation options, such as the ability to set your own start time.

Signed-off-by: F.N. Claessen <claessen@seita.nl>
@nhoening
Copy link
Copy Markdown
Contributor

I suggest that the code forecasts by definition from the end of available data.

I'll assume you mean the UI code, which is what I would choose (and not, say the API code). I'd prefer to just expand the form with a few basic customisation options, such as the ability to set your own start time.

Okay, the forecasting API endpoint has a start field, which we should use. Understood.

Options I see:

  • The dialogue gets the ability to choose the start time. Actually not very easy (adding a date picker).
  • We implement my idea of the dropdown in the dialogue. For this, the frontend needs to know the event_start of the most recent belief (so it can set the start of forecasts to that time if the user chooses that option) - and it can atually get it from the stats on the same page ("Last recorded")! 👍 Potential problem: uploading new data doesn't update the stats (nor the graphs). That is something that should actually happen, maybe a bug?

Flix6x added 2 commits March 11, 2026 16:45
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x
Copy link
Copy Markdown
Contributor

Flix6x commented Mar 11, 2026

image

The datetime selector might show up differently for you, because it depends on the user's browser and locale, see https://developer.mozilla.org/en-US/docs/Web/HTML/Reference/Elements/input/datetime-local.

@nhoening
Copy link
Copy Markdown
Contributor

Okay, can we preset the datetime selector to the time f the last event, since it is already available, at least?

I want to keep it possible for this feature that the user does not have to think a lot.

Flix6x added 2 commits March 11, 2026 17:24
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
@Flix6x Flix6x requested a review from nhoening March 11, 2026 16:28
Flix6x and others added 2 commits March 11, 2026 17:39
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Nicolas Höning <nicolas@seita.nl>
Copy link
Copy Markdown
Contributor

@nhoening nhoening left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question for my understanding (about forecasts in general): For training data - which data source/s is/are used? All of them? All beliefs that are not forecasts? Even forecasts?

@Flix6x
Copy link
Copy Markdown
Contributor

Flix6x commented Apr 4, 2026

Question for my understanding (about forecasts in general): For training data - which data source/s is/are used? All of them? All beliefs that are not forecasts? Even forecasts?

  • For autoregressors: everything known prior to the current viewpoint, except data sourced by forecasters.
  • For other input: everything known prior to the current viewpoint.

This currently happens in data.models.forecasting.pipelines.base.BasePipeline.load_data_all_beliefs.

I could imagine we'd also want to exclude data sources by schedulers. So far we haven't encounter the need, presumably because the sensors we forecast are not also the sensors we schedule.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Forecasting button Fetch new sensor data when saving sensors_to_show

4 participants