Skip to content

Commit 7002539

Browse files
joshuaunityFlix6xnhoening
authored
feat: Adapting flex-config to SensorsToShowSchema (#1904)
* refactor: refactored SensorsToShowSchema to validate into new shape Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: debugging - work in progress Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: debug errors Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fixed cahrts failing to render Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: udpate test case with new schema changes Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * tests: adapting more testcases to new schema shape Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fix failing test Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fixed failing api due to logic oversight Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: handle asset plot entry Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: add changelog entry Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * Update documentation/changelog.rst Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/data/schemas/utils.py Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/data/schemas/generic_assets.py Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/data/schemas/generic_assets.py Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/data/schemas/generic_assets.py Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * fix: fixed schema bugs Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fixed skipped validation step Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: add docstring to schema functions Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: little changes Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * tests: expanding test case Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: more backward compatibility refactoring Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: update docstring for SensorsToShowSchema Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: support for old sensor to show format for flatenen functions Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * tests: change test reference asset Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * test: apply fixture to test case due to asset resource not found Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: multiple followups across docs and schema based on PR request changes Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fixed failing test Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: update tpy accoutn data relating to sensors_to_show Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * tests: update test cases Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: refactored util function for backward compatibility Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * tests: fixed failing tests - phase 2 Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: Fix failing pipeline due to code indentation Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * Feat/clean up duplicate flattening implementation (#1984) * refactor: move extract_sensors_from_flex_config to schemas/generic_assets.py Signed-off-by: F.N. Claessen <claessen@seita.nl> * delete: internal import no longer needed Signed-off-by: F.N. Claessen <claessen@seita.nl> * refactor: rename util method to flatten_sensors_to_show Signed-off-by: F.N. Claessen <claessen@seita.nl> * refactor: get rid of duplicate implementation for flattening sensors to show Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: make doctest out of example Signed-off-by: F.N. Claessen <claessen@seita.nl> * style: rst-style docstrings Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: support multiple flex-config field names Signed-off-by: F.N. Claessen <claessen@seita.nl> * feat: test flatten function Signed-off-by: F.N. Claessen <claessen@seita.nl> --------- Signed-off-by: F.N. Claessen <claessen@seita.nl> * Update documentation/views/asset-data.rst Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/data/schemas/generic_assets.py Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Feat: Graph modal upgrade - flex-config integration (#1926) * chore: multiple updates Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: stabilized leeft side of modal and kicked off work onteh right side, including dissabling logic Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: completed left side of graph modal Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: work in progress Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: more stabilization work as well as reactivation of broken features Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: reorder tabs Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fixed bug whre options keep gettgin added teh configType dropdown. Also added some docstring Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fixed error where graph cant be removed Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: ixed issue with graph titles not being editable Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fixed broken units dropdown as well as some other refactoring to revive other broken features Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: Major refactor phase 1 Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: Major refactor phase 2 Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * feat; new util function to find an asset site_asset Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: add extra info icon to form elements Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: update writeup Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fix failing util function due to wrong formatting allowing trailing zeros Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fixed bug where a new sensor cant be added to an existing graph Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: Fixed graphs to properly display subcharts and mixed charts containg fixed and dynamic value plots Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: removed unused code Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: post resolving conflicts refactoring Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * tests: remove unsupported edgecase Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: fix issue where unsuppoerted flexocntext fields are sent to API, causing validation error Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * feat: up to date asset data for graph modals Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: Fix bug where you cant remove sensors from a grpah with multiple sensors Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: amek all asset ref inputs equal size Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * Update flexmeasures/ui/templates/assets/asset_graph.html Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/ui/templates/assets/asset_graph.html Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * fix: fixed error when trying to remove polts thats not a array of sensors Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: Fixed bug where the options to select a felxConfig field disables the first option, in turn the first option coudlnt be selected Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: imporve fuction to improve readability and also fix edgecase bug Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: Fixed bug where sensors filter sectoin doesnt change after a selected card has been remvoed, still shows 'Add to X graph' Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: udpate temp sensor naming sceme Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: persisten root asset selection Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: implement new data format in toy data for testing Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: removed unused values Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: rename util function name Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: adding comments Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * feat: dynamically render graph title on 'add plot' button Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: remvoed duplicate asset name in graph ledgend Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: fetch sensors based on site asset instead of current asset Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * Dynamically change ledgen titles Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: little change to wording Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: transfer initialization check into component for rendering graphs Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * Update flexmeasures/ui/templates/assets/asset_graph.html Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/ui/templates/assets/asset_graph.html Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/ui/templates/assets/asset_graph.html Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/ui/templates/assets/asset_graph.html Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/ui/templates/assets/asset_graph.html Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/ui/templates/assets/asset_graph.html Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * Update flexmeasures/ui/templates/assets/asset_graph.html Co-authored-by: Nicolas Höning <nicolas@seita.nl> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> * chore: name change Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: backward compatibility support and imporvement UI data representation Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * fix: Fix issue where subcharts are split up into single cahrts on the frontend Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: Improve asset identification in plot headers and enhance sensor rendering logic Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * refactor: Simplify asset plot card rendering and improve code readability Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * style: Adjust disabled input width and clean up class names in asset plot card Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * style: Update plot header class for improved font size consistency Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * chore: add changelog entry Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> * update OpenAPI specs Signed-off-by: Nicolas Höning <nicolas@seita.nl> * address a flake8, mypy, precommit errors Signed-off-by: Nicolas Höning <nicolas@seita.nl> --------- Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> Signed-off-by: Nicolas Höning <nicolas@seita.nl> Co-authored-by: Nicolas Höning <nicolas@seita.nl> * style: black Signed-off-by: F.N. Claessen <claessen@seita.nl> --------- Signed-off-by: joshuaunity <oghenerobojosh01@gmail.com> Signed-off-by: JDev <45713692+joshuaunity@users.noreply.github.com> Signed-off-by: F.N. Claessen <claessen@seita.nl> Signed-off-by: Nicolas Höning <nicolas@seita.nl> Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Co-authored-by: Nicolas Höning <nicolas@seita.nl> Co-authored-by: F.N. Claessen <claessen@seita.nl>
1 parent d4b76ee commit 7002539

23 files changed

Lines changed: 2330 additions & 758 deletions

documentation/changelog.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,9 @@ New features
8484
* Support creating schedules with only information known prior to some time, now also via the CLI (the API already supported it) [see `PR #1871 <https://www.github.com/FlexMeasures/flexmeasures/pull/1871>`_].
8585
* Added capability to update an asset's parent from the UI [`PR #1957 <https://www.github.com/FlexMeasures/flexmeasures/pull/1957>`_]
8686
* Add ``fields`` param to the asset-listing endpoints, to save bandwidth in response data [see `PR #1884 <https://www.github.com/FlexMeasures/flexmeasures/pull/1884>`_]
87+
* Support for flex-config in the ``SensorsToShowSchema`` [see `PR #1904 <https://www.github.com/FlexMeasures/flexmeasures/pull/1904>`_]
88+
* Upgrade graph modal to support flex-config references in plots [see `PR #1926 <https://www.github.com/FlexMeasures/flexmeasures/pull/1926>`_]
89+
8790

8891
.. note:: For backwards-compatibility, the new ``fields`` parameter will only be fully active, i.e. also returning less fields per default, in v0.32. Set ``FLEXMEASURES_API_SUNSET_ACTIVE=True`` to test the full effect now.
8992

documentation/views/asset-data.rst

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -108,19 +108,20 @@ Editing the graphs dashboard
108108

109109
Click the "Edit Graph" button to open the graph editor.
110110

111-
Use the "Add Graph" button to create graphs. For each graph, you can select one or more sensors, from all available sensors associated with the asset, including public sensors, and add them to your plot.
111+
Use the "Add Graph" button to create graphs. For each graph, you can select one or more sensors, from all available sensors associated with the asset, including public sensors, and add them to your plot.
112112

113-
You can overlay data from multiple sensors on a single graph. To do this, click on an existing plot and add more sensors from the available options on the right.
113+
In addition, you can add an asset's flex-config to the graph, as long as the value of that config is a sensor(e.g. `[{"title":"Power","plots":[{"sensor":2}]},{"title":"Costs","plots":[{"sensors":[5,6]}]}]`).
114114

115-
Finally, it is possible to set custom titles for any sensor graph by clicking on the "edit" button right next to the default or current title.
115+
Finally, it is possible to set custom titles for any graph by clicking on the "edit" button right next to the default or current title.
116116

117117
.. image:: https://github.com/FlexMeasures/screenshots/raw/main/screenshot-asset-editgraph.png
118118
:align: center
119119
.. :scale: 40%
120120
121121
|
122122
123-
Internally, the asset has a `sensors_to_show` field, which controls which sensor data appears in the plot. This can also be set by a script. The accepted format is a dictionary with a graph title and a lists of sensor IDs (e.g. `[{"title": "Power", "sensor": 2}, {"title": "Costs", "sensors": [5,6] }]`).
123+
Internally, the asset has a `sensors_to_show` field, which controls which sensor data appears in the plot. This can also be set by a script or through the API.
124+
The accepted format is a dictionary with a graph title followed by a plot containing senors or asset flex-config reference (e.g. `[{"title":"Power","plots":[{"sensor":2}]},{"title":"Costs","plots":[{"sensor":5},{"asset":10,"flex-model":"soc-min"},]}]`).
124125

125126

126127
Showing daily KPIs

flexmeasures/api/v3_0/assets.py

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@
5050
GenericAssetSchema as AssetSchema,
5151
GenericAssetIdField as AssetIdField,
5252
GenericAssetTypeSchema as AssetTypeSchema,
53+
SensorsToShowSchema,
5354
)
5455
from flexmeasures.data.schemas.scheduling import AssetTriggerSchema
5556
from flexmeasures.data.services.scheduling import (
@@ -63,9 +64,6 @@
6364
)
6465
from flexmeasures.api.common.schemas.users import AccountIdField
6566
from flexmeasures.api.common.schemas.assets import default_response_fields
66-
from flexmeasures.utils.coding_utils import (
67-
flatten_unique,
68-
)
6967
from flexmeasures.ui.utils.view_utils import clear_session, set_session_variables
7068
from flexmeasures.auth.policy import check_access
7169
from flexmeasures.data.schemas.sensors import (
@@ -913,7 +911,7 @@ def get_chart_data(self, id: int, asset: GenericAsset, **kwargs):
913911
tags:
914912
- Assets
915913
"""
916-
sensors = flatten_unique(asset.validate_sensors_to_show())
914+
sensors = SensorsToShowSchema.flatten(asset.validate_sensors_to_show())
917915
return asset.search_beliefs(sensors=sensors, as_json=True, **kwargs)
918916

919917
@route("/<id>/auditlog")

flexmeasures/cli/data_add.py

Lines changed: 20 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1826,19 +1826,31 @@ def create_asset_with_one_sensor(
18261826
db.session.flush()
18271827
battery = discharging_sensor.generic_asset
18281828
battery.sensors_to_show = [
1829-
{"title": "Prices", "sensor": day_ahead_sensor.id},
1829+
{"title": "Prices", "plots": [{"sensor": day_ahead_sensor.id}]},
18301830
{
18311831
"title": "Power flows",
1832-
"sensors": [production_sensor.id, discharging_sensor.id],
1832+
"plots": [
1833+
{"sensors": [production_sensor.id, discharging_sensor.id]},
1834+
],
18331835
},
18341836
]
18351837

18361838
# the site gets a similar dashboard (TODO: after #1801, add also capacity constraint)
18371839
building_asset.sensors_to_show = [
1838-
{"title": "Prices", "sensor": day_ahead_sensor.id},
1840+
{
1841+
"title": "Prices",
1842+
"plots": [
1843+
{
1844+
"asset": building_asset.id,
1845+
"flex-context": "consumption-price",
1846+
}
1847+
],
1848+
},
18391849
{
18401850
"title": "Power flows",
1841-
"sensors": [production_sensor.id, discharging_sensor.id],
1851+
"plots": [
1852+
{"sensors": [production_sensor.id, discharging_sensor.id]},
1853+
],
18421854
},
18431855
]
18441856

@@ -1878,10 +1890,10 @@ def create_asset_with_one_sensor(
18781890

18791891
process = shiftable_power.generic_asset
18801892
process.sensors_to_show = [
1881-
{"title": "Prices", "sensor": day_ahead_sensor.id},
1882-
{"title": "Inflexible", "sensor": inflexible_power.id},
1883-
{"title": "Breakable", "sensor": breakable_power.id},
1884-
{"title": "Shiftable", "sensor": shiftable_power.id},
1893+
{"title": "Prices", "plots": [{"sensor": day_ahead_sensor.id}]},
1894+
{"title": "Inflexible", "plots": [{"sensor": inflexible_power.id}]},
1895+
{"title": "Breakable", "plots": [{"sensor": breakable_power.id}]},
1896+
{"title": "Shiftable", "plots": [{"sensor": shiftable_power.id}]},
18851897
]
18861898

18871899
db.session.commit()

flexmeasures/conftest.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -548,6 +548,31 @@ def create_assets(
548548
),
549549
)
550550
db.session.add(sensor)
551+
scc_sensor = Sensor(
552+
name="site-consumption-capacity",
553+
generic_asset=asset,
554+
event_resolution=timedelta(minutes=15),
555+
unit="MW",
556+
)
557+
db.session.add(scc_sensor)
558+
cc_sensor = Sensor(
559+
name="consumption-capacity",
560+
generic_asset=asset,
561+
event_resolution=timedelta(minutes=15),
562+
unit="MW",
563+
)
564+
db.session.add(cc_sensor)
565+
pc_sensor = Sensor(
566+
name="production-capacity",
567+
generic_asset=asset,
568+
event_resolution=timedelta(minutes=15),
569+
unit="MW",
570+
)
571+
db.session.add(pc_sensor)
572+
db.session.flush() # assign sensor IDs
573+
asset.flex_model["consumption-capacity"] = {"sensor": cc_sensor.id}
574+
asset.flex_model["production-capacity"] = {"sensor": pc_sensor.id}
575+
asset.flex_context["site-consumption-capacity"] = {"sensor": scc_sensor.id}
551576
assets.append(asset)
552577

553578
# one day of test data (one complete sine curve)

flexmeasures/data/migrations/versions/e690d373a3d9_copy_Power_Price_Weather_time_series_data_to_TimedBeliefs_table.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -118,7 +118,7 @@ def copy_time_series_data(
118118
# Copy in batches and report on progress
119119
for i in range(len(results) // batch_size + 1):
120120
if i > 0:
121-
print(f" - done copying {i*batch_size} rows...")
121+
print(f" - done copying {i * batch_size} rows...")
122122

123123
insert_values = []
124124
for values in results[i * batch_size : (i + 1) * batch_size]:

flexmeasures/data/models/charts/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ def chart_type_to_chart_specs(chart_type: str, **kwargs) -> dict:
1919
for chart_type, chart_specs in getmembers(belief_charts)
2020
if isfunction(chart_specs) or isinstance(chart_specs, dict)
2121
}
22+
2223
# Create chart specs
2324
chart_specs_or_fnc = belief_charts_mapping[chart_type]
2425
if isfunction(chart_specs_or_fnc):

0 commit comments

Comments
 (0)