Enable PartitionRouter and IncrementingCountCursor in the Low-Code Framework

[Carl Witt (carlwitt)]

It feels like a simple problem to solve – just maintain a cursor for each partition.
I'm fetching various event streams from an /events/{stream} endpoint. I use the partition router to iterate the stream names. The endpoint has a query parameter startSeq that I can use for incremental sync.

The yaml created by the low code builder looks like this:

  - type: DeclarativeStream
    name: Events
    retriever:
      type: SimpleRetriever
      requester:
        type: HttpRequester
        url: >-
          {{config['knime_business_hub_api_endpoint'] }}/events/adhoc/{{
          stream_partition['stream'] }}
      // ...
      partition_router:
        type: ListPartitionRouter
        values:
          - catalog-repository-items
          - accounts-users
          - accounts-teams
          - accounts-groups
          - ai
        cursor_field: stream
    // ...
    incremental_sync:
      type: IncrementingCountCursor
      start_value: "0"
      cursor_field: id
      start_value_option:
        type: RequestOption
        field_name: startSeq
        inject_into: request_parameter

Which leads to

Failed to save ... due to the following error: 'Encountered an error while discovering streams. Error: The low-code framework does not currently support usage of a PartitionRouter and an IncrementingCountCursor at the same time. Please specify only one of these options for stream Events.'