Skip to content

Commit 6248efd

Browse files
authored
Allow mapping over map results (#4302)
1 parent 13cd412 commit 6248efd

File tree

2 files changed

+13
-1
lines changed

2 files changed

+13
-1
lines changed

docs/book/how-to/steps-pipelines/dynamic_pipelines.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -123,7 +123,10 @@ def map_reduce():
123123
```
124124

125125
Key points:
126-
- `step.map(...)` fans out a step over sequence-like inputs.
126+
- `step.map(...)` fans out a step over sequence-like inputs. These inputs can be either
127+
- a single list-like output artifact (see the code sample above)
128+
- a list of output artifacts.
129+
- the output of a `.map(...)` or `.product(...)` call if the respective step only returns a single output artifact
127130
- Steps can accept lists of artifacts directly as inputs (useful for reducers).
128131
- You can pass the mapped output directly to a downstream step without loading in the orchestration environment.
129132

src/zenml/execution/pipeline/dynamic/runner.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -642,6 +642,15 @@ def expand_mapped_inputs(
642642
"wrap your input with the `unmapped(...)` function.",
643643
key,
644644
)
645+
elif (
646+
isinstance(value, Sequence)
647+
and value
648+
and all(isinstance(item, OutputArtifact) for item in value)
649+
):
650+
# List of step output artifacts, in this case the mapping is over
651+
# the items of the list
652+
mapped_input_names.append(key)
653+
mapped_inputs.append(tuple(value))
645654
elif isinstance(value, Sequence):
646655
logger.warning(
647656
"Received sequence-like data for step input `%s`. Mapping over "

0 commit comments

Comments
 (0)