parallel_chat and parallel_chat_structured return different outputs shapes

[bakaburg1]

Summary

parallel_chat() and parallel_chat_structured() currently return different kinds of objects:

• parallel_chat() returns a list of Chat objects, NULLs, or error objects
• parallel_chat_structured() returns a structured data object, typically a data frame

This makes downstream code more complex than it needs to be, because callers have to branch on which function they used and handle two different result shapes.

Why this matters

The two functions are very close in intent:

• both submit multiple prompts in parallel
• both can fail on individual rows
• both may need per-row recovery logic

But because the return shapes differ, callers cannot treat them as drop-in alternatives.

That makes it harder to:

• write generic post-processing
• salvage parse failures
• retry only failing rows
• keep uniform logging and metadata handling

Proposed direction

A more uniform design would be for both functions to return a list of Chat objects, with the raw assistant text preserved even when structured parsing fails.

For parallel_chat_structured(), that would mean:

• returning the raw unprocessed string on parse failure
• keeping enough metadata to allow ad hoc parsing or recovery later
• not collapsing the result directly into a data frame too early

That way callers could inspect and salvage failed turns instead of losing the raw output.

Even better

An even cleaner API would be to let parallel_chat() accept an optional type argument:

• if type is absent, return normal chat outputs
• if type is present, return structured outputs

That would make the structured and unstructured paths true drop-ins with a single entry point.

Example

A model such as:

https://openrouter.ai/openai/gpt-oss-20b:free

can return JSON-like output that is close to valid but not always perfectly parseable. In those cases, preserving the raw string would let callers recover the response with custom parsing logic.

[hadley]

The whole point of those functions is that they return different shapes of output. They are not designed to be drop in alternatives. So I don't really understand the point of your proposal. Maybe if you provided a concrete example of what you are trying to do it would be easier for me to understand?

[bakaburg1]

I'm developing a general LLM solver to integrate with the Vitals package for various benchmarking tasks. It features batched parallelisation and caching. It allows users to choose between a structured or regular parallel_chat depending on whether an Ellmer-typed structure is passed among the arguments.

So here was the bottleneck. I hoped to seamlessly drop in one of the two parallel_chat functions based solely on the presence of the structured type argument.

But then I discovered they returned results in different shapes, so I was forced to make a lot of glue code to adapt the outputs, since vitals requires chat objects (I had to reconstruct them for parallel structured).

Also, if parallel_chat_structured fails the parsing, since only a simple data frame is returned, there is no way to recover the raw string or obtain more information about the failure reason.

I think that having both of them return the same chat object would simplify the interface. Even better if there was just one function whose behaviour depends on whether the structured type argument is present or not, as I was trying to do.

An additional feature could be an argument to select whether to return just the result or entire chat objects.