[feature] tool call support in vlm

[jordane95]

User problem

looks like no support for tool use in current vlm dataset, which is critical for visual agentic model. would be great to support it

Desired outcome

Add support for tool use in vlm training

Alternatives considered

No response

Affected area

area:data

Urgency / use case

Blocking current work

Extra context

No response

[cuichenx]

Hi @jordane95 if the dataset has tool calling, then we support finetuning on HF datasets with the correct chat template. Could you clarify what the request is? Are you requesting a tool calling tutorial?