Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,13 @@ description: "Learn how to connect Arcade to your LLM in Python"
import { Steps, Tabs, Callout } from "nextra/components";
import { SignupLink } from "@/app/_components/analytics";

<!-- Editorial: Voice and Tone - Changed "you'll" to "you will" for more direct tone; Structure - Added intro line explaining what reader will learn -->

# Connect Arcade to your LLM

Arcade tools work alongside an LLM. To make that work, you need a small piece of glue code called a "harness." The harness orchestrates the back-and-forth between the user, the model, and the tools. In this guide, you'll build one so you can wire Arcade into your LLM-powered app.
Connect Arcade to your LLM in Python by building a harness that orchestrates tool execution.

Arcade tools work alongside an LLM. To make that work, you need a small piece of glue code called a "harness." The harness orchestrates the back-and-forth between the user, the model, and the tools. In this guide, you will build one so you can wire Arcade into your LLM-powered app.

<GuideOverview>
<GuideOverview.Outcomes>
Expand Down Expand Up @@ -94,7 +98,7 @@ OPENROUTER_MODEL=YOUR_OPENROUTER_MODEL
</Callout>

<Callout type="info">
In this example, you're using OpenRouter to access the model, as it makes it
In this example, you are using OpenRouter to access the model, as it makes it
straightforward to use any model from multiple providers with a single API.

OpenRouter is compliant with the OpenAI API specification, so you can use it
Expand Down Expand Up @@ -132,7 +136,7 @@ llm_client = OpenAI(

### Select and retrieve the tools from Arcade

In this example, you're implementing a multi-tool agent that can retrieve and send emails, as well as send messages to Slack. While a harness can expose a broad catalog of tools to the LLM, it's best to limit that set to what's relevant for the task to keep the model efficient.
In this example, you are implementing a multi-tool agent that can retrieve and send emails, as well as send messages to Slack. While a harness can expose a broad catalog of tools to the LLM, it's best to limit that set to what's relevant for the task to keep the model efficient.

```python filename="main.py"
# Define the tools for the agent to use
Expand Down Expand Up @@ -182,11 +186,7 @@ def authorize_and_run_tool(tool_name: str, input: str):
return json.dumps(result.output.value)
```

This helper function adapts to any tool in the catalog and will make sure that the authorization requirements are met before executing the tool. For more complex agentic patterns, this is generally the best place to handle interruptions that may require user interaction, such as when the tool requires a user to approve a request, or to provide additional context.

### Write a helper function that handles the LLM's invocation

There are many orchestration patterns that can be used to handle the LLM invocation. A common pattern is a ReAct architecture, where the user prompt will result in a loop of messages between the LLM and the tools, until the LLM provides a final response (no tool calls). This is the pattern we will implement in this example.
This helper function adapts to any tool in the catalog and ensures that the authorization requirements satisfy the tool before executing it. Different orchestration patterns can handle the LLM invocation. A common pattern is a ReAct architecture, where the user prompt will result in a loop of messages between the LLM and the tools, until the LLM provides a final response (no tool calls). This is the pattern this example implements.

To avoid the risk of infinite loops, limit the number of turns (in this case, a maximum of 5). This is a parameter that you can tune to your needs. Set it to a value that is high enough to allow the LLM to complete its task but low enough to prevent infinite loops.

Expand Down Expand Up @@ -244,11 +244,11 @@ def invoke_llm(
return history
```

These two helper functions form the core of your agentic loop. Notice that authorization is handled outside the agentic context, and the tool execution is passed back to the LLM in every case. Depending on your needs, you may want to handle tool orchestration within the harness and pass only the final result of multiple tool calls to the LLM.
These two helper functions form the core of your agentic loop. Notice that the harness handles authorization outside the agentic context, and the tool execution result goes back to the LLM in every case. Depending on your needs, you may want to handle tool orchestration within the harness and pass only the final result of multiple tool calls to the LLM.

### Write the main agentic loop

Now that you've written the helper functions, write an agentic loop that interacts with the user. The core pieces of this loop are:
Now that you have written the helper functions, write an agentic loop that interacts with the user. The core pieces of this loop are:

1. Initialize the conversation history with the system prompt
2. Get the user input and add it to the conversation history
Expand Down Expand Up @@ -314,7 +314,7 @@ With the selection of tools above, you should be able to get the agent to effect
## Next Steps

- Learn more about using Arcade with a [framework](/guides/agent-frameworks) or [MCP client](/guides/tool-calling/mcp-clients).
- Learn more about how to [build your own MCP Servers](/guides/create-tools/tool-basics/build-mcp-server).
- Learn more about how to [build your own MCP servers](/guides/create-tools/tool-basics/build-mcp-server).

## Example code

Expand Down Expand Up @@ -468,4 +468,4 @@ def chat():
if __name__ == "__main__":
chat()
```
</details>
</details>