GitHub - byt3bl33d3r/figaro: Orchestrate fleets of Claude Code & Claude Computer Use agents across containers, VMs, and physical devices. Live desktop streaming, intelligent task delegation, and multi-channel human-in-the-loop gateway. Schedule tasks with self-healing and self-learning.

Figaro Logo

Figaro orchestrates fleets of Claude computer use agents that automate workflows on full desktop environments. Agents run inside containerized Linux desktops or connect to any VNC-accessible machine -- remote servers, cloud VMs, or physical workstations. All desktops are live-streamed to a central dashboard, and a supervisor agent handles task delegation. Everything can be managed conversationally through external channels like Telegram.

Figaro can connect to any accessible desktop -- local machines, remote servers, cloud VMs, or physical workstations running macOS, Windows, or Linux. Desktops are added from the UI with a connection URL (vnc://, rdp://, ssh://, telnet://, ws://, or wss://), and the supervisor agent can observe and interact with any connected desktop via screenshots, typing, clicking, and key presses. Desktop streaming uses Apache Guacamole (guacd + guacamole-common-js) for protocol-agnostic remote access.

The system is built for long-running tasks that take minutes to hours. All services communicate over NATS (pub/sub + JetStream for durable task events). A supervisor agent handles task optimization and delegation.

You can also manage everything by chatting with the supervisor agent through the gateway -- for example, via Telegram. Send it natural language instructions to create tasks, schedule recurring jobs, check worker status, or ask questions about running tasks. The supervisor understands the full system and can delegate work to workers, inspect desktops via VNC, and report back results, all through a conversational interface.

UI Action shot

Figaro Dashboard

Quick Start

curl -fsSL https://raw.githubusercontent.com/byt3bl33d3r/figaro/main/install.sh | bash

Or clone and run directly:

git clone https://github.com/byt3bl33d3r/figaro.git && cd figaro && ./install.sh

By default the install script uses the prod-local overlay, Figaro will be available at http://localhost:8000.

Prerequisites

Docker and Docker Compose -- on Linux the install script will install Docker automatically via get.docker.com. On macOS you must install Docker Desktop manually before running the script.
Claude credentials (~/.claude.json and ~/.claude/.credentials.json) -- created by running claude and signing in. On macOS (assuming you have a Claude Code subscription) once logged in, you can export the credentials file for use in containers with the following command:
```
security find-generic-password -s "Claude Code-credentials" -w > ~/.claude/.credentials.json
```

Optional

An OpenAI API key (used for patchright-cli transcription functionality)
A Telegram bot token (enables chatting with the supervisor agent via Telegram for task submission, status checks, and notifications). Create a bot via @BotFather and set GATEWAY_TELEGRAM_BOT_TOKEN and GATEWAY_TELEGRAM_ALLOWED_CHAT_IDS in your .env

Advanced Setup

Note

The install.sh script does this for you.

cp .env.example .env

# Generate an encryption key for VNC password storage (required)
echo "FIGARO_ENCRYPTION_KEY=$(openssl rand -hex 16)" >> .env

# Optional: set an OPENAPI_API_KEY
# OPENAI_API_KEY=sk-proj-example
# Optional: set Telegram gateway variables for notifications and task submission
# GATEWAY_TELEGRAM_BOT_TOKEN=your-bot-token
# GATEWAY_TELEGRAM_ALLOWED_CHAT_IDS=["your-chat-id"]

VNC passwords are encrypted at rest in PostgreSQL using pgcrypto. The FIGARO_ENCRYPTION_KEY is used for symmetric encryption and must be set before starting the orchestrator. The install.sh script generates this automatically.

The base docker/docker-compose.yml defines all shared services but does not expose ports. Choose an overlay for your deployment scenario:

# Production, localhost-only (recommended) -- ports bound to 127.0.0.1
docker compose -f docker/docker-compose.yml -f docker/docker-compose.prod-local.yml up --build

# Development -- localhost ports + a desktop service for testing VNC
docker compose -f docker/docker-compose.yml -f docker/docker-compose.dev.yml up --build

Caution

The docker/docker-compose.prod.yml overlay binds ports to 0.0.0.0, making NATS and the orchestrator accessible from any network interface. Only use this if you understand the security implications and have appropriate firewall rules in place. Please read Security to understand known attack surface.

docker compose -f docker/docker-compose.yml -f docker/docker-compose.prod.yml up --build

Open http://localhost:8000.

This starts PostgreSQL, NATS (port 8443), guacd (Guacamole daemon), the orchestrator (port 8000), 2 workers, 2 supervisors, and the gateway. The supervisor service uses the same figaro-worker binary started with the --supervisor flag.

Scaling

docker compose -f docker/docker-compose.yml -f docker/docker-compose.prod-local.yml up --build --scale worker=4 --scale supervisor=3

Teardown

docker compose -f docker/docker-compose.yml -f docker/docker-compose.prod-local.yml down       # Stop services
docker compose -f docker/docker-compose.yml -f docker/docker-compose.prod-local.yml down -v    # Stop and remove all data

Connecting External Desktops

Figaro is not limited to its own containerized workers. Any machine with a VNC server can be connected as a desktop.

From the UI

Click "Add Desktop" in the dashboard header. Provide a worker ID, a desktop URL, and select the OS type. Supported URL schemes:

vnc://user:password@hostname:5901 -- direct TCP VNC connection
rdp://user:password@hostname:3389 -- RDP connection
ssh://user:password@hostname:22 -- SSH connection
telnet://user:password@hostname:23 -- telnet connection
ws://hostname:6080 -- WebSocket (legacy noVNC-compatible)
wss://hostname:6080 -- WebSocket over TLS (legacy noVNC-compatible)

Credentials can be embedded in the URL or entered separately. Connected desktops appear in the live desktop grid and can be viewed, screenshotted, and interacted with by the supervisor agent's VNC tools. SSH and telnet connections are accessible via the supervisor's terminal tools (ssh_run_command, telnet_run_command).

From Environment Variables

Pre-configure desktops at startup via the FIGARO_DESKTOP_WORKERS environment variable:

FIGARO_DESKTOP_WORKERS='[{"id": "mac-studio", "novnc_url": "vnc://user:pass@192.168.1.50:5900", "metadata": {"os": "macos"}}]'

Agent Upgrade Path

Desktop-only entries act as placeholders. When a worker agent connects with a matching ID, the desktop is automatically upgraded to a full agent worker capable of receiving tasks. When the agent disconnects, the desktop reverts to view-only mode rather than disappearing. This means you can pre-register your desktops and have agents attach and detach dynamically.

This is useful when you want to connect existing physical machines or VMs to Figaro without running the full containerized worker stack. For example, you can point Figaro at your Mac Mini's VNC server, and the supervisor can already observe and interact with its screen. Later, run the figaro-worker agent on that machine to give it full task execution capabilities -- the desktop entry upgrades seamlessly without any reconfiguration. If you install patchright-cli on that machine it becomes particularly handy: it's a standalone browser automation CLI that speaks the same claude-agent-sdk protocol, so you can drop it onto any machine with a browser and instantly level up your browser automation tasks.

Scheduled Tasks

Figaro Scheduled Tasks

Figaro supports cron-like scheduling for recurring tasks. Scheduled tasks are managed through the UI or directly by chatting to Figaro via Telegram or whatever channel is configured in the Gateway.

Each scheduled task has:

Prompt -- the task instruction sent to the agent
Cron expression -- standard cron syntax for scheduling (e.g., 0 9 * * * for daily at 9 AM)
Start URL -- optional URL the worker's browser should navigate to before starting
Enabled/disabled toggle -- pause and resume without deleting
Manual trigger -- execute immediately regardless of schedule or enabled state
Notify on completion via Gateway -- send a notification to configured gateway channels (e.g., Telegram) when the task completes or fails
Self-learning -- optional automatic prompt optimization after each run

Scheduled tasks are assigned to supervisors, which delegate to workers. The scheduler checks for due tasks every 60 seconds and assigns them to idle supervisors (or queues them if none are available).

NATS API

figaro.api.scheduled-tasks                # List all scheduled tasks
figaro.api.scheduled-tasks.get            # Get a specific scheduled task
figaro.api.scheduled-tasks.create         # Create a new scheduled task
figaro.api.scheduled-tasks.update         # Update an existing scheduled task
figaro.api.scheduled-tasks.delete         # Delete a scheduled task
figaro.api.scheduled-tasks.toggle         # Enable or disable a scheduled task
figaro.api.scheduled-tasks.trigger        # Trigger immediate execution

Self-Healing

When a task fails, the orchestrator can automatically create a "healer" task that analyzes the failure and retries with an improved approach. This is designed for browser automation where failures are often transient -- an element didn't load in time, a page layout changed, or the agent navigated to the wrong place.

How It Works

A worker publishes an error to figaro.task.{id}.error
The orchestrator evaluates whether to heal based on a priority chain:
- Task-level: the self_healing flag on the specific failed task
- Scheduled task-level: the self_healing field on the associated scheduled task (if any)
- System-level: FIGARO_SELF_HEALING_ENABLED environment variable (default: true)
If healing is enabled and retries remain (< FIGARO_SELF_HEALING_MAX_RETRIES, default: 2), a healer task is created
The healer task contains the original prompt, the error message, and the conversation history (last 50 messages)
A supervisor analyzes the failure and decides:
- Recoverable (element not found, timing issues, navigation errors): delegates to a worker with an improved prompt. May use VNC tools to inspect the desktop state first.
- Unrecoverable (invalid credentials, service down, fundamental approach failure): explains why and does not retry.
If the retried task also fails and retries remain, another healer is created (up to max_retries)

Loop Prevention

Tasks with source="healer" or source="optimizer" are never healed -- only original tasks trigger healing
The retry chain is tracked via original_task_id and retry_number to enforce the max retries limit

Configuration

Variable	Default	Description
`FIGARO_SELF_HEALING_ENABLED`	`true`	Enable automatic retry of failed tasks
`FIGARO_SELF_HEALING_MAX_RETRIES`	`2`	Maximum healer retries per task chain

Self-Learning

Scheduled tasks with self_learning enabled automatically optimize their prompts after each completed run. The system analyzes how the task actually executed and rewrites the prompt to be more effective next time.

How It Works

A worker completes a scheduled task
The orchestrator checks: does the task have a scheduled_task_id, source="scheduler", and self_learning=True?
If so, it retrieves the conversation history, filters to key message types (assistant, tool_result, result), and caps at the last 50 messages
An optimization task is created with the current prompt and conversation history, then assigned to an idle supervisor
The supervisor analyzes what happened during execution and calls update_scheduled_task to save an improved prompt
The next scheduled run uses the improved prompt

Design Details

Optimization runs fire-and-forget as a background task -- failures are logged but don't affect the main task completion flow
Optimization runs after every completed scheduled task run (no throttling)
The supervisor is instructed to only update the prompt field, preserving schedule, enabled state, and start URL settings
Self-learning is toggled per scheduled task via a checkbox in the UI

Agent Memory

Figaro has a persistent memory system that lets agents store and retrieve learned knowledge across task executions. Memories are stored in PostgreSQL with hybrid search (BM25 full-text via ParadeDB + pgvector semantic similarity), accessed via NATS request/reply through the agents' Python sandbox.

How It Works

Both supervisors and workers have access to memory functions (figaro.save_memory(), figaro.search_memories(), figaro.list_memories(), figaro.delete_memory()) available in their python_exec tool via a built-in figaro module. Claude Code's built-in Memory tool is disabled in favor of this custom implementation.

The supervisor workflow integrates memory at two points:

Before delegation: searches memories for relevant context (site tips, user preferences, past errors) and injects it into the worker's prompt
After task completion: saves any new learnings as memories for future tasks

Memory Structure

Each memory has:

Field	Description
`content`	Free-text memory content
`collection`	Category for organization (`sites`, `users`, `errors`, `workflows`, or `default`)
`metadata`	Arbitrary JSON key-value pairs
`embedding`	1536-dimensional vector (OpenAI `text-embedding-3-small`) for semantic search

Standard collections:

sites -- site-specific knowledge (URLs, login flows, navigation patterns, cookie banners)
users -- user preferences and requirements
errors -- failure patterns and known fixes
workflows -- multi-step procedures for recurring tasks

Search

Memory search uses Reciprocal Rank Fusion (RRF) combining BM25 full-text search and pgvector cosine similarity. BM25 receives 2x weight in the fusion. If no OpenAI API key is configured, embeddings are skipped and search falls back to BM25 only.

Memories are deduplicated by (collection, content_hash) -- saving the same content to the same collection updates the existing record.

NATS API

figaro.api.memories.save                  # Save a memory (upsert by content hash)
figaro.api.memories.search                # Hybrid BM25 + vector search
figaro.api.memories.list                  # List memories (optional collection filter)
figaro.api.memories.delete                # Delete a memory by ID

Configuration

Variable	Default	Description
`OPENAI_API_KEY`	--	OpenAI API key for embedding generation (optional, degrades to BM25-only search)

Gateway

The gateway is a channel-agnostic messaging service that routes messages between external communication channels (Telegram, future: WhatsApp, Slack, etc.) and the orchestration system. Users can send natural language instructions to create tasks, schedule jobs, check status, or ask questions -- all through a conversational interface.

Messages can include attachments (images, documents, etc.) up to 20 MB, which are forwarded to the supervisor agent alongside the text prompt.

Voice Message Transcription

The gateway supports voice message transcription. When a user sends a voice or audio message, it is automatically transcribed to text and processed as a regular task -- no special handling needed from the user's perspective.

User sends a voice/audio message via Telegram
The gateway downloads the audio file from Telegram
Audio is converted to raw PCM (16-bit, 16kHz, mono) via ffmpeg
PCM data is streamed in 100ms chunks over a WebSocket to Claude's STT endpoint (/api/ws/speech_to_text/voice_stream)
The endpoint returns transcript segments which are assembled into the final text
The transcribed text follows the exact same path as a typed message -- published to NATS for task creation and supervisor delegation

Authentication uses a Claude CLI OAuth token read from the credentials file (~/.claude/.credentials.json). The token is loaded fresh on every request, so credential rotation is handled automatically. ffmpeg must be available in the gateway container's PATH (included in the Docker image).

Configuration

Variable	Default	Description
`GATEWAY_TELEGRAM_BOT_TOKEN`	--	Telegram bot token (create via @BotFather)
`GATEWAY_TELEGRAM_ALLOWED_CHAT_IDS`	--	Allowed Telegram chat IDs (JSON array)
`GATEWAY_STT_BASE_URL`	`wss://claude.ai`	STT WebSocket base URL
`GATEWAY_STT_CREDENTIALS_PATH`	`~/.claude/.credentials.json`	Path to Claude credentials file for OAuth token

Security

Caution

Figaro has no authentication, no TLS, and no authorization by design. If you can reach NATS, you own the system. Do not expose any Figaro services to untrusted networks.

Figaro is designed for trusted, isolated environments -- private Docker networks, and in "production" over encrypted overlay networks like Tailscale, Headscale, or Nebula. See SECURITY.md for a full catalogue of known security trade-offs, attack vectors, and the reasoning behind them.

Architecture

                     +--------------------+
                     |    NATS Server     |
                     |  (+ JetStream)     |
                     |  (+ WebSocket)     |
                     +---------+----------+
       +----------+-----------+-----------+--------------+
       |          |           |           |              |
  +----+----+ +---+----+ +---+-----+ +---+----------+ +-+----------+
  | Worker  | |Orchestr| |Supervis | |   Gateway    | |    UI      |
  |  (x N)  | |  ator  | |   or    | |  (channels)  | |   (SPA)    |
  +---------+ +---+----+ +---------+ +--------------+ +------------+
                  |
             +----+---------------+
             |       guacd        |
             | (Apache Guacamole) |
             +--------------------+

All services communicate via NATS (pub/sub + JetStream for durable task events). The UI connects to NATS via WebSocket (nats.ws) for both real-time events and mutations (request/reply). HTTP endpoints are minimal: GET /api/config (NATS URL discovery), GET /api/guacamole/token (encrypted connection tokens), and WS /guacamole/webSocket (Guacamole WebSocket tunnel via guapy).

Services

Orchestrator (`figaro/`)

FastAPI application that manages task lifecycle, worker registry, and scheduling. Serves the built UI as static files. Handles all NATS API request/reply operations (task CRUD, scheduled tasks, help requests, VNC/SSH/telnet proxy). Mounts a guapy Guacamole WebSocket server for desktop streaming. Persists state to PostgreSQL with Alembic migrations.

Stack: Python 3.14, FastAPI, SQLAlchemy (async), asyncpg, asyncvnc, guapy, asyncssh, telnetlib3, Pillow

Worker (`figaro-worker/`)

A Claude computer use agent that executes browser automation tasks via the claude-agent-sdk. The agent sees and interacts with a desktop like a human would -- taking screenshots, moving the mouse, clicking elements, typing text, and navigating between applications. It has its own set of skills (custom tools) and can run shell commands. Task progress is streamed to JetStream.

The worker is a standalone service that can run on any machine with a desktop environment. In Docker, it runs inside a containerized Linux desktop (Fluxbox + TigerVNC + Chromium + noVNC) provided by the container image, but it can also run directly on a physical or virtual machine -- see Connecting External Desktops.

The same binary also runs as a supervisor when started with the --supervisor flag. In supervisor mode, the agent receives complex tasks and delegates them to workers. It can directly observe and interact with any connected desktop via VNC tools -- taking screenshots, typing, clicking, and pressing keys -- without delegating a full task. It can also execute commands on workers with SSH or telnet connections. Uses SDK-native custom tools backed by NATS request/reply for delegation and task management. Supports blocking delegation with inactivity-based timeouts.

Stack: Bun (compiled native binary), @anthropic-ai/claude-agent-sdk, NATS

Legacy Python implementations of both the worker and supervisor exist in legacy/ for reference.

Gateway (`figaro-gateway/`)

Channel-agnostic messaging gateway that routes messages between external communication channels and the orchestration system. Currently supports Telegram. Implements a Channel protocol for adding new channels (WhatsApp, Slack, etc.) with minimal boilerplate.

Stack: Python 3.12+, python-telegram-bot, NATS

UI (`figaro-ui/`)

React single-page application providing a dashboard with live desktop grid (Guacamole viewers), event stream, chat input for task submission, scheduled task management, and help request handling. Connects directly to NATS via WebSocket for all communication. Desktop streaming uses guacamole-common-js with auto-reconnect and display scaling.

Stack: React 18, TypeScript, Vite, Zustand, Tailwind CSS, guacamole-common-js, nats.ws

Shared NATS Library (`figaro-nats/`)

Reusable Python package providing a typed NATS client wrapper (NatsConnection) with JSON serialization, auto-reconnect, and methods for Core NATS and JetStream operations. Also provides subject constants and stream configuration.

Stack: Python 3.12+, nats-py, Pydantic

Patchright CLI (`patchright-cli/`)

Browser automation CLI tool built on Patchright (a Playwright fork for undetected browser automation). Uses a daemon-per-session architecture with Unix socket communication. Installed inside worker containers to provide browser automation capabilities.

Stack: Python 3.14, Patchright, OpenAI SDK (for audio transcription)

Development Setup

Dev Container (Recommended)

The repository includes a VS Code Dev Container configuration with all dependencies pre-installed:

Python 3.14, Node.js, Bun, uv
Desktop environment (VNC) for local testing
Docker-outside-of-Docker for container builds
Claude Code CLI

Open the repository in VS Code and select "Reopen in Container."

Manual Setup

Each service uses uv for Python dependency management:

# Shared NATS library (install first -- other services depend on it)
cd figaro-nats && uv sync --frozen

# Orchestrator
cd figaro && uv sync --frozen

# Worker + Supervisor (Bun)
cd figaro-worker && bun install

# Gateway
cd figaro-gateway && uv sync --frozen

# UI
cd figaro-ui && npm install

Running Services Individually

# Orchestrator (port 8000)
cd figaro && uv run figaro

# Worker
cd figaro-worker && bun run dev

# Supervisor (same binary, --supervisor flag)
cd figaro-worker && bun run dev -- --supervisor

# Gateway
cd figaro-gateway && uv run figaro-gateway

# UI dev server (port 3000)
cd figaro-ui && npm run dev

Database Migrations

cd figaro
uv run alembic upgrade head                              # Apply migrations
uv run alembic revision --autogenerate -m "description"  # Create new migration

Configuration

Environment Variables

Variable	Service	Description	Default
`FIGARO_HOST`	Orchestrator	Bind address	`0.0.0.0`
`FIGARO_PORT`	Orchestrator	Listen port	`8000`
`FIGARO_DATABASE_URL`	Orchestrator	PostgreSQL connection string	--
`FIGARO_NATS_URL`	Orchestrator	NATS server URL	`nats://localhost:4222`
`FIGARO_NATS_WS_URL`	Orchestrator	NATS WebSocket URL for UI	`ws://localhost:8443`
`FIGARO_STATIC_DIR`	Orchestrator	Path to built UI files	--
`FIGARO_SELF_HEALING_ENABLED`	Orchestrator	Auto-retry failed tasks	`true`
`FIGARO_SELF_HEALING_MAX_RETRIES`	Orchestrator	Max healing retries per task chain	`2`
`FIGARO_GUACD_HOST`	Orchestrator	Guacamole daemon hostname	`localhost`
`FIGARO_GUACD_PORT`	Orchestrator	Guacamole daemon port	`4822`
`FIGARO_ENCRYPTION_KEY`	Orchestrator	AES-256-CBC key for Guacamole tokens (auto-generated if not set)	--
`FIGARO_VNC_PASSWORD`	Orchestrator	VNC password for worker desktops	--
`FIGARO_VNC_PORT`	Orchestrator	VNC display port on workers	`5901`
`WORKER_NATS_URL`	Worker	NATS server URL	--
`WORKER_ID`	Worker	Unique worker identifier	--
`WORKER_NOVNC_URL`	Worker	External noVNC URL for UI	--
`SUPERVISOR_NATS_URL`	Worker (supervisor mode)	NATS server URL	--
`SUPERVISOR_ID`	Worker (supervisor mode)	Unique supervisor identifier	hostname
`SUPERVISOR_MODEL`	Worker (supervisor mode)	Claude model to use	`claude-opus-4-6`
`SUPERVISOR_CLAUDE_CODE_PATH`	Worker (supervisor mode)	Path to Claude Code CLI binary	--
`SUPERVISOR_MAX_TURNS`	Worker (supervisor mode)	Max conversation turns	--
`SUPERVISOR_DELEGATION_INACTIVITY_TIMEOUT`	Worker (supervisor mode)	Inactivity timeout for delegated tasks (seconds)	`600`
`GATEWAY_NATS_URL`	Gateway	NATS server URL	--
`GATEWAY_TELEGRAM_BOT_TOKEN`	Gateway	Telegram bot token	--
`GATEWAY_TELEGRAM_ALLOWED_CHAT_IDS`	Gateway	Allowed Telegram chat IDs (JSON array)	--
`GATEWAY_STT_BASE_URL`	Gateway	STT WebSocket base URL	`wss://claude.ai`
`GATEWAY_STT_CREDENTIALS_PATH`	Gateway	Path to Claude credentials file for STT OAuth token	`~/.claude/.credentials.json`
`OPENAI_API_KEY`	Orchestrator	OpenAI API key for memory embeddings (optional)	--
`VITE_NATS_WS_URL`	UI	NATS WebSocket URL	`ws://localhost:8443`

NATS Configuration

The NATS server runs with JetStream enabled, WebSocket on port 8443 (no TLS), and HTTP monitoring on port 8222. See docker/nats.conf for the full configuration.

NATS Subject Design

Registration and Presence (Core NATS)

figaro.register.worker                    # Worker registration
figaro.register.supervisor                # Supervisor registration
figaro.register.gateway                   # Gateway registration
figaro.deregister.{type}.{id}             # Graceful disconnect
figaro.heartbeat.{type}.{id}              # Periodic liveness

Task Assignment (Core NATS, point-to-point)

figaro.worker.{worker_id}.task            # Assign task to specific worker
figaro.supervisor.{supervisor_id}.task    # Assign task to specific supervisor

Task Events (JetStream, TASKS stream)

figaro.task.{task_id}.assigned            # Task assigned
figaro.task.{task_id}.message             # Streaming SDK output
figaro.task.{task_id}.complete            # Task completed
figaro.task.{task_id}.error               # Task failed

Help Requests (Core NATS)

figaro.help.request                       # New help request
figaro.help.{request_id}.response         # Response to help request

Broadcasts (Core NATS)

figaro.broadcast.workers                  # Updated workers list
figaro.broadcast.supervisors              # Updated supervisors list
figaro.broadcast.task_healing             # Task healing event

API (NATS request/reply)

figaro.api.delegate                       # Delegate task to worker
figaro.api.workers                        # List workers
figaro.api.tasks                          # List tasks
figaro.api.tasks.get                      # Get task
figaro.api.tasks.create                   # Create task
figaro.api.tasks.search                   # Search tasks
figaro.api.supervisor.status              # Supervisor status
figaro.api.scheduled-tasks                # List scheduled tasks
figaro.api.scheduled-tasks.{get,create,update,delete,toggle,trigger}
figaro.api.help-requests.respond          # Respond to help request
figaro.api.help-requests.dismiss          # Dismiss help request
figaro.api.vnc                            # VNC operations (screenshot, type, key, click)
figaro.api.ssh                            # SSH operations (run_command)
figaro.api.telnet                         # Telnet operations (run_command)
figaro.api.memories.save                  # Save a memory
figaro.api.memories.search                # Search memories (hybrid BM25 + vector)
figaro.api.memories.list                  # List memories
figaro.api.memories.delete                # Delete a memory

Gateway (Core NATS)

figaro.gateway.{channel}.send             # Send message via channel
figaro.gateway.{channel}.task             # Task from channel
figaro.gateway.{channel}.question         # Ask question via channel
figaro.gateway.{channel}.register         # Channel gateway registers availability

Message Flows

Task Submission (UI to Worker)

UI sends NATS request to figaro.api.tasks.create
Orchestrator creates the task, claims an idle worker, publishes to figaro.worker.{id}.task
Worker executes the task with the Claude Agent SDK
Worker streams messages via JetStream (figaro.task.{id}.message)
Worker publishes completion via JetStream (figaro.task.{id}.complete)
Orchestrator updates the database and sets the worker idle

Help Request Flow (Human-in-the-Loop)

Worker or supervisor publishes to figaro.help.request
Orchestrator creates the help request and broadcasts to the UI
Gateway routes the help request to a channel (e.g., Telegram)
First responder (UI or channel) replies
Response is routed back to the requesting agent

Testing

Python Services

cd figaro && uv run pytest              # Orchestrator
cd figaro-nats && uv run pytest         # Shared library
cd figaro-gateway && uv run pytest      # Gateway

Linting and formatting:

uv run ruff check .
uv run ruff format .

UI

cd figaro-ui
npm run test           # Run tests
npm run test:watch     # Watch mode
npm run build          # TypeScript check + production build

Worker (Bun)

cd figaro-worker && bun test

Contributing

Pull requests are welcome! Issues are disabled on this repository -- if you've found a bug or have a feature request, please start a thread in Discussions first. Once the approach is agreed upon, feel free to open a PR.

Credits

OpenClaw for design inspiration around some agent control primitives (e.g. loop control, cost tracking etc..)
Apache Guacamole & Guapy
QMD for the agent memory RAG implementation

Name		Name	Last commit message	Last commit date
Latest commit History 42 Commits
.devcontainer		.devcontainer
.github/workflows		.github/workflows
docker		docker
docs		docs
figaro-gateway		figaro-gateway
figaro-nats		figaro-nats
figaro-ui		figaro-ui
figaro-worker		figaro-worker
figaro		figaro
patchright-cli		patchright-cli
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
CLAUDE.md		CLAUDE.md
LICENSE.md		LICENSE.md
README.md		README.md
SECURITY.md		SECURITY.md
TODO.md		TODO.md
install.sh		install.sh

Folders and files

Latest commit

History

Repository files navigation

Table of Contents

UI Action shot

Quick Start

Prerequisites

Optional

Advanced Setup

Scaling

Teardown

Connecting External Desktops

From the UI

From Environment Variables

Agent Upgrade Path

Scheduled Tasks

NATS API

Self-Healing

How It Works

Loop Prevention

Configuration

Self-Learning

How It Works

Design Details

Agent Memory

How It Works

Memory Structure

Search

NATS API

Configuration

Gateway

Voice Message Transcription

Configuration

Security

Architecture

Services

Orchestrator (figaro/)

Worker (figaro-worker/)

Gateway (figaro-gateway/)

UI (figaro-ui/)

Shared NATS Library (figaro-nats/)

Patchright CLI (patchright-cli/)

Development Setup

Dev Container (Recommended)

Manual Setup

Running Services Individually

Database Migrations

Configuration

Environment Variables

NATS Configuration

NATS Subject Design

Registration and Presence (Core NATS)

Task Assignment (Core NATS, point-to-point)

Task Events (JetStream, TASKS stream)

Help Requests (Core NATS)

Broadcasts (Core NATS)

API (NATS request/reply)

Gateway (Core NATS)

Message Flows

Task Submission (UI to Worker)

Help Request Flow (Human-in-the-Loop)

Testing

Python Services

UI

Worker (Bun)

Contributing

Credits

About

Resources

License

Security policy

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Contributors 1

Orchestrator (`figaro/`)

Worker (`figaro-worker/`)

Gateway (`figaro-gateway/`)

UI (`figaro-ui/`)

Shared NATS Library (`figaro-nats/`)

Patchright CLI (`patchright-cli/`)

Packages