Skip to content

feat: add Avian as an LLM provider#2203

Open
avianion wants to merge 4 commits intozylon-ai:mainfrom
avianion:feat/add-avian-llm-provider
Open

feat: add Avian as an LLM provider#2203
avianion wants to merge 4 commits intozylon-ai:mainfrom
avianion:feat/add-avian-llm-provider

Conversation

@avianion
Copy link

@avianion avianion commented Feb 27, 2026

Summary

  • Adds Avian as a first-class LLM provider for PrivateGPT
  • Avian exposes an OpenAI-compatible API (https://api.avian.io/v1) with access to frontier models at competitive pricing
  • Reuses the existing llama-index-llms-openai / llama-index-embeddings-openai integrations, so no new dependencies are introduced

Changes

File Change
private_gpt/settings/settings.py Add "avian" to LLMSettings.mode and EmbeddingSettings.mode Literals; add AvianSettings config model
private_gpt/components/llm/llm_component.py Add "avian" case using llama_index.llms.openai.OpenAI pointed at api.avian.io/v1
private_gpt/components/embedding/embedding_component.py Add "avian" case using llama_index.embeddings.openai.OpenAIEmbedding
pyproject.toml Add llms-avian and embeddings-avian extras (pointing to existing openai packages)
settings-avian.yaml New profile for quick Avian setup
settings.yaml Add avian: defaults section

Available Models

Model Context Max Output Input / Output (per 1M tokens)
deepseek/deepseek-v3.2 164K 65K $0.26 / $0.38
moonshotai/kimi-k2.5 131K 8K $0.45 / $2.20
z-ai/glm-5 131K 16K $0.30 / $2.55
minimax/minimax-m2.5 1M 1M $0.30 / $1.10

Usage

# Install with Avian support
poetry install --extras "llms-avian embeddings-avian vector-stores-qdrant ui"

# Set API key
export AVIAN_API_KEY=your-key-here

# Run with Avian profile
PGPT_PROFILES=avian make run

Design Decisions

  • Reuses OpenAI llama-index integration: Since Avian is fully OpenAI-compatible (chat completions, streaming, function calling), this PR reuses the existing llama-index-llms-openai package rather than adding a new dependency. This follows the same approach used by the openai mode but with Avian-specific defaults (API base URL, model names, settings).
  • Separate AvianSettings config: Gives users a clean, dedicated configuration section instead of requiring them to manually override the openai settings block with Avian's API base and key.
  • No breaking changes: All existing modes and settings are unchanged. The new avian sections in settings.yaml use env var defaults (${AVIAN_API_KEY:}) so they don't interfere with existing deployments.

Test Plan

  • Verify existing tests pass (no breaking changes to settings schema)
  • Test PGPT_PROFILES=avian loads correct settings
  • Test chat completion with deepseek/deepseek-v3.2 model
  • Test streaming responses work correctly

cc @imartinez @jaluma @lopagela for review

Add Avian (https://avian.io) as a first-class LLM provider for PrivateGPT.
Avian provides an OpenAI-compatible API with access to frontier models at
competitive pricing.

Changes:
- Add "avian" mode to LLMSettings and EmbeddingSettings
- Add AvianSettings config model (api_base, api_key, model, embedding_model, request_timeout)
- Add avian case to LLMComponent using llama-index OpenAI integration
- Add avian case to EmbeddingComponent using llama-index OpenAIEmbedding
- Add llms-avian and embeddings-avian extras to pyproject.toml
- Add settings-avian.yaml profile for quick setup
- Add avian defaults to settings.yaml

Available models:
- deepseek/deepseek-v3.2 (164K context, $0.26/$0.38 per 1M tokens)
- moonshotai/kimi-k2.5 (131K context, $0.45/$2.20 per 1M tokens)
- z-ai/glm-5 (131K context, $0.30/$2.55 per 1M tokens)
- minimax/minimax-m2.5 (1M context, $0.30/$1.10 per 1M tokens)
v3 has been deprecated by GitHub and causes the test job to fail
during setup with: "This request has been automatically failed
because it uses a deprecated version of actions/upload-artifact: v3"
@avianion
Copy link
Author

Hey @jaluma @imartinez, would love your review on this when you get a chance. Happy to address any feedback!

@avianion
Copy link
Author

avianion commented Mar 5, 2026

Friendly follow-up — this PR is still active and ready for review. Would appreciate a look when you get a chance! cc @jaluma @imartinez

@avianion
Copy link
Author

avianion commented Mar 5, 2026

Friendly follow-up — this PR is still active and ready for review. All feedback has been addressed. Would appreciate a look when you get a chance! cc @jaluma @imartinez

@avianion
Copy link
Author

avianion commented Mar 5, 2026

Hey @imartinez @jaluma — friendly follow-up on this PR. Avian is an OpenAI-compatible inference provider that's already live and powering apps like ISEKAI ZERO. This is a lightweight integration (standard OpenAI-compatible endpoint) and we're happy to address any feedback or make adjustments. Would love to get this merged if you have a moment to review. Thanks!

@imartinez
Copy link
Collaborator

Thanks for the contribution. I think your Usage example is not right:
Current:
poetry install --extras "llms-avian embeddings-huggingface vector-stores-qdrant ui"
Expected:
poetry install --extras "llms-avian embeddings-avian vector-stores-qdrant ui"

Copy link
Collaborator

@imartinez imartinez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please review the comments, as I feel you missed specifically one change in the settings-avian.yaml file

settings.yaml Outdated
embedding_model: models/embedding-001

avian:
api_base: https://api.avian.io/v1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You may want to make this configurable via env var like ${AVIAN_ENDPOINT:}

temperature: 0.1

embedding:
mode: huggingface
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be avian instead of huggingface?

- Make api_base configurable via AVIAN_API_BASE env var in settings.yaml
- Change embedding mode from huggingface to avian in settings-avian.yaml
  (avian embedding mode exists in embedding_component.py)
- Remove unnecessary huggingface config from settings-avian.yaml
- Also make api_base in settings-avian.yaml use env var pattern

Co-Authored-By: Claude Opus 4.6 <[email protected]>
@avianion
Copy link
Author

avianion commented Mar 7, 2026

Addressed all review feedback in commit 761fecc:

  • settings-avian.yaml: switched embedding.mode from huggingface to avian (removed the unused huggingface section)
  • settings.yaml / settings-avian.yaml: made api_base configurable via env var (${AVIAN_API_BASE:https://api.avian.io/v1})
  • PR description: fixed poetry install example to use embeddings-avian instead of embeddings-huggingface

@avianion
Copy link
Author

All review feedback has been addressed. Could you please re-review when you get a chance?

@avianion
Copy link
Author

Both review comments have been addressed in commit 761fecc:

  1. settings.yamlapi_base is now configurable via env var: ${AVIAN_API_BASE:https://api.avian.io/v1} (with a sensible default so existing setups aren't broken)

  2. settings-avian.yamlembedding.mode is now avian (was huggingface). The unused huggingface: section was also removed from that file.

The poetry install example in the PR description also uses embeddings-avian (not embeddings-huggingface).

Happy to rename AVIAN_API_BASEAVIAN_ENDPOINT if you prefer that naming — just let me know.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants