Conversation
Add Avian (https://avian.io) as a first-class LLM provider for PrivateGPT. Avian provides an OpenAI-compatible API with access to frontier models at competitive pricing. Changes: - Add "avian" mode to LLMSettings and EmbeddingSettings - Add AvianSettings config model (api_base, api_key, model, embedding_model, request_timeout) - Add avian case to LLMComponent using llama-index OpenAI integration - Add avian case to EmbeddingComponent using llama-index OpenAIEmbedding - Add llms-avian and embeddings-avian extras to pyproject.toml - Add settings-avian.yaml profile for quick setup - Add avian defaults to settings.yaml Available models: - deepseek/deepseek-v3.2 (164K context, $0.26/$0.38 per 1M tokens) - moonshotai/kimi-k2.5 (131K context, $0.45/$2.20 per 1M tokens) - z-ai/glm-5 (131K context, $0.30/$2.55 per 1M tokens) - minimax/minimax-m2.5 (1M context, $0.30/$1.10 per 1M tokens)
v3 has been deprecated by GitHub and causes the test job to fail during setup with: "This request has been automatically failed because it uses a deprecated version of actions/upload-artifact: v3"
|
Hey @jaluma @imartinez, would love your review on this when you get a chance. Happy to address any feedback! |
|
Friendly follow-up — this PR is still active and ready for review. Would appreciate a look when you get a chance! cc @jaluma @imartinez |
|
Friendly follow-up — this PR is still active and ready for review. All feedback has been addressed. Would appreciate a look when you get a chance! cc @jaluma @imartinez |
|
Hey @imartinez @jaluma — friendly follow-up on this PR. Avian is an OpenAI-compatible inference provider that's already live and powering apps like ISEKAI ZERO. This is a lightweight integration (standard OpenAI-compatible endpoint) and we're happy to address any feedback or make adjustments. Would love to get this merged if you have a moment to review. Thanks! |
|
Thanks for the contribution. I think your Usage example is not right: |
imartinez
left a comment
There was a problem hiding this comment.
Please review the comments, as I feel you missed specifically one change in the settings-avian.yaml file
settings.yaml
Outdated
| embedding_model: models/embedding-001 | ||
|
|
||
| avian: | ||
| api_base: https://api.avian.io/v1 |
There was a problem hiding this comment.
You may want to make this configurable via env var like ${AVIAN_ENDPOINT:}
settings-avian.yaml
Outdated
| temperature: 0.1 | ||
|
|
||
| embedding: | ||
| mode: huggingface |
There was a problem hiding this comment.
Shouldn't it be avian instead of huggingface?
- Make api_base configurable via AVIAN_API_BASE env var in settings.yaml - Change embedding mode from huggingface to avian in settings-avian.yaml (avian embedding mode exists in embedding_component.py) - Remove unnecessary huggingface config from settings-avian.yaml - Also make api_base in settings-avian.yaml use env var pattern Co-Authored-By: Claude Opus 4.6 <[email protected]>
|
Addressed all review feedback in commit 761fecc:
|
|
All review feedback has been addressed. Could you please re-review when you get a chance? |
|
Both review comments have been addressed in commit 761fecc:
The poetry install example in the PR description also uses Happy to rename |
Summary
https://api.avian.io/v1) with access to frontier models at competitive pricingllama-index-llms-openai/llama-index-embeddings-openaiintegrations, so no new dependencies are introducedChanges
private_gpt/settings/settings.py"avian"toLLMSettings.modeandEmbeddingSettings.modeLiterals; addAvianSettingsconfig modelprivate_gpt/components/llm/llm_component.py"avian"case usingllama_index.llms.openai.OpenAIpointed atapi.avian.io/v1private_gpt/components/embedding/embedding_component.py"avian"case usingllama_index.embeddings.openai.OpenAIEmbeddingpyproject.tomlllms-avianandembeddings-avianextras (pointing to existing openai packages)settings-avian.yamlsettings.yamlavian:defaults sectionAvailable Models
deepseek/deepseek-v3.2moonshotai/kimi-k2.5z-ai/glm-5minimax/minimax-m2.5Usage
Design Decisions
llama-index-llms-openaipackage rather than adding a new dependency. This follows the same approach used by theopenaimode but with Avian-specific defaults (API base URL, model names, settings).AvianSettingsconfig: Gives users a clean, dedicated configuration section instead of requiring them to manually override theopenaisettings block with Avian's API base and key.aviansections insettings.yamluse env var defaults (${AVIAN_API_KEY:}) so they don't interfere with existing deployments.Test Plan
PGPT_PROFILES=avianloads correct settingsdeepseek/deepseek-v3.2modelcc @imartinez @jaluma @lopagela for review