Skip to content

Add PQS (Prompt Quality Score) to Prompt Management and Testing#107

Open
OnChainAIIntel wants to merge 1 commit intopromptslab:mainfrom
OnChainAIIntel:add-pqs-prompt-quality-score
Open

Add PQS (Prompt Quality Score) to Prompt Management and Testing#107
OnChainAIIntel wants to merge 1 commit intopromptslab:mainfrom
OnChainAIIntel:add-pqs-prompt-quality-score

Conversation

@OnChainAIIntel
Copy link
Copy Markdown

What this adds

PQS (Prompt Quality Score) — a pre-inference prompt scoring system, added as a row in the Prompt Management and Testing table.

Why it fits

PQS is complementary to existing entries in this section. Quick differentiation:

  • Promptfoo tests outputs against assertions (post-inference QA)
  • PQS scores inputs before inference runs (pre-flight quality gate)
  • DSPy / TextGrad optimize prompts via LLM calls; PQS scores them on structured dimensions without requiring optimization loops

PQS scores any prompt on 8 dimensions using 5 established frameworks (PEEM, RAGAS, MT-Bench, G-Eval, ROUGE). Published on GitHub Marketplace as "PQS Check" for CI-based gating.

Ecosystem

MIT licensed. Happy to rework the description or move to a different section if a better fit exists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant