ai4c CLI strategy: trust, drift, and hybrid command model

[cmungall]

Context

We’re debating whether ai4c should be:

1. a full Python CLI/framework,
2. mostly just wrappers over existing CLIs (linkml-*, deep-research-client, etc.), or
3. a null model with no framework (just markdown instructions + agents).

The pain point is real: generated templates (justfile/ODK-like) drift when users or agents customize them.

Why this matters

A markdown-only model gives a narrative contract, but not an executable one.
A small standard command surface gives trust primitives:

• predictability (same command names / intent across projects),
• verifiability (runnable outputs, not just prose),
• reproducibility (human + agent reruns),
• auditability (what exactly ran),
• safer defaults (dry-run, validate-before-apply),
• less prompt ambiguity during handoffs.

Proposal (lean + agent-first)

Use a hybrid model with single source of truth in config:

• ai4c.yaml defines tasks/datatypes/entry resolution.
• ai4c executes shell-native tasks (thin router over existing CLIs).
• optional generated just facade calls ai4c (never authoritative).

To avoid drift:

• generated file lives at .ai4c/commands.just and is always overwriteable,
• user/agent custom logic lives in project.just only,
• top-level justfile imports both.

Decision questions

1. What must be stable across projects: command names, or only outcomes?
2. How much project-specific customization is expected in practice?
3. Should customization be mostly declarative (ai4c.yaml) vs hook scripts?
4. Should CI call ai4c directly or continue through just?
5. What is the minimum universal v1 command set?

Suggested v1 scope

• Minimal ai4c task router + aliases: validate, research run, render, export.
• Shell-first execution, structured logging, no heavy plugin framework.
• Optional ai4c just sync to regenerate .ai4c/commands.just.

Out of scope for v1

• Rich plugin ecosystem.
• Full template-managed project lifecycle.
• Domain-specific bespoke validators in core.

[github-actions]

Summary & Analysis

What this issue is about

This issue frames the key architectural decision for the ai4c CLI tool: how much framework should sit between curation workflows and the underlying tools (linkml-*, deep-research-client, etc.)? The core tension is stability vs. flexibility: a rigid framework prevents drift but limits customization; a markdown-only model is flexible but unverifiable.

Connection to related issues

This issue is essentially the implementation answer to #40 ("Unify all the different ontology pipelines"), which identified that cognitive fragmentation across mondo, cl, efo, uberon, namo deployments is causing real harm (agents failing due to project-specific config drift, poor instructions, inconsistent tooling). Issue #40 asked what to unify; this issue proposes how.

It also connects directly to:

• Change the actions template for claude and dragon-ai to use ODK docker image #58 (ODK docker image for actions templates) — same template-drift problem at the CI layer
• Provenance tracking for AI-assisted curation: challenges and design considerations #62 (Provenance tracking) — the auditability point in this issue ("what exactly ran") is a prerequisite for meaningful provenance; ai4c with structured logging would satisfy that requirement

The failure patterns in #60 (training analysis: 260 PRs across 5 ontologies) are the direct motivation — without a stable, predictable command surface, agents can't reliably reproduce valid workflows across projects.

---

Analysis of the three options

Option	Stability	Flexibility	Agent-friendliness	Drift risk
Full Python CLI/framework	High	Low	High (predictable)	Low
just wrappers	Medium	High	Medium	High — customized justfiles fork from generated ones
Markdown-only / null model	Low	Highest	Low (unverifiable)	N/A — there's nothing to drift

The drift problem with just wrappers is real and well-known from the ODK experience: ODK generates Makefile sections that users then modify, and those modifications become "sacred" local state that can't safely be overwritten by ODK upgrades. ODK addressed this with the src/ontology/Makefile (generated, overwritable) + top-level Makefile (user-owned) split — exactly the pattern proposed here for .ai4c/commands.just vs project.just.

---

Assessment of the hybrid proposal

The proposed model is architecturally sound and mirrors battle-tested patterns:

1. ai4c.yaml as single source of truth — analogous to ODK's [project]-odk.yaml. Declarative config that drives generation is much more maintainable than generated imperative code.

2. Thin router over existing CLIs — right call for v1. Avoids reimplementing linkml-validate, deep-research-client, etc. while providing a stable command surface.

3. .ai4c/commands.just always-overwriteable — this is the key insight. The problem with ODK-style templates isn't the generation, it's the assumption that generated files are "owned" by users. Making the generated file explicitly non-authoritative (always overwriteable) breaks that assumption cleanly.

4. project.just for user/agent custom logic — gives agents a clearly-scoped place to add project-specific commands without touching the stable layer.

---

Suggested answers to the decision questions

1. What must be stable: command names, not just outcomes. Agents and humans need to know that ai4c validate means the same thing across projects — this is what allows shared training, shared instructions (see Unify all the different ontology pipelines #40), and shared agent behavior.

2. Project-specific customization: In practice, the variation is mostly in inputs (which files, which schemas) not in behavior (what validate means). Declarative ai4c.yaml handles this naturally.

3. Customization should be declarative first (ai4c.yaml), with hook scripts as escape hatch. This mirrors the lesson from Overarching Training Recommendations for AI Ontology Agents #60: agents that try to customize behavior programmatically (elaborate workarounds) fail more than agents that work within declared constraints.

4. CI should call ai4c directly, with just as a human convenience layer. This gives CI reproducibility without dependence on justfile state.

5. Minimum v1 command set — the proposed validate, research run, render, export seems right. Consider adding ai4c status (show what would run, dry-run mode) as a trust primitive for agents before they commit to an action.

---

One additional consideration: schema-based config validation

If ai4c.yaml is the single source of truth, it should itself be validated against a LinkML schema. This closes the loop: the tool that validates ontologies should validate its own config. This also gives agents a precise spec to write against when generating or modifying ai4c.yaml files — reducing hallucination risk vs. free-form YAML.

---

Recommended next step

Given the connection to #40 (unify pipelines), a minimal v1 that just handles validate + structured logging across the existing ontology repos would immediately address the most painful part of the fragmentation problem without requiring a full framework build.

@dragon-ai-agent please investigate what a minimal v1 ai4c CLI would need to implement to unify validation workflows across the ontology projects mentioned in issue #40 (mondo, cl, efo, uberon, namo). Look at what each project currently uses for validation (make targets, justfile targets, linkml-validate calls) and propose a minimal ai4c.yaml schema and ai4c validate command interface that could work across all of them.