Skip to content

Latest commit

ย 

History

History
355 lines (272 loc) ยท 17.4 KB

File metadata and controls

355 lines (272 loc) ยท 17.4 KB

Codomyrmex Module System Overview

Version: v1.2.3 | Status: Active | Last Updated: March 2026

Codomyrmex is built on a modular architecture where each module provides specific functionality while maintaining clear interfaces and minimal coupling. This design allows for flexible composition, independent development, and easy extensibility.

๐Ÿ—๏ธ Module Architecture Principles

1. Self-Contained Modules

Each module is a complete unit with:

  • Its own dependencies (managed via pyproject.toml extras)
  • Comprehensive tests (tests/)
  • API documentation (API_SPECIFICATION.md)
  • Usage examples (USAGE_EXAMPLES.md)
  • Security considerations (SECURITY.md)

2. Standardized Interfaces

All modules follow consistent patterns:

  • MCP Tools: Model Context Protocol specifications for AI integration
  • Public APIs: Well-defined functions and classes for programmatic access
  • CLI Integration: Commands accessible through the main Codomyrmex CLI
  • Configuration: Environment variables and config file support

3. Layered Dependencies

Modules are organized in dependency layers to prevent circular dependencies:

graph TD
    subgraph sg_2ba8632d46 [Application Layer]
        InteractiveShell["Interactive<br/>Shell"]
        SystemDiscovery["System Discovery<br/>& Orchestration"]
    end

    subgraph sg_d55324be2d [Service Layer]
        AIAgents["AI Agents<br/>& Benchmarks"]
        Documentation["Documentation<br/>& Education"]
        CICD["CI/CD<br/>(incl. build)"]
        Coding["Coding<br/>(static analysis, patterns)"]
    end

    subgraph sg_6572b4b746 [Core Layer]
        ModelOps["Model Ops<br/>(eval, registry, optimization)"]
        DataViz["Data Visualization<br/>& Export"]
        Testing["Testing<br/>(workflow, chaos)"]
        GitOperations["Git Operations<br/>& Merge Resolution"]
    end

    subgraph sg_5f13f7c061 [Foundation Layer]
        LoggingMonitoring["Logging &<br/>Monitoring"]
        EnvironmentSetup["Environment<br/>Setup"]
        MCP["Model Context<br/>Protocol"]
        Telemetry["Telemetry<br/>(metrics, dashboard)"]
        Events["Events<br/>(streaming, notifications)"]
    end

    %% Dependencies flow upward
    AIAgents -.-> LoggingMonitoring
    AIAgents -.-> EnvironmentSetup
    AIAgents -.-> MCP

    Documentation -.-> LoggingMonitoring

    CICD -.-> LoggingMonitoring
    CICD -.-> Coding

    Coding -.-> LoggingMonitoring

    ModelOps -.-> LoggingMonitoring
    DataViz -.-> LoggingMonitoring
    Testing -.-> LoggingMonitoring
    GitOperations -.-> LoggingMonitoring

    InteractiveShell -.-> SystemDiscovery
    SystemDiscovery -.-> LoggingMonitoring
Loading

Key Principles:

  • Upward Dependencies Only: Higher layers depend on lower layers, never the reverse
  • Foundation Services: Used by all modules for consistency
  • Clear Layer Boundaries: Each layer has distinct responsibilities

Related Documentation:

๐Ÿ“ฆ Module Categories

๐Ÿ—๏ธ Foundation Modules

Essential infrastructure used by all other modules

Module Purpose Key Features
logging_monitoring Centralized logging system Structured logging, multiple formats, log aggregation
environment_setup Environment validation Dependency checking, API key management, setup automation, dependency resolution
model_context_protocol AI communication standard Standardized LLM interfaces, tool specifications, auto-discovery
terminal_interface Rich terminal interactions Colored output, progress bars, interactive prompts
config_management Configuration management Centralized config loading, deployment, monitoring, secret management
database_management Data persistence Database operations, migrations, backups, performance monitoring, data lineage
concurrency Distributed synchronization Locks, semaphores, thread-safe primitives, channels, rate limiting
dependency_injection IoC container Service registration, constructor injection, lifecycle scoping
serialization Data serialization JSON, YAML, binary formats, schema validation, streaming I/O
telemetry Observability OpenTelemetry traces, spans, context propagation, metrics collection, dashboards
events Event system Event bus, pub/sub, event handlers, replay, dead letter, streaming, notifications
validation Data validation Schema validation, input sanitization, constraint checking, shared schema registry

โš™๏ธ Core Functional Modules

Primary capabilities that users directly interact with

Module Purpose Key Features
coding Code execution and review Multi-language sandboxing, code review, static analysis, pattern matching
data_visualization Charts and plots Static/interactive plots, multiple formats, customizable styling, multi-format export
git_operations Version control automation Git workflows, branch management, commit automation, merge conflict resolution
security Security scanning and audit Vulnerability detection, compliance checking, threat modeling, vulnerability scanner, governance
scrape Web scraping Content extraction, crawling, data collection
documents Document processing Document parsing, metadata extraction, search indexing, RAG chunking
cache Caching infrastructure In-memory caching, distributed cache, multi-strategy invalidation
compression Data compression Multiple algorithms, streaming compression, file archiving
encryption Encryption utilities Symmetric/asymmetric encryption, key management, secure storage, digital signing
networking Network utilities HTTP clients, WebSocket support, network diagnostics, service mesh
search Full-text search TF-IDF indexing, fuzzy matching, relevance ranking, hybrid BM25+semantic search
crypto Cryptography Cryptography utilities, currency, encoding, protocols, steganography
static_analysis Static analysis Code parsing, pattern matching, syntax trees

๐Ÿ”ง Service Modules

Higher-level services that orchestrate core modules

Module Purpose Key Features
documentation Documentation generation Website generation, API docs, tutorial creation, education & tutoring
api API infrastructure OpenAPI/Swagger specs, REST API building, versioning, rate limiting
ci_cd_automation CI/CD pipeline management Pipeline orchestration, deployment automation, monitoring, build synthesis
containerization Container management Docker lifecycle, image building, Kubernetes orchestration
logistics Orchestration and scheduling Task orchestration, workflow scheduling, resource allocation
cloud Cloud integrations Cloud provider APIs, resource management, deployment, cost management
auth Authentication OAuth, API keys, token management, access control
deployment Deployment strategies Blue/green, canary, rolling deployments
edge_computing Edge deployment IoT gateways, edge inference, offline-first patterns
model_ops ML operations Model lifecycle, experiment tracking, serving, evaluation, registry, optimization, feature store
orchestrator Workflow execution DAG execution, step composition, retry policies, scheduling
testing Test infrastructure Fixtures, data generators, workflow validation, chaos engineering
performance Performance monitoring Metrics collection, benchmarking, optimization guidance
calendar_integration Event management Calendar providers, event scheduling
release Release management Versioning, changelog generation, release coordination
docs_gen Documentation generation Template scaffolding, source-to-docs conversion
calendar Event management Calendar providers, event scheduling

๐ŸŽฎ Application Modules

User-facing applications and interfaces

Module Purpose Key Features
system_discovery System exploration Module scanning, capability discovery, health monitoring
terminal_interface Interactive exploration Command shell, foraging interface, guided discovery
cli Command-line interface CLI commands, scripting support, interactive shell, shell completion
website Website generation Static site generation, templating, local server, accessibility
audio Audio processing Transcription, synthesis, audio analysis
video Video processing Video analysis, frame extraction, transcription
email Email communication Email parsing, IMAP/SMTP integration

๐Ÿ›ก๏ธ Secure Cognitive Modules

Autonomous security and economic capabilities

Module Purpose Key Features
identity Identity & Verification 3-Tier personas, bio-cognitive verification, revocation
wallet Self-Custody Secure key management, Natural Ritual recovery, key rotation, smart contracts
defense Active Defense Exploit detection, context poisoning, rabbit hole containment
market Anonymous Markets Reverse auctions, demand aggregation, anonymous bidding
privacy Privacy & Anonymity Crumb scrubbing, mixnet simulation, dynamic blacklists
dark Dark mode & PDF ops Dark mode utilities, PDF processing, steganography

๐Ÿง  Intelligence Modules

AI reasoning and agent capabilities

Module Purpose Key Features
agents Agentic framework integrations Jules, Claude, Codex integration, agent orchestration, benchmarks
cerebrum Case-based reasoning Bayesian inference, active inference, reasoning engine
fpf Feed-Parse-Format Pipeline Combinator engine, transformation pipelines, composition
skills Skills framework Skill definitions, execution, chaining
agentic_memory Long-term agent memory Retrieval, persistence, contextual recall, memory compression
graph_rag Knowledge graph RAG Graph-enhanced retrieval, knowledge extraction
llm LLM infrastructure Language model integration, provider abstraction, Ollama support, multimodal, safety filtering
prompt_engineering Prompt management Template versioning, optimization, evaluation, A/B testing
vector_store Embeddings storage Similarity search, HNSW indexing, retrieval
tool_use Tool registry & composition Tool validation, chain abstraction, workflow tools

๐Ÿ”— Integration Modules

Standards, protocols, and IDE integrations

Module Purpose Key Features
model_context_protocol AI communication standard Standardized LLM interfaces, tool specifications
module_template Module creation template Template for creating new modules with standard structure
ide IDE integrations Antigravity, Cursor, VS Code automation and control
plugin_system Plugin architecture Plugin loading, registration, lifecycle management
collaboration Multi-agent collaboration Swarm coordination, shared context, team workflows
feature_flags Feature toggles Feature gates, rollout strategies, A/B routing

๐Ÿš€ Advanced Modules

Specialized capabilities for advanced use cases

Module Purpose Key Features
spatial 3D/4D modeling and visualization 3D model creation, manipulation, rendering, world models
physical_management Physical system simulation System monitoring, resource management, performance tracking
bio_simulation Ant colony simulation Pheromone foraging, genetic algorithms, colony dynamics
embodiment Physical/robotic integration Hardware interfaces, sensor fusion, motor control
evolutionary_ai Genetic algorithms Evolutionary optimization, population dynamics, fitness
quantum Quantum computing Algorithm primitives, circuit simulation, qubit operations
simulation General simulation framework Agent-based modeling, discrete event simulation
networks Network graph analysis Graph metrics, community detection
formal_verification Formal verification Model checking, theorem proving

๐Ÿ› ๏ธ Utility Modules

Common utilities and development tools

Module Purpose Key Features
exceptions Exception hierarchy Centralized exception classes, error handling, error codes
utils Common utilities Helper functions, CLI helpers, shared utilities, hashing, retry, i18n
templating Template engine Jinja2-style templating, code generation, scaffolding
tests Test infrastructure Unit tests, integration tests, test utilities
examples Code examples Usage examples, tutorials, demo applications
meme Information dynamics Memetic warfare, meme propagation, evolution modeling
finance Financial operations Double-entry bookkeeping, tax compliance, forecasting
relations CRM & social graphs Contact management, social network analysis, graph metrics
maintenance System health Health checks, cleanup routines, diagnostics

๐Ÿ”„ Module Communication Patterns

1. Direct API Calls

Modules expose public functions and classes:

from codomyrmex.data_visualization import create_line_plot
from codomyrmex.agents import generate_code_snippet

# Direct function calls
plot_result = create_line_plot(x_data, y_data, title="Sample Plot")
code_result = generate_code_snippet("Create a factorial function", "python")

2. Event-Driven Integration

Modules can publish and subscribe to events:

from codomyrmex.logging_monitoring import get_logger
logger = get_logger(__name__)
logger.info("Code generation completed", extra={"event_type": "code_generated"})

3. Pipeline Composition

Modules can be chained together in processing pipelines:

def enhance_code_pipeline(source_code):
    # 1. Analyze existing code (static_analysis is now in coding)
    analysis = coding.static_analysis.analyze_code(source_code)

    # 2. Generate improvements with AI
    improvements = agents.suggest_improvements(source_code, analysis)

    # 3. Validate generated code
    validation = coding.validate_code(improvements)
    return validation

๐Ÿ“‹ Module Development Guidelines

Creating a New Module

  1. Use the Template:

    cp -r src/codomyrmex/module_template src/codomyrmex/my_new_module
  2. Follow the Structure:

    my_new_module/
    โ”œโ”€โ”€ __init__.py                 # Module initialization
    โ”œโ”€โ”€ README.md                   # Module overview and usage
    โ”œโ”€โ”€ API_SPECIFICATION.md        # API documentation
    โ”œโ”€โ”€ MCP_TOOL_SPECIFICATION.md   # AI integration specs
    โ”œโ”€โ”€ CHANGELOG.md               # Version history
    โ”œโ”€โ”€ SECURITY.md                # Security considerations
    โ”œโ”€โ”€ pyproject.toml              # Project dependencies
    โ”œโ”€โ”€ my_module_code.py          # Implementation
    โ”œโ”€โ”€ docs/                      # Extended documentation
    โ”‚   โ”œโ”€โ”€ technical_overview.md  # Architecture details
    โ”‚   โ””โ”€โ”€ tutorials/             # Usage tutorials
    โ””โ”€โ”€ tests/                     # Comprehensive tests
        โ”œโ”€โ”€ README.md              # Testing instructions
        โ””โ”€โ”€ test_my_module.py      # Test implementation
    
  3. Implement Standard Interfaces:

    • Public API functions with clear signatures
    • MCP tools for AI integration
    • Configuration via environment variables
    • Comprehensive error handling and logging
  4. Document Everything:

    • API specifications with examples
    • Usage tutorials for common scenarios
    • Integration patterns with other modules
    • Security considerations and best practices

๐Ÿ” Module Health and Monitoring

Module Status Checking

from codomyrmex.system_discovery import SystemDiscovery

discovery = SystemDiscovery()
status = discovery.check_module_health()

print(f"Modules working: {status['working_count']}/{status['total_count']}")
print(f"Dependencies satisfied: {status['dependencies_ok']}")
print(f"Tests passing: {status['tests_passing']}")

This modular architecture enables Codomyrmex to be both powerful and flexible, allowing users to leverage individual capabilities or compose complex workflows across multiple modules. The standardized interfaces and clear dependency relationships make it easy to extend, maintain, and integrate with external systems.

Navigation Links