|
| 1 | +# Devito Copilot Instructions |
| 2 | + |
| 3 | +## Repository Overview |
| 4 | + |
| 5 | +Devito is a Python package for implementing optimized stencil computation (finite differences, image processing, machine learning) from high-level symbolic problem definitions. It builds on SymPy and employs automated code generation and just-in-time compilation to execute optimized computational kernels on CPUs, GPUs, and clusters. |
| 6 | + |
| 7 | +**Key Facts:** |
| 8 | +- Production-stable package (Development Status 5) for scientific computing |
| 9 | +- Python 3.10-3.13 support (current development uses 3.12.3) |
| 10 | +- ~50MB repository with extensive test coverage |
| 11 | +- Primary languages: Python (core), C/C++ (generated code) |
| 12 | +- Main dependencies: NumPy, SymPy, cgen, codepy, psutil |
| 13 | +- Supports multiple compilation backends: C, OpenMP, OpenACC, MPI |
| 14 | + |
| 15 | +## Build and Development Environment |
| 16 | + |
| 17 | +### Prerequisites and Installation |
| 18 | + |
| 19 | +**System Dependencies (Ubuntu/Debian):** |
| 20 | +```bash |
| 21 | +sudo apt update |
| 22 | +sudo apt install -y python3-numpy python3-sympy python3-psutil python3-pip |
| 23 | +sudo apt install -y gcc g++ flake8 |
| 24 | +``` |
| 25 | + |
| 26 | +**Development Installation (preferred method):** |
| 27 | +```bash |
| 28 | +# Install with all dependencies - requires network access |
| 29 | +pip install -e .[tests,extras] --break-system-packages |
| 30 | + |
| 31 | +# Alternative: conda environment (recommended for full development) |
| 32 | +conda env create -f environment-dev.yml |
| 33 | +conda activate devito |
| 34 | +``` |
| 35 | + |
| 36 | +**Docker Development (if dependencies fail):** |
| 37 | +```bash |
| 38 | +docker-compose up devito # Starts Jupyter on port 8888 |
| 39 | +# OR |
| 40 | +docker build . --file docker/Dockerfile.devito --tag devito_img |
| 41 | +``` |
| 42 | + |
| 43 | +### Environment Variables (Critical for Compilation) |
| 44 | +```bash |
| 45 | +export DEVITO_ARCH="gcc" # gcc, clang, icx, custom |
| 46 | +export DEVITO_LANGUAGE="openmp" # C, openmp, CXX, CXXopenmp |
| 47 | +export OMP_NUM_THREADS=2 # For parallel tests |
| 48 | +``` |
| 49 | + |
| 50 | +### Build and Test Commands |
| 51 | + |
| 52 | +**Linting (ALWAYS run before committing):** |
| 53 | +```bash |
| 54 | +flake8 --builtins=ArgumentError . |
| 55 | +# Configuration in pyproject.toml: max-line-length=90, specific ignore rules |
| 56 | +``` |
| 57 | + |
| 58 | +**Testing:** |
| 59 | +```bash |
| 60 | +# Core tests (required, ~10-15 minutes) |
| 61 | +pytest -k "not adjoint" -m "not parallel" --maxfail=5 tests/ |
| 62 | + |
| 63 | +# Adjoint tests (advanced feature tests) |
| 64 | +pytest -k "adjoint" -m "not parallel" tests/ |
| 65 | + |
| 66 | +# Example tests (integration validation) |
| 67 | +pytest examples/ |
| 68 | + |
| 69 | +# Specific test file |
| 70 | +pytest tests/test_operator.py -v |
| 71 | +``` |
| 72 | + |
| 73 | +**Configuration Check:** |
| 74 | +```bash |
| 75 | +python3 -c "from devito import configuration; print(''.join(['%s: %s \n' % (k, v) for (k, v) in configuration.items()]))" |
| 76 | +``` |
| 77 | + |
| 78 | +### Common Build Issues and Solutions |
| 79 | + |
| 80 | +1. **Import errors for cgen/codepy:** Install via pip or use conda environment |
| 81 | +2. **Compiler not found:** Set DEVITO_ARCH to match installed compiler |
| 82 | +3. **OpenMP issues on macOS:** Install with `brew install llvm libomp` |
| 83 | +4. **Network timeouts during pip install:** Use conda or Docker alternatives |
| 84 | +5. **Permission issues:** Use `--user` or `--break-system-packages` flags |
| 85 | + |
| 86 | +## Project Architecture and Layout |
| 87 | + |
| 88 | +### Core Directory Structure |
| 89 | +``` |
| 90 | +devito/ # Main source package |
| 91 | +├── __init__.py # Package initialization, configuration setup |
| 92 | +├── arch/ # Architecture-specific compilation backends |
| 93 | +├── builtins/ # Built-in mathematical functions |
| 94 | +├── core/ # Core IR (Intermediate Representation) classes |
| 95 | +├── data/ # Data structures and memory management |
| 96 | +├── finite_differences/# Finite difference operators and stencils |
| 97 | +├── ir/ # Intermediate representation passes |
| 98 | +├── mpi/ # MPI parallelization support |
| 99 | +├── operator/ # Operator compilation and generation |
| 100 | +├── passes/ # Compiler optimization passes |
| 101 | +├── symbolics/ # Symbolic computation utilities |
| 102 | +├── tools/ # Utility functions and helpers |
| 103 | +└── types/ # Core types (Grid, Function, Equation, etc.) |
| 104 | +
|
| 105 | +tests/ # Test suite (comprehensive coverage) |
| 106 | +examples/ # Tutorials and demonstrations |
| 107 | +├── seismic/ # Seismic modeling examples |
| 108 | +├── cfd/ # Computational fluid dynamics |
| 109 | +├── misc/ # Miscellaneous examples |
| 110 | +└── userapi/ # User API demonstrations |
| 111 | +
|
| 112 | +scripts/ # Utility scripts |
| 113 | +docker/ # Docker configurations |
| 114 | +benchmarks/ # Performance benchmarks |
| 115 | +``` |
| 116 | + |
| 117 | +### Key Configuration Files |
| 118 | +- `pyproject.toml`: Build configuration, dependencies, flake8 settings |
| 119 | +- `pytest.ini`: Test configuration and options |
| 120 | +- `requirements*.txt`: Dependency specifications for different environments |
| 121 | +- `environment-dev.yml`: Conda environment specification |
| 122 | +- `.github/workflows/`: CI/CD pipelines |
| 123 | + |
| 124 | +## Continuous Integration and Validation |
| 125 | + |
| 126 | +### GitHub Workflows (All Must Pass) |
| 127 | +1. **pytest-core-nompi.yml**: Core functionality tests across Python 3.10-3.13, multiple GCC versions |
| 128 | +2. **pytest-core-mpi.yml**: MPI parallel execution tests |
| 129 | +3. **pytest-gpu.yml**: GPU computation tests (OpenACC/CUDA) |
| 130 | +4. **flake8.yml**: Code style validation |
| 131 | +5. **examples.yml**: Integration tests via example execution |
| 132 | +6. **tutorials.yml**: Jupyter notebook validation |
| 133 | + |
| 134 | +### Pre-commit Validation Steps |
| 135 | +```bash |
| 136 | +# 1. Lint code |
| 137 | +flake8 --builtins=ArgumentError . |
| 138 | + |
| 139 | +# 2. Run core tests |
| 140 | +pytest -k "not adjoint" -m "not parallel" --maxfail=5 tests/ |
| 141 | + |
| 142 | +# 3. Check configuration |
| 143 | +python3 -c "from devito import configuration; print(configuration)" |
| 144 | + |
| 145 | +# 4. Test example (optional but recommended) |
| 146 | +python examples/seismic/acoustic/acoustic_example.py |
| 147 | +``` |
| 148 | + |
| 149 | +### Test Execution Times |
| 150 | +- Core tests: 10-15 minutes |
| 151 | +- Full test suite: 30-45 minutes |
| 152 | +- Individual test files: 1-5 minutes |
| 153 | +- Linting: <1 minute |
| 154 | +- Examples: 5-10 minutes each |
| 155 | + |
| 156 | +## Development Guidelines |
| 157 | + |
| 158 | +### Code Style (Enforced by CI) |
| 159 | +- **Line length**: 90 characters maximum |
| 160 | +- **Indentation**: 4 spaces (no tabs) |
| 161 | +- **Imports**: Grouped (stdlib, third-party, devito) and alphabetically sorted |
| 162 | +- **Docstrings**: NumPy-style documentation required for public APIs |
| 163 | +- **Naming**: CamelCase classes, snake_case methods/variables |
| 164 | + |
| 165 | +### Making Changes |
| 166 | +1. **Always run linting first**: `flake8 --builtins=ArgumentError .` |
| 167 | +2. **Test specific areas**: Run relevant tests before committing |
| 168 | +3. **Environment setup**: Ensure DEVITO_ARCH and DEVITO_LANGUAGE are set |
| 169 | +4. **Documentation**: Update docstrings for API changes |
| 170 | +5. **Examples**: Add/update examples for new features |
| 171 | + |
| 172 | +### Critical Dependencies |
| 173 | +- **SymPy**: Symbolic mathematics (versions 1.12-1.14 supported) |
| 174 | +- **NumPy**: Numerical arrays (version 2.x) |
| 175 | +- **cgen/codepy**: C code generation and compilation |
| 176 | +- **psutil**: System monitoring and resource management |
| 177 | + |
| 178 | +### Common Workarounds |
| 179 | +- **Star imports**: F403/F405 warnings are expected in `__init__.py` due to API design |
| 180 | +- **Lambda expressions**: E731 warnings are sometimes acceptable for configuration |
| 181 | +- **Import order**: Follow the pattern in `devito/__init__.py` |
| 182 | + |
| 183 | +## Agent-Specific Instructions |
| 184 | + |
| 185 | +**Trust these instructions** - they are validated and current. Only search/explore if: |
| 186 | +1. Instructions appear outdated or incorrect |
| 187 | +2. New functionality is not covered |
| 188 | +3. Specific error messages require investigation |
| 189 | + |
| 190 | +**Time-saving tips:** |
| 191 | +1. Use Docker if pip installation fails due to network issues |
| 192 | +2. Set environment variables before running any compilation-dependent tests |
| 193 | +3. Start with small test files when debugging |
| 194 | +4. Check configuration output to verify environment setup |
| 195 | +5. Use `--maxfail=5` to stop test runs early on systematic failures |
| 196 | + |
| 197 | +**File modification priorities:** |
| 198 | +1. Core types and operators: `devito/types/`, `devito/operator/` |
| 199 | +2. Mathematical operations: `devito/finite_differences/`, `devito/symbolics/` |
| 200 | +3. Compilation backends: `devito/arch/`, `devito/passes/` |
| 201 | +4. Always update corresponding tests in `tests/` |
0 commit comments