Project for PEI evaluation 25/26
Data storage service providing dual-database persistence layer for the NWDAF platform. Consumes processed network data from Kafka and stores it in both time-series (InfluxDB) and analytical (ClickHouse) databases, while exposing REST APIs for data retrieval.
- Python 3.13
- FastAPI - REST API framework
- InfluxDB - Time-series database for metrics
- ClickHouse - Columnar analytical database
- Apache Kafka (confluent_kafka) - Message streaming consumer
- Docker & Docker Compose - Containerization
- uvicorn - ASGI server
- pytest - Testing framework
Dual Database Strategy:
- InfluxDB: Optimized for time-series queries, network metrics storage, fast writes
- ClickHouse: Optimized for analytical queries, aggregations, OLAP workloads
- Kafka consumer: Subscribes to
network.data.processedtopic - Dual writes: Simultaneously stores data in InfluxDB and ClickHouse
- REST API endpoints: Query historical network data
- Cell metadata retrieval
- Time-range queries
- Latency data access
- Batch pagination support
- Data models: Network performance metrics (RSRP, SINR, RSRQ, latency, CQI, datarate)
- Integration: Provides data to ML service for training and processor for metadata
docker-compose up -d- InfluxDB: Port 8086
- ClickHouse: Port 8123 (HTTP), Port 9000 (TCP)
/cells- List all cells/cells/{cell_id}- Get cell metadata/latency- Query latency data/metrics- Retrieve network metrics
Kafka (network.data.processed) →
Storage Service →
├── InfluxDB (time-series metrics)
└── ClickHouse (analytical queries)
- Historical network performance analysis
- Training data for ML models
- Cell metadata for processor windowing
- Time-range queries for analytics
- Real-time metrics dashboards
Configure via .env:
- Database connection strings
- Kafka broker URLs
- Service ports
- Retention policies
pytest tests/