Smart Pricing AI – Template Project Structure

This repository serves as an MLOps architecture template for a dynamic pricing system.
⚠️ It is not runnable as-is. The implementation details have been partially removed for confidentiality. In the codebase, you will see markers like:

### --- OMITTED --- ###

Project Layout

• sp_worker contains the ML pipelines orchestrated by Airflow, It means smart_pricing_worker
• api contains the FastAPI service that reads the data generated by sp_worker.
• deploy.sh at the root sets up shared volume, deploys sp_worker and then api.

Conceptual Workflow

• sp_worker runs as a background process inside a Docker container with Airflow. It generates AI predictions, data analytics, and other artifacts.
• Files are saved locally with timestamps to allow historical record-keeping. A cleanup process periodically removes old files to prevent uncontrolled storage growth.
• api runs in a separate Docker container with access to the same shared volume. It exposes endpoints to retrieve the locally saved files (predictions, analytics, logs, updates, status, etc.).
• deploy.sh orchestrates deployment of both containers and the shared volume.

smart_pricing/
├── .gitignore
├── deploy.sh                  # Root deploy script to set up both API and sp_worker
├── sample.py
├── api/                       # FastAPI service reading data generated by sp_worker
│   ├── .env                   # Environment file (must be created following .env.example)
│   ├── .env.example           # Template for environment variables
│   ├── deploy.sh              # Deploy script for API
│   ├── docker-compose.yml
│   ├── Dockerfile
│   ├── pyproject.toml
│   ├── uv.lock
│   ├── app/
│   │   ├── main.py            # FastAPI entry point
│   │   └── __init__.py
│   ├── config/                # Configuration and logging
│   ├── controllers/           # API route handlers
│   │   └── endpoints/         # Endpoint-specific logic
│   ├── models/                # Pydantic data models
│   ├── services/              # Business logic services
│   └── utils/                 # Helper functions for the API
├── sp_worker/                 # ML/ETL worker orchestrated by Airflow
│   ├── .dockerignore
│   ├── .env
│   ├── .env.example           # Template for environment variables
│   ├── deploy.sh              # Deploy script for Airflow worker
│   ├── docker-compose.yml
│   ├── Dockerfile
│   ├── Dockerfile.mlflow
│   ├── pyproject.toml
│   ├── README.md
│   ├── .vscode/
│   ├── dags/                  # Airflow DAGs
│   │   ├── smart_pricing_dag_A.py
│   │   ├── smart_pricing_dag_B.py
│   │   └── smart_pricing_dag_utils.py
│   ├── logs/                  # Airflow logs (inside container)
│   ├── plugins/               # Airflow plugins (if any)
│   └── src/                   # Source code of the worker
│       ├── config/            # Logging, paths, settings
│       ├── data_analytics/    # Data analysis pipelines
│       ├── data_ingestion/    # Scripts to fetch/load external data
│       ├── features/          # Feature engineering code
│       ├── ml/                # Machine learning pipelines
│       │   └── baseline/      # Baseline ML experiments
│       │       ├── type_a/    # Type A ML pipelines
│       │       └── type_b/    # Type B ML pipelines
│       │   └── optimized_baseline/
│       └── utils/
│           └── extra_data/    # Feather files for input data:
│               ├── channel_categories.feather
│               ├── manual_channel_roomtype_mapping.feather
│               └── manual_prop_id_event_area_mapping.feather

Deployment Instructions

1. Pre-requisites:
   • .env file created following .env.example.
   • Three feather files in sp_worker/src/utils/extra_data/.
   • Docker installed and running.
2. Run deployment from root:

./deploy.sh

• Creates shared Docker volume airflow_shared-data.
• Sets correct permissions for Airflow (UID 50000) and API (UID 1000).
• Deploys sp_worker (Airflow + ML pipelines) and API (FastAPI).

Services Overview

Service	Description	URL / Access
Airflow	Orchestration of DAGs and AI pipelines	http://localhost:8080/
MLflow	Track ML experiments, logs, and metrics	http://localhost:5000/
FastAPI	API that reads data produced by sp_worker	http://localhost:8005/docs

Airflow Usage

• Toggle DAGs ON to start AI automation.
• DAG Graph view shows task dependencies.
• Logs are accessible for debugging and monitoring.
• Runs inside Docker container; data and logs are persistent in volume.

MLflow Usage

• All experiments tracked inside sp_worker/src/ml/.
• Monitor parameters, metrics, and training outcomes.

FastAPI Usage

• Reads shared data generated by sp_worker.
• Interactive API documentation: http://localhost:8005/docs.
• Runs inside Docker container; uses shared volume to read data.

Run sample.py to test endpoints:

• Wait until the Airflow DAGs finish successfully.
• Then execute: python sample.py
• This will make requests to the FastAPI endpoints and verify that the generated data is accessible.

Notes

• All services run inside Docker containers ; host setup is minimal.
• Shared data structure is automatically managed by deploy.sh.
• .env file and feather files are mandatory to avoid runtime errors.
• Root deploy.sh is the entry point for deploying everything.
• Run sample.py only after Airflow DAGs are completed to test the API.