Integrate OpenTelemetry from adit-radis-shared (#191)

* Integrate OpenTelemetry from adit-radis-shared

Use the shared telemetry module from adit-radis-shared to set up
OpenTelemetry traces, metrics, and logs. Telemetry is initialized
in manage.py and asgi.py before Django loads, and the OTel logging
handler is added conditionally in settings.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Update uv.lock for adit-radis-shared openobserve branch

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* Add OpenObserve and OTel collector Docker infrastructure

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* uv lock

* uv lock missing bracket in merge conflict

* Move observability stack to centralized openradx-observability project

Remove OpenObserve and OTel collector services from RADIS's Docker
Compose stack. Telemetry is now sent directly to the centralized
collector via OTEL_EXPORTER_OTLP_ENDPOINT.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Move OTEL_EXPORTER_OTLP_ENDPOINT out of compose, keep in env file

Remove the OTLP endpoint mapping from docker-compose.base.yml since
the observability stack is now centralized in openradx-observability.
Add docker-compose.observability.yml.example as an opt-in overlay
referencing the env variable, and ignore the active copy in .gitignore.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Use otel-collector.local endpoint in example.env

Point OTEL_EXPORTER_OTLP_ENDPOINT to the collector on the shared
openradx-observability network instead of host.docker.internal.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Rename observability overlay to override

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Pin adit-radis-shared to 0.20.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

Author: samuelvkwong

.gitignore

@@ -143,6 +143,9 @@ pyvenv.cfg
 # Certificate files
 *.pem
 
+# Override overlay (opt-in, copied from .example)
+docker-compose.override.yml
+
 # RADIS specific
 backups/
 models/

docker-compose.override.yml.example

@@ -0,0 +1,21 @@
+x-observability: &observability
+  networks:
+    - default
+    - openradx-observability
+  environment:
+    OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-}
+    OTEL_PYTHON_DJANGO_EXCLUDED_URLS: "health/,static/.*"
+
+networks:
+  openradx-observability:
+    external: true
+
+services:
+  init:
+    <<: *observability
+  web:
+    <<: *observability
+  default_worker:
+    <<: *observability
+  llm_worker:
+    <<: *observability

example.env

@@ -109,6 +109,11 @@ REPORT_LLM_PROVIDER_URL="http://host.docker.internal:11434/v1"
 # 'cli generate-example-reports'.
 REPORT_LLM_PROVIDER_API_KEY="ollama"
 
+# OpenTelemetry Configuration
+# Set this to the OTLP HTTP endpoint of the centralized openradx-observability stack.
+# See https://github.com/openradx/openradx-observability for setup instructions.
+OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector.local:4318
+
 # Docker swarm mode does not respect the Docker Proxy client configuration
 # (see https://docs.docker.com/network/proxy/#configure-the-docker-client),
 # but we can set those environment variables manually.

manage.py

@@ -24,6 +24,11 @@ def initialize_debugger():
 def main():
     os.environ.setdefault("DJANGO_SETTINGS_MODULE", "radis.settings.development")
 
+    # Initialize OpenTelemetry before Django loads to ensure all requests are traced
+    from adit_radis_shared.telemetry import setup_opentelemetry
+
+    setup_opentelemetry()
+
     initialize_debugger()
 
     try:

pyproject.toml

@@ -7,7 +7,7 @@ version = "0.0.0"
 readme = "README.md"
 requires-python = ">=3.12,<4.0"
 dependencies = [
-    "adit-radis-shared @ git+https://github.com/openradx/adit-radis-shared.git@0.19.1",
+    "adit-radis-shared @ git+https://github.com/openradx/adit-radis-shared.git@0.20.0",
     "adrf>=0.1.9",
     "aiofiles>=24.1.0",
     "asyncinotify>=4.2.0",

radis/asgi.py

@@ -12,9 +12,15 @@
 
 import os
 
-from django.core.asgi import get_asgi_application
-
 os.environ.setdefault("DJANGO_SETTINGS_MODULE", "radis.settings.development")
+
+# Initialize OpenTelemetry before Django loads to ensure all requests are traced
+from adit_radis_shared.telemetry import setup_opentelemetry  # noqa: E402
+
+setup_opentelemetry()
+
+from django.core.asgi import get_asgi_application  # noqa: E402
+
 django_asgi_app = get_asgi_application()
 
 from channels.routing import ProtocolTypeRouter  # noqa: E402

radis/settings/base.py

@@ -12,6 +12,7 @@
 
 from pathlib import Path
 
+from adit_radis_shared.telemetry import add_otel_logging_handler, is_telemetry_active
 from environs import env
 
 # During development and calling `manage.py` from the host we have to load the .env file manually.
@@ -266,6 +267,9 @@
     "root": {"handlers": ["console"], "level": "ERROR"},
 }
 
+if is_telemetry_active():
+    add_otel_logging_handler(LOGGING)
+
 # Internationalization
 # https://docs.djangoproject.com/en/3.0/topics/i18n/