[Feature] OpenTelemetry Tracing

[Siddhant-K-code]

Summary

Add OpenTelemetry (OTEL) distributed tracing support for debugging and observability.

Motivation

Production deployments need end-to-end tracing to debug latency issues and understand request flow across services.

Components

• OTEL SDK integration - Instrument all API endpoints
• Span creation - Create spans for each pipeline stage
• Context propagation - Support W3C Trace Context headers
• Exporter configuration - OTLP, Jaeger, Zipkin

Trace Structure

distill.request (root span)
├── distill.dedup
│   ├── distill.embedding
│   └── distill.clustering
├── distill.compress
│   ├── distill.extractive
│   └── distill.placeholder
└── distill.summarize

Configuration

# distill.yaml
telemetry:
  tracing:
    enabled: true
    exporter: otlp
    endpoint: localhost:4317
    sample_rate: 0.1  # 10% sampling

Acceptance Criteria

• All pipeline stages traced
• Context propagation works across services
• Configurable sampling rate
• Works with Jaeger, Zipkin, OTLP collectors