feat: add per-pod GPU SM utilization metrics for time-slicing workloads

[zbennett10]

Closes #587

Summary

When CUDA time-slicing is active, multiple pods share a single physical GPU. Standard DCGM per-device metrics (dcgm_fi_dev_gpu_util) report aggregate utilization for the whole device — you cannot tell how much of the GPU proxy, embeddings, or inference pods are each consuming.

This PR adds an opt-in ProcessPodCollector that attributes GPU SM utilization to individual pods by joining:

1. NVML nvmlDeviceGetProcessUtilization() — per-PID SM utilization sampled directly from the CUDA driver
2. Kubelet pod-resources gRPC API — maps GPU UUIDs to (pod, namespace, container) tuples
3. /proc/<pid>/cgroup — links NVML PIDs back to container identities

New metric

# HELP dcgm_fi_dev_sm_util_per_pod SM utilization attributed to a pod (time-slicing)
# TYPE dcgm_fi_dev_sm_util_per_pod gauge
dcgm_fi_dev_sm_util_per_pod{gpu="0",uuid="GPU-abc123",pod="synapse-proxy-...",namespace="synapse-staging",container="proxy"} 42

One gauge is emitted per (pod, namespace, container, gpu_uuid) tuple. The value is the NVML SM utilization percentage (0–100).

Enabling

Standalone DaemonSet

spec:
  template:
    spec:
      hostPID: true
      containers:
        - name: dcgm-exporter
          env:
            - name: DCGM_EXPORTER_ENABLE_PER_POD_GPU_UTIL
              value: "true"
          volumeMounts:
            - name: pod-resources
              mountPath: /var/lib/kubelet/pod-resources
              readOnly: true
      volumes:
        - name: pod-resources
          hostPath:
            path: /var/lib/kubelet/pod-resources
            type: Directory

With GPU Operator (v24.x+) ClusterPolicy

spec:
  dcgmExporter:
    perPodGPUUtil:
      enabled: true

See companion PR in NVIDIA/gpu-operator that wires this through ClusterPolicy -> NVIDIA/gpu-operator#2178

Test plan

• Unit tests pass (go test ./internal/pkg/collector/... -run TestProcessPodCollector -v) — 10 test cases, all PASS, no GPU required
• go build ./... on ubuntu-latest (CI)
• Integration tested in our AWS environment: verified dcgm_fi_dev_sm_util_per_pod emitted with correct pod/namespace/container labels when time-slicing is active