Merge pull request #335 from santoshsahturing/calendar_gym_turing

calendar gym environment from turing

Author: burtenshaw

docs/environments.md

@@ -437,5 +437,25 @@ The OpenEnv community has built a catalog of ready-to-run environments that cove
 </div>
 
 
+<div class="environment-grid">
+  <div class="environment-card">
+    <div class="environment-card__body">
+      <span class="environment-card__tag">Calendar Gym</span>
+      <p class="environment-card__description">
+        This environment exposes a Calendar Gym tools through the OpenEnv reset/step/state interface. The server runs a FastAPI app that serves the OpenEnv endpoints.
+      </p>
+    </div>
+    <div class="environment-card__links">
+      <a class="environment-card__icon" href="https://huggingface.co/spaces/TuringEnterprises/calendar-gym/blob/main/README.md" target="_blank" rel="noreferrer noopener" aria-label="RLVE Gym docs">
+        <svg viewBox="0 0 24 24" aria-hidden="true" focusable="false">
+          <path d="M6 3c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h12c1.1 0 2-.9 2-2V9l-6-6H6zm8 1.5L18.5 9H14V4.5z" fill="currentColor"/>
+        </svg>
+      </a>
+      <a class="environment-card__icon environment-card__icon--hf" href="https://huggingface.co/spaces/TuringEnterprises/calendar-gym" target="_blank" rel="noreferrer noopener" aria-label="RLVE Gym on Hugging Face">
+        <img src="https://huggingface.co/front/assets/huggingface_logo-noborder.svg" alt="" aria-hidden="true" />
+      </a>
+    </div>
+  </div>
+</div>
 
 > Want to publish your own environment? Head over to the [Build Your Own Environment](environment-builder.md) guide for a step-by-step walkthrough.

envs/calendar_env/Dockerfile

@@ -0,0 +1,43 @@
+# Use Python 3.11 slim image as base
+FROM python:3.11-slim
+
+ENV API_PORT=8004
+
+# Set working directory
+WORKDIR /app
+
+# Set environment variables
+ENV PYTHONDONTWRITEBYTECODE=1 \
+  PYTHONUNBUFFERED=1 \
+  PYTHONPATH=/app
+
+# Install system dependencies
+RUN apt-get update && apt-get install -y \
+  gcc \
+  && rm -rf /var/lib/apt/lists/*
+
+# Copy requirements first for better caching
+COPY requirements.txt .
+
+# Install Python dependencies
+RUN pip install --no-cache-dir --upgrade pip && \
+  pip install --no-cache-dir -r requirements.txt
+
+# Copy application code
+COPY . .
+
+# Place seed database where the server expects it
+COPY server/mcp_databases /app/mcp_databases
+
+# Create directory for database files
+RUN mkdir -p /app/mcp_databases
+
+# Expose port 8004
+EXPOSE 8004
+
+# Health check
+HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
+  CMD python -c "import requests; requests.get('http://localhost:8004/health')" || exit 1
+
+# Run the application
+CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "8004"]

envs/calendar_env/README.md

@@ -0,0 +1,77 @@
+---
+title: Calendar Environment Server
+emoji: 📅
+colorFrom: blue
+colorTo: green
+sdk: docker
+pinned: false
+app_port: 8004
+base_path: /docs
+tags:
+  - openenv
+---
+# Calendar Environment
+
+This environment exposes a Calendar Gym tools through the OpenEnv reset/step/state interface. The server runs a FastAPI app that serves the OpenEnv endpoints.
+
+## Server Setup
+
+### Docker (Recommended)
+
+```bash
+cd envs/calendar_env
+docker build -t calendar-env:latest .
+docker run --rm -p 8004:8004 calendar-env:latest
+curl http://localhost:8004/health
+```
+On Server health success response will be:
+`{"status":"healthy","service":"calendar-env"}`
+
+### Without Docker
+
+```bash
+cd envs/calendar_env
+python3 -m venv venv
+source venv/bin/activate
+pip install -r requirements.txt
+uvicorn server.app:app --host 0.0.0.0 --port 8004
+```
+
+## Client Setup
+
+### Quick Start (Demo)
+
+For a quick demo, simply update `llm_api_key` in `scenario_config.json` and run:
+```bash
+python client.py --scenario scenario_config.json
+```
+The existing config includes a sample scenario for testing.
+
+### Configure Scenario
+
+To customize for your use case, edit `scenario_config.json` and update these fields:
+
+**llm variables:**
+- `llm_api_key` - Your OpenAI/Anthropic/Google API key (or set via env var)
+- `llm_model` - Model name (e.g., `gpt-4o-mini`, `claude-3-5-sonnet-20241022`)
+- `llm_provider` - Provider: `openai`, `anthropic`, or `google`
+
+**Scenario Variables**
+- `user_prompt` - Task for the agent to complete
+- `system_prompt` - Instructions for agent behavior
+- `context` - The auth headers for gym like (x-access-token)
+- `seed_database_file` - Path to SQL file for custom data
+- `verifiers` - SQL queries to validate task completion
+- `expected_tools` - Tools agent should use (for tracking)
+
+### Run Client
+
+**Run scenario-based benchmark:**
+```bash
+python client.py --scenario scenario_config.json 
+```
+
+Output will be saved to `response_output/` folder with execution details, tool calls, and verification results.
+
+**Notebook Evaluation:**
+For interactive evaluation and testing, use the: [`Jupyter notebook`](client_notebooks/OpenEnv_and_mcp_Single_Gym_Client_Meta_Turing.ipynb)

envs/calendar_env/__init__.py

@@ -0,0 +1,35 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+"""Calendar Environment package exports."""
+
+from typing import Any
+from .models import (
+    CalendarAction,
+    CalendarObservation,
+    MCPAction,
+    MCPObservation,
+    ListToolsAction,
+    ToolCallAction,
+)
+
+__all__ = [
+    "CalendarAction",
+    "CalendarObservation",
+    "CalendarEnv",
+    "MCPAction",
+    "MCPObservation",
+    "ListToolsAction",
+    "ToolCallAction",
+]
+
+
+def __getattr__(name: str) -> Any:
+    if name == "CalendarEnv":
+        from .client import CalendarEnv as _CalendarEnv
+
+        return _CalendarEnv
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")