[OPIK-4961] [P SDK] [M2] Python SDK: Add project_name parameter to dataset, prompt, and experiment methods#5684
[OPIK-4961] [P SDK] [M2] Python SDK: Add project_name parameter to dataset, prompt, and experiment methods#5684
Conversation
…tests - Introduced the `project_name` field in the `Dataset` class to associate datasets with specific projects. - Updated dataset initialization in tests and CLI to include the new `project_name` parameter when creating datasets. - Added a corresponding `project_name` property with getter functionality in the `Dataset` class.
github-actions
bot
added
python
Python SDK Unit Tests Results (Python 3.11)2 308 tests 2 294 ✅ 56s ⏱️ For more details on these failures, see this check. Results for commit 2f954f0. ♻️ This comment has been updated with latest results. |
Python SDK Unit Tests Results (Python 3.14)2 308 tests 2 294 ✅ 38s ⏱️ For more details on these failures, see this check. Results for commit 2f954f0. ♻️ This comment has been updated with latest results. |
Python SDK Unit Tests Results (Python 3.13)2 308 tests 2 294 ✅ 51s ⏱️ For more details on these failures, see this check. Results for commit 2f954f0. ♻️ This comment has been updated with latest results. |
Python SDK Unit Tests Results (Python 3.10)2 308 tests 2 294 ✅ 58s ⏱️ For more details on these failures, see this check. Results for commit 2f954f0. ♻️ This comment has been updated with latest results. |
Python SDK Unit Tests Results (Python 3.12)2 308 tests 2 294 ✅ 53s ⏱️ For more details on these failures, see this check. Results for commit 2f954f0. ♻️ This comment has been updated with latest results. |
![baz-reviewer[bot]](https://avatars.githubusercontent.com/in/933528?s=60&v=4){kind=small}
Python SDK E2E Tests Results (Python 3.11)238 tests - 8 236 ✅ - 8 10m 31s ⏱️ + 2m 22s Results for commit 2f954f0. ± Comparison against base commit 8e24a51. This pull request removes 13 and adds 5 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
Python SDK E2E Tests Results (Python 3.12)238 tests - 8 236 ✅ - 8 9m 20s ⏱️ + 1m 23s Results for commit 2f954f0. ± Comparison against base commit 8e24a51. This pull request removes 13 and adds 5 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
Python SDK E2E Tests Results (Python 3.13)238 tests - 8 236 ✅ - 8 9m 13s ⏱️ +52s Results for commit 2f954f0. ± Comparison against base commit 8e24a51. This pull request removes 13 and adds 5 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
Python SDK E2E Tests Results (Python 3.10)238 tests - 8 236 ✅ - 8 13m 16s ⏱️ + 4m 56s Results for commit 2f954f0. ± Comparison against base commit 8e24a51. This pull request removes 13 and adds 5 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
Python SDK E2E Tests Results (Python 3.14)238 tests - 8 236 ✅ - 8 8m 42s ⏱️ +38s Results for commit 2f954f0. ± Comparison against base commit 8e24a51. This pull request removes 13 and adds 5 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
andrescrz
left a comment
andrescrz
left a comment
There was a problem hiding this comment.
LGTM, feel free to get another extra review from @alexkuzmik
I guess this still needs Fern to be generated with the latest API changes, hence tests are currently failing.
…related tests - Introduced the `project_name` parameter throughout SDK methods for dataset, experiment, evaluation suite, and prompt operations. - Updated retrieval, creation, and search functions to handle `project_name` correctly. - Added missing TODO comments to integrate `project_name` with the backend when applicable. - Synchronized test cases to validate new `project_name` behavior.
github-actions
bot
added
the
documentation
|
🌿 Preview your docs: https://opik-preview-3f5178dd-6e60-46c6-84d9-4dcfe18bf230.docs.buildwithfern.com/docs/opik No broken links found 📌 Results for commit 8f9c4d4 |
| def get_dataset_id( | ||
| rest_client: OpikApi, dataset_name: str, project_name: Optional[str] | ||
| ) -> str: | ||
| try: | ||
| dataset_id = rest_client.datasets.get_dataset_by_identifier( | ||
| dataset_name=dataset_name | ||
| dataset_name=dataset_name, project_name=project_name | ||
| ).id |
There was a problem hiding this comment.
get_dataset_id forwards project_name even when None, serialized as null and violating the API schema — should we omit project_name when None and only pass it if present?
rest_client.datasets.get_dataset_by_identifier(id, project_name=project_name) => rest_client.datasets.get_dataset_by_identifier(id, **({'project_name': project_name} if project_name is not None else {}))
Finding type: Type Inconsistency | Severity: 🔴 High
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents:
In sdks/python/src/opik/api_objects/dataset/rest_operations.py around lines 218 to 224,
the get_dataset_id function currently calls
rest_client.datasets.get_dataset_by_identifier with project_name even when project_name
is None, causing a null to be sent and the server to reject the request. Change the call
so project_name is only included when it is not None — e.g., build kwargs =
{'dataset_name': dataset_name} and add 'project_name': project_name only if project_name
is not None, or call get_dataset_by_identifier(dataset_name=dataset_name) when
project_name is None. Ensure behavior is unchanged for non-None project_name.
…t methods - Adjusted method parameter order to make `project_name` optional and consistent with SDK conventions. - Added `project_name` resolution logic for improved dataset ID retrieval.
…t streaming methods - Updated `delete_dataset` to accept an optional `project_name` parameter, with default resolution logic. - Extended dataset streaming and related classes to handle `project_name`. - Added unit tests to validate `project_name` behavior in `delete_dataset`.
… and search methods - Updated prompt creation, retrieval, and history functions (`create_prompt`, `get_prompt`, `get_prompt_history`) to include `project_name` handling. - Extended ChatPrompt methods with `project_name` support. - Refactored relevant classes and tests to validate `project_name` resolution and propagation.
…relevant tests - Added `project_name` parameter to chat prompt creation, retrieval, search, and history methods (`create_chat_prompt`, `get_chat_prompt`, `get_chat_prompt_history`, `search_prompts`). - Updated chat prompt-related tests to validate `project_name` propagation. - Fixed typos in code examples for `get_prompt_history` and `get_chat_prompt_history`.
…, and version view methods - Updated dataset creation, filtering, and version view methods (`create_dataset`, `get_items`, `get_version_view`) to include `project_name` handling. - Extended E2E tests to validate `project_name` propagation and expected behavior. - Added assertions for `project_name` comparison in dataset verifications.
…xperiments, and verifiers - Added `project_name` handling to evaluation suite creation, retrieval, and propagation methods. - Updated experiments, verifiers, and rest operations to support `project_name`. - Enhanced E2E tests to validate `project_name` propagation and assertions within evaluation scenarios.
| client: opik_client.Opik, experiment_name: str, project_name: Optional[str] = None | ||
| ) -> experiment.Experiment: | ||
| experiments = client.get_experiments_by_name(name=experiment_name) | ||
| experiments = client.get_experiments_by_name( | ||
| name=experiment_name, project_name=project_name | ||
| ) |
There was a problem hiding this comment.
get_experiments_data_by_name accepts project_name but doesn't forward it to rest_client.experiments.stream_experiments, should we forward project_name into the stream call or filter results in the experiment REST operations?
Finding type: Breaking Changes | Severity: 🔴 High
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents:
In sdks/python/src/opik/evaluation/rest_operations.py around lines 14-18,
get_experiment_with_unique_name now accepts project_name but the underlying experiment
lookup still ignores project scoping. Fix the root cause in the experiment REST
operations: open sdks/python/src/opik/api_objects/experiment/rest_operations.py and
modify the function that implements get_experiments_data_by_name (or the streaming
helper) to either forward the project_name into
rest_client.experiments.stream_experiments (add it as a query/parameter to the API call)
or, if the backend cannot accept it, filter the streamed/parsed experiment results by
project_name before returning. Also review
sdks/python/src/opik/api_objects/opik_client.py (opik_client.get_experiments_by_name) to
ensure it forwards project_name to the experiment REST function. Make sure unit tests
cover the case evaluate(..., project_name=...) so experiments from other projects are
not returned.
| rest_client=self._rest_client, name=name, project_name=project_name | ||
| ) | ||
|
|
||
| return experiment.Experiment( |
There was a problem hiding this comment.
experiment.Experiment construction is duplicated across getters, including project_name, should we extract _build_experiment_from_public(public_experiment) and reuse it?
Finding type: Code Dedup and Conventions | Severity: 🟢 Low
Want Baz to fix this for you? Activate Fixer
| else: | ||
| experiment = rest_operations.get_experiment_with_unique_name( | ||
| client=client, experiment_name=experiment_name | ||
| client=client, experiment_name=experiment_name, project_name=project_name | ||
| ) | ||
|
|
||
| dataset_ = client.get_dataset(name=experiment.dataset_name) | ||
| dataset_ = client.get_dataset( | ||
| name=experiment.dataset_name, project_name=project_name | ||
| ) |
There was a problem hiding this comment.
Should we default client.get_dataset's project_name to experiment.project_name when project_name isn't provided, e.g. client.get_dataset(..., project_name=project_name or experiment.project_name)?
Finding type: Use stable client APIs | Severity: 🔴 High
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents:
In sdks/python/src/opik/evaluation/evaluator.py around lines 633 to 640 inside the
evaluate_experiment function, the code calls
client.get_dataset(name=experiment.dataset_name, project_name=project_name) which can be
None when the caller provided experiment_id. Change this call to default to the
experiment's project when caller did not provide one: use
client.get_dataset(name=experiment.dataset_name, project_name=project_name or
experiment.project_name). Ensure the fallback only applies after experiment is resolved
(i.e., experiment variable is available).
2 participants
Details
Add project_name parameter to dataset, prompt, and experiment methods in the Python SDK, so that users can work with project-scoped entities.
Change checklist
Issues
AI-WATERMARK
AI-WATERMARK: [yes|no]
Testing
Fixed existing tests
Documentation
Updated docstrings where appropriate