fix(opensearch): Update should clear projects and personas when they are empty

[acaprau]

Description

Currently empty projects and personas are treated as None, meaning no update required, when really an empty list should mean clear these respective fields from the chunk.

This is a followup to #8680 (comment).

How Has This Been Tested?

I went through the effort of adding external dependency unit tests for this function of the old document index interface. Hopefully these fixtures can be used to easily add additional tests when needed. This took me more time and effort than I wanted, please clap.

Additional Options

• [Optional] Please cherry-pick this PR to the latest release version.
• [Optional] Override Linear Check

---

Summary by cubic

Fixes document updates so empty user_projects/personas clear those fields in OpenSearch and Vespa. Filters for project/persona no longer match cleared chunks.

• Bug Fixes
  • In update_single, treat empty lists as different from None; send empty sets to clear fields and None when not provided.
  • Added external dependency tests for the old DocumentIndex to verify clearing and filter behavior on both backends.

^{Written for commit 892e1c7. Summary will update on new commits.}

[greptile-apps]

Greptile Summary

Fixes a bug where empty lists for user_projects and personas were incorrectly treated as None (no update needed), when they should actually clear those fields from document chunks.

• Changed OpenSearch condition from checking truthiness (if user_fields.user_projects) to explicit None check (if user_fields.user_projects is not None) on lines 386 and 393
• Added clarifying comments to Vespa implementation (logic was already correct there)
• Added comprehensive test coverage with new external dependency unit test that validates clearing works for both OpenSearch and Vespa
• All TODOs properly formatted with ticket references per project standards

Confidence Score: 5/5

• Safe to merge - focused bug fix with comprehensive test coverage
• Clean, minimal change that fixes a real bug with proper test validation. The fix correctly distinguishes between empty lists (clear fields) and None (no update). No side effects or risks identified.
• No files require special attention

Important Files Changed

Filename	Overview
backend/onyx/document_index/opensearch/opensearch_document_index.py	Fixed bug where empty lists for user_projects and personas were treated as None instead of clearing the fields
backend/tests/external_dependency_unit/document_index/test_document_index_old.py	Comprehensive new test file with fixtures for both OpenSearch and Vespa, validates that empty lists clear fields as expected

_{Last reviewed commit: ac49a02}

[cubic-dev-ai]

1 issue found across 4 files

Confidence score: 4/5

• There’s a moderate risk of test failures: module-scoped fixtures in backend/tests/external_dependency_unit/document_index/test_document_index_old.py depend on function-scoped tenant_context, which can trigger pytest ScopeMismatch errors.
• This is limited to the test suite and doesn’t imply a runtime regression, so the overall merge risk remains low.
• Pay close attention to backend/tests/external_dependency_unit/document_index/test_document_index_old.py - fix fixture scoping to avoid ScopeMismatch errors.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="backend/tests/external_dependency_unit/document_index/test_document_index_old.py">

<violation number="1" location="backend/tests/external_dependency_unit/document_index/test_document_index_old.py:60">
P2: Module-scoped fixtures cannot depend on a function-scoped fixture. `tenant_context` should be module-scoped (or removed from those fixtures) to avoid pytest `ScopeMismatch` errors.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: Module-scoped fixtures cannot depend on a function-scoped fixture. tenant_context should be module-scoped (or removed from those fixtures) to avoid pytest ScopeMismatch errors.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At backend/tests/external_dependency_unit/document_index/test_document_index_old.py, line 60:

<comment>Module-scoped fixtures cannot depend on a function-scoped fixture. `tenant_context` should be module-scoped (or removed from those fixtures) to avoid pytest `ScopeMismatch` errors.</comment>

<file context>
@@ -0,0 +1,290 @@
+    yield f"test_index_{uuid.uuid4().hex[:8]}"  # Test runs here.
+
+
[email protected](scope="function")
+def tenant_context() -> Generator[None, None, None]:
+    """Sets up tenant context for testing."""
</file context>

[github-actions]

🖼️ Visual Regression Report

Project	Changed	Added	Removed	Unchanged	Report
admin	2	0	0	107	View Report
exclusive	0	0	0	8	✅ No changes