Draft
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR upgrades the query adapter algorithm by refactoring its implementation for multi-threaded triplet extraction, weight optimization via L-BFGS, and adds gradient-based unit tests. It also extends typing for float tensors, refines the CLI bench command with better error handling and reranker support, and enhances progress bar outputs in document insertion.
- Refactored
_query_adapter.pyto introduce helper functions (_extract_triplets,_optimize_query_target,_compute_query_adapter_grad, etc.), multi-threading, and weight optimization with SciPy’sminimize. - Added
FloatTensoralias in_typing.pyand atest_query_adapter_gradintests/test_query_adapter.pyto validate the gradient. - Improved the CLI bench command (
_cli.py,_bench.py), including aprescorestep, optional reranker integration, and user-friendly import errors. - Minor enhancement: show inserted document IDs in the progress bar during
_insert.py.
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_query_adapter.py | Added gradient correctness test using scipy.check_grad and increased num_evals for stability |
| src/raglite/_typing.py | Introduced FloatTensor alias for 3-D float arrays |
| src/raglite/_query_adapter.py | Full refactor of query-adapter optimization logic, threading, and weight learning |
| src/raglite/_insert.py | Show document ID in insertion progress bar |
| src/raglite/_cli.py | Guard bench imports, expose reranker option |
| src/raglite/_bench.py | Added prescore hook and reranker support in evaluator |
Comments suppressed due to low confidence (1)
src/raglite/_query_adapter.py:80
- [nitpick] The
_compute_query_adapterhelper now implements the core transform logic for both 'dot' and 'cosine' metrics but lacks direct unit tests. Consider adding tests that verify its output under known inputs.
def _compute_query_adapter(
Contributor
|
Related to issue: #152 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes:
_bench.py._bench.py.insert_documents._query_adapter.py.