Skip to content

Commit 84bd2fe

Browse files
rbs333nkanu17
andauthored
Feat/sql redis query (#467)
# Spec for SQLQuery class Make sql-like commands available to be translated into Redis queries via redisvl to cut down on syntax overhead for engineers. Ex: ```py from redisvl.query import SQLQuery from redisvl.index import SearchIndex redis_index = SearchIndex.from_existing( "my_book_index", redis_url="my_redis_connection" ) sql_query = SQLQuery(""" SELECT title, author, price FROM my_book_index WHERE category = "scify" """ ) response = redis_index.query(sql_query) ``` This code would then produce the equivalent redis query to be executed against the database: ```py FT.search my_book_index "@category:{scify}" LOAD 3 @title @author @price DIALECT 2 ``` Expose a method on the object: `.redis_query_string()` such that you can easily inspect the resulting redis query constructed from SQLQuery class invocation. Ex: ```py from redisvl.query import SQLQuery sql_str = """ SELECT user, credit_score, job, age FROM user_simple WHERE age > 17 """ sql_query = SQLQuery(sql_str) sql_query.redis_query_string(redis_url="redis://localhost:6379") # result: # 'FT.SEARCH user_simple "@Age:[(17 +inf]" RETURN 4 user credit_score job age' ``` # Packaging and dependencies In order to use the `SQLQuery` class, user will have to install the optional dependency on [sql-redis](https://github.com/redis-developer/sql-redis). This can be accomplished with the command `pip install redisvl[sql-redis]`. # Tested/supported operators | Datatype | Operator | Tested | SQL Example | Redis Result | |----------|----------|--------|-------------|--------------| | tag | = | ✅ | `SELECT title, category FROM {index} WHERE category = 'electronics'` | `FT.SEARCH test_index "@category:{electronics}" RETURN 2 title category` | | | != | ✅ | `SELECT title, category FROM {index} WHERE category != 'electronics'` | `FT.SEARCH test_index "-@category:{electronics}" RETURN 2 title category` | | | IN | ✅ | `SELECT title, category FROM {index} WHERE category IN ('books', 'accessories')` | `FT.SEARCH test_index "@category:{books\|accessories}" RETURN 2 title category` | | numeric | > | ✅ | `SELECT title, price FROM {index} WHERE price > 100` | `FT.SEARCH test_index "@price:[(100 +inf]" RETURN 2 title price` | | | >= | ✅ | `SELECT title, price FROM {index} WHERE price >= 25 AND price <= 50` | `FT.SEARCH test_index "@price:[25 +inf] @price:[-inf 50]" RETURN 2 title price` | | | = | ✅ | `SELECT title, price FROM {index} WHERE price = 45` | `FT.SEARCH test_index "@price:[45 45]" RETURN 2 title price` | | | != | ✅ | `SELECT title, price FROM {index} WHERE price != 45` | `FT.SEARCH test_index "-@price:[45 45]" RETURN 2 title price` | | | < | ✅ | `SELECT title, price FROM {index} WHERE price < 50` | `FT.SEARCH test_index "@price:[-inf (50]" RETURN 2 title price` | | | <= | ✅ | `SELECT title, price FROM {index} WHERE price >= 25 AND price <= 50` | `FT.SEARCH test_index "@price:[25 +inf] @price:[-inf 50]" RETURN 2 title price` | | | BETWEEN | ✅ | `SELECT title, price FROM {index} WHERE price BETWEEN 40 AND 60` | `FT.SEARCH test_index "@price:[40 60]" RETURN 2 title price` | | text | = | ✅ | `SELECT title, name FROM {index} WHERE title = 'laptop'` | `FT.SEARCH test_index "@title:laptop" RETURN 2 title name` | | | != | ✅ | `SELECT title, name FROM {index} WHERE title != 'laptop'` | `FT.SEARCH test_index "-@title:laptop" RETURN 2 title name` | | | prefix | ✅ | `SELECT title, name FROM {index} WHERE title = 'lap*'` | `FT.SEARCH test_index "@title:lap*" RETURN 2 title name` | | | suffix | ✅ | `SELECT title, name FROM {index} WHERE name = '*book'` | `FT.SEARCH test_index "@name:*book" RETURN 2 title name` | | | fuzzy | ✅ | `SELECT title, name FROM {index} WHERE title = '%laptap%'` | `FT.SEARCH test_index "@title:%laptap%" RETURN 2 title name` | | | phrase | ✅ | `SELECT title, name FROM {index} WHERE title = 'gaming laptop'` | `FT.SEARCH test_index "@title:gaming laptop" RETURN 2 title name` | | | phrase with stopword | ✅ | `SELECT title, name FROM {index} WHERE title = 'laptop and keyboard'` | `FT.SEARCH test_index "@title:laptop keyboard" RETURN 2 title name` | | | IN | ❌ | `SELECT title, name FROM {index} WHERE title IN ('Python', 'Redis')` | NOT SUPPORTED | | vector | vector_distance | ✅ | `SELECT title, vector_distance(embedding, :vec) AS score FROM {index} LIMIT 3` | `FT.SEARCH test_index "*=>[KNN 3 @Embedding $vector AS score]" PARAMS 2 vector $vector DIALECT 2 RETURN 2 title score LIMIT 0 3` | | | cosine_distance | ✅ | `SELECT title, cosine_distance(embedding, :vec) AS vector_distance FROM {index} LIMIT 3` | `FT.SEARCH test_index "*=>[KNN 3 @Embedding $vector AS vector_distance]" PARAMS 2 vector $vector DIALECT 2 RETURN 2 title vector_distance LIMIT 0 3` | | date | >, >=, =, !=, <, <=, IN, BETWEEN | | | TODO | | geo | = | | | TODO | # Tested/supported aggregation reducer functions | Reducer | Tested | SQL Example | Redis Result | |---------|--------|-------------|--------------| | COUNT | ✅ | `SELECT COUNT(*) as total FROM {index}` | `FT.AGGREGATE test_index "*" GROUPBY 0 REDUCE COUNT 0 AS total` | | COUNT | ✅ | `SELECT category, COUNT(*) as count FROM {index} GROUP BY category` | `FT.AGGREGATE test_index "*" LOAD 1 category GROUPBY 1 @category REDUCE COUNT 0 AS count` | | COUNT_DISTINCT | ✅ | `SELECT category, COUNT_DISTINCT(title) as unique_titles FROM {index} GROUP BY category` | `FT.AGGREGATE test_index "*" LOAD 2 category title GROUPBY 1 @category REDUCE COUNT_DISTINCT 1 @title AS unique_titles` | | SUM | ✅ | `SELECT SUM(price) as total FROM {index}` | `FT.AGGREGATE test_index "*" LOAD 1 price GROUPBY 0 REDUCE SUM 1 @price AS total` | | SUM | ✅ | `SELECT category, SUM(price) as total_price FROM {index} GROUP BY category` | `FT.AGGREGATE test_index "*" LOAD 2 category price GROUPBY 1 @category REDUCE SUM 1 @price AS total_price` | | MIN | ✅ | `SELECT MIN(price) as min_price FROM {index}` | `FT.AGGREGATE test_index "*" LOAD 1 price GROUPBY 0 REDUCE MIN 1 @price AS min_price` | | MIN | ✅ | `SELECT category, MIN(price) as min_price FROM {index} GROUP BY category` | `FT.AGGREGATE test_index "*" LOAD 2 category price GROUPBY 1 @category REDUCE MIN 1 @price AS min_price` | | MAX | ✅ | `SELECT MAX(price) as max_price FROM {index}` | `FT.AGGREGATE test_index "*" LOAD 1 price GROUPBY 0 REDUCE MAX 1 @price AS max_price` | | MAX | ✅ | `SELECT category, MAX(price) as max_price FROM {index} GROUP BY category` | `FT.AGGREGATE test_index "*" LOAD 2 category price GROUPBY 1 @category REDUCE MAX 1 @price AS max_price` | | AVG | ✅ | `SELECT category, AVG(price) as avg_price FROM {index} GROUP BY category` | `FT.AGGREGATE test_index "*" LOAD 2 category price GROUPBY 1 @category REDUCE AVG 1 @price AS avg_price` | | STDDEV | ✅ | `SELECT STDDEV(price) as price_stddev FROM {index}` | `FT.AGGREGATE test_index "*" LOAD 1 price GROUPBY 0 REDUCE STDDEV 1 @price AS price_stddev` | | QUANTILE | ✅ | `SELECT category, QUANTILE(price, 0.5) as median_price FROM {index} GROUP BY category` | `FT.AGGREGATE test_index "*" LOAD 2 price category GROUPBY 1 @category REDUCE QUANTILE 2 @price 0.5 AS median_price` | | TOLIST | ✅ | `SELECT category, ARRAY_AGG(title) as titles FROM {index} GROUP BY category` | `FT.AGGREGATE test_index "*" LOAD 2 category title GROUPBY 1 @category REDUCE TOLIST 1 @title AS titles` | | FIRST_VALUE | ✅ | `SELECT category, FIRST_VALUE(title) as first_title FROM {index} GROUP BY category` | `FT.AGGREGATE test_index "*" LOAD 2 category title GROUPBY 1 @category REDUCE FIRST_VALUE 1 @title AS first_title` | # TODO: subsequent work that will follow in separate PRs - DATE datatype support - ISMISSING / EXISTS null check support - Advanced text search BM25 etc. - Advanced hybrid search - GEO datatype support --------- Co-authored-by: Nitin Kanukolanu <nitinkanukolanu@gmail.com>
1 parent 00211d5 commit 84bd2fe

File tree

12 files changed

+3893
-101
lines changed

12 files changed

+3893
-101
lines changed

docs/api/query.rst

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -231,3 +231,23 @@ MultiVectorQuery
231231
:inherited-members:
232232
:show-inheritance:
233233
:exclude-members: add_filter,get_args,highlight,return_field,summarize
234+
235+
236+
SQLQuery
237+
========
238+
239+
.. currentmodule:: redisvl.query
240+
241+
242+
.. autoclass:: SQLQuery
243+
:members:
244+
:show-inheritance:
245+
246+
.. note::
247+
SQLQuery requires the optional ``sql-redis`` package. Install with:
248+
``pip install redisvl[sql-redis]``
249+
250+
.. note::
251+
SQLQuery translates SQL SELECT statements into Redis FT.SEARCH or FT.AGGREGATE commands.
252+
The SQL syntax supports WHERE clauses, field selection, ordering, and parameterized queries
253+
for vector similarity searches.

docs/user_guide/02_hybrid_queries.ipynb

Lines changed: 118 additions & 96 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)