[Querqy] How can we add different analyzer for the query and the matching field in Querqy query?

[rakeshroshann-ai]

I’m looking for some help. How can we apply a different analyzer for the query and the matching field?

query analyzer: default_synonym
field analyzer: default

Without querqy:

{
  "multi_match": {
    "query": "{{query}}",
    "operator": "or",
    "minimum_should_match": "-24%",
    "type": "cross_fields",
    "analyzer": "default_synonym",
    "tie_breaker": 0.7,
    "fields": ["keywords^10", "name.default^5"]
  }
}

with Querqy:

{
  "querqy": {
    "matching_query": {
      "query": "{{query}}"
    },
    "query_fields": ["keywords^10", "name.default^5"],
    "rewriters": ["replace", "common_rules"],
    "minimum_should_match": "-24%",
    "tie_breaker": 0.7
  }
}

querqy version: querqy-elasticsearch-1.7.es8171.0

[rakeshroshann-ai]

@renekrie I’d appreciate your suggestion on this.

[renekrie]

Hey @rakeshroshann-ai,

You cannot set an analyzer in Querqy. Can you let me know why you want to set an analyzer? Maybe there is an alternative way of achieving the same goal in Querqy.

[akbodapati]

Hi @renekrie,
We want to apply different analyzers on the query and the matching field specifically for synonym matching.
We have synonyms_set stored in Elasticsearch, and we have defined a custom analyzer of type "synonym_graph" which uses this set. We want to apply this analyzer only on the query side, so that synonym expansion happens during query analysis, and correct products are returned

Example Use Case
While searching for “vegan milk”.
We want the query analyzer to rewrite this into ("vegan milk" OR "plant based milk").
The product title field itself is analyzed without synonyms, to prevent unintended matches and index bloat.

We looked into using the Common Rules Rewriter for synonyms, but it doesn’t handle stemming, asciifolding, plurals etc which we can mention in a custom elasticsearch analyzer

Could you suggest how we can achieve this in Querqy?

[renekrie]

Hey @akbodapati,

Querqy also applies synonyms at query time as we think that this is the much better approach.

Querqy does not allow to specify a Lucene analyzer at query time (but of course it applies the per-field analyzers from the index mappings).

To be honest, I don't quite see how stemmers should be applied before synonyms in standard Elasticsearch. Let's assume you have defined a synonym for "spy => spy, secret agent" in your synonym token filter and a query "spy" is being passed. A preceding English (porter-like) stemmer would turn "spy" into "spi" and the lookup in the synonym filter would fail and not expand "spy", same for "spies". You'd need to put "spi => spi, secret agent" into your synonyms to make that work. (I hope I don't missing anything here.)

If we hadn't applied the stemmer analyzer before synonyms, we'd at least match the token for "spy" and we'd have to maintain a second rule for the plural "spies => spies, secret agent".

It would be great if we didn't have to maintain synonyms for both singular and plural. That's why we've created a solution in Query that can deal with singular/plural matching when looking up synonyms in Common Rules (ans it's not using analyzers). This is only available for German so far and - as far as I remember - only in Querqy for Solr (but it could be ported the Elasticsearch version quickly). I'm not sure if an implementation for German would be of any use to you? Other languages could be added as there is some form of abstraction but one would have to implement some language specifics. (For German we chose a lemmatization approach instead of stemming as stemming often is too imprecise.)

The only other missing thing that you mention is ASCII-folding. You could write a custom query rewriter that does just this and apply it before the Common Rules rewriter but, to be honest, most teams that I've seen just ignore this or consciously keep diacritics etc. as they think they are too important to be removed (it depends a bit on the language).