Using limit with merge while using document search can give more results than limit.

[FranseFrikandel]

Performing a document search with multiple fields and using merge, the total amount of results often exceeds the set limit. This probably happends since the limit is enforced for each field seperately, and only after are the results merged. This can mess with pagination.

Following code shows it happening:

const FlexSearch = require("flexsearch");

names = ["foo", "bar"];

const search_idx = new FlexSearch.Document({
    tokenize: 'full',
    document: {
        id: 'id',
        tag: 'category',
        index: ['name_1', 'name_2', 'name_3']
    }
});

for (let i = 0; i < 30; i++) {
    search_idx.add({ id: i, name_1: names[i % 2], name_2: names[(i + 1) % 2], name_3: names[i % 2] })
}

// We will have more than 10 results
res = search_idx.search("foo", { merge: true, limit: 10, offset: 0 })
console.log(res)
console.log(res.length)

Along with that, in my own application, this is causing results to be duplicated when using offset. Weirdly, I haven't yet been able to replicate that behaviour in a more minimal example though. Here are the end and start of my 2 queries overlapping:

  ...
  { id: 1177, field: [ 'name_en' ] },
  { id: 322, field: [ 'name_en' ] },
  { id: 323, field: [ 'name_en' ] },
  { id: 357, field: [ 'name_en' ] }
]
[
  { id: 1177, field: [ 'sku' ] },
  { id: 322, field: [ 'sku' ] },
  { id: 323, field: [ 'sku' ] },
  { id: 357, field: [ 'sku' ] },
  ...

So it's apparently related to the ordering in the 2 seperate fields. These duplicates are beyond the offset for the name_en field, but before the sku field.