Possibility of including TREC DL 2019 and TREC DL 2020

[liyongkang123]

Hi,
As the BEIR library sees wider adoption, I’m wondering if it would be possible to add more datasets to better support it. A typical candidate would be TREC DL 2019 and TREC DL 2020. Since we already have MS MARCO, integrating these two datasets shouldn’t be too difficult.

I greatly appreciate your help. If you don’t have time, maybe I could process them into the same format, and then you could just upload them to the website.

Many thanks.

[liyongkang123]

I’ve created a code repository to deal with this.

Anyone who needs it can use it.

[thakur-nandan]

Hi @liyongkang123, the TREC-DL 2019 is present in the msmarco dataset under the test split.

Regards,
Nandan

[YuchenHui22314]

Hi @thakur-nandan When using the BEIR msmarco for benchmarking. Should we use "dev" or "test"?