We use Indri as our primary evaluation platform because it has convenient support for various models and TREC-isms. Going forward, it would be good to have similar evaluation support for ElasticSearch.
- Find or create an ElasticSearch client program that can take an Indri query file, run against ElasticSearch with a particular similarity model and parameters, and output a TREC-formatted results file for consumption by trec_eval
- Ideally, this will be easily parallelizable