Baselines
The following baselines were run using the combined, short topics on the biocaddie_all index using the baselines/ scripts to sweep parameter combinations and the mkeval.sh script to perform LOOCV for each desired metric.
| Model | MAP | NDCG | P@20 | NDCG@20 | Notes |
|---|---|---|---|---|---|
| TFIDF | 0.2524 | 0.459 | 0.4643 | 0.4144 | |
| Okapi | 0.2548 | 0.466 | 0.5+ | 0.4414+ | |
| QL (JM) | 0.2573 | 0.5159 | 0.5524+ | 0.4886+ | |
| QL (Dir) | 0.2837+ | 0.5306+ | 0.5667+ | 0.5054+ | Map mu=1000, NDCG mu=500, P_20 mu=500, NDCG_20 mu=500 |
| QL (TS) | 0.2794 | 0.5391+ | 0.531 | 0.4702 |
Feedback models
| Model | MAP | NDCG | P@20 | NDCG@20 | P@100 | NDCG@100 | Notes |
|---|---|---|---|---|---|---|---|
| QL (Dir) | 0.2837 | 0.5306 | 0.5667 | 0.5054 | 0.3981 | 0.4541 | Map mu=1000, NDCG mu=500, P_20 mu=500, NDCG_20 mu=500 |
| Pubmed | 0.3076+ | 0.5853+ | 0.5381 | 0.4855 | 0.4248 | 0.4501 | Using mu=2500 for pubmed, sweeping mu for biocaddie |
| Wikipedia | 0.2947 | 0.5956+ | 0.5738 | 0.4956 | 0.4062 | 0.4487 | Using mu=2500 for wikipedia, sweeping mu for biocaddie |