...
+ indicates significant improvement over TFIDF baselines (p < 0.05)
Results when using the combined, orig topics on the biocaddie_all index using the baselines/ scripts to sweep parameter combinations and the mkeval.sh script to perform LOOCV for each desired metric.
| Model | MAP | NDCG | P@20 | NDCG@20 | P@100 | NDCG@100 | Notes | Date |
|---|---|---|---|---|---|---|---|---|
| TFIDF | 0.1661 | 0.3749 | 0.395 | 0.3234 | 0.2505 | 0.2962 | Sweep b and k1 | 5/17/17 |
| Okapi | 0.2894 | 0.5629+ | 0.5643 | 0.4927+ | 0.4029 | 0.4516 | Sweep b, k1, k3 | 5/17/17 |
| QL (JM) | 0.2661 | 0.5311 | 0.5381 | 0.4853 | 0.3852 | 0.437 | Sweep lambda | 5/17/17 |
| QL (Dir) | 0.2976+ | 0.5656+ | 0.5643 | 0.5004 | 0.4119 | 0.457 | Sweep mu | 5/17/17 |
| QL (TS) | 0.2931+ | 0.5653+ | 0.5714 | 0.4827 | 0.4024 | 0.4507 | Sweep mu and lambda | 5/17/17 |
Feedback/SDM models
Green rows are runs with Garrick's framework, all others are with Craig's framework.
...