...
| No Format |
|---|
cd ~/biocaddie Rscript scripts/compare_treccds.R <collection> <from model> <to model> <topic> <year> |
Results (rm3 compared to be updatedtfidf baseline)
| Model | MAP | NDCG | P@20 | NDCG@20 | P@100 | NDCG@100 | Notes | Date |
|---|---|---|---|---|---|---|---|---|
| tfidf | 0.0957 | 0.3017 | 0.3 | 0.2353 | 0.1857 | 0.194 | Sweep b and k1 | 06/05/17 |
| Okapi | 0.0552- | 0.1831- | 0.23- | 0.1824- | 0.128- | 0.1477- | Sweep b, k1, k3 | 06/05/17 |
| QL (JM) | 0.0971 | 0.2901 | 0.3167 | 0.2708 | 0.1827 | 0.212 | Sweep lambda | 06/05/17 |
| QL (Dir) | 0.1026 | 0.3002 | 0.3217 | 0.2705 | 0.1937 | 0.2197+ | Sweep mu | 06/05/17 |
| QL (TS) | 0.0976 | 0.3001 | 0.295 | 0.2555 | 0.1857 | 0.2145 | Sweep mu and lambda | 06/05/17 |
| RM3 | 0.1026 | 0.3002 | 0.3217 | 0.2705 | 0.1937 | 0.2197+ | Sweep mu, fbDocs, fbTerms, and lambda | 06/06/17 |
| No Format |
|---|
root@integration-1:~/biocaddie# Rscript scripts/compare_treccds.R combined tfidf dir orig 2015
[1] "map 0.0957 0.1026 p= 0.2837"
[1] "ndcg 0.3017 0.3002 p= 0.5381"
[1] "P_20 0.3 0.3217 p= 0.2352"
[1] "ndcg_cut_20 0.2353 0.2705 p= 0.0834"
[1] "P_100 0.1857 0.1937 p= 0.2639"
[1] "ndcg_cut_100 0.194 0.2197 p= 0.0396"
root@integration-1:~/biocaddie# Rscript scripts/compare_treccds.R combined tfidf jm orig 2015
[1] "map 0.0957 0.0971 p= 0.4594"
[1] "ndcg 0.3017 0.2901 p= 0.7606"
[1] "P_20 0.3 0.3167 p= 0.3204"
[1] "ndcg_cut_20 0.2353 0.2708 p= 0.1038"
[1] "P_100 0.1857 0.1827 p= 0.5918"
[1] "ndcg_cut_100 0.194 0.212 p= 0.1283"
root@integration-1:~/biocaddie# Rscript scripts/compare_treccds.R combined tfidf okapi orig 2015
[1] "map 0.0957 0.0552 p= 0.9997"
[1] "ndcg 0.3017 0.1831 p= 1"
[1] "P_20 0.3 0.23 p= 0.9979"
[1] "ndcg_cut_20 0.2353 0.1824 p= 0.9991"
[1] "P_100 0.1857 0.128 p= 0.9999"
[1] "ndcg_cut_100 0.194 0.1477 p= 0.9993"
root@integration-1:~/biocaddie# Rscript scripts/compare_treccds.R combined tfidf two orig 2015
[1] "map 0.0957 0.0976 p= 0.432"
[1] "ndcg 0.3017 0.3001 p= 0.5467"
[1] "P_20 0.3 0.295 p= 0.5667"
[1] "ndcg_cut_20 0.2353 0.2555 p= 0.2152"
[1] "P_100 0.1857 0.1857 p= 0.5"
[1] "ndcg_cut_100 0.194 0.2145 p= 0.0652"
root@integration-1:~/biocaddie/scripts# ./compare_treccds.R combined tfidf dir orig 2015
[1] "map 0.0957 0.1026 p= 0.2837"
[1] "ndcg 0.3017 0.3002 p= 0.5381"
[1] "P_20 0.3 0.3217 p= 0.2352"
[1] "ndcg_cut_20 0.2353 0.2705 p= 0.0834"
[1] "P_100 0.1857 0.1937 p= 0.2639"
[1] "ndcg_cut_100 0.194 0.2197 p= 0.0396" |
2. TREC CDS 2016 collection (http://trec-cds.appspot.com/2016.html)
...