...
Model | MAP | NDCG | P@20 | NDCG@20 | P@100 | NDCG@100 | Notes | Date |
---|---|---|---|---|---|---|---|---|
classic tfidf | 0.3282 | 0.5824 | 0.6867 | 0.5478 | 0.5013 | 0.5018 | No parameters | 07/10/17 |
classic tfidf (tfidf-snowball) | 0.3375 | 0.5944 | 0.6667 | 0.5256 | 0.4987 | 0.5002 | No parameters | 07/10/17 |
BM25 | 0.3543 | 0.6105 | 0.7467 | 0.5917 | 0.506 | 0.5186 | Sweep b, k1 | 07/10/17 |
BM25 (bm25-snowball) | 0.3764+ | 0.6239 | 0.73 | 0.6006 | 0.5413+ | 0.539+ | Sweep b, k1 | 07/10/17 |
QL (JM) | 0.3382 | 0.6022 | 0.7233 | 0.571 | 0.5 | 0.4996 | Sweep lambda | 07/10/17 |
QL (JM) (jm-snowball) | 0.3448 | 0.6058 | 0.67 | 0.5813 | 0.4987 | 0.5289+ | Sweep lambda | 07/10/17 |
QL (Dir) | 0.3675 | 0.6163 | 0.6567 | 0.5664 | 0.5213 | 0.522 | Sweep mu | 07/10/17 |
QL (Dir) (dir-snowball) | 0.3776 | 0.6315 (p-value=0.0534) | 0.7033+ | 0.6006+ | 0.5307 | 0.5365 | Sweep mu | 07/10/17 |
Rocchio | 0.4044 | 0.6417 | 0.6967 | 0.5403 | 0.492 | 0.4912 | Sweep b, k1, fbTerms, fbDocs, fbOrigWeight | 07/10/17 |
Rocchio (rocchio-snowball) | 0.3959 | 0.6052- | 0.7267 | 0.598+ | 0.5453+ | 0.525 | Sweep b, k1, fbTerms, fbDocs, fbOrigWeight | 07/10/17 |
...
No Format |
---|
thphan@biocaddie-dev:/data/thphan/biocaddie$ Rscript scripts/new/compare.R test tfidf-snowball dir-snowball short biocaddie Please enter run methods for comparison: 0: both are Indri 1: both are Lucene 2: from is Indri, to is Lucene 3: from is Lucene, to is Indri 1 [1] "map 0.3375 0.3776 p= 0.0387" [1] "ndcg 0.5944 0.6315 p= 0.0072" [1] "P_20 0.6667 0.7033 p= 0.1042" [1] "ndcg_cut_20 0.5256 0.6006 p= 0.0046" [1] "P_100 0.4987 0.5307 p= 0.0652" [1] "ndcg_cut_100 0.5002 0.5365 p= 0.0207" thphan@biocaddie-dev:/data/thphan/biocaddie$ Rscript scripts/new/compare.R test tfidf-snowball jm-snowball short biocaddie Please enter run methods for comparison: 0: both are Indri 1: both are Lucene 2: from is Indri, to is Lucene 3: from is Lucene, to is Indri 1 [1] "map 0.3375 0.3448 p= 0.2782" [1] "ndcg 0.5944 0.6058 p= 0.2069" [1] "P_20 0.6667 0.67 p= 0.475" [1] "ndcg_cut_20 0.5256 0.5813 p= 0.0161" [1] "P_100 0.4987 0.4987 p= 0.5" [1] "ndcg_cut_100 0.5002 0.5289 p= 0.0117" thphan@biocaddie-dev:/data/thphan/biocaddie$ Rscript scripts/new/compare.R test tfidf-snowball bm25-snowball short biocaddie Please enter run methods for comparison: 0: both are Indri 1: both are Lucene 2: from is Indri, to is Lucene 3: from is Lucene, to is Indri 1 [1] "map 0.3375 0.3764 p= 0.0284" [1] "ndcg 0.5944 0.6239 p= 0.011" [1] "P_20 0.6667 0.73 p= 0.0331" [1] "ndcg_cut_20 0.5256 0.6006 p= 0.0045" [1] "P_100 0.4987 0.5413 p= 0.0326" [1] "ndcg_cut_100 0.5002 0.539 p= 0.0149" |
Compare results between unstemmed and stemmed indexes:
No Format |
---|
thphan@biocaddie-dev:/data/thphan/biocaddie$ Rscript scripts/new/compare.R test tfidf tfidfrocchio-snowball short biocaddie Please enter run methods for comparison: 0: both are Indri 1: both are Lucene 2: from is Indri, to is Lucene 3: from is Lucene, to is Indri 1 [1] "map 0.3282 0.33753959 p= 0.14630427" [1] "ndcg 0.5824 0.59446052 p= 0.14542869" [1] "P_20 0.6867 0.66677267 p= 0.8081189" [1] "ndcg_cut_20 0.5478 0.5256598 p= 0.87150424" [1] "P_100 0.5013 0.49875453 p= 0.58191152" [1] "ndcg_cut_100 0.5018 0.5002525 p= 0.56522733" |
Compare results between unstemmed and stemmed indexes:
No Format |
---|
thphan@biocaddie-dev:/data/thphan/biocaddie$ Rscript scripts/new/compare.R test dirtfidf dirtfidf-snowball short biocaddie Please enter run methods for comparison: 0: both are Indri 1: both are Lucene 2: from is Indri, to is Lucene 3: from is Lucene, to is Indri 1 [1] "map 0.36753282 0.37763375 p= 0.08421463" [1] "ndcg 0.61635824 0.63155944 p= 0.05341454" [1] "P_20 0.65676867 0.70336667 p= 0.0011808" [1] "ndcg_cut_20 0.56645478 0.60065256 p= 0.02228715" [1] "P_100 0.52135013 0.53074987 p= 0.19425819" [1] "ndcg_cut_100 0.5225018 0.53655002 p= 0.06455652" thphan@biocaddie-dev:/data/thphan/biocaddie$ Rscript scripts/new/compare.R test jmdir jmdir-snowball short biocaddie Please enter run methods for comparison: 0: both are Indri 1: both are Lucene 2: from is Indri, to is Lucene 3: from is Lucene, to is Indri 1 [1] "map 0.33823675 0.34483776 p= 0.26030842" [1] "ndcg 0.60226163 0.60586315 p= 0.33580534" [1] "P_20 0.72336567 0.677033 p= 0.88850011" [1] "ndcg_cut_20 0.5715664 0.58136006 p= 0.25510222" [1] "P_100 0.55213 0.49875307 p= 0.551942" [1] "ndcg_cut_100 0.4996522 0.52895365 p= 0.00260645" thphan@biocaddie-dev:/data/thphan/biocaddie$ Rscript scripts/new/compare.R test bm25jm bm25jm-snowball short biocaddie Please enter run methods for comparison: 0: both are Indri 1: both are Lucene 2: from is Indri, to is Lucene 3: from is Lucene, to is Indri 1 [1] "map 0.35433382 0.37643448 p= 0.03172603" [1] "ndcg 0.61056022 0.62396058 p= 0.09453358" [1] "P_20 0.74677233 0.7367 p= 0.85488885" [1] "ndcg_cut_20 0.5917571 0.60065813 p= 0.27752551" [1] "P_100 0.5065 0.54134987 p= 0.020955" [1] "ndcg_cut_100 0.51864996 0.5395289 p= 0.0026" thphan@biocaddie-dev:/data/thphan/biocaddie$ Rscript scripts/new/compare.R test bm25 bm25-snowball short biocaddie Please enter run methods for comparison: 0: both are Indri 1: both are Lucene 2: from is Indri, to is Lucene 3: from is Lucene, to is Indri 1 [1] "map 0.3543 0.3764 p= 0.0317" [1] "ndcg 0.6105 0.6239 p= 0.0945" [1] "P_20 0.7467 0.73 p= 0.8548" [1] "ndcg_cut_20 0.5917 0.6006 p= 0.2775" [1] "P_100 0.506 0.5413 p= 0.0209" [1] "ndcg_cut_100 0.5186 0.539 p= 0.0441" thphan@biocaddie-dev:/data/thphan/biocaddie$ Rscript scripts/new/compare.R test rocchio rocchio-snowball short biocaddie Please enter run methods for comparison: 0: both are Indri 1: both are Lucene 2: from is Indri, to is Lucene 3: from is Lucene, to is Indri 1 [1] "map 0.4044 0.3959 p= 0.6841" [1] "ndcg 0.6417 0.6052 p= 0.9667" [1] "P_20 0.6967 0.7267 p= 0.2037" [1] "ndcg_cut_20 0.5403 0.598 p= 0.0465" [1] "P_100 0.492 0.5453 p= 0.0035" [1] "ndcg_cut_100 0.4912 0.525 p= 0.0625" |