...
- Clarity (Cronen-Townshend, 2002)
- Drift ( Shtok, Kurland, Carmel, 2009)
- Deviation (Perez-Iglesia and Araujo, 2010)
- Absolute/relative divergence (Lv & Zhai)
Preliminary results
The following predictors were found to be significant from (overfitted) analysis of test data. These are predictors for fbOrigWeight
- deviation (R^2=0.49, p<0.001)
- varSCQ (R^2=0.37, p <0.01)
- drift (R^2=0.37, p<0.01)
The following linear models were found to have the best fit for the RM3 fbOrigWeight:
- fbOrigWeight^(1/2) ~ drift + varSCQ (R^2=0.6269, p<0.0001)
- fbOrigWeight^(1/2) ~ drift + deviation (R^2=0.5801, p<0.001)
- fbOrigWeight^(1/2) ~ varSCQ + deviation(R^2=0.6496, p<0.001)
Using the "varSCQ + deviation" model to estimate fbOrigWeight for the held-out query using leave-one-query-out crossvalidation, we see NDCG=0.59 compared to NDCG=0.56. This is a significant difference with 1-tailed t-test and p < 0.01
References
Carmel, D., & Yom-Tov, E. (2010). Estimating the Query Difficulty for Information Retrieval. Synthesis Lectures on Information Concepts, Retrieval, and Services, 2(1), 1–89. http://doi.org/10.2200/S00235ED1V01Y201004ICR015
...