CS 848-002 (Fall 2016)

Student Reading and Research Presentations

1.     G. V. Cormack et al., The Efficient Construction of Large Test Collections, SIGIR 1998.

2.     I. Soboroff and S. Robertson, Building a Filtering Test Collection for TREC 2002, SIGIR 2003.

3.     D. D. Lewis et al., A New Benchmark Collection for Text Categorization Research, 5 J. Mach. Learn. Res. 361 (2004).


4.     M. Sanderson and H. Joho, Forming Test Collections with No System Pooling, SIGIR 2004.

5.     E. Voorhees, Variance in Relevance Judgments and the Measurement of Retrieval Effectiveness, 36 Info. Proc. & Mgmt. 697 (2000).

6.     P. Bailey et al., Relevance Assessment:  Are Judges Interchangeable and Does It Matter?, SIGIR 2008.

7.     A. Roegiest and G. V. Cormack, Impact of Review-Set Selection on Human Assessment for Text Classification, SIGIR 2016.

8.     A. Turpin et al., The Benefits of Magnitude Estimation Relevance Assessments for Information Retrieval Evaluation, SIGIR 2015.

9.     H. Roitblat et al., Document Categorization in Legal Electronic Discovery:  Computer Classification vs. Manual Review, 61 JASIST 70 (2010).


10.  M. R. Grossman and G. V. Cormack, Technology-Assisted Review in E-Discovery Can Be More Effective and More Efficient Than Exhaustive Manual Review, 17 Rich. J.L. & Tech 11 (2011).

11.  K. Schieneman and T. Gricks, The Implications of Rule 26(g) on the Use of Technology-Assisted Review, 7 Fed. Courts L. Rev. 239 (2013).

12.  M. R. Grossman and G.V. Cormack, Comments on ‘The Implications of Rule 26(g) on the Use of Technology-Assisted Review’, 7 Fed. Courts L. Rev. 385 (2014).

13.  W. Webber et al., Sequential Testing in Classifier Evaluation Yields Biased Estimates of Effectiveness, SIGIR 2013.


14.  M. Bagdouri et al., Towards Minimizing the Annotation Cost of Certified Text Classification, CIKM 2013.


15.  G. V. Cormack and M. R. Grossman, Evaluation of Machine-Learning Protocols for Technology-Assisted Review, SIGIR 2014.

16.  G. V. Cormack and M. R. Grossman, Engineering Quality and Reliability in Technology-Assisted Review, SIGIR 2016.

17.  G. V. Cormack and T. R. Lynam, On-Line Supervised Spam Filter Evaluation, TOIS 2007.


18.  D. Sculley and G.M. Wachman, Relaxed Online SVMs for Spam Filtering and Errata, SIGIR 2007.

19.  G. V. Cormack and A. Kolcz, Spam Filtering Evaluation with Imprecise Ground Truth, SIGIR 2009.

20.  B. Wallace et al., Active Learning for Biomedical Citation Screenings, KDD 2010.