Comparing four Technology-Assisted Review (TAR) protocols on the RCV1-v2 dataset

Addendum to Guest Blog: Talking Turkey

Gordon V. Cormack and Maura R. Grossman

+ Continuous Active Learning [acttoplite]
× Simple Active Learning [untoplite]
✳ Simple Passive Learning (keyword seed) [pastoplite]
☐ Simple Passive Learning (random training) [pasranlite]

Results as one pdf

Click here (103 pages).

Results as separate pdfs

C11.pdfC12.pdfC13.pdfC14.pdfC15.pdfC16.pdfC17.pdfC18.pdfC21.pdf
C22.pdfC23.pdfC24.pdfC31.pdfC32.pdfC33.pdfC34.pdfC41.pdfC42.pdf
C151.pdfC152.pdfC171.pdfC172.pdfC173.pdfC174.pdfC181.pdfC182.pdfC183.pdf
C311.pdfC312.pdfC313.pdfC331.pdfC411.pdfC1511.pdfCCAT.pdfE11.pdfE12.pdf
E13.pdfE14.pdfE21.pdfE31.pdfE41.pdfE51.pdfE61.pdfE71.pdfE121.pdf
E131.pdfE132.pdfE141.pdfE142.pdfE143.pdfE211.pdfE212.pdfE311.pdfE312.pdf
E313.pdfE411.pdfE511.pdfE512.pdfE513.pdfECAT.pdfG15.pdfG151.pdfG152.pdf
G153.pdfG154.pdfG155.pdfG156.pdfG157.pdfG158.pdfG159.pdfGCAT.pdfGCRIM.pdf
GDEF.pdfGDIP.pdfGDIS.pdfGENT.pdfGENV.pdfGFAS.pdfGHEA.pdfGJOB.pdfGMIL.pdf
GOBIT.pdfGODD.pdfGPOL.pdfGPRO.pdfGREL.pdfGSCI.pdfGSPO.pdfGTOUR.pdfGVIO.pdf
GVOTE.pdfGWEA.pdfGWELF.pdfM11.pdfM12.pdfM13.pdfM14.pdfM131.pdfM132.pdf
M141.pdfM142.pdfM143.pdfMCAT.pdf

Note: For ease of comparison with our SIGIR study, the graphs above portray the use of training-set sizes (for simple active and simple passive learning) that minimize 75% recall effort. For easy comparison with Webber's results, graphs portraying training-set sizes that minimize 80% recall effort (denoted cfr80 by Webber) are available here.