Question: 7. Below is a table showing how two human judges rated the relevance of a set of 12 documents to a particular information need (0=

7. Below is a table showing how two human judges rated the relevance of a set of 12 documents to a particular information need (0= nonrelevant, 1= relevant ). Let us assume that you've written an IR system that for this query returns the ranking list: {doc2,doc5, doc6, doc7, doc8, doc 12}.[25 points ] 1) Calculate precision, recall, and F1 of your system if a document is considered relevant only if the two judges agree they are relevant. 2) Calculate precision, recall, and F1 of your system if a document is considered relevant if either judge thinks it is relevant. 3) Now if we use the average rating of two judges as the relevance judgments, Compute the DCG for the ranking list
Step by Step Solution
There are 3 Steps involved in it
Get step-by-step solutions from verified subject matter experts
