Question: 7. Below is a table showing how two human judges rated the relevance of a set of 12 documents to a particular information need (0=

 7. Below is a table showing how two human judges rated

7. Below is a table showing how two human judges rated the relevance of a set of 12 documents to a particular information need (0= nonrelevant, 1= relevant ). Let us assume that you've written an IR system that for this query returns the ranking list: {doc2,doc5, doc6, doc7, doc8, doc 12}.[25 points ] 1) Calculate precision, recall, and F1 of your system if a document is considered relevant only if the two judges agree they are relevant. 2) Calculate precision, recall, and F1 of your system if a document is considered relevant if either judge thinks it is relevant. 3) Now if we use the average rating of two judges as the relevance judgments, Compute the DCG for the ranking list

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer
Step: 1 Unlock blur-text-image
Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock
Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!