Question: Problem 5 . EvaluationBelow is a table showing how two human judges rated the relevance of a set of 1 2 documents to a particular
Problem EvaluationBelow is a table showing how two human judges rated the relevance of a set of documents to a particular information need nonrelevant, relevant Let us assume that youve written an IR system that for this query returns the set of documents docIDJudgeJudge Calculate the kappa measure between the two judges; Calculate precision, recall, and F of your system if a document is considered relevantonly if the two judges agree; Calculate precision, recall, and F of your system if a document is considered relevantif either judge thinks it is relevant.Hint: create a confusion matrix
Step by Step Solution
There are 3 Steps involved in it
1 Expert Approved Answer
Step: 1 Unlock
Question Has Been Solved by an Expert!
Get step-by-step solutions from verified subject matter experts
Step: 2 Unlock
Step: 3 Unlock
