Suppose we have a collection of 20 documents, d, d2,..., d20, which have been judged for...
Fantastic news! We've Found the answer you've been seeking!
Question:
Transcribed Image Text:
Suppose we have a collection of 20 documents, d, d2,..., d20, which have been judged for relevance to a query. A 3-point relevance scale was used, so relevant documents have been divided into Perfect, Good and just Relevant results. Weights for these levels are shown below for NDCG(Normalized Discounted Cumulative Gain): Perfect 3 Good 2 Relevant 1 Non-relevant 0 Consider the result lists retrieved for the three different information needs shown as below respectively: Result Q Result_Q2 Result Q3 = <3,0,2,2,0> = = <3, 2, 2, 2, 0, 2, 0, 1 > < 0,2,0,3> 1. Assume there are totally 10 relevant documents in the collection. What are the precision and recall for result list Result-Q2? Draw the interpolated precision-recall curve. 2. What is the precision @4 of each result list? (4) (3) 3. What is the average precision of each result list? and what is MAP for this IR system if there are only these three information needs in the test collection? (4) 4. What is the perfect ranking for Result Q = < 3,2,2,2,0, 2, 0, 1 >? And calculate the Ideal Discounted Cumulative Gain (DCG) for this set of documents. (5) 5. To measure/evaluate information retrieval (IR) effectiveness, what are the three elements required for a test collection, so the performance of the IR system could be compared? (3) 6. What is Kappa Measure? (2) 7. For a particular information need if Judge 1 rated the relevance of a set of 5 documents as Result = < < R, N, R, R, N > and Judge 2 rated as Result2 = < R, R, R, N, N >. Calculate the Kappa measure if the expected chance agreement ratio P(E) is 0.5. (4) Suppose we have a collection of 20 documents, d, d2,..., d20, which have been judged for relevance to a query. A 3-point relevance scale was used, so relevant documents have been divided into Perfect, Good and just Relevant results. Weights for these levels are shown below for NDCG(Normalized Discounted Cumulative Gain): Perfect 3 Good 2 Relevant 1 Non-relevant 0 Consider the result lists retrieved for the three different information needs shown as below respectively: Result Q Result_Q2 Result Q3 = <3,0,2,2,0> = = <3, 2, 2, 2, 0, 2, 0, 1 > < 0,2,0,3> 1. Assume there are totally 10 relevant documents in the collection. What are the precision and recall for result list Result-Q2? Draw the interpolated precision-recall curve. 2. What is the precision @4 of each result list? (4) (3) 3. What is the average precision of each result list? and what is MAP for this IR system if there are only these three information needs in the test collection? (4) 4. What is the perfect ranking for Result Q = < 3,2,2,2,0, 2, 0, 1 >? And calculate the Ideal Discounted Cumulative Gain (DCG) for this set of documents. (5) 5. To measure/evaluate information retrieval (IR) effectiveness, what are the three elements required for a test collection, so the performance of the IR system could be compared? (3) 6. What is Kappa Measure? (2) 7. For a particular information need if Judge 1 rated the relevance of a set of 5 documents as Result = < < R, N, R, R, N > and Judge 2 rated as Result2 = < R, R, R, N, N >. Calculate the Kappa measure if the expected chance agreement ratio P(E) is 0.5. (4)
Expert Answer:
Answer rating: 100% (QA)
Here are the answers to the questions 1 Result Q has 2 Perfect results and 2 Good results out of a t... View the full answer
Related Book For
Understanding Basic Statistics
ISBN: 9781111827021
6th Edition
Authors: Charles Henry Brase, Corrinne Pellillo Brase
Posted Date:
Students also viewed these finance questions
-
Python and most Python libraries are free to download or use, though many users use Python through a paid service. Paid services help IT organizations manage the risks associated with the use of...
-
answer all questions as instructed below. attend all questions. 4 Computer Vision (a) Explain why such a tiny number of 2D Gabor wavelets as shown in this sequence are so efficient at representing...
-
In Exercises 1126, determine whether each equation defines y as a function of x. x + y = 25
-
Rank each set of compounds in order of increasing basicity. (a) NaOH, NH3, CH3NH2, PhNH2 (b) Aniline, p-methylaniline, p-nitroaniline (c) Aniline, pyrrole, pyridine, piperidine (d) Pyrrole,...
-
What is the hit ratio? Starting from power on, the following byte-addressed cache references are recorded. Address O 4 16 132 232 160 1024 30 140 3100 180 2180
-
Three parallel-plate capacitors each store the same amount of charge. Capacitor 1 has a plate area \(A\) and a plate separation \(d\). Capacitor 2 has a plate area \(2 A\) and a plate separation...
-
Given the balance sheets and selected data from the income statement of SMG Industries that follow, answer parts (a)(c). a. Calculate the firms operating cash flow (OCF) for the year ended December...
-
what happens when ou use an aggregation function in a calculated column?
-
The main trade - offs within the warehouse does NOT include . . . . . . Group of answer choices High customer service level versus lower inventory Higher cost versus lower customer service level...
-
Boardwalk Electronics manufactures 300,000 circuit boards per month. A random sample of 3,000 boards is inspected every week for five characteristics. During a recent week, three defects were found...
-
In September 2021 Kevin sells a drawing for 2,000. He bought the drawing in February 2015 for 50,000 when it was thought (incorrectly) to be by a famous artist. Compute the allowable loss.
-
Terry bought a house for 65,000 on 1 October 2001 and occupied the house as his principal private residence. He lived in the house until 1 October 2007 when he went to stay with relatives in...
-
In June 2019 Katie acquired a 5-year option to buy a piece of land. The option cost her 10,000. In June 2021 she sold the option for 8,000. Compute the chargeable gain.
-
Karen bought a house in 1994 for 42,000. In November 1995 she spent 18,000 on dividing the house into two self-contained flats. In September 2019 she sold one of the flats for 95,000, at which time...
-
For the estimation, the CEO starts with P(S), the prior probability of the new product?s success and the track records of the consultant?s precious forecasts: Find the following. P(F|S) = % favorable...
-
Assume today is the 21st of February. Using the information below, FT Extract, answer the following questions (parts i and ii). You work for a US company that is due to receive 250 million in June...
-
Brown India Limited manufactures office tables. Normal capacity of the factory is 60,000 tables per annum. Following are the cost and inventory details for the year 200506. Required: Carry out the...
-
Shoppers Departmental Stores Ltd. furnishes the following details of purchase, sale etc. of its garments section for the year 200506. Determine the value of inventory of the garments section as on...
-
KABU Enterprises Ltd. buys a building for the purpose of investment. It issues 1,00,000 equity shares of its company, which are quoted on the day of the deal at Rs. 155 per share at the NSE. The...
Study smarter with the SolutionInn App