Question: 4. [10] Suppose we have two predictive models, M1 and M2 and run 10 rounds of 10-fold cross-validation test using the two models. In each

4. [10] Suppose we have two predictive models, M1 and M2 and run 10 rounds of 10-fold cross-validation test using the two models. In each round, a training set and a testing set are determined based on the 10-fold cross-validation method, and the same training set (re- spectively testing set is used to build (respectively to test the two models. The error rates obtained from the test are given in the following table. Perform a t-test at a significance level of 0.001 to determine if one model is significantly better than the other. A t-distribution table is provided as an attachment to this assignment. 1 3 4 5 6 7 9 10 Models round 2 MMI 30.5 32.2 20.7 20.6 31.0 41.0 27.7 26.0 21.5 26.0 M. 22.4 14.5 22.4 19.6 20.7 20.4 22.1 19.4 16. 35.0

Step by Step Solution

There are 3 Steps involved in it

1 Expert Approved Answer

Step: 1 Unlock blur-text-image

Question Has Been Solved by an Expert!

Get step-by-step solutions from verified subject matter experts

Step: 2 Unlock

Step: 3 Unlock

Students Have Also Explored These Related Databases Questions!

Suppose we have two predictive models, M_1 and M_2 and run 10 rounds of 10-fold cross-validation test using the two models. In each round, a training set and a testing set are determined based on the...

Suppose you built two classifier models M1 and M2 from the same training dataset and tested them on the same test dataset using 10-fold cross-validation. The error rates obtained over 10 iterations...

Problem 1 (20 points) Consider the following confusion matrix. predicted class C1 (positive) C2 (negative) actual class C1 (positive) 28 72 C2 (negative) 328 781 Compute sensitivity, specificity,...

Problem 2 (10 points) Suppose you built two classifier models MI and M2 from the same training dataset and tested them on the same test dataset using 10-fold cross- validation. The error rates...

You are asked to evaluate the performance of two classification models, M1 and M2. The following table shows the probabilities obtained by applying the models to the test set: Plot the ROC curve for...

A real estate company has built two predictive models for estimating the selling price of a house. Using a small test data set of 10 observations, it tries to assess how the prediction models would...

Type or paste question here A real estate company has built two predil tive models for estimating the selling price of a house. Using a small test data set of 10 observations, it tries to assess how...

CSCI 5525 MACHINE LEARNING, Fall 2017, Prof Schrater Homework 1 September 27, 2017 1. For data (x, y) with a joint distribution p(x, y) = p(y|x)p(x), the expected loss of a function f (x) to model y...

BE562: Problem Set 3 Fall 2016 Due 10/21/2016 8PM 1. Hidden Markov Models and Protein Structure (20 pts) One biological application of hidden Markov models is to determine the secondary structure...

Nolton Products uses standard costing. It allocates manufacturing overhead (both variable and fixed) to products on the basis of standard direct manufacturing labour-hours (DLH). Nolton develops its...

Accounting X Onn. In the UK the area of tangible fixed assets is covered by FRS 15, Tangible Fixed Assets, with IAS 16, Property, plant Additional information: 1. Account receivables include...

CA9.7 (LO 1) Ethics (Cost of Land vs. BuildingEthics) Tones Company purchased a warehouse in a downtown district where land values are rapidly increasing. Gerald Carter, controller, and Wilma Ankara,...

Bohemia Industries By Smith and Jones (updated in 2014 by Martin Quinn) At the beginning of September, Paul Owen, the new manager of a division of Bohemia Industries, received the August income...

4. Do you consider yourself a. mostly focused on the future? b. mostly focused on the present? _______

9. In terms of style, are you a. solid and mainstream? b. always on the cutting edge? _______

6. When change occurs, are you a. typically out in front leading the way? b. typically holding back and waiting to see what will happen? _______