Consider two classification concepts given in Figure 1, where x = [-6,6] x -4,4, y={-1, +1}...
Fantastic news! We've Found the answer you've been seeking!
Question:
Transcribed Image Text:
Consider two classification concepts given in Figure 1, where x = [-6,6] x -4,4, y={-1, +1} and p(yr) = {0,1} is defined in the drawing. A) 4 (-4,3) + (2, 1) - 0 (-2, -1) + + B) 4 (-4, 3) 0 (-1,-2) (2, 0) + 0 6 0 Figure 1: Two concepts where examples that fall within any of the three 3 x 3 (panel A) or 1 x 1 (panel B) squares are labeled positive and the remaining examples (outside each of the squares but within X) are labeled negative. The position of the point x=(1, 2) in the upper left-hand corner for each square is shown in the picture. Consider horizontal axis to be 21 and vertical axis as 22. Your experiments in this question will rely on generating a data set of size n = {250, 1000, 10000} drawn from a uniform distribution in X and labeled according to the rules from Figure 1; e.g., P(Y = 1x) = 0.99 if x that was randomly drawn is inside any of the three squares in either of the two panels, and P(Y = 1|x) = 0.03 otherwise (notice that this introduces a certain amount of asymmetric noise in the data). The goal of the following two problems will be to train and evaluate classifiers created from the data generated in this way. You can use any library you want in this assignment and do programming in Python, MATLAB, or R. Your code should be easy to run for each question and sub-question below so that we can replicate your results to the maximum extent possible. Consider single-output feed-forward neural networks with one or two hidden layers such that the number of hidden neurons in each layer is h = {1, 4, 12} and h = {0, 3}, respectively, with h = 0 meaning that there is no second hidden layer. Consider one of the standard objective functions as your optimization criterion and use early stopping and regularization as needed. Consider a hyperbolic tangent activation function in each neuron and the output but you are free to experiment with others if you'd like to. For each of the architectures, defined by a parameter combination (h1, h2), evaluate the performance of each model using classification accuracy, balanced accuracy, and area under the ROC curve as your performance criteria. To evaluate the performance of your models use cross-validation. However, to evaluate the performance of performance evaluation, generate another very large data set on a fine grid in X. Then use the predictions from your trained model on all these points to determine the "true" performance. You can threshold your predictions in the middle of your prediction range (i.e., at 0.5 if you are predicting between 0 and 1) to determine binary predictions of your models and to then compare those with true class labels you generated on the fine grid. Provide meaningful comments about all aspects of this exercise (performance results for different network architectures, accuracy in and of cross-validation, true accuracy when you disregard class-label noise, run time, etc.). The comments should not just re-state the results but rather capture trends and give reasoning as to why certain behavior was observed. Consider two classification concepts given in Figure 1, where x = [-6,6] x -4,4, y={-1, +1} and p(yr) = {0,1} is defined in the drawing. A) 4 (-4,3) + (2, 1) - 0 (-2, -1) + + B) 4 (-4, 3) 0 (-1,-2) (2, 0) + 0 6 0 Figure 1: Two concepts where examples that fall within any of the three 3 x 3 (panel A) or 1 x 1 (panel B) squares are labeled positive and the remaining examples (outside each of the squares but within X) are labeled negative. The position of the point x=(1, 2) in the upper left-hand corner for each square is shown in the picture. Consider horizontal axis to be 21 and vertical axis as 22. Your experiments in this question will rely on generating a data set of size n = {250, 1000, 10000} drawn from a uniform distribution in X and labeled according to the rules from Figure 1; e.g., P(Y = 1x) = 0.99 if x that was randomly drawn is inside any of the three squares in either of the two panels, and P(Y = 1|x) = 0.03 otherwise (notice that this introduces a certain amount of asymmetric noise in the data). The goal of the following two problems will be to train and evaluate classifiers created from the data generated in this way. You can use any library you want in this assignment and do programming in Python, MATLAB, or R. Your code should be easy to run for each question and sub-question below so that we can replicate your results to the maximum extent possible. Consider single-output feed-forward neural networks with one or two hidden layers such that the number of hidden neurons in each layer is h = {1, 4, 12} and h = {0, 3}, respectively, with h = 0 meaning that there is no second hidden layer. Consider one of the standard objective functions as your optimization criterion and use early stopping and regularization as needed. Consider a hyperbolic tangent activation function in each neuron and the output but you are free to experiment with others if you'd like to. For each of the architectures, defined by a parameter combination (h1, h2), evaluate the performance of each model using classification accuracy, balanced accuracy, and area under the ROC curve as your performance criteria. To evaluate the performance of your models use cross-validation. However, to evaluate the performance of performance evaluation, generate another very large data set on a fine grid in X. Then use the predictions from your trained model on all these points to determine the "true" performance. You can threshold your predictions in the middle of your prediction range (i.e., at 0.5 if you are predicting between 0 and 1) to determine binary predictions of your models and to then compare those with true class labels you generated on the fine grid. Provide meaningful comments about all aspects of this exercise (performance results for different network architectures, accuracy in and of cross-validation, true accuracy when you disregard class-label noise, run time, etc.). The comments should not just re-state the results but rather capture trends and give reasoning as to why certain behavior was observed.
Expert Answer:
Related Book For
Introduction to Operations Research
ISBN: 978-1259162985
10th edition
Authors: Frederick S. Hillier, Gerald J. Lieberman
Posted Date:
Students also viewed these programming questions
-
Q1. You have identified a market opportunity for home media players that would cater for older members of the population. Many older people have difficulty in understanding the operating principles...
-
CANMNMM January of this year. (a) Each item will be held in a record. Describe all the data structures that must refer to these records to implement the required functionality. Describe all the...
-
For brands is it more important to create big idea that is required to drive a customer connection or can it be stifled by corporate requirements to measure performance with automation and generate...
-
From May 15 through June 30, 2012, American Airlines flew 152 flights from Chicago, Illinois, to Austin, Texas. Of these, 22 arrived late. Treat this as a random sample and conduct a test with the...
-
Indicate whether each statement is required in IFRS or in ASPE, or in both. IFRS ASPE Both 1. Statement of Financial Position 2. Statement of Comprehensive Income 3. Income Statement 4. Statement of...
-
Describe corporate ethics, the SarbanesOxley Act of 2002, and corporate compliance.
-
Moe Electronics produces APP switches for NASA. The current assembly capacity is 100 switches per hour. It takes 40 minutes to receive the necessary component from the previous workstation. Completed...
-
Primare Corporation provided the following data for last month's manufacturing operations. Purchases of raw materials Indirect materials used in production Direct labor Manufacturing overhead applied...
-
Use Eq.1, the data in Table 1 below for Mygas field, GB =G(B, -Bei)+We (1) Table 1. GpBg/(Bg- Bgi) Gp, MMSCF cuft/SCF Bg, Bg-Bgi, GpBg, MMcuft p, psia cuft/SCF MMMcuft 3200 0.0053 0.0000 0.0000 2925...
-
Does the phenomenon of interference help to explain the unreliability of autobiographical memory? Why?
-
The three fluid-filled tubes in the inner ear that are responsible for our sense of balance are known as the ____________________.
-
Wundt described psychology as the study of conscious experience, a perspective he called____________.
-
One of the main criticisms of biological therapies is that they treat the symptoms of mental disorder without uncovering and treating the underlying problems from which people are suffering. Do you...
-
If researchers learned how to control emotional responses so that targeted emotions could be caused or prevented, what ethical concerns m.ight arise? Under what circumstances, if any, should such...
-
Which of the following would cause a decrease in cash? a. An increase in the Average Collection Period from 15 days to 30 days b. Selling off fixed assets for more than the book value c. An increase...
-
The following cost information was provided to you for analysis: September 12,000 Units Produced Costs: TIC TAC TOE TING August 10,000 P80,000 70.000 60.000 50,000 How much is the fixed cost per...
-
(a) Among 880 smart phones sold by a retailer, 72 required repairs under the warranty. Estimate the probability that a new phone, which has just been sold, will require repairs under the warranty....
-
A car rental agency has 19 compact cars and 12 intermediate-size cars. If four of the cars are randomly selected for a safety check, what is the probability of getting two of each kind?
-
With reference to Exercise 3.34, suppose that the questionnaire filled in by one of the 150 persons is to be double-checked. If it is chosen in such a way that each questionnaire has a probability of...
Study smarter with the SolutionInn App