In this problem, we will implement logistic regression for classifying two classes (zero and one) from...
Fantastic news! We've Found the answer you've been seeking!
Question:
Transcribed Image Text:
In this problem, we will implement logistic regression for classifying two classes (zero and one) from MNIST. You may not use any routines from PyTorch other than the ones that help download the data. Use Numpy to code up the optimization algorithm. (a) (0 points) First, prepare the dataset. Select all the samples belonging to the first two classes from MNIST's training dataset, this will be your training dataset. Similarly, create a validation dataset for the two classes by picking samples corresponding to the first two classes from the validation set of the MNIST dataset. You can subsample input images from 28×28 to 14×14 if you need. (b) (10 points) Logistic regression solves for 1 n (1+ Yi argmin Clog (1 + e¯yi (wxi+wo)) n wЄRd,woЄR i=1 入 + ½ (|||| + + w%2³) (1) where x = R196. Set y₁ = 1 for MNIST images labeled zero and y; = -1 for MNIST images labeled one. Initialize the weights randomly for both the following parts but make sure that they are the same for both gradient descent and gradient descent with Nesterov's acceleration in part (c). You can try a few different values of X and pick the one that gives the best validation error for the following parts. Optimize the objective in (1) using gradient descent (note, not stochastic gradient descent) and plot the training loss as a function of the number of parameter updates on a semi-log scale (log scale on the Y-axis). This plot should be a straight line. As we saw in the class, the slope of this line should be about --¹ for gradient descent. Compute the slope of the line in your plot and mention it clearly. (c) (5 points) Write down the Hessian of the loss function in (1). Without assuming any special A conditions about the dataset {(xi, Yi)} i=1,...,., is this problem strongly convex? What is the best strong convexity parameter for the loss function in (1)? In this problem, we will implement logistic regression for classifying two classes (zero and one) from MNIST. You may not use any routines from PyTorch other than the ones that help download the data. Use Numpy to code up the optimization algorithm. (a) (0 points) First, prepare the dataset. Select all the samples belonging to the first two classes from MNIST's training dataset, this will be your training dataset. Similarly, create a validation dataset for the two classes by picking samples corresponding to the first two classes from the validation set of the MNIST dataset. You can subsample input images from 28×28 to 14×14 if you need. (b) (10 points) Logistic regression solves for 1 n (1+ Yi argmin Clog (1 + e¯yi (wxi+wo)) n wЄRd,woЄR i=1 入 + ½ (|||| + + w%2³) (1) where x = R196. Set y₁ = 1 for MNIST images labeled zero and y; = -1 for MNIST images labeled one. Initialize the weights randomly for both the following parts but make sure that they are the same for both gradient descent and gradient descent with Nesterov's acceleration in part (c). You can try a few different values of X and pick the one that gives the best validation error for the following parts. Optimize the objective in (1) using gradient descent (note, not stochastic gradient descent) and plot the training loss as a function of the number of parameter updates on a semi-log scale (log scale on the Y-axis). This plot should be a straight line. As we saw in the class, the slope of this line should be about --¹ for gradient descent. Compute the slope of the line in your plot and mention it clearly. (c) (5 points) Write down the Hessian of the loss function in (1). Without assuming any special A conditions about the dataset {(xi, Yi)} i=1,...,., is this problem strongly convex? What is the best strong convexity parameter for the loss function in (1)?
Expert Answer:
Answer rating: 100% (QA)
Lets break down the problem into parts and address each one a Prepare the dataset import numpy as np from torchvision import datasets transforms Load ... View the full answer
Related Book For
Smith and Roberson Business Law
ISBN: 978-0538473637
15th Edition
Authors: Richard A. Mann, Barry S. Roberts
Posted Date:
Students also viewed these programming questions
-
On January 1, 2014, Iron Mountain Ski Corporation purchased a new snow-grooming machine for $50,000. The machine is estimated to have a 10-year life with a $2,000 salvage value. What journal entry...
-
Until It Sleeps Corporation expects an EBIT of $14,000 every year forever. Until It Sleeps currently has no debt, and its cost of equity is 16 percent. The firm can borrow at 9 percent. If the...
-
For the following exercises, determine whether the two vectors u and v are equal, where u has an initial point P 1 and a terminal point P 2 and v has an initial point P 3 and a terminal point P 4 . P...
-
Dayton Industries is contemplating some operational changes to reduce its overall costs of quality. The company believes that if it upgrades one component of its product at an additional cost of...
-
Azure Isle Resort has a pricing structure for vacationers in one of its three dwelling categories: the hotel, villas, and beach bungalows. The base price is for staying in the hotel. Beach bungalows...
-
15. If y = 3x-4x+2 then slope at x = 1 is: (1) 1 (2) 2 (3) 4 (4) -2 16. sin240 = ? (1) - 2 (3) 3 1/2 17. Value of sin 2 is (4) (2) 12 (1) 23 (2) (3) 90 180 (4) 0 18. The rate of mass of the gas...
-
A tennis ball is mounted on a spring that is suspended from a ceiling in John's dorm room. For entertainment, stress relief, boredom or just for a pure conversation piece, the tennis ball can be...
-
1. (25 points) Consider two firms, 1 and 2, producing an identical good simul- taneously. This good has market demand given by the demand function y = (12-p)/3, where p is price, and y = y + y2 is...
-
Consider the following functions. f(-3)=10 and g(-3)=1 tep 1. Find (f+g)(-3)
-
In an exhaust fan, a set of gears is used to reduce the speed from a motor to the fan system (see Figure 1.1). The 20-tooth pinion gear on the motor has a diametral pitch of 5 and rotates at 1150 rpm...
-
Zack and Leah live in a community property state. Zack owns a policy on his life, the proceeds of which were paid to his daughter from a prior marriage. Zack and Leah used community funds to purchase...
-
Marc, age 67, is a single taxpayer. In 2023, he received $200,000 of interest income from an investment in city of Houston bonds and $20,000 of social security benefits. He also incurred $14,500 of...
-
On August 15, Year 1, Dryco Corp., a calendar-year S corporation, acquired 100% of the stock of an active C corporation. As a result of the acquisition, as of what date will Dryco's S election...
-
Listed here are the costs associated with the production of 1,000 drum sets manufactured by TrueBeat. Costs 1. Plastic for casing-$18,000 2. Wages of assembly workers-$88, 000 3. Property taxes on...
-
Nike manufactures shoes and sportswear. How has the Internet changed the way this company communicates with its suppliers and retail customers?
-
Central Adjustment Bureau allegedly threatened Consumer with a lawsuit, service at his office, and attachment and sale of his property to collect a debt when it did not intend to take such actions...
-
Distinguish between an assignment of rights and a delegation of duties.
-
Civil Code 1719, subdivision (a) provides in part that any person who draws a check that is dishonored due to insufficient funds shall be liable to the payee for the amount owing upon the check and...
-
Role of controller, role of chief financial officer. George Jimenez is the controller at Balkin Electronics, a manufacturer of devices for the computer industry. The company may promote him to chief...
-
In the following, assume that all growth and discount rates are stated in real terms. 1. Assume the Eurozone inflation-adjusted average growth in capital stock is 3.0 percent per annum into...
-
Which conclusion presented by Ryan about the top-down approach and the bottom-up approach is most likely correct? A. Conclusion 1. B. Conclusion 2. C. Conclusion 3. Use the following information to...
Study smarter with the SolutionInn App