Machine Learning Fundamentals A Concise Introduction 1st Edition Hui Jiang - Solutions

Unlock your understanding of "Machine Learning Fundamentals: A Concise Introduction, 1st Edition by Hui Jiang" with our comprehensive solutions and resources. Access the answers key and solutions manual online, featuring step-by-step answers and solved problems to enhance learning. Discover chapter solutions and a detailed test bank, ensuring thorough preparation for every topic. With our instructor manual and textbook solutions available in PDF format, you can easily navigate through complex questions and answers. Embrace the convenience of free downloads, and elevate your study sessions with expertly crafted resources designed for optimal comprehension.

3.20 VC-dimension of sine functions. Consider the hypothesis family of sine functions(Example 3.16): fx ! sin(!x) : ! 2 Rg .(a) Show that for any x 2 R the points x; 2x; 3x and 4x cannot be shattered by this family of sine functions.(b) Show that the VC-dimension of the family of sine functions is
3.19 VC-dimension of a vector space of real functions. Let F be a nite-dimensional vector space of real functions on Rn, dim(F) = r < 1. Let H be the set of hypotheses:H = ffx: f(x) 0g: f 2 Fg:Show thatd, the VC-dimension of H, is nite and that d r. (Hint: select an arbitrary set of m = r + 1
3.18 VC-dimension of ellipsoids. What is the VC-dimension of the set of all ellipsoids in Rn?
3.17 VC-dimension of closed balls in Rn. Show that the VC-dimension of the set of all closed balls in Rn, i.e., sets of the form fx 2 Rn : kx ???? x0k2 rg for some x0 2 Rn and r 0, is less than or equal to n + 2.
3.16 VC-dimension of axis-aligned squares and triangles.(a) What is the VC-dimension of axis-aligned squares in the plane?(b) Consider right triangles in the plane with the sides adjacent to the right angle both parallel to the axes and with the right angle in the lower left corner. What is the
3.15 VC-dimension of subsets. What is the VC-dimension of the set of subsets I of the real line parameterized by a single parameter : I = [; +1][[+2;+1)?
3.14 VC-dimension of nite hypothesis sets. Show that the VC-dimension of a nite hypothesis set H is at most log2 jHj.
3.13 VC-dimension of union of k intervals. What is the VC-dimension of subsets of the real line formed by the union of k intervals?
3.12 Rademacher complexity. Professor Jesetoo claims to have found a better bound on the Rademacher complexity of any hypothesis set H of functions taking values in f????1; +1g, in terms of its VC-dimension VCdim(H). His bound is of the form Rm(H) O????VCdim(H)m. Can you show that Professor
3.11 Rademacher complexity of regularized neural networks. Let the input space be X = Rn1 . In this problem, we consider the family of regularized neural networks dened by the following set of functions mapping X to R:H =8 0. Use the previous questions to derive an upper bound on the Rademacher
3.10 Rademacher complexity of prediction vector. Let S = (x1; : : : ; xm) be a sample of size m and x h: X ! R.(a) Denote by u the vector of predictions of h for S: u = h(x1) ...h(xm). Give an upper bound on the empirical Rademacher complexity bR S(H) of H =fh;????hg in terms of kuk2 (Hint:
3.9 Rademacher complexity of intersection of concepts. Let H1 and H2 be two families of functions mapping X to f0; 1g and let H = fh1h2 : h1 2 H1; h2 2 H2g. Show that the empirical Rademacher complexity of H for any sample S of size m can be bounded as follows:bR S(H) bR S(H1) + bR S(H2):Chapter
3.8 Rademacher identities. Fix m 1. Prove the following identities for any 2 R and any two hypothesis sets H and H0 of functions mapping from X to R:(a) Rm(H) = jjRm(H):(b) Rm(H + H0) = Rm(H) + Rm(H0):(c) Rm(fmax(h; h0) : h 2 H; h0 2 H0g) Rm(H) + Rm(H0);where max(h; h0) denotes the function x
3.7 Two function hypothesis class. Let H be a hypothesis set reduced to two functions:H = fh????1; h+1g and let S = (x1; : : : ; xm) X be a sample of size m.(a) Assume that h????1 is the constant function taking value ????1 and h+1 the constant function taking the value +1. What is the
3.6 Singleton hypothesis class. Consider the trivial hypothesis set H = fh0g.(a) Show that Rm(H) = 0 for any m > 0.(b) Use a similar construction to show that Massart's lemma (theorem 3.7) is tight.
3.5 Finer Rademacher upper bound. Show that a ner upper bound on the Rademacher complexity of the family G can be given in terms of ES[(G; S)], where (G; S) is the number of ways to label the points in sample S.
3.4 Lower bound on growth function. Prove that Sauer's lemma (theorem 3.17) is tight, i.e., for any set X of m > d elements, show that there exists a hypothesis class H of VC-dimension d such that H(m) =Pd i=0????m i.
3.3 Growth function of linear combinations. A linearly separable labeling of a set X of vectors in Rd is a classication of X into two sets X+ and X???? with X+ =fx 2 X: w x > 0g and X???? = fx 2 X: w x < 0g for some w 2 Rd.Let X = fx1; : : : ; xmg be a subset of Rd.(a) Let fX+;X????g be a
3.2 Growth function and Rademacher complexity of thresholds in R. Let H be the family of threshold functions over the real line: H = fx 7! 1x : 2 Rg[fx 7!1x : 2 Rg. Give an upper bound on the growth function m(H). Use that to derive an upper bound on Rm(H).
3.1 Growth function of intervals in R. Let H be the set of intervals in R. The VC-dimension of H is 2. Compute its shattering coecient H(m), m 0.Compare your result with the general bound for growth functions.
2.14 In this exercise, we generalize the notion of noise to the case of an arbitrary loss function L: Y Y ! R+.(a) Justify the following denition of the noise at point x 2 X:noise(x) = min y02Y Ey[L(y; y0)jx]:What is the value of noise(x) in a deterministic scenario? Does the denition match the
2.13 Learning with an unknown parameter. In example 2.9, we showed that the concept class of k-CNF is PAC-learnable. Note, however, that the learning algorithm is given k as input. Is PAC-learning possible even when k is not provided? More generally, consider a family of concept classes fCsgs where
2.12 Bayesian bound. Let H be a countable hypothesis set of functions mapping X to f0; 1g and let p be a probability measure over H. This probability measure represents the prior probability over the hypothesis class, i.e. the probability that a particular hypothesis is selected by the learning
2.11 Senate laws. For important questions, President Mouth relies on expert advice.He selects an appropriate advisor from a collection of H = 2;800 experts.(a) Assume that laws are proposed in a random fashion independently and identically according to some distribution D determined by an unknown
2.10 Consistent hypotheses. In this chapter, we showed that for a nite hypothesis set H, a consistent learning algorithm A is a PAC-learning algorithm. Here, we consider a converse question. Let Z be a nite set of m labeled points. Suppose that you are given a PAC-learning algorithm A. Show that
2.9 Learning union of intervals. Give a PAC-learning algorithm for the concept class C2 formed by unions of two closed intervals, that is [a; b][[c; d], with a; b; c; d 2 R.Extend your result to derive a PAC-learning algorithm for the concept class Cp formed by unions of p 1 closed intervals,
2.8 Learning intervals. Give a PAC-learning algorithm for the concept class C formed by closed intervals [a; b] with a; b 2 R.
2.7 Learning in the presence of noise | general case. In this question, we will seek a result that is more general than in the previous question. We consider a nite hypothesis set H, assume that the target concept is in H, and adopt the following noise model: the label of a training point received
2.6 Learning in the presence of noise | rectangles. In example 2.4, we showed that the concept class of axis-aligned rectangles is PAC-learnable. Consider now the case where the training points received by the learner are subject to the following noise: points negatively labeled are unaected by
2.5 Triangles. Let X = R2 with orthonormal basis (e1; e2), and consider the set of concepts dened by the area inside a right triangle ABC with two sides parallel to the axes, with A????????!B=kA????????!Bk = e1 and A????!C=kA????!Ck = e2, and kA????????!Bk=kA????!Ck =for some positive real 2 R+.
2.4 Non-concentric circles. Let X = R2 and consider the set of concepts of the form c = fx 2 R2 : jjx ???? x0jj rg for some point x0 2 R2 and real number r. Gertrude, an aspiring machine learning researcher, attempts to show that this class of concepts may be (; )-PAC-learned with sample
2.3 Concentric circles. Let X = R2 and consider the set of concepts of the form c = f(x; y) : x2 + y2 r2g for some real number r. Show that this class can be(; )-PAC-learned from training data of size m (1=) log(1=).
2.2 PAC learning of hyper-rectangles. An axis-aligned hyper-rectangle in Rn is a set of the form [a1; b1] : : : [an; bn]. Show that axis-aligned hyper-rectangles are PAC-learnable by extending the proof given in Example 2.4 for the case n = 2.
2.1 Two-oracle variant of the PAC model. Assume that positive and negative examples are now drawn from two separate distributions D+ and D????. For an accuracy (1 ???? ), the learning algorithm must nd a hypothesis h such that:P xD+[h(x) = 0] and P xD????[h(x) = 1] : (2.25)Thus, the
3. Generate cluster diagrams for the two values of k that you selected in Exercise 2.Which one of these do you believe is the best result? Why?
2. Use the techniques described in this chapter to select two possible optimal values for k for the clustering problem you coded in Exercise 1. Justify your answer.
1. Using the college dataset from this chapter, perform clustering that looks at the average faculty salary and annual tuition rates for schools located in Indiana. Choose k=3 and produce a visualization of your clusters.
3. Now, attempt to improve the predictive accuracy of the income prediction model by using the extreme gradient boosting approach. This time, instead of explicitly setting the tuning parameters, have caret evaluate two values per hyperparameter in order to select the combination that provides the
2. Attempt to improve the accuracy of the income prediction random forest model by doing some additional parameter tuning. What improvement in predictive accuracy were you able to achieve?
1. Research the tuning parameters available for other learning methods with the caret package. What parameters may be tuned for each one of the following techniques?a. k-nearest-neighbor (with the knn package)b. Generalized linear models (with the glm package)c. Naïve Bayes (with the naive_bayes
3. You recently built three machine learning models to perform a classification task and found that the models have the ROC curves shown in Figure 9.18.a. Which model performs the best against your data?b. How would you choose between models A and C?improving the performance of a machine learning
2. Consider the following confusion matrix:Compute the following values:a. Predictive accuracy (pa)b. Probability of expected agreement (pe )c. Kappa ( )d. Precisione. Recallf. F-score g. Sensitivity h. Specificity i. False positive rate j. True positive rate k. False negative rate l. True negative
1. You are building a machine learning model using an original dataset of 10,000 observations.The dataset includes 10 independent variables and 1 dependent variable.The independent variables are a mixture of categorical and numeric data, while the dependent variable is a binary value.If you used
3. The C5.0 algorithm discussed in this chapter takes a different approach to building decision trees. Use the C50 package in R to build a decision tree model of the building permit dataset using the same features that we used in this chapter. What results did you achieve? How do they differ from
2. Attempt to improve the accuracy of the building permit model by including additional features in the decision tree. What improvement in predictive accuracy were you able to achieve?
1. Use the decision tree that we built in the case study (shown in Figure 8.8) to predict the income level for each of the following people:a. A married 30-year-old woman with 16 years of educationb. A divorced 45-year-old man with 12 years of educationc. A married 40-year-old woman with 8 years of
3. In Chapter 5, we used logistic regression to predict the income of prospective customers.Using the same income dataset, attempt to improve upon the predictive accuracy of the previous model by using a naïve Bayes approach. Just like we did in Chapter 5, limit your data to only the categorical
2. The gym is soliciting a new member who is a male teenager who does not own a home.a. Compute Likelihood(Drop-in | Male, Teenager, Homeowner)b. Compute Likelihood(Standard | Female, Adult, Homeowner)c. Compute Likelihood(Elite | Female, Adult, Homeowner)d. Which membership level is this person
1. The gym is soliciting a new member who is a female adult homeowner.a. Compute Likelihood(Drop-in | Female, Adult, Homeowner)b. Compute Likelihood(Standard | Female, Adult, Homeowner)c. Compute Likelihood(Elite | Female, Adult, Homeowner)d. Which membership level is this person most likely to
2. Modify the code used for the donation data use case to incorporate categorical variables into the model. What impact does this have on the accuracy of the model?
1. Examine the following figure. The square near the center of the diagram represents a new, unlabeled point. Using the k-nearest neighbors algorithm, what class would you assign the point using each of the following parameters?a. k = 1b. k = 3c. k = 7d. k = 15
5. Extend the logistic regression model from the income prediction use case to include the continuous variables as well.a. Create and examine a correlation plot for these variables. Do they exhibit multicollinearity?b. Examine the summary statistics for the continuous variables. Do you observe any
4. After improving your model in exercise 3, you use the model to make predictions for employees that were not in the original training set. You obtain the following 10 predictions:1 2 3 4 5 0.1465268 0.9588654 0.9751363 0.4956821 0.8601916 6 7 8 9 10 0.3984430 0.2268064 0.8490515 0.9527210
3. After developing the model in exercise 2, you obtained additional information about the individuals in the study. Specifically, you learned how many prior times each person participated in the screening program. You incorporate that information into your model and obtain these
2. You are working with a healthcare provider who provides patients with a free annual health screening. The provider would like to better understand the factors that drive participation in the screening program. You use logistic regression to develop a model that predicts participation based upon
1. Consider each one of the following problems. Would this problem best be approached as a regression problem or a classification problem?a. Predicting the restaurant chain that someone is most likely to visit based upon their age, number of children, ZIP code, and income levelb. Predicting the
5. After building the regression model in exercise 3, you return to the same dataset and want to know whether the age of a student at application time is also a contributing factor to their GPA. You add this element to a multiple regression model and receive the results shown here:Call:lm(formula =
4. Returning to the bicycle rental dataset, use R to create a simple regression model designed to predict the realfeel temperature based upon the air temperature.Explain your model and describe how well it fits the data.
3. You are working with college admission data and trying to determine whether you can predict a student’s future GPA based upon their college admission test score. The test is scored on a scale of 0–100, while GPA is measured on a scale of 0.0–4.0.Call:lm(formula = gpa ~ test)Residuals:Min
2. Using the blood pressure dataset from the use case in this chapter, produce a correlation plot. Use the corrplot.mixed function and generate a plot that shows the correlation coefficients visually above the diagonal and numerically below the diagonal.Provide an interpretation of your results.
1. You are working with a movie production company to evaluate the potential success of new feature films. As you begin your work, you gather data elements about all feature films released in the past 10 years. Identify five data elements that you think would be useful to gather for analysis.
3. In the vehicles dataset, discretize the co2emissions variable using the High value for emission levels at or above 500 grams per mile and Low for emission levels below this mark. Using the discretized variable for the strata, generate a stratified random sample of 2 percent of the dataset. Show
2. Using the min-max normalization approach, normalize the values of the co2emissions variable in the vehicles dataset so that they fall between the values of 1 and 10. Show the descriptive statistics for the original and normalized variables.
1. For all manual transmission vehicles in the vehicles dataset, list the descriptive statistics for the drive, make, model, and class variables only.
3. Explore the RStudio environment, as explained in this chapter. Create a file called chicken.R that contains the following R script:install.packages("tidyverse")library(tidyverse)ggplot(data=ChickWeight) +geom_smooth(mapping=aes(x=Time, y=weight, color=Diet))Execute your code. It should produce a
2. Visit the rstudio.com website. Download and install the current version of RStudio for your computer.
1. Visit the r-project.org website. Download and install the current version of R for your computer.
1. Consider each of the following machine learning problems. Would the problem be best approached as a classification problem or a regression problem? Provide a rationale for your answer.a. Predicting the number of fish caught on a commercial fishing voyageb. Identifying likely adopters of a new
2. The index (3.1.8) becomes E(W,b) = κ=1n i=1(yκi − (Wxκ )i − bi )2, where yκi is the ith component of the κth target yκ . In order to find normal equations, defineˆW=⎛⎜⎜⎜⎜⎜⎝w11 w12 . . . w1n w21 w22 . . . w2n......wd1 wd2 . . . wdn b1 b2 . . . bn⎞⎟⎟⎟⎟⎟⎠,
1. We have (∇f )i = ∂i(cj xj ) = ci and (∇g)i = ∂i(xjAjkxk) = (Aik + (A)ik)xk.
2. The index (3.1.8) becomes E(W,b) = κ=1n i=1(yκi − (Wxκ )i − bi )2, where yκi is the ith component of the κth target yκ . In order to find normal equations, defineˆW=⎛⎜⎜⎜⎜⎜⎝w11 w12 . . . w1n w21 w22 . . . w2n......wd1 wd2 . . . wdn b1 b2 . . . bn⎞⎟⎟⎟⎟⎟⎠,
1. We have (∇f )i = ∂i(cj xj ) = ci and (∇g)i = ∂i(xjAjkxk) = (Aik + (A)ik)xk.
Use the joint distribution of RBMs in Eq. (15.8) to prove the conditional independence of RBMs, and further derive that both Pr¹hj = 1 j vº and Pr¹vi = 1 j hº can be computed with a sigmoid function.
Following the idea of the VAEs in Section 13.4, use the variational distribution in Eq. (15.6) to derive a proxy function for the likelihood function of the LDA model (i.e., p????W; , , ). By maximizing this proxy function iteratively, derive a learning algorithm for all LDA parameters.
For the Bayesian network in Figure 15.24, design a sampling scheme to generate samples to estimate the following conditional distributions:I p¹x1, x2 j ˆ x6, ˆ x7ºI p¹x3, x7 j ˆ x4, ˆ x5º
If all conditional probabilities of the causal Bayesian network in Figure 15.12 are unknown, what types of data do you need to estimate these probabilities? How will you collect them?
Given the causal Bayesian network in Figure 15.12, calculate the following probabilities:a. Pr¹W = 1ºb. Pr¹L = 1 jW = 1º and Pr¹L = 1 jW = 0ºc. Pr¹L = 1 j R = 1º and Pr¹R = 0 j L = 0º
Assume three binary random variablesa, b, c 2 f0, 1g have the following joint distribution:By direct evaluation, show that this distribution has the property that a and c are marginally independent (i.e., p¹a, cº = p¹aºp¹cº), but a and c become dependent when conditioned on b (i.e., p¹a,
Assume three binary random variablesa, b, c 2 f0, 1g have the following joint distribution:By direct evaluation, show that this distribution has the property that a and c are marginally dependent (i.e., p¹a, cº , p¹aºp¹cº), but a and c become independent when conditioned on b (i.e., p¹a,
Replace the sigmoid function in Eq. (14.28) with a softmax function, and formulate a Gaussian process for a multiclass pattern-classification problem.
Derive Laplace’s method for a Gaussian process in binary classification.
Show the procedure to derive the conditional distribution in Eq. (14.27).
Following the ideas in Example 14.3.1, derive the variational Bayesian learning procedure for the Gaussian mixture hidden Markov models (HMMs) in Section 12.4, as shown in Figure 12.15.
Assume we choose the same prior distribution for a GMM as in Example 14.3.1. Use the EM algorithm to derive the MAP estimation for GMMs:a. Give the formulae to update all GMM parameters MAP iteratively.b. If we approximate the true posterior distribution of a GMM p¹jxº by another approximate
Following the ideas in Example 14.3.1, derive a variational distribution for a multivariate Gaussian model using the variational Bayesian method. Compare the derived variational distribution with the exact posterior distribution in Example 14.2.2.
Use Laplace’s method to conduct Bayesian learning for the log-linear models in Section 11.4.
Use Laplace’s method to conduct Bayesian learning for the probit regression in Section 11.4.
Consider a linear regression model from x 2 Rn to y 2 R: y = w|x + ", where w 2 Rn is the model parameter, and " is an independent zero-mean Gaussian noise " N????0, 2. Assume we choose a Gaussian distribution as the prior of the model parameter w: p¹wº = N????w0, 0. Assuming we have
Show the procedure to derive the following two steps in Bayesian learning of a multivariate Gaussian model:a. Completing the square: IM i=1 with x = xi. N (xi - ) 2(x; 1) = (x x) (x x) + N( x) ( x), i=1 b. (x-x)(x; - x) = tr(((x-x)(x; - x))1) = tr(NS), with S (x-x)(x; - x). =
Show the procedure to derive the updating formulae in Eqs. (14.7) and (14.8) for the mean n and variance2 n of Example 14.1.1.
Derive an SGD algorithm for a GAN to train a CNN-based deep generative model for image generation, using a convolution-layer-based encoder and a deconvolution-layer-based decoder [148].
Derive a stochastic gradient-descent (SGD) algorithm for VAE to train a convolutional neural network(CNN)–based deep generative model for image generation, using a convolution-layer-based encoder and a deconvolution-layer-based decoder [148].
Compute the gradient for the ICA log-likelihood function in Eq. (13.9), and derive a gradient-descent method for the MLE of ICA.
Factor analysis can be viewed as an infinite mixture model in Q12.5, where the factor z is considered to be the continuous mixture index, and p¹zº and p¹xjzº are viewed as mixture weights and component models, respectively. Extend the EM algorithm for infinite mixture models in Q12.5 to derive
Derive the conditional distribution p¹zjxº for factor analysis.
Derive the conditional distribution in Eq. (13.7) for probabilistic PCA models.
Show the procedure to derive Eq. (13.4) for liner Gaussian models.
Assume a joint distribution p¹x, yº of two random vectors x 2 Rn and y 2 Rn is a linear Gaussian model defined as follows:p¹xº = N????x, ????1, where 2 Rn is the mean vector; 2 Rnn is the precision matrix; and p¹y j xº = N????y Ax +b, L????1, where A 2 Rnn, b 2 Rn, and L 2 Rnn is the
When the HMM algorithms are run on long sequences, underflow errors often occur because we need to multiply many small positive numbers. To address this issue, we often represent all forward and backward probabilities in the logarithm domain. In this case, we can do arithmetic operations as
Derive an efficient method to compute the gradient of the log-likelihood function for the following mixture models:a. Gaussian mixture models: @@ ln p¹xº, where p¹xº is given in Eq. (12.6).b. Hidden Markov models: @@ ln p¹oº, where p¹oº is given in Eq. (12.15).

Showing 300 - 400 of 816

Machine Learning Fundamentals A Concise Introduction 1st Edition Hui Jiang - Solutions

Step by Step Answers