Data Mining And Predictive Analytics 2nd Edition Daniel T Larose, Chantal D Larose - Solutions

Discover comprehensive solutions and answers to questions from "Data Mining And Predictive Analytics 2nd Edition" by Daniel T Larose and Chantal D Larose. Access our online answers key, solution manual, and solutions PDF to understand solved problems with step-by-step answers. Our resource offers a valuable test bank and chapter solutions, ideal for students and educators seeking detailed textbook insights. Enhance your learning with the instructor manual and free download options, ensuring you have all the tools for success in predictive analytics.

3.15 A GSS that cross-classified income in thousands of dollars (
3.16 A study on educational aspirations of high school students (S. Crysdale, Int. J.Compar. Sociol. 16: 19-36, 1975) measured aspirations with the scale (some high school, high school graduate, some college, college graduate). The student counts in these categories were (9, 44, 13, 10) when family
3.17 Refer to Table 2.13 on homosexual sex and premarital sex.a. Construct and interpret a mosaic plot.b. Obtain a 95% confidence interval for gamma. Interpret the association.
3.18 Table 3.14 shows the results of a retrospective study comparing radiation therapy with surgery in treating cancer of the larynx. The response indicates whether the cancer was controlled for at least two years following treatment. Table 3.15 shows SAS output. Some R output looks like:>
3.19 A study in the Department of Wildlife Ecology at the University of Florida sampled wild common carp fish from a wetland in central Chile. One analysis investigated whether the fish muscle had lead pollutant and whether there was evident malforma-tion in the fish. Of 25 fish without lead, 7 had
3.20 Seneta and Phipps (2001) described a medical study that compared subjects with nonacute appendicitis and with acute appendicitis in terms of whether they suffered severe right abdominal pain. Such severe pain was reported by 5 of the 15 nonacute cases and by 1 of the 16 acute cases. The
3.21 Analyze Table 3.1 using the Bayesian approach with independent uniform prior distributions.a. Specify the posterior distribution of (π₁, π₂).b. Using software or your own simulation, estimate the posterior mean of the dif-ference of proportions and find a 95% equal-tail posterior
3.22 Refer to the table (11, 0/0, 1) analyzed with Bayesian methods in Section 3.6.4.Using simulation, estimate P(π₁ > π₂|y₁, n₁; y₂, n₂) for independent beta(α₁ α₂)priors having (a) α₁ = α₂ = 2, (b) α₁ = α₂ = 1, and (c) α₁ = α₂ = 0.50. Interpret.
3.23 Table 3.16 cross-classifies votes in the 2000 and 2004 U.S. presidential elections.Treating the two rows as independent binomials and using uniform priors, generate the posterior distribution of the odds ratio. Plot it, and find a 95% equal-tail or HPD posterior interval. What is the
3.24 Is θ the midpoint of commonly used confidence intervals for the odds ratio θ? Why or why not?
3.25 For comparing two binomial samples with fixed sample sizes, show that the stan-dard error (3.1) of a log odds ratio increases when, for either sample, the absolute difference of proportions of successes and failures increases. [Hint: Show that the asymptotic variance is minimized when each
3.26 Using the delta method as in Section 3.1.6, show that the Wald confidence interval for the logit of a binomial parameter is log[/(1 - ㄤ)] ± Ζα/2/√ηπ (1 – π).Explain how to use this interval to obtain one for itself. [Newcombe (2001) noted that the sample logit is also the midpoint
3.27 For two parameters, a confidence interval for 01 - 02 based on single-sample estimate and interval (l₁, u₁) for θ₁, i = 1, 2, is(01-02 - √(01-11)² + (12-02)², 01 - 02 + √(41 – 01)² + (02-12)2).Newcombe (1998b) proposed an interval for πι π₂ using the score interval (li,
3.28 For multinomial sampling, use the asymptotic variance of log to show that for Yule's Q (Exercise 2.38) the asymptotic variance of √n(Q-Q) is(Σ; Σπ¹) (1 – Q²)2/4 (Yule 1900, 1912).
3.29 For multinomial probabilities π = (πι, π2, ...) with a contingency table of ar-bitrary dimensions, consider a measure of form g(π) = ν/δ. Show that the asymptotic variance of √n[g(t) - g(x)] is σ² = [Σ; πη; – (Σ₁ πίηί)²]/84, where ni = δ(δν/δπ₁) –
3.30 Show that x2 = n Σ; Σ; (Pij - Pi+P+j)²/Pi+P+j = n Σ₁ Σ; Pi+P+j (aij - 1)2 for the sample association factors {aij}. Thus, X2 can be large when n is large, regardless of whether the association is practically important. Explain why this test, like other tests, merely indicates the degree
3.31 For a 2 x 2 table, consider H0: πιι = 02, πι2 = π21 = θ(1 – θ), π22 = (1 – θ)2.a. Show that the marginal distributions are identical and that independence holds.b. For a multinomial sample, under Ho show that e = (p1+ + P+1)/2.c. Explain how to test Ho. Show that df = 2 for the
3.32 For testing independence, show that X2 ≤ n min(I – 1, J-1). Hence V2 =X2/[n min(I - 1, J - 1)] falls between 0 and 1 (Cramér 1946). [For 2 x 2 tables, X2/n is often called phi-squared; it equals Goodman and Kruskal's tau of Exercise 2.39. Other measures based on X2 include the contingency
3.33 For a 1 x 2 table (i.e., a single binomial Y based on n trials, with probabilities π and 1 – π), consider testing Ηο: π = πο.a. Show that the Pearson residuals are(γ – ηπο)/√ηπο and – (у – пπο)/√n(1 – πο), which have differing absolute values when πο ≠
3.34 For a 2 x 2 table, show that:a. The four Pearson residuals may take different values.b. All four standardized residuals have the same absolute value. (This is sensible, since df = 1.)c. The square of each standardized residual equals X2.
3.35 Use a partitioning argument to explain why G2 for testing independence cannot increase after combining two rows (or two columns) of a contingency table.[Hint: Explain why G2 for full table = G2 for collapsed table +G2 for table of the two rows that are combined in the collapsed table.]
3.36 Assume independence, and let pij = nij/n and tij = Pi+P+j•a. Show that pij and tij are unbiased for πij = πi+N+j.b. Show that var(pij) = πί+ π+j(1 – πί + π+j)/n.c. Using E(pi+P+j)² = E(p²+)E(p) and E(p+ E(p+) = var(pi+) + [E(pi+)]², show that var(ij) = {πί+ Π+j [πί+(1 –
3.37 Consider an I × J table with ordered columns and unordered rows. Ridits (Bross 1958) are data-based column scores. The jth sample ridit is the average cumulative proportion within category j, j-1 P₁ = ++ (2) P+j.k=1 The sample mean ridit in row i is R₁ = ∑;jPjli. Show that Σ; P+jfj =
3.38 Show that the sample value of the uncertainty coefficient (2.13) satisfies-G2/2n (p+; log p+j). [Haberman (1982) gave its standard error.]
3.39 Of six candidates for three managerial positions, denote the females by F1, F2, F3 and the males by M1, M2, M3.a. Identify the 20 possible combinations of candidates that could be selected. Con-struct the contingency table for the actual sample, which is (F2, M1, M3).b. Let p₁ denote the
3.40 When a test statistic has a continuous distribution, the P-value has a null uniform dis-tribution, P(P-value < α) = a for 0 < a < 1. For Fisher's exact test, explain why P(P-value < α) < α. [Hint: P(P-value < α) = E[P(P-value ≤ anı+, n+1, n)].]
3.41 Note 3.3 showed moments of the hypergeometric distribution (3.17). Letting p =n+1/n, show that n₁₁ has the same mean as a binomial random variable for n1+trials with success probability p, and that it has its variance multiplied by a finite population correction factor (n - n1+)/(n - 1).
3.42 For the tea-tasting data (Table 3.9), construct the null distributions of the ordinary P-value and the mid P-value for Fisher's exact test with Ha: 0 > 1. Find and compare their expected values.
3.43 In Section 3.5.6 we analyzed a 2 x 2 table having entries (3, 0/0, 3). Explain why the unconditional P-value, evaluated at π = 0.50, is related to Fisher conditional P-values for various tables by 6 Σk=0 2 P(X² ≥ 6) = P(X² ≥ 6|n+1 = k)P(n+1 = k).32 Thus, the unconditional P-value of 35
3.44 For testing Ηο: πι = π₂ with two binomial variates y₁ and y2, a "reasonable" test would not reject Ho if y₁ = y2 = 0. Since as π₁ and π₂ approach 0, the probability of this converges to 1 even if π₁ ≠ π2, explain why any such test is biased, potentially having power less
3.45 For independent uniform prior distributions for two binomial parameters, show that r = πι/π2 has prior density g(r) = ½ for 0 ≤ r ≤ 1 and g(r) = 1/2r2 for r > 1.
3.46 Explain why a Bayesian HPD interval is sensible for πι π₂ but not usually forπι/π2.
3.47 Consider a particular choice of Dirichlet means {Yij = E(π¡j) = α¡j/K} for the Bayes estimator (1.19) extended to two-way tables. Show that the total mean squared error is 2[K/(n + K)]² [Σ(πιj - Vij)²] + [n/(n + K)]² [1 - Σπ]divided by n. Show that the value of K that minimizes
For each of the predictors, find the log posterior odds ratio, and explain the contribution of this predictor to the probability of a malignant tumor.
Find the naïve Bayes classifications for each of the combinations in Exercise 31.
(Optional) Assess the validity of the conditional independence assumption, using calculations similarly to Table 14.5.
For each of the combinations in the previous exercise, find the posterior odds ratio.
Using your results from the previous exercise, find the maximum a posteriori classification of tumor class, for each of the following combinations:a. Mitoses=low and Clump Thickness=low.b. Mitoses=low and Clump Thickness=high.c. Mitoses=high and Clump Thickness=low.d. Mitoses=high and Clump
Construct the joint conditional probabilities, similarly to Table 14.4.
Find the posterior probability that the tumor is malignant, given that clump thickness is (i) high and (ii) low.
Find the posterior probability that the tumor is malignant, given that mitoses is (i) high and (ii) low.
Find the conditional probabilities for each of the predictors, given that the tumor is malignant. Then find the conditional probabilities for each of the predictors, given that the tumor is benign.
Find the prior probabilities for each of the predictors and the target variable. Find the complement probabilities of each.
Consider using only two predictors, mitoses and clump thickness, to predict tumor class. Categorize the values for mitoses as follows: Low =1 and High=2–10. Categorize the values for clump thickness as follows: Low =1–5 and High=6–10. Discard the original variables and use these categorized
Compute the probabilities by which the Bayes net model classifies the fourth instance from the test file movies_test.arff. Do your calculations result in a positive classification as reported by WEKA?
Revisit the WEKA naïve Bayes example. Calculate the probability that the first instance in movies_test.arff is “pos” and “neg.” Do your calculations agree with those reported by WEKA leading to a negative classification?
Provide the MAP classification for season given that a warm coat was purchased, in the clothing purchase example in the Bayesian network section.
Find the naïve Bayes classifier for the following customers. Use the empirical distribution where necessary.a. Belongs to neither plan, with 400 day minutes.b. Belongs to the International Plan only, with 400 minutes.c. Belongs to both plans, with 400 minutes.d. Belongs to both plans, with zero
Verify the empirical distribution results referred to in the text, of the numbers of records within the certain margins of error of 800 minutes, for each of churners and non-churners.
Calculate the naïve Bayes classification for all four possible combinations of International Plan and Voice Mail Plan membership, using the 25.31% 74.69% balancing.
Compute the posterior odds ratio for each of the combinations of International Plan and Voice Mail Plan membership, using the balanced data set.
What are the two main considerations when building a Bayesian network?
Describe the intrinsic relationship among the variables in a Bayesian network.
Explain the difference in assumptions between naïve Bayes classification and Bayesian networks.
Explain what is meant by working with the empirical distribution. Describe how this can be used to estimate the true probabilities.
Extra credit: Investigate the mixture idea for the continuous predictor mentioned in the text.
Describe the process for using continuous predictors in Bayesian classification, using the concept of distribution.
Explain why the log posterior odds ratio is useful. Provide an example.
When is the naïve Bayes classification the same as the MAP classification? What does this mean for the naïve Bayes classifier, in terms of optimality?
What is meant by conditional independence? Provide an example of events that are conditionally independent. Now provide an example of events that are not conditionally independent.
Explain why the MAP classification is impractical to apply directly for any interesting real-world data mining application.
Explain why we cannot avoid altering, even slightly, the character of the data set, when we apply balancing.
Describe what balancing is, and when and why it may be needed. Also, describe two techniques for achieving a balanced data set, and explain why one method is preferred.
Explain the interpretation of the posterior odds ratio. Also, why do we need it?
Explain in plain English what is meant by the maximum a posteriori classification.
Why would we expect, in most data mining applications, the maximum a posteriori estimate to be close to the maximum-likelihood estimate?
Explain the difference between the prior and posterior distributions.
Describe the differences between the frequentist and Bayesian approaches to probability.
Use cluster membership to predict rating. One way to do this would be to construct a histogram of rating based on cluster membership alone. Describe how the relationship you uncovered makes sense, based on your earlier profiles.
Which clustering solution do you prefer, and why?
Rerun the k-means algorithm with k=3.
Develop clustering profiles that clearly describe the characteristics of the cereals within the cluster.
Using all of the variables, except name and rating, run the k-means algorithm with k=5 to identify clusters within the data.
Confirm the calculations for the second pass and third pass for MSB, MSE, and pseudo-F for step 4 of the example given in the chapter.
Which clustering solution do you think is preferable? Why?
Refer to Exercise 8. Show that the ratio of the between-cluster variation to the within-cluster variation increases with each pass of the algorithm.
Once again identify the clusters in Exercise 6 data, this time by applying the k-means algorithm, with k=3. Try using initial cluster centers as far apart as possible.
Refer to Exercise 6. Show that the ratio of the between-cluster variation to the within-cluster variation increases with each pass of the algorithm.
Suppose that we have the following data: a bcde f ghi j (2,0) (1,2) (2,2) (3,2) (2,3) (3,3) (2,4) (3,4) (4,4) (3,5) Identify the cluster by applying the k-means algorithm, with k=2. Try using initial cluster centers as far apart as possible.
Suppose that we have the following data (one variable). Use single linkage to identify the clusters. Data: 0 0 1 3 3 6 7 9 10 10
Repeat Exercises 21–31 using the red wines only. Compare your principal component profiles and the performance of your regression models.
Discuss the question of whether we should add principal components to increase the predictive power of the regression model and/or reduce the standard error. Arm yourself for this discussion by incrementing your optimal number of components by 1, running the PCA, performing the regression with the
Provide a table showing the correlation coefficients of each principal component with each other principal component. Comment.
Run a multiple regression of the principal components predictors on quality.a. Get the VIF measures of the components. Comment.b. Compare the regression standard error for the two regression models. Comment.c. Compare the R2 for the two regression models. Comment.
Compare your principal components in one-to-one manner with your EDA earlier. Discuss.
Proceed to apply PCA to the predictors, using varimax rotation. In the output, suppress factor loadings less than |0.5|.a. Provide both the unrotated and the rotated component matrices.b. Use the results in (a) to demonstrate how varimax rotation eases interpretability of the components.c. Report
Determine the optimal number of components to extract, usinga. The Eigenvalue Criterion;b. The Proportion of Variance Explained Criterion;c. The Minimum Communality Criterion;d. The Scree Plot Criterion;e. Try to arrive at a consensus among the four criteria as to the optimal number of components
Clearly explain how PCA can solve the problem of collinear predictors.
Run a multiple regression of the predictors on quality. Get the variance inflation factor (VIF) measures of the predictors. Explain what these mean in the context of this problem. Explain whether they support or undermine the need for PCA in this problem.
Suppose we eventually would like to perform linear regression analysis of quality versus these predictors. Clearly explain why we should beware of using a set of predictors that are highly correlated.
Construct a matrix plot of the predictors. Provide a table showing the correlation coefficients of each predictor with each other predictor. Color code your table so that the reader can easily see at a glance which are the strongest correlations. What you are doing here is doing EDA for the
Standardize the predictors.
Construct a useful user-defined composite using the predictors. Describe situations where the composite would be more appropriate or useful than the principal components, and vice versa. Use the wine_quality_training data set, available at the textbook web site, for the remaining exercises. The
Apply PCA using varimax rotation, with your chosen number of components. Write up a short profile of the first few components extracted.
Based on the information from the previous exercise, make a decision about how many components you shall extract.
How many components should be extracted according toa. The Eigenvalue Criterion?b. The Proportion of Variance Explained Criterion?c. The Scree Plot Criterion?d. The Communality Criterion?
Now, suppose we are interested in estimating the number of home runs, based on the other numerical variables in the data set. So all the other numeric variables will be our predictors. Investigate whether sufficient variability exists among the predictors to perform PCA.
First, filter out all batters with fewer than 100 at bats. Next, standardize all the numerical variables using z-scores.
Based on the following rotated component matrix:a. Provide a quick profile of the first four components.b. If we extracted an eighth component, describe how the first component would change.c. What is your considered opinion on the usefulness of applying PCA on this data set?
Based on the following scree plot, how many components should be extracted using the scree plot criterion? Now, based on the three criteria, work toward a decision on the number of components to extract. Scree plot Component number 1 2 3 4 5 6 7 8 9 10 11 Eigenvalue 1.1 1.0 0.9

Showing 2000 - 2100 of 4107

First
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
Last

Data Mining And Predictive Analytics 2nd Edition Daniel T Larose, Chantal D Larose - Solutions

Step by Step Answers