Questions and Answers of Measurement Theory In Action

There are a large number of personality differences that, to date, have little or no means of assessment. While some of these constructs are well defined, others suffer the disgrace of poor construct
Assume that you are currently working as a(n)a. clinical psychologist for a local parole board, orb. industrial/organizational psychologist for the U.S. Postal Service, orc. school psychologist
For this exercise, identify two different measures of the same construct. For example, you could identify two different measures of the personality construct, agreeableness. Also, obtain the test
Caleb knew he had to pay a visit to Dr. Zavala, the instructor of his psychometrics course. Caleb’s grade on the first exam was, shall we say, less than impressive. While his performance on the
Sheila was frustrated. Although she was happy with both the topic and the constructs she had chosen to examine in her senior honors thesis, she had hit several roadblocks in determining what measures
Mark each of the following as observed score (X), true score (T), or error (E) 1. During a timed, two-hour exam, both of Celia’s mechanical pencils ran out of lead causing her temporary distress
Using the Spearman–Brown prophecy formula provided in Case Study 5.2, estimate Sheila’s reliability for the dogmatism scale if she used only one third of the number of original items. Is this an
It didn't make sense. It just didn't. How could the reliability be so low? Chad scratched his head and thought. Chad had agreed to help analyze the data from his graduate advisor's most
Olivia and Gavin had been studying for their upcoming psychometrics midterm for over an hour, but the past few minutes had been less than productive. While the two thought they both had a good
Using the data set “Reliability.sav”, perform the reliability analyses outlined below. The scales provided here include a depression scale (14 items, V1–V14), a life satisfaction scale (10
What is the CVI for the five-item test in question 10 prior to deletion of any items due to low CVR?Question 10:Imagine the case in which 14 SMEs were asked to provide CVR ratings for a five-item
In her years of experience as a clinical therapist, Juanita had come to suspect that some of her clients seemed to share a common trait.Specifically, a significant portion of her clients expressed
Reflecting for a moment on the results of his ambitious undertaking, Lester smiled. His boss at the Testing and Personnel Services Division of this large midwestern city had assigned him the task of
For each of the following tests, identify two different samples of people who would have the expertise to serve as subject matter experts (SMEs) for providing judgments regarding the content validity
Given the limited number of items that can be included on a test or inventory, test developers must often make difficult decisions regarding the proportion of items that can be used to assess each
For this exercise, choose an appropriate sample of at least ten individuals to act as SMEs for this scale. Ask these SMEs to familiarize themselves with the proposed dimensions of the scale. Then ask
“This will be a cinch,” Cecilia had thought when she first received the assignment to conduct a criterion-related validity study. She’d been a human resource (HR) specialist at Joyco for only
Principal Andrew Dickerson of Mountain Central High School had a hunch. Actually, it was more like a strong suspicion. At nearly 15%, the student dropout rate in his high school was well above the
For each of the criteria presented in items 1–3, identify at least two psychological or cognitive measures that might serve as useful predictors in a criterion-related validation study. 1. Grades
Use the data set “Bus driver.sav” to correlate the possible predictors with the job performance measures and then answer the following questions. 1. Overall, how highly are the possible
A sales manager hoping to improve the selection process for the position of product sales compiled the data file “Sales.sav.” The manager administered several tests to her current employees and
Ever since learning about the concept in her psychology class, Khatera had known that she would complete her thesis on the construct of emotional intelligence. Since her initial introduction to the
DiAnn wasn’t too surprised to see Edgar arrive shortly after her office hours began. The material recently presented by the instructor in the psychological testing course for which she served as
Imagine that you have recently developed the following construct measures. For each of these newly developed instruments, identify two actual measures that could be used to examine the new
An industrial/organizational psychologist developed a personality based measure to assess the integrity of potential job applicants. The measure she developed was intended to mask the purpose of the
Assume you wanted to carry out a meta-analysis to determine how effective typing software is in improving typing speed and accuracy. What is the best way to get started in conducting such a meta
How do you decide which studies to include or exclude? What information to code?
Raul, a second-year master’s student, was very excited that his thesis committee had just approved his proposal to conduct a meta-analysis looking at what predicts employees’ satisfaction with
Ming-Yu, a new PhD student, was just given her first assignment as a graduate research assistant for Professor Riggs. Professor Riggs was a quantitative psychologist who studied the effects of sport
Individually or in small groups of three to five, students will select a topic on which to perform a meta-analysis. They will then outline, in detail, the steps to be taken if they were to actually
In Table 10.3, you will find data from the Albemarle Paper Co. v. Moody (1975) Supreme Court case. The case involved looking at the use of meta-analysis (more specifically, validity generalization)
Larry had just completed his master’s degree in industrial and organizational (I/O) psychology and obtained his first job with the human resources department of a large local school district. The
Joelle, a life span developmental psychology graduate student, recently completed an internship at an adult day care facility. The majority of the facility’s clientele were elderly individuals who
Using the data set of Mersman and Shultz (1998), recreate the results presented in Figures 11.3 and 11.4. In addition, recreate the regression equations presented in the module overview. Finally, run
Jaime was rightly proud of the midterm exam he created for the course in which he served as teaching assistant, PSY 451: Introduction to Forensic Psychology. The instructor, Dr. Dan Kellemen, had
“It’ll be easy money.” So said the director of faculty development, when trying to convince Dr. Patricia Lonergan to present a brief overview on test development to a group of interested
Students sometimes complain that items on a test are vague, exceedingly difficult, or even unrelated to the topics presented in the course. Here’s your opportunity to see what it’s like to create
Andrew, a third-year graduate student, was enrolled in a PhD program in quantitative psychology. He had recently obtained a highly competitive summer internship with a Fortune 500 company in its
Linda, a second-year master’s student, had agreed to help out the department of psychology with its outcomes assessment process. In exchange for her work on the project, the department chair agreed
Selected items (13 to be exact) from a 50-item multiple-choice test given to an undergraduate organizational behavior class are presented in Table 13.4. Look through the test to get a sense of the
Alexius, a fifth-year doctoral student, had agreed to sit on the committee that was restructuring the doctoral comprehensive exams for his department. It seemed every year students complained about
Lui-Ping (most people just called her Jasmin), a recent master’s graduate, had decided to return home to Malaysia after graduation. After a short job search, she obtained a job with the ministry of
The Minnesota Multiphasic Personality Inventory (MMPI) is one of the earliest and best-known empirically derived tests. Graham (1977, 1999) presented a detailed description of the development of the
“I knew this topic wasn’t any good, but none of you listened to me, did you?” asked Doug, half in jest. He and four other students in his graduate test construction seminar had recently begun
Use the data file “Joinership rational ratings.sav” to do the following: 1. Compute the mean and standard deviation of each item. 2. Determine a single cut score that you believe reflects
1. Using only those items retained following Exercises 15.2 and 15.3, conduct a reliability analysis of each dimension of the scale (as represented by the factors that emerged in Exercise 15.3).
Ryan, a first-year PhD student, was teaching his own discussion sections of introductory psychology for the first time. At Ryan’s university, the introductory psychology class consists of a
Dora was very excited about her committee’s recent approval of her proposal for her master’s thesis project. After several arduous sets of revisions, she was finally ready to collect her data.
Reflecting concerns that standardized testing was unfair, the psychology department at South East State University (SESU) decided six years ago to eliminate the requirement that prospective students
Although he had worked in human resources (HR) for a little more than a year, Connor Maxfield had been unexpectedly promoted to oversee the selection of new employees into MiniCorp after his boss
Perform a multiple regression analysis using pescore as the dependent variable and each of the six predictors identified in Exercise 8.2 as the independent variables. Choose “enter” as the
Because volunteers are unpaid, the work motivation of these individuals can be complex. It is possible that individuals’ work motivation depends on their perceptions of their own ability, along
Use the data set discussed in Exercise 17.2 to answer the following items. 1. Conduct a multiple regression analysis using all nine predictors. Choose method equals “enter.” Write out the
Andra was on a roll, or at least she had been until she looked at the results of her exploratory factor analysis (EFA). At the request of the chair of the psychology department, she had been working
“Will she ever be happy?” grumbled Chin, mostly to herself. Chin was, of course, referring to her thesis advisor. Everything, it seemed, revolved around her thesis right now. She and her fiancé
In an article published in the Journal of Personality, Marsh (1990) provided evidence that confirmatory factor analysis (CFA) can be very useful in analyzing a multitrait-multimethod (MTMM)
Module 11 discussed the idea of measurement equivalence in the context of cross-cultural research. CFA work has been very important in determining the measurement equivalence of instruments across
Elena, a first-year educational measurement graduate student, vaguely remembered a discussion of item response theory (IRT) in her undergraduate tests and measurements class, but never thought that
Dr. Agars was recently hired by the state of California as a senior research analyst in the Department of Corrections (DoC). His degree was in industrial and organizational (I/O) psychology, with a
1. The Web site http://www.sscicentral.com provides information on several IRT programs, including BILOG-MG, PARSCALE, and IRTPro. The Web site has a student version of IRTPro available for
Scott, a second-year graduate student in educational measurement, had just gotten off the phone with his sister Gail, who had recently finished her nursing degree. Gail and her friend, Tammy, had
The Educational Testing Service identified differential item functioning (DIF) on an analogy question from the SAT exam. The question was an analogy that asked: “Strawberry: Red as (a) peach:
Your advisor wants to develop a new measure of negative affectivity to complement existing measures that are traditional response formats. Your advisor believes that existing measures are limited
Structured interviews are some of the most popular measurement devices used for hiring purposes because they tap into managers’ affinity to employment interviews but they also do so in a way that
Benjamin, a college senior who had a dual major in psychology and marketing, decided he wanted to complete his undergraduate honors thesis in the area of consumer psychology. Specifically, he was
Amanda, a graduate student who had just completed her first year in an industrial and organizational psychology PhD program, was excited because she had just gotten her first consulting job. She was
Outline a scaling study similar to the example that is provided at the end of the overview section of the module. Select a construct (other than school-related social anxiety) and answer the
Using the data from the “Bus Driver.sav” data set, scale the 10 task items on the three dimensions of “Frequency,” “Relative Time Spent,”and “Importance.” Use Table 3.1 to fill in the
Some measures of a particular construct are better than others. Discuss what you believe to be the three most important issues that should be considered prior to item writing that may affect the
As new interns in the human resource department at SAVECO, Juan and Barbara were excited to receive their first major assignment. The president of the company had just asked them to assess the job
The time had come, at last. After countless years as a student, Janie had been teaching her first undergraduate college course—an introductory statistics course—for nearly five weeks. Now the
1. Is the current test related to any other demographic information such as age, education level, or work experience? How about the proposed test?2. The complainant (with ID #450) is suggesting that
1. What descriptive information can we provide to the EEOC regarding the current MC test being used? How about the proposed one?2. Create appropriate graphs to describe the current and proposed MC
Megan, a second-year clinical psychology graduate student, had just gotten her first assignment in her graduate internship placement in the Community Counselling Center (CCC). Dr. Chavez had given
Professor Ullman had just given the first examination in her introductory psychological statistics class. She passed the results on to Rudy, one of her graduate teaching assistants, so that he could
Most tests are administered to identify some form of individual differences. These can include inter-individual differences, intraindividual differences, or both. Again, looking at the Armed Services
As noted in the overview, there are typically numerous constituents in any given testing process. These constituents may include the test takers, test developers, test users, and, more broadly,
The Armed Services Vocational Aptitude Battery (ASVAB) was discussed in the module overview. Nearly one million people take this test each year, many of them high school students. The test consists
A local volunteer referral agency was interested in using “tests” to place volunteer applicants in the volunteer organizations it served. In order to do so, however, the agency needed to assess
Professor Gilbert, an educational testing professor at a local state university, was contacted by a small school district that had decided to implement a Talented and Gifted (TAG) program for
The professor or some other “moderator” will summarize the initial set of ratings for the class. You will then be provided with the results and be allowed to make changes to your initial ratings
“Existential Crisis” of the television show, The Good Place, the characters Tahani and Jason discuss the way Jason rated individuals who auditioned for his dance team. Ratings were conducted on
The “Geoscience attitudes.sav” data set asks students about their attitudes toward the geosciences (archaeology, geography, and geology). The data set has 13 attitude items and 137 respondents.
Extracted communalities and eigenvalues can be computed from an unrotated factor matrix. An extracted eigenvalue is the sum of the squared loadings of the items on a factor. An extracted communality
Imagine you have been asked to develop a personality test that measures antisocial behaviors. The measure will be used for three different purposes. The first purpose will be to identify high school
Find two articles that have used exploratory factor analysis in the development of a psychological scale. Study these two articles, focusing on the documentation of the procedures that were used in
Just as in any other statistical technique, there is a wide variety in terms of the practice and reporting of CFA. Find two studies that use CFA techniques to analyze the structure of a psychological
Either individually or in small groups, perform a literature search to find a recent empirical article that provides an example of the application of IRT to an applied testing situation. IRT
In the module overview, we discussed the key elements of IRT and how it can be applied to computer adaptive testing and differential item functioning. We mentioned that many large-scale tests are now
The Web page https://scholarworks.umass.edu/pare/vol4/iss1/6/ by Ronald Hambleton and H. Jane Rogers is titled “Item Bias Review.” The page provides a number of questions test creators can ask
Either individually or in small groups, perform a literature search to find a recent empirical article that provides an example of the application of IRT to an applied testing situation that
Find two empirical studies that use generalizability theory to evaluate a test or measure (make sure to choose different sets of authors). Study those two articles, paying particular focus on the
One of the leading centers of research on GT has been Center for Advanced Studies in Measurement and Assessment at the University of Iowa, where Dr. Robert Brennan has worked for many years. Check
Assume you wanted to determine the degree of inter-rater agreement between two forensic psychologists who were each rating 100 potential parolees in terms of their potential for committing additional
What do you expect to learn as a student in a psychological testing course?
What will likely be your major stake in the testing process once you finish your measurement course?
What alternative “test” to the APGAR could an obstetrics nurse or doctor use to assess newborn functioning? What would be the advantages and disadvantages compared to the APGAR test shown in
Who are the major constituents or stakeholders in the psychological testing process?
What is the major purpose of examining inter-individual differences via test scores?
What are the different types of intra-individual differences?

Showing 1 - 100 of 260