1. The Web site http://www.sscicentral.com provides information on several IRT programs, including BILOG-MG, PARSCALE, and IRTPro. The Web site has a student version of IRTPro available for download. 

2. Once IRTPro is downloaded and installed on your computer, start the program. You should get a screen that looks like Figure 20.3a. From the menu, select “Open” and from type of file choose “IRTPro Data file” as the type of file. Choose “Chapter 20 data.ssig.” It is possible to open regular ASCII data files in the student version but at this point, it is easier for you to open a file already prepared for you. After opening this file, you should see the data set open on your screen (see Figure 20.3b). This data set is comprised of ten items from the GMA test used in Chapter 16. Next click on the “Analysis” tab on the Command Menu at the top of the screen and then choose “Unidimensional Analysis.” Next, choose the “Items” tab in the middle of the screen. At this point you should see what appears in Figure 20.3c.

 Add all of the items from the list of variables on the column in the left to the “items” column on the right (use the shift key to easily move them over in one time). Click on the “models” tab and you can see that the 2PL model will be run for all ten items. 

Click “Run” and the program will initiate the parameter estimation. If your computer is fast, it should take less than one second before the output is completed (see Figure 20.3d). 

3. Examine the output file. There are a lot of interesting bits of information there. You can see the a and b parameters for each item. Beware, the column that has the c parameter should be ignored at this point. This program uses c to signify a different value than what we typically think of as the guessing parameter. That value, when you estimate the 3PL model, will be signified with the letter g for IRTPro. Notice which items are most discriminating (large values of a) and which items are relatively easy and which are relatively difficult. From this output file, if you click on the Analysis tab this time, you will see a Graphics command. Click on that and you will see the IRFs for each item (see Figure 20.3e). 

4. Now that you have estimated the 2PL model for these ten items, you can rerun the program using the 3PL model and the 1PL Rasch model. To rerun the 3PL model, go back to the models section and where you see 2PL for each item, right-click and you will be given the option to change to the 3PL model. Rerun the analysis and see how the parameter estimates change (noting that the g parameter in the output is what we normally refer to as the c parameter in the text). If you want to run the Rasch model, make sure the model chosen is the 2PL model and click the button “constraints.” From there, highlight all of the values in the a parameter column (i.e., selecting the a parameters for all ten items). Right-click over these highlighted a parameters and you will be given an open to “set parameters equal.” Click on that option and then you will be running the Rasch model. You can verify that you have run the Rasch model because in the output all of the a parameter estimates should be equal across items. 

Questions 

a. What are the basic descriptive statistics for the data? (N, average number correct, number of items, etc.) 

b. Which items were most and least discriminating and how did that change across models? 

c. If you had to choose just three items for a test that provided the most information at low ability, which would you choose? What about for high ability? 

d. Which of the three models seems to best fit the data? What did you base your answer on?

Figure 20.3

Transcribed Image Text:

(a) (c) Initial IRTPro Computer Programs Window IRTPRO KTPro Computer Program Window far Running the Analys (e) brigentel (b) IRTP Graphics for model pathyes (d) IKTP Data File for Chapter 20 datsgfile IRTPro Opt for 2P1. model www