For this assignment, you'll be provided with details about how this a Dean's position review is would be performed and weigh in on what types of reliability and validity evidence you would want to collect about the process to ensure its accuracy. We'll focus on exploring the process being used and how we would go about determining that it is an accurate process $($or not$).$

Background information:

Job details of the typical administrator position:

K$-$State's Provost's $($a position above the Dean and under the President of the University$)$ statement on expected performance areas for Deans:

"Most fundamentally, I expect a dean or other administrative leader to:

$(1)$ lead their college or unit on a

positive trajectory,

$(2)$ provide leadership at the university level,

$(3)$ demonstrate administrative

effectiveness, and

$(4)$ provide a strong example of positive, ethical leadership through outstanding

personal qualities."

Raters:

The Dean puts together a document on the things s$/$he has done in the past $5$ years $($or however long since the last review$).$ It includes written summaries and data to demonstrate outcomes such as his$/$her own research scholarship, the research productivity and awards of all faculty in the College, number of undergraduate and graduate students in the College and major awards they have won, committees the Dean has served on$,$ other examples of leadership, diversity and inclusion efforts, and his plan for the next $5$ years $($or other interval if the evaluation is positive$).$

Faculty $-$ faculty can complete an anonymous survey to rate the Dean on various performance dimensions and provide written feedback. There are options to not rate or click "unable to observe" on the survey

Staff $-$ similar to the faculty survey, staff can also rate the Dean on various areas of performance and provide written feedback.

Panel review $-$ a panel $($of mostly department heads$)$ would be organized to oversee the review, compile input, and make a recommendation to the Provost regarding the Dean's level of performance. Part of the recommendation includes whether to retain the Dean or not! $($talk about high stakes$)$

Independent of the data from these various sources, please provide your input on what reliability and validity evidence would be important to look at to ensure the accuracy of the approach above.

What types of reliability evidence should be collected? How could that data be collected and explored? What would you want to find if the ratings are reliable?

Hint, there are four possible types of reliability

How would you go about collecting content validity evidence to ensure the various rating forms are collecting the appropriate information? What are $3$ things the performance ratings should measure based on the information above on what Dean performance should look like?

What type of convergent validity evidence should be collected? That is$,$ what types of objective measures could be collected to show the performance ratings are related to other indicators of performance?

What is one thing you would change, remove, or add about the process to try to make it more reliable and$/$or more valid? Why?

Finally, based on the information about this approach, would you recommend a similar approach to evaluate upper$-$level leadership or C$-$Suite leadership $($CEO$,$ CFO, etc.$)$ at your organization? Why or why not?