Understanding Student Ratings
This introduction to the student ratings tool is to help faculty and administrators capitalize on its features to promote student learning and evaluate faculty progress in achieving this aim.
Watch this video for detailed instructions on how to understand student ratings. Additionally, there are links to certain video segments with their associated topics provided below the video.
The context for the video is found at the bottom of this page.
Understanding and interpreting uncertainty bands
45:04 Response pattern affects shape of uncertainty band (opens in a new tab)
59:33 How are we to view the uncertainty band; what are it’s components? (opens in a new tab)
1:12:56 What are the sources of variability in the uncertainty band? (opens in a new tab)
An index to the contents of the video recording of the presentation and discussion
0:00 Brent Webb provides background on the work of the task force (opens in a new tab)
1:35 Issues around previous instrument that gave rise to new survey (opens in a new tab)
3:11 Survey overview (Burlinggame) (opens in a new tab)
4:42 7-year timeline overview (opens in a new tab)
6:04 Task force charge (opens in a new tab)
7:51 How the old system guided formation of the new (opens in a new tab)
9:43 Issues addressed by the task force I (opens in a new tab)
12:33 Issues addressed by the task force II (opens in a new tab)
13:02 Sample” instructor” item from the New Instrument (opens in a new tab)
14:17 Sample “AIMs” item from the New Instrument (opens in a new tab)
14:51 Improvements in Reporting (opens in a new tab)
15:18 Sample semester report for a class (opens in a new tab)
18:15 “AIMs items” discussed (opens in a new tab)
19:55 Questions (opens in a new tab)
19:58 What does the composite score “cover”? (opens in a new tab)
21:58 What does the composite score for course “cover”? (opens in a new tab)
23:28 What does the composite score represent? (opens in a new tab)
24:44 How do we deal with the reliability band; is there any guidance on that? (opens in a new tab)
28:35 How do we “frame” and “sell” this to our students correctly? (opens in a new tab)
29:25 Why not make survey participation required of students? (opens in a new tab)
34:36 Two slides to provide insight into how “trends” might appear (opens in a new tab)
32:54 (this is partly in response to the comment and expression of concern). (opens in a new tab)
34:54 Multi-year report: Positive Change (opens in a new tab)
35:14 Multi-year report: Negative Change (opens in a new tab)
35:40 Questions addressed by W-2015 pilot (Reese) (opens in a new tab)
- Can there be a single reliable composite score?
- How does the new composite score relate to the old global instructor item?
40:53 Summary of what the pilot looked like (participation and response rate) (opens in a new tab)
41:50 Can 5-items be summed into a composite? (slide) (opens in a new tab)
42:02 What is the estimated reliability of the composite? (slide) (opens in a new tab)
43:07 Relationship of Items/Composite to old global (opens in a new tab)
43:18 Comment from Brent Webb about course size (opens in a new tab)
44:14 Comment about reliability improving with aggregation over time (opens in a new tab)
45:04 Response pattern affects shape of uncertainty band (opens in a new tab)
46:28 Summary recommendations (future) (opens in a new tab)
47:20 Questions (opens in a new tab)
51:42 When will faculty receive training on this? (opens in a new tab)
59:33 How are we to view the uncertainty band; what are it’s components? (opens in a new tab)
1:03:47 Will there eventually be a “trend” component built into the instrument? (opens in a new tab)
1:08:56 Comment/caution about how the new instrument is presented (opens in a new tab)
1:10:44 Question about how the grades component of the new survey is derived (opens in a new tab)
1:12:56 What are the sources of variability in the uncertainty band? (opens in a new tab)
1:18:16 What do we do about inappropriate or even profane student comments? (opens in a new tab)
1:25:21 Brent Webb provides concluding comments (opens in a new tab)
On September 18, 2015, department chairs were given a preview of the new student ratings tool. Two members of the Student Ratings Task Force, Professors Gary Burlingame (Psychology) and Shane Reese (Statistics), as well as members of the Academic Vice President’s Council, presented an overview of the development process, discussed the new instrument, and answered questions. The recording has been edited to reduce the nearly two-hour meeting to a reasonable length for viewing.