What is the Assessment Quality Dashboard in Insights?
The Assessment Quality Dashboard provides instructors, instructional designers, and others interested in understanding Quiz efficacy, a tool to identify poorly performing assessments and take informed action to iteratively improve on those assessments. It compiles Learners’ responses and provides metrics that highlight underperforming Quizzes and Quiz items.
The dashboard aims to answer the following questions:
- Which Quizzes or questions might need a review?
- What additional information is required to support interventions or iterations of assessments?
The goal is for instructors and instructional designers improving the quality of their assessments to support future use of their activities.
Why did we update the Assessment Quality Dashboard?
Initially released in 2019, the Assessment Quality Dashboard has received plenty of feedback on what is and isn’t working. The feedback focused on five major areas, and those became the goals of the project:
- Performance Improvement
- Increased Accessibility
- Extended Browser Support
- At-risk item identification
- Enhanced Filters
What can you expect in the December release?
With the December release (20.21.12,) the updated experience for the dashboard is available to turn on via configuration variable. The default value will be OFF, though administrators will be able to turn it ON once they’re ready. Please note that this experience will not have an opt-in experience, it will be either ON or OFF.
Upon the first load, the dashboard will display interactive cards, highlighting the number of quizzes included in the dashboard, Quizzes with low average scores, and the distribution of Quiz reliability. This is all followed by a table, listing all Quizzes and calculations available. The Org Unit filter will update which courses are included or excluded from all calculations and data displayed. Interacting with the cards will filter the data table, instantly highlighting quizzes that might need review.
Figure: An example of the Assessment Quality Dashboard.
The dashboard settings provide an area to define thresholds for poor performance as well as which columns should be included in the table, allowing each user to customize their view to their own preferences.
Quiz Item Analysis is available once an individual Quiz is selected from the data table. The new metrics provide details on the Quiz questions for the selected quiz. Again, the cards at the top of the dashboard are interactive, filtering the data table below to reflect the card selected. Thresholds for each indicator can be customized per user in the settings area of the dashboard, just like on the main page. Quiz questions performing below the thresholds set will be flagged, with the low-performing metric highlighted in the table.
Figure: Quiz Item Analysis in the Assessment Quality Dashboard.
Selecting the Quiz item shows the prompt for an item, as well as answer options, feedback, and hints. Editing the Quiz or Quiz questions will require navigation to the Quiz tool.
Figure: An example of the quiz question preview.
What is next?
Continued updates to this experience, most notably the ability to aggregate and surface flagged items on the dashboard for immediate review. Keep an eye on the 90-day preview and release notes for updates on when these features will be released in the product.