Product News Product blogs

Course Learning Outcome Evaluation (CLOE) Walkthrough

Updated August 13th, 2019

The Course Learning Outcome Evaluation (CLOE) data export is one of the Advanced Data Sets which is now available in the Data Hub. This data export provides learner performance data for any assessment activity which is aligned to a learning objective with a defined achievement threshold. This is very useful data when you are interested in evaluating and tracking learner performance at the course level or at aggregated levels. However, many people have had questions about how to configure the assessment activities to trigger that data to be included in the export. The video talks about the requirements for CLOE, some special considerations for using the data export and the process to associate learning activities with learning objectives at the course offering level.

In addition to the video, we also have an Excel pivot table template available to help you get started with your analysis of this data (attached below). To use the template, simply run your CLOE data export from the Data Hub and download the completed export file. Cut and paste the completed export (CSV) file contents into the Paste Data tab of the Excel spreadsheet. Then recalculate the pivot tables and view the results. There are step-by-step instructions for using the template on the first tab of the spreadsheet. In addition to the Excel pivot table template, a document of special considerations and a recommended competency structure diagram are provided.

Figure: Simple Competency Org Structure Diagram.

CLOE Template Usage Considerations

When running the CLOE data set, you can enter any parent org unit ID value or a specific course offering org unit ID. For example, you could enter a course template org unit ID to get all course offerings under a template. Or you could
enter a department org unit ID or a semester org unit ID to get all courses in that department or that semester.
The CLOE data export will provide achievement of learning objectives from the course template org unit level along
with evaluation results of assessment activities contained within the course offering.
Rubrics should be created within the course offering for rubric evaluation data to be included.
A grade item must be attached to the assessment activity for the grade data columns to be populated.
Competency status must be Approved for achievement data to be tracked and reported.
Points and custom points rubrics can be reversed in order – text only rubrics cannot
Rubrics should be associated on the Objectives tab of the learning activity (quiz, assignment or discussion), not on the Properties tab unless you use more than one rubric. If you use a separate rubric for grading versus outcomes assessment, then the rubric should be on the Properties tab, and the outcomes assessment rubric should be on the Objectives tab.
All activities associated to the learning objective need to complete before it will show on the data export.
Tip: Learning objectives and activities should have a one to one relationship.
All learners must have an assigned Org Defined ID.
Numerically graded items must have assigned Grade Points and Max Grade Points values.
If the activity has not been evaluated for a learner, then the Activity Grade % value must be blank.
After the data is pasted in the working copy of the file, the pivot tables and charts can be filtered to exclude blanks
(and optionally #N/A values). Don't do this in the template as it will exclude data the next time the template is used.
The Evaluations by Value chart and the Evaluations by Activity by Value display the same data in two different views. Evaluations by Value shows the student count as a stacked bar chart whereas the Evaluations by Activity by Value shows the achievement count as a clustered vertical histogram.
The Achievement by Outcome—Count and Achievement by Outcome—Percentage also show the same data set, just presented in two slightly different views. The first shows the count of students who have achieved and not achieved each outcome, and the second shows the percentage of students who have achieved and not achieved each outcome.
The configuration variable d2l.Tools.AdvancedDataSets.ProgramOrgUnitTypeId must be configured in your learning environment.
Review the Brightspace Analytics—Administrator Guide in the Brightspace Community for additional information about setting up and granting permissions for Data Hub and CLOE.
If you need to clear out the data file to use it with a different data set, do not clear the rows. There are calculated fields in the far right-hand columns. Clear cells A2:AK430.
When pasting data, do not paste entire rows or entire columns. Pasting entire rows will wipe out the calculated fields. Pasting entire columns will remove the column headers. Paste data starting in the cell A2.