By Megan Walker, Product Manager, Data and Analytics, Product Management D2L
Originally published 26 June 2018
Description of the Change
In 10.7.11 (1804), we released the Daily Differentials in Brightspace Data Sets (BDS). Previously in Data Hub when you selected a BDS, you were only able to download the full data set that included data from all time or in the case of data heavy data sets like Session History, the last 2-3 years. Now, you have the option of downloading that same full extract or a daily differential which contains all the inserts and updates made to your data (aka the differences) since the last successful run which occurs every 24 hours.
Until October 1st, you will continue to be able to access refreshed full and differential extracts daily if you have had Data Hub and the Brightspace Data Sets since before 10.7.11. After Oct 1st, all clients will receive a weekly full and daily differential extract. The only exception is for those who have purchased the hourly differential add-on who will see daily full extracts and hourly differential extracts.
Why did we make this change?
- Differentials have a faster run time which allows data sets to be generated in shorter times. This makes smaller files available to you sooner. With differentials there is no need to truncate and reload the complete data sets daily. This results in faster processing time for you to ingest the data.
When to use a Full extract?
- When a data set includes soft deletes, there is no need to load the full data set in to resolve deletions. A soft-delete is defined as information that has been removed from the Brightspace Application and is no longer available in the UI but tracked in the database. For data sets without soft delete information, a full extract will be required to understand deletions. Consult the Analytics Admin Guide to learn with data sets include soft deletes. You can also learn more about soft deletes here.
Who is impacted by the change?
- All clients who currently have Datahub and the Brightspace Data Sets will be impacted by this release.
When will the change take effect?
- Daily differentials are already available but fulls will be generated weekly opposed to daily as of Oct 1, 2018.
Do I have the option of opting out of this change?
- The data set generation will be updated across the board for all clients. There isn't an opt-out for this change.
What should I consider at my end to ensure adoption of these changes?
- Please review any queries you have that are written...
How can I provide feedback associated with this change?
- Please provide any ongoing feedback as a response to this blog post.