Preventing Data Set files from being zipped?

Options
Jeremy.N.518
Jeremy.N.518 Posts: 12 ๐ŸŒฑ

My organization has blocked .zip files for security purposes. However, it seems as if all reports run from the Data Hub are only exported as .zip files.

Is there any way, such as a configuration setting, that would allow me to export the BDS/ADS reports as native .csv files instead of .zip files?

Best Answer

  • Jeffrey.M.844
    Jeffrey.M.844 Posts: 67
    edited July 2023 Answer โœ“
    Options

    Hi @Jeremy.N.518,

    How's it going!? I hope all is well ๐Ÿ™‚

    Unfortunately โ€” as you pointed out โ€” all downloads from Data Hub are zip, and the zip package contains a csv file. There aren't any configuration variables or settings that you can use (currently) to change that behavior.

    One thing I wanted to point out here, although it may not help you much with your particular situation, is that data sets can also be retrieved through our open API. If you're interested in learning more about the API, we have a Developer Platform and entire section dedicated to Data Hub and Data Export Framework.

    Having said that, our API routes retrieve the same zip packages, so this won't necessarily help in and of itself.

    What you may want to do is follow up with your IT team and see if the policy can be updated to make either of the following exceptions

    a) allow downloading of zips from a particular source

    b) allow a particular machine to download zips

    if a, you could then retrieve only zips from Brightspace. if b, you could task a designated machine (manually or through API) with pulling the data sets using the API and then build out some connection points to that machine for retrieving the unzipped csv files that were gathered.

    Finally, although raised for slightly different reasons, I would recommend you (and others interested in new functionality to support csv download) upvote the following feature request in the Product Ideas Exchange (PIE): Advanced Dataset to return JSON or unzipped CSV (D8158). Perhaps an extension of that idea or a new idea all on its own โ€” feel free to steal it and post it as yours โ€” would be to have the data hub offer an 'manage files' option where the file gets places in Brightspace and you would unzip from within the application (then download). Again, to be clear, that last thought is not a current feature, but rather an idea/concept for how we might be able offer alternative download options through further development.

    I hope that helps, and do post back here if you end up logging a new PIE item in case other community members wish to upvote it!

    Jeffrey

Answers

  • Renee.J.194
    Renee.J.194 Posts: 114 โœจ
    Options

    We use a middleware, called Jitterbit to

    1. Handle the API calls to retrieve the Brightspace datasets.
    2. Unzip the files.
    3. Upsert them to the appropriate tables in our institutional data lake.

    This has served us well for the most part.