Dataset extracting issue in ADF - status code 400 Bad Request
I built a pipeline in Azure data factory to download bright space datasets in batches.
Some of the datasets extracting gave me HTTP 400 Bad request error.
What confused me is some of the datasets extracts worked, some didn't.
The download link URL is like this:
https://XXX.brightspace.com/d2l/api/lp/1.45/datasets/bds/SchemaId/plugins/PluginId/extracts/extractID
I have noticed that when the extract ID are numbers, the pipeline works fine. I am able to download these datasets. Example:
https://XXX.brightspace.com/d2l/api/lp/1.45/datasets/bds/dcef8789-1e6b-4d98-ac76-588583b3ba30/plugins/6d6cd2d8-c714-41fd-9465-f797dfd69c76/extracts/1699858856
When the extract ID contains letters, the process always fails. Example:
https://XXX.brightspace.com/d2l/api/lp/1.45/datasets/bds/435ee960-871f-484f-8e66-44886dea08f8/plugins/d18ed567-e0a3-4fb7-912f-84d294620830/extracts/MTY5OTg1OTEwMA
The error message is:
"Http request failed with client error, status code 400 BadRequest, please check your activity settings."
I couldn't figure out how to solve this. It makes no sense because if the url is problematic, the whole pipeline should fail. But it wasn't the case.
Did anyone experience the same?
Any help is appreciated.
BTW, what is the meaning/difference between extract IDs that consist numbers or letters? (Just try to understand the logic )
Thanks,
Vikki
Antwoorden
-
I'm having the exact same problem - was a fix found for this?
-
I'm experiencing the same issue. Has anyone found a solution yet?
-
Hi all
I was sweeping through Community and found this question wasn't answered. I attempted a test and wasn't able to reproduce the issue you described. I suspect that, judging by the version number included and the age of the original post, an invalid version number (too high) was used. I've seen cases of higher-than-advertised version numbers used successfully and unsuccessfully in the past. I've had poor results trying to stay ahead of the latest version numbers, myself, and don't prefer relying on them. If in fact that's the issue, I would instead recommend pulling the version numbers for LE and LP API calls as a regular part of your code and storing the results in global variables.
Presuming you're using a valid/current API version number, here's how I tested BDS downloads via API:
- Start with creating an App ID/KEY pair in Brightspace (this approach is required for https://apitesttool.desire2learnvalence.com/). If this is already created then just log into your site.
- In a separate browser tab, log into APITestTool and enter the site URL (excluding "https://"), and the App ID/KEY values.
- Click "Authenticate" so that your User ID/KEY can be obtained by APITestTool.
- With "GET" selected below, enter this API Route to obtain a SchemaId and PluginId: "/d2l/api/lp/1.49/datasets/bds/" (obtain the current version number from https://docs.valence.desire2learn.com/res/dataExport.html). Click "Submit".
- Choose your SchemaId and PluginId from the ouput you received, insert them into this API Route: "/d2l/api/lp/(version)/datasets/bds/(schemaId)/plugins/(pluginId)/extracts", and click "Submit" again.
- From the output above, I get a DownloadLink containing letters in the ExtractId. When I cut-and-paste that link to a new browser tab I receive the respective download (zipped).
If I alter the ExtractId so that it's 10 characters (shorter than expected) or just numeric then I get a 404 Not Found message instead of a 400 Bad Request. Usually a "Bad Request" is caused by incorrect formatting of the data (JSON) block, however this particular route doesn't require one.
Hopefully that's helpful information for future readers of this thread!
Best regards,
Matt
Learning Administration Manager
