The plug-in interface for IBulkFileRetriever is used by BulkManager to support the retrieval of bulk files on remote servers.
The interface implementation is D2L.IM.IPSIS.Bulk.Handlers.IBulkFileRetrieverHandler.
The following configuration list provides a starting point for any IPSIS implementation:
IPSIS, Template - Bulk File Retriever
- BulkFileRetrieverSetLocalPathToD2LTempHandler (SortOrder = 10)
- BulkFileRetrieverUriHandler (SortOrder = 20)
BulkFileRetrieverSetLocalPathToD2LTempHandler
Sets the local path to a unique directory in instance temp directory using the original filename from the request.
Implementation
D2L.IM.IPSIS.Default.Bulk.Handlers.BulkFileRetrieverSetLocalPathToD2LTempHandler
Expected behavior
The handler sets the local save path to {D2L instance temp directory}\{generated unique directory}\{filename defined in remotefileurl}.
Error behavior
If the local path does not exist, the handler throws an exception and inserts an error log into the database.
Configuration note
Since this handler sets the download path, it should be placed before a download handler (e.g. BulkFileRetrieverUriHandler or BulkFileRetrieverHttpsGetAcceptAllCertsHandler).
BulkFileRetrieverUriHandler
This handler downloads files with URI's starting with http:, https:, ftp:, and file:.
Implementation
D2L.IM.IPSIS.Default.Bulk.Handlers.BulkFileRetrieverUriHandler
Expected behavior
The handler performs the following tasks:
- Updates the status for the IBulkFileUrl to downloading.
- Sets the username and password if one is passed for the URI.
- Creates an URI connection that supports ftp, http, and https and file. This behavior assumes there is no authentication required for the connection if none is passed in the request. If a username and password are supplied in the request (they're populated by the BulkFileRetrieverRetrieveAuthenticationCredientialsHandler), the system uses BasicAuth to connect to the endpoint.
- Downloads the file.
- Updates the status for the IBulkFileUrl to downloaded.
Error behavior
- If the local path does not exist, a Path does not exist exception is thrown and the handler inserts an error log into the logging database.
- If an invalid username and password combination is received, an authentication exception is thrown and the handler inserts an error log into the logging database.
- If the URL is unreachable, a URI format exception is thrown and the handler inserts an error log into the logging database.
- If a timeout occurs, a timeout exception is thrown and the handler inserts an error log into the logging database.
Configuration notes
- The BulkFileRetrieverUriHandler downloads the files, and as such should be placed after any handlers which configure the outgoing request (ex. adding credentials, setting the download directory, etc).
- This handler should not be combined with any other handler responsible for file download (BulkFileRetrieverHttpsGetAcceptAllCertsHandler).
BulkFileRetrieverHttpsGetAcceptAllCertsHandler
This handler downloads files that have a URI with a prefix of https, http, ftp, and file. If it is using SSL, it ignores all certificate errors. This is useful for things like self-signed certificates, that might only be used internally or in development environments.
Implementation
D2L.IM.IPSIS.Default.Bulk.Handlers.BulkFileRetrieverHttpsGetAcceptAllCertsHandler
Expected behavior
This is the concrete class that implements IBulkFileRetrieverHandler. The handler performs the following tasks:
- The handler receives a IBulkFileRetriever request which contains a BulkFileUrl.
- Updates the IBulkFileUrl status to Downloading.
- This behavior assumes there is no authentication required for the connection if none is passed in the request.
- Opens a https connection to the remote URL located in the IBulkFileUrl object and performs a GET request.
- Downloads the file and saves it to the local save path.
- Updates the file status to Downloaded.
Error behavior
- If an invalid username and password combination is passed, an authentication exception is thrown and the handler inserts an error log into the logging database.
- If the URL is unreachable, the handler throws an authentication exception and inserts an error log into the logging database.
- If a timeout occurs, the handler throws a timeout exception and inserts an error log into the logging database.
Configuration notes
- This handler actually downloads the files, and as such should be placed after any handlers which configure the outgoing request (i.e. adding credentials, setting the download directory, etc.).
- This should not be combined with any other handler responsible for file download (BulkFileRetrieverHttpsGetAcceptAllCertsHandler).
BulkFileRetrieverConfigVariableDefinedDownloadPathHandler
Sets the local path to a directory defined in the Config Variable Browser using the original filename from the request.
Implementation
D2L.IM.IPSIS.Default.Bulk.Handlers.BulkFileRetrieverConfigVariableDefinedDownloadPathHandler
Expected behavior
- If the config variable is defined, the handler sets the local save path to {d2l.Integrations.IPSIS>LocalBulkFileDownloadPath}\{SystemId}.{TransactionId}\{filename defined in remotefileurl}.
- The transactionId is stripped of any characters that aren't allowed in paths, and all '\' .
- If the path does not exist, the system tries to create it first.
Error behavior
- If the config variable is not defined, an exception is thrown.
- Exceptions could also be thrown if, for example, the drive is invalid in the config variable.
Configuration note
First handler in the list replacing alternatives to defining the location for the file destination.
BulkFileRetrieverRetrieveAuthenticationCredientialsHandler
Sets the username and password on the request based on the file URL and information in the endpoints table.
Implementation
D2L.IM.IPSIS.Default.Bulk.Handlers.BulkFileRetrieverRetrieveAuthenticationCredientialsHandler
Expected behavior
The handler performs the following tasks:
- Grabs a list of endpoints marked under the type BulkFileDownload from the endpoint table.
- If the remote file URL begins with any of those endpoints (e.g. "http://mysite.com/cookies/fun.xml" starts with "http://mysite.com"), the handler grabs the credentials from that endpoint entry, and applies them to the request.
- If the remote url matches no endpoints the handler does nothing.
Known limitations
Because of schema constraints, the system currently supports only one BulkFileDownload endpoint per source system.
Error behavior
If the handler finds more than one endpoint which matches the remote url, it throws a BulkProcessingFailedException with a descriptive error message (including the file name that failed).
Configuration note
This handler should be placed before any handler which actually downloads the file.