This post describes how to leverage Dataverse in order to upload large files from a Dynamics 365 form to a server
Note: the solution described below leverages the Dataverse File column to store the large file
Problem Description
Referring to Figure 1,
1.) A large file is placed on a Dynamics 365 form (assuming the internet browser used can handle the large file). This is achieved via a ‘file upload’ control on the form (it’s not described in this post)
2.) Within the form’s custom JavaScript, the file is converted into base64, placed in a JSON request and sent to the custom API (i.e. C# plugin class) UploadFileToExternalSystem (see appendix)
This works fine for files under 100MB. However the power platform rejects files over 100MB (well, it seems to be around this value) and generates the error message ‘The page was not displayed because the request entity is too large’.
The next section presents a solution for uploading files which are over 100MB
Solution Overview
Referring to Figure 2,
1.) A large file (over 100MB) is uploaded to a Dynamics 365 form
2.) Within the form’s custom JavaScript, a call is made to generate a record in a custom dataverse table ‘File Staging’. (This table contains a File column to temporarily store the large file)
3.) The record id of the record just created is retrieved
4.) Within the form’s custom JavaScript, a call is made to ‘uploadFile()’. This function stores the large file in the cpl_temporaryfile column of the ‘File Staging’ record
5.) Rather than the base64 file contents being placed in the JSON request, the id of the newly created ‘File Staging’ record is placed in the request
6.) Within the custom API ‘UploadFileToExternalSystem’ class, the C# function ‘DownloadFile()’ is called. This retrieves the file from the column cpl_temporaryfile in the ‘File Staging’ record
7.) Since the complete file is now available within the custom API, the file is deleted from dataverse (to save space)
Solution Detail
1.) Create the table called ‘File Staging’ in dataverse with a File column called ‘Temporary File’ to temporarily store the large file
In this example, the size of the File column is 500MB. This can’t be selected within the designer, so the following (unsupported) steps need to be taken. (the same can be achieved with a http Post, see Appendix)
- Create an unmanaged solution containing the newly created table and File column
- Export it as an unmanaged solution
- In Customizations.xml, update the maximum file size to 524288 Kb (500 x 1024 x 1024)
- In Solution.xml, update the solution version
- Re-import the unmanaged solution
2.) Create the uploadFile() TypeScript function (Figure 5). This function calls the actions IntializeFileBlocksUpload, UploadBlock & CommitFileBlockUpload
3.) Create the DownloadFile() C# function (Figure 6). This function calls the actions InitializeFileBlockDownload & DownloadBlock
4.) Create the DeleteFile() C# function. The code for this function is listed in Reference #2
For Part 2 Download large files via Dataverse
Appendix
The custom API UploadFileToExternalSystem is defined as follows
There are two input parameters
- Document Request
- Document Request Type
The ‘Document Request’ contains the following values (which are passed to the custom API as a JSON)
- relatedEntityName
- relatedEntityId
- files
- name
- extension
- size
- sizeInBytes
- mimeType
- lastModifiedDate
- securityClassification
- dcpGenerated
- fileStagingId
- documentGenerationMethod
- content
The Document Request Type can contain the following values
- UpdateDocument
- GetRelatedEntities
References
1.) https://learn.microsoft.com/en-us/power-apps/developer/data-platform/file-attributes?tabs=sdk
2.) https://learn.microsoft.com/en-us/power-apps/developer/data-platform/file-column-data?tabs=sdk