Uploading large files via Dataverse – Part 1/2

Introduction

Figure 1

Referring to Figure 1,

1.) A large file is placed on a Dynamics 365 form (assuming the internet browser used can handle the large file)

2.) Within the form’s custom JavaScript, the file is converted into base64, placed in a JSON request and sent to the custom API UploadFileToExternalSystem()

This works fine for files under 100MB. However the power platform rejects files over 100MB (well, it seems to be around this value) and generates the error message ‘The page was not displayed because the request entity is too large’.

The next section presents a solution for uploading files which are over 100MB

Solution Overview

Figure 2

Referring to Figure 2,

1.) A large file (over 100MB) is uploaded to a Dynamics 365 form

2.) Within the form’s custom JavaScript, a call is made to generate a record in a custom dataverse table ‘File Staging’. (This table contains a File column to temporarily store the large file)

3.) The record id of the record just created is retrieved

4.) Within the form’s custom JavaScript, a call is made to ‘uploadFile()’. This function stores the large file in the cpl_temporaryfile column of the ‘File Staging’ record

5.) Rather than the base64 file contents being placed in the JSON request, the id of the newly created ‘File Staging’ record is placed in the request

6.) Within the custom API ‘UploadFileToExternalSystem()’, the C# function ‘DownloadFile()’ is called. This retrieves the file from the column cpl_temporaryfile in the ‘File Staging’ record

7.) Since the complete file is now available within the custom API, the file is deleted from dataverse (to save space)

Solution Detail

1.) Create the table called ‘File Staging’ in dataverse with a File column called ‘Temporary File’ to temporarily store the large file

Figure 3

In this example, the size of the File column is 500MB. This can’t be selected within the designer, so the following (unsupported) steps need to be taken. (the same can be achieved with a http Post, see Apendix)

  • Create an unmanaged solution containing the newly created table and File column
  • Export it as an unmanaged solution
  • In Customizations.xml, update the maximum file size to 524288 Kb (500 x 1024 x 1024)
  • In Solution.xml, update the solution version
  • Re-import the unmanaged solution
Figure 4

2.) Create the uploadFile() TypeScript function. This function calls the actions IntializeFileBlocksUpload, UploadBlock & CommitFileBlockUpload

Figure 5

3.) Create the DownloadFile() C# function. This function calls the actions InitializeFileBlockDownload & DownloadBlock

Figure 6

4.) Create the DeleteFile() C# function. The code for this function is listed in Reference #2

For Part 2 https://dustinminer.com/2024/05/27/downloading-large-files-via-dataverse-part-2-2/

Appendix

Figure 7

References

1.) https://learn.microsoft.com/en-us/power-apps/developer/data-platform/file-attributes?tabs=sdk

2.) https://learn.microsoft.com/en-us/power-apps/developer/data-platform/file-column-data?tabs=sdk

3.) https://learn.microsoft.com/en-us/power-apps/developer/data-platform/webapi/samples/file-operations