Upload and Download Enhancements

There is a new method in place for processing file uploads, and then serving those files to you when viewing the download link. This new method should vastly improve performance for the end user, as well as reduce server resources greatly.

Here is an explanation of how we used to do it and how we are doing it now.

Old method

Upload

  • File data is sent to our server as a FileStream
  • Entire file is read into a byte array in memory
  • The byte array is then scanned via clamAV
  • If the file is to be encrypted server side, then the byte array is passed into the encryption method and a new byte array is returned
  • The byte array is then written to a file

Download

  • Read the file into a byte array in memory
  • If the file was encrypted and we have the key, then the byte array is passed into the decryption method and a new byte array is returned
  • The byte array is then passed as a buffered response to the client

New method

Upload

  • File data is sent to our server as a FileStream
  • The file stream is then scanned via clamAV byte by byte
  • If the file is to be encrypted server side, the stream is then encrypted in byte buffers (4kb buffer) and then directly written to the file after each buffer is processed
  • If the file isn't to be encrypted, then the stream is written directly to the file

Download

  • A FileStream is created for the local file copy
  • If the upload was encrypted, then the stream is decrypted in byte buffers (4kb buffer) and then directly written to the response output stream
  • If the upload wasn't encrypted, the FileStream is sent directly to the client in a buffered response

What does this mean?

For uploads, instead of creating multiple copies of the file in byte arrays, the file is streamed directly to disk, encrypting it along the way if necessary. For downloads, the file is streamed directly to the client, decrypting it if necessary.