FilesAPI.upload_bytes very slow

[thomafred]

Is your feature request related to a problem? Please describe.
The file upload function FilesAPI.upload_bytes is very slow. Currently uploading 10 files concurrently, but upload speed no more than 4 MB/s (my internet-connection supports 10 MB/s).

The function FilesAPI.upload is however quite fast.

Describe the solution you'd like
The library should be able to parallelize file upload similar to the blob-client in the Azure SDK.

Describe alternatives you've considered
Attempted to parallelize using the Multiprocessing-library:

from multiprocessing import Pool

NUM_WORKERS = 10

# {.... }

def file_upload_cb(file_name):
  global cdf_client

  with open(file_name, 'rb') as f:
    cdf_client.files.upload_bytes(f, external_id=fname) # Use file-name as external ID for demo-purpose

with Pool(NUM_WORKERS) as pool:
  pool.map(file_upload_cb, list_of_files)

Additional context

In the picture below, the blue line is the download speed from Azure Blob-storage, the red line is the total upload-speed for 10 concurrent file-uploads using the Cognite FileApi.

[thomafred]

Seems that it is the function FilesAPI.upload_bytes which is slow. The function FilesAPI.upload is sufficiently fast. I have updated the issue to reflect this

[haakonvt]

@thomafred Istill observe this issue, but only on windows. What OS, SDK version and python version are you using?