Can multiple stream uploads be active at the same time?

[Niproblema]

For example; there is a huge file upload that is going slowly in the first stream. Does that mean that all other streams, which might be significantly faster and more important, have to wait for the first one to finish, or may other streams be processed in parallel?

I am not sure if I am looking at the correct thing, but this excerpt might indicate that due to write lock, only one stream might be active at the time in either direction. Is this correct?

        private async Task<bool> SendInternalAsync(WatsonMessage msg, long contentLength, Stream stream, CancellationToken token)
        {
            if (msg == null) throw new ArgumentNullException(nameof(msg));
            if (!Connected) return false;
            
            if (contentLength > 0  && (stream == null || !stream.CanRead))
            {
                throw new ArgumentException("Cannot read from supplied stream.");
            }

            if (token == default(CancellationToken))
            {
                CancellationTokenSource linkedCts = CancellationTokenSource.CreateLinkedTokenSource(token, _Token);
                token = linkedCts.Token;
            }

            bool disconnectDetected = false;
             
            if (_Client == null || !_Client.Connected)
            {
                disconnectDetected = true;
                return false;
            }
                 
            await _WriteLock.WaitAsync(token).ConfigureAwait(false);

            try
            { 
                await SendHeadersAsync(msg, token).ConfigureAwait(false); 
                await SendDataStreamAsync(contentLength, stream, token).ConfigureAwait(false);

                _Statistics.IncrementSentMessages();
                _Statistics.AddSentBytes(contentLength);
                return true;
            }
            catch (TaskCanceledException)
            {
                return false;
            }
            catch (OperationCanceledException)
            {
                return false;
            }
            catch (Exception e)
            {
                _Settings.Logger?.Invoke(Severity.Error,
                    _Header + "failed to write message to " + _ServerIp + ":" + _ServerPort + ":" +
                    Environment.NewLine +
                    e.ToString() +
                    Environment.NewLine);

                disconnectDetected = true;
                return false;
            }
            finally
            {
                _WriteLock.Release(); 

                if (disconnectDetected)
                {
                    Connected = false;
                    Dispose();
                }
            }
        }

[jchristn]

Correct, you don't want multiple concurrent writers to a stream. The data would then be interleaved. You would want to use multiple separate connections.

[Niproblema]

This makes perfect sense, but it would be super useful for me. For example, I have multiple network streams, users uploading their data, which is proxied to dedicated service. Without concurrent streams a single user might block upload for everyone else indefinitely also.
I don't expect you to fix this, since it's a single client after all, but do you have a suggestion how I could solve that?

[Niproblema]

@jchristn Don't worry I won't bother you anymore, but can you check my last question :)

[jchristn]

Hi @Niproblema I will try and spend some time investigating this today. Cheers

[Niproblema]

@jchristn Thanks, I really appreciate it! Let me elaborate,
I use

        private static MemoryStream GetStream(long mb)
        {
            MemoryStream memoryStream = new MemoryStream();
            memoryStream.Seek(mb * 1024 * 1024, SeekOrigin.Begin);
            memoryStream.WriteByte(0);
            memoryStream.Position = 0;
            return memoryStream;
        }

to simulate a large file, then i stream it from client to server or vice versa (fails in both cases). As soon as the recipient tries to read the stream by args.DataStream.CopyToAsync(Stream, CancellationToken).ConfigureAwait(false); the client calls disonnect and the copy fails due to stream closing. Let me know if you can replicate.

[jchristn]

Having a hard time reproducing this... Here is a test program I wrote which sets the max proxied stream size to 32. A client is instantiated which sends 10 messages, where each message is i * 10 bytes in length, so after the third iteration, the underlying data stream is passed. Gist

And the resultant output:

Could you provide an example of this so I can reproduce?

[Niproblema]

Hi, sorry for the delay. Eventually I managed to transfer streams over 10GB. Thanks !