NSFS | Add checks for request/response aborted in namespace_fs read_object_stream()

[romayalon]

Signed-off-by: Romy [email protected]

Explain the changes

1. Object_sdk - Added an abort controller property and added 2 functions:
   1.1. setup_abort_controller() - setup event listeners on the req/res that will call abort() of the abort_controller.
   1.2. throw_if_aborted() - will check if the abort_controller got the aborted signal and will throw error when true.
2. S3_get_object -
   2.1. Called setup_abort_controller() in order to hang the event listeners on each get_object request/response.
   2.2. Removed the previous event listeners code.
3. Namespace FS -
   3.1. Added a call to object_sdk.throw_if_aborted() after each await in read_object_stream(). (see Note)
   3.2. Passed object_sdk.abort_controller.signal parameter to wait_finished() and to wait_drain().

NOTE -
The whole purpose of this new abort_controller functionality was to fix the following situation -

1. A Client sends s3_get_object request, NooBaa is writing a buffer to the response, and the client reads it very slowly.
2. NooBaa in the meantime reads the next data from the fs and holds a new buffer, but NooBaa is not yet writing it to the socket and is awaiting for wait_drain() (because the client reads the data very slowly).
3. The user calls abort on the request, and the first buffer is released.
4. NooBaa still awaits on wait_drain() and therefore still holding the second buffer.

Issues: Fixed #xxx / Gap #xxx

1. Fixed #HPO 659: Mix read/write with md5 checking halts all I/O on all metalLB IP's and does not recover unless noobaa pod restart #6934 hopefully

Testing Instructions:

• Doc added/updated
• Tests added

[guymguym]

another round of comments. apologies for making it iterative.

[guymguym]

So I just noticed that the aborted event is being deprecated since nodejs 16 (we are still on 14).
https://nodejs.org/dist/latest-v16.x/docs/api/http.html#event-aborted
And the docs recommend listening for the close event. LOL.

[guymguym]

Seems like we need to do this:

        req.once('close', () => {
            if (req.destroyed) {
                dbg.log0('request aborted', req.url);
                this.abort_controller.abort(new Error('request aborted ' + req.url));
            }
        });

[guymguym]

Ok, so this is a gap for moving to node 16, because on node 14 we can't check the req.destroyed state yet.

FYI @liranmauda and @nimrod-becker - we need to track this so that once we switch to node 16 we make sure to include this fix, which @romayalon left as a comment in the code.

[guymguym]

@romayalon I resolved the previous comments, but the deprecation of the abort event on the request is probably best to fix...

[guymguym]

Great work @romayalon. Super artistic and accurate as usual! Godspeed.

[romayalon]

@guymguym Thank you for your great help and profound code review!