Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor current implementation of storage jobs GetOutput #31

Open
jwreagor opened this issue Jul 27, 2017 · 0 comments
Open

Refactor current implementation of storage jobs GetOutput #31

jwreagor opened this issue Jul 27, 2017 · 0 comments
Labels
Milestone

Comments

@jwreagor
Copy link
Contributor

Currently, our implementation utilizing Manta's GetJobOutput() only returns a result set size header. The response is a stream to accommodate tail -f behavior and it doesn't look like we're returning the path to the Manta Job result object.

To illustrate, here's an actual request/response which includes the result object path...

GET /testacct/jobs/aabbccdd-0859-ccc2-d267-a2b4a0258eac/live/out HTTP/1.1
Host: us-east.manta.joyent.com
User-Agent: manta-go client API
Accept: */*
Authorization: Signature ...
Date: Thu, 27 Jul 2017 17:33:02 UTC
Accept-Encoding: gzip
Connection: close
HTTP/1.1 200 OK
Content-Type: text/plain
Date: Thu, 27 Jul 2017 17:33:02 GMT
Server: Manta
x-request-id: 142c4d40-7231-1de7-8bcf-2536c4d80c1b
x-response-time: 126
x-server-name: 3d2bcd91-5dd9-4e23-83a5-794144eab9fd
Transfer-Encoding: chunked
Connection: Close

/testacct/jobs/aabbccdd-0859-ccc2-d267-a2b4a0258eac/stor/reduce.1.359fdfda-53f1-40df-84d7-46b5040ea48a

From the above example, storage.GetJobOutputOutput{} (ripe for a rename) should include a field called ObjectPath which holds the value of "/testacct/jobs/aabbccdd-0859-ccc2-d267-a2b4a0258eac/stor/reduce.1.359fdfda-53f1-40df-84d7-46b5040ea48a".

Also, if we want to return the object, we could fire a subsequent Objects.Get and attach the returned storage.GetObjectOutput{} which contains an ObjectReader to the actual results. It might make sense to do this on behalf of the user at the cost of a second round trip (but that's debatable).

@jwreagor jwreagor added the manta label Dec 1, 2017
@jwreagor jwreagor added this to the 1.x milestone Jan 9, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant