diff --git a/src/http-gateways/trustless-gateway.md b/src/http-gateways/trustless-gateway.md index 5f9c5d141..949e2b0bf 100644 --- a/src/http-gateways/trustless-gateway.md +++ b/src/http-gateways/trustless-gateway.md @@ -67,15 +67,15 @@ Same as in :cite[path-gateway], but with limited number of supported response ty ### `Accept` (request header) -A Client SHOULD sent this HTTP header to leverage content type negotiation +A Client SHOULD send this HTTP header to leverage content type negotiation based on section 12.5.1 of :cite[rfc9110]. -Below response types MUST to be supported: +Below response types MUST be supported: - [application/vnd.ipld.raw](https://www.iana.org/assignments/media-types/application/vnd.ipld.raw) - A single, verifiable raw block to be returned. -Below response types SHOULD to be supported: +Below response types SHOULD be supported: - [application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car) - Disables IPLD/IPFS deserialization, requests a verifiable CAR stream to be @@ -86,7 +86,7 @@ Below response types SHOULD to be supported: - A verifiable :cite[ipns-record] (multicodec `0x0300`). A Gateway SHOULD return HTTP 400 Bad Request when running in strict trustless -mode (no deserialized responses) and `Accept` header is missing. +mode (no deserialized responses) and `Accept` header is missing. ## Request Query Parameters @@ -126,7 +126,7 @@ When the terminating entity at the end of the specified content path: specified byte range of that entity. - When dealing with a sharded UnixFS file (`dag-pb`, `0x70`) and a non-zero - `from` value, the UnixFS data and `blocksizes` determine the + `from` value, the UnixFS data and `blocksizes` determine the corresponding starting block for a given `from` offset. - cannot be interpreted as a continuous array of bytes (such as a DAG-CBOR/JSON @@ -163,14 +163,14 @@ that includes enough blocks for the client to understand why the requested returned: - If the requested `entity-bytes` resolves to a range that partially falls - outside of the entity's byte range, the response MUST include the subset of + outside the entity's byte range, the response MUST include the subset of blocks within the entity's bytes. - This allows clients to request valid ranges of the entity without needing to know its total size beforehand, and it does not require the Gateway to buffer the entire entity before returning the response. - If the requested `entity-bytes` resolves to a zero-length range or falls - fully outside of the entity's bytes, the response is equivalent to + fully outside the entity's bytes, the response is equivalent to `dag-scope=block`. - This allows client to produce a meaningful error (e.g, in case of UnixFS, leverage `Data.blocksizes` information present in the root `dag-pb` block). @@ -366,8 +366,8 @@ Gateway implementations SHOULD decide on the implicit default ordering or other parameters, and use it in responses when client did not explicitly specify any matching preference. -A Gateway MAY choose to implement only some of the parameters and return HTTP -400 Bad Request or 406 Not Acceptable when client requested a response with +A Gateway MAY choose to implement only some parameters and return HTTP +400 Bad Request or 406 Not Acceptable when a client requested a response with unsupported content type variant. A Client MUST verify `Content-Type` returned with CAR response before diff --git a/src/ipips/ipip-0412.md b/src/ipips/ipip-0412.md index b8bd5cc00..93f96314c 100644 --- a/src/ipips/ipip-0412.md +++ b/src/ipips/ipip-0412.md @@ -1,13 +1,19 @@ --- title: "IPIP-0412: Signaling Block Order in CARs on HTTP Gateways" date: 2023-05-15 -ipip: proposal +ipip: ratified editors: - name: Marcin Rataj github: lidel url: https://lidel.org/ + affiliation: + name: Protocol Labs + url: https://protocol.ai/ - name: Jorropo github: Jorropo + affiliation: + name: Protocol Labs + url: https://protocol.ai/ relatedIssues: - https://github.com/ipfs/specs/issues/348 - https://github.com/ipfs/specs/pull/330 @@ -29,7 +35,7 @@ We want to make it easier to build light-clients for IPFS. We want them to have low memory footprints on arbitrary sized files. The main pain point preventing this is the fact that CAR ordering isn't specified. -This require to keeping some kind of reference either on disk, or in memory to +This requires keeping some kind of reference either on disk, or in memory to previously seen blocks for two reasons. 1. Blocks can arrive out of order, meaning when a block is consumed (data is @@ -52,7 +58,7 @@ This IPIP aims to improve the status quo. CAR content type ([`application/vnd.ipld.car`](https://www.iana.org/assignments/media-types/application/vnd.ipld.car)) already supports `version` parameter, which allows gateway to indicate which -CAR flavour is returned with the response. +CAR flavor is returned with the response. The proposed solution introduces two new parameters for the content type headers in HTTP requests and responses: `order` and `dups`. @@ -61,7 +67,7 @@ The `order` parameter allows the client to indicate its preference for a specific block order in the CAR response, and the `dups` parameter specifies whether duplicate blocks are allowed in the response. -A Client SHOULD sent `Accept` HTTP header to leverage content type negotiation +A Client SHOULD send `Accept` HTTP header to leverage content type negotiation based on section 12.5.1 of :cite[rfc9110] to get the preferred response type. More details in Section 5. (CAR Responses) of :cite[trustless-gateway]. @@ -184,24 +190,16 @@ between clients and Trustless Gateways using various combinations of `order` and Relevant tests were added to [gateway-conformance](https://github.com/ipfs/gateway-conformance) test suite -in [#87](https://github.com/ipfs/gateway-conformance/pull/87). - - - -Below are CIDs, CARs, and short summary of each fixture. - -TODO: -1. a CAR with blocks for a small file in DFS order -2. a CAR with blocks for a small file with one block appearing twice - -Tests for duplicates use a fixture where a directory contains two files that -are the same. If `dups=n`, then there are no duplicates. If `dups=y`, then the -blocks of the file are sent twice, by the order they show up in the DAG. - -The same fixture is used for testing `order=dfs`. +in [#87](https://github.com/ipfs/gateway-conformance/pull/87), and include the below fixture. + +- `bafybeihchr7vmgjaasntayyatmp5sv6xza57iy2h4xj7g46bpjij6yhrmy` + ([CAR](https://github.com/ipfs/gateway-conformance/raw/v0.3.0/fixtures/trustless_gateway_car/dir-with-duplicate-files.car)) + - An UnixFS directory with two files that are the same (same CID). + - If `dups=n`, then there should be no duplicate blocks in the returned CAR. + - If `dups=y`, then the blocks of the file are sent twice. + - The same fixture can be used for testing `order=dfs` and checking if blocks that belong to files arrive in the DFS order. + - It is encouraged to also test DFS order with HAMT fixture such as `bafybeidbclfqleg2uojchspzd4bob56dqetqjsj27gy2cq3klkkgxtpn4i` + ([CAR](https://github.com/ipfs/gateway-conformance/raw/v0.3.0/fixtures/trustless_gateway_car/single-layer-hamt-with-multi-block-files.car)) ### Copyright