Skip to content

Commit

Permalink
ipip-412: add fixtures and final editorials
Browse files Browse the repository at this point in the history
  • Loading branch information
lidel committed Aug 4, 2023
1 parent 1d0d1dc commit 0b1d0e2
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 31 deletions.
18 changes: 9 additions & 9 deletions src/http-gateways/trustless-gateway.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,15 +67,15 @@ Same as in :cite[path-gateway], but with limited number of supported response ty

### `Accept` (request header)

A Client SHOULD sent this HTTP header to leverage content type negotiation
A Client SHOULD send this HTTP header to leverage content type negotiation
based on section 12.5.1 of :cite[rfc9110].

Below response types MUST to be supported:
Below response types MUST be supported:

- [application/vnd.ipld.raw](https://www.iana.org/assignments/media-types/application/vnd.ipld.raw)
- A single, verifiable raw block to be returned.

Below response types SHOULD to be supported:
Below response types SHOULD be supported:

- [application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car)
- Disables IPLD/IPFS deserialization, requests a verifiable CAR stream to be
Expand All @@ -86,7 +86,7 @@ Below response types SHOULD to be supported:
- A verifiable :cite[ipns-record] (multicodec `0x0300`).

A Gateway SHOULD return HTTP 400 Bad Request when running in strict trustless
mode (no deserialized responses) and `Accept` header is missing.
mode (no deserialized responses) and `Accept` header is missing.

## Request Query Parameters

Expand Down Expand Up @@ -126,7 +126,7 @@ When the terminating entity at the end of the specified content path:
specified byte range of that entity.

- When dealing with a sharded UnixFS file (`dag-pb`, `0x70`) and a non-zero
`from` value, the UnixFS data and `blocksizes` determine the
`from` value, the UnixFS data and `blocksizes` determine the
corresponding starting block for a given `from` offset.

- cannot be interpreted as a continuous array of bytes (such as a DAG-CBOR/JSON
Expand Down Expand Up @@ -163,14 +163,14 @@ that includes enough blocks for the client to understand why the requested
returned:

- If the requested `entity-bytes` resolves to a range that partially falls
outside of the entity's byte range, the response MUST include the subset of
outside the entity's byte range, the response MUST include the subset of
blocks within the entity's bytes.
- This allows clients to request valid ranges of the entity without needing
to know its total size beforehand, and it does not require the Gateway to
buffer the entire entity before returning the response.

- If the requested `entity-bytes` resolves to a zero-length range or falls
fully outside of the entity's bytes, the response is equivalent to
fully outside the entity's bytes, the response is equivalent to
`dag-scope=block`.
- This allows client to produce a meaningful error (e.g, in case of UnixFS,
leverage `Data.blocksizes` information present in the root `dag-pb` block).
Expand Down Expand Up @@ -366,8 +366,8 @@ Gateway implementations SHOULD decide on the implicit default ordering or
other parameters, and use it in responses when client did not explicitly
specify any matching preference.

A Gateway MAY choose to implement only some of the parameters and return HTTP
400 Bad Request or 406 Not Acceptable when client requested a response with
A Gateway MAY choose to implement only some parameters and return HTTP
400 Bad Request or 406 Not Acceptable when a client requested a response with
unsupported content type variant.

A Client MUST verify `Content-Type` returned with CAR response before
Expand Down
42 changes: 20 additions & 22 deletions src/ipips/ipip-0412.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
---
title: "IPIP-0412: Signaling Block Order in CARs on HTTP Gateways"
date: 2023-05-15
ipip: proposal
ipip: ratified
editors:
- name: Marcin Rataj
github: lidel
url: https://lidel.org/
affiliation:
name: Protocol Labs
url: https://protocol.ai/
- name: Jorropo
github: Jorropo
affiliation:
name: Protocol Labs
url: https://protocol.ai/
relatedIssues:
- https://github.com/ipfs/specs/issues/348
- https://github.com/ipfs/specs/pull/330
Expand All @@ -29,7 +35,7 @@ We want to make it easier to build light-clients for IPFS. We want them to have
low memory footprints on arbitrary sized files. The main pain point preventing
this is the fact that CAR ordering isn't specified.

This require to keeping some kind of reference either on disk, or in memory to
This requires keeping some kind of reference either on disk, or in memory to
previously seen blocks for two reasons.

1. Blocks can arrive out of order, meaning when a block is consumed (data is
Expand All @@ -52,7 +58,7 @@ This IPIP aims to improve the status quo.
CAR content type
([`application/vnd.ipld.car`](https://www.iana.org/assignments/media-types/application/vnd.ipld.car))
already supports `version` parameter, which allows gateway to indicate which
CAR flavour is returned with the response.
CAR flavor is returned with the response.

The proposed solution introduces two new parameters for the content type headers
in HTTP requests and responses: `order` and `dups`.
Expand All @@ -61,7 +67,7 @@ The `order` parameter allows the client to indicate its preference for a
specific block order in the CAR response, and the `dups` parameter specifies
whether duplicate blocks are allowed in the response.

A Client SHOULD sent `Accept` HTTP header to leverage content type negotiation
A Client SHOULD send `Accept` HTTP header to leverage content type negotiation
based on section 12.5.1 of :cite[rfc9110] to get the preferred response type.

More details in Section 5. (CAR Responses) of :cite[trustless-gateway].
Expand Down Expand Up @@ -184,24 +190,16 @@ between clients and Trustless Gateways using various combinations of `order` and

Relevant tests were added to
[gateway-conformance](https://github.com/ipfs/gateway-conformance) test suite
in [#87](https://github.com/ipfs/gateway-conformance/pull/87).

<!-- TODO after conformance release is tagged
A detailed list of compliance checks for `order` and `dupes` can be found in
[`v0.TODO.0/trustless_gateway_car_test.go`](https://github.com/ipfs/gateway-conformance/blob/v0.TODO.0/tests/trustless_gateway_car_test.go) or later.
-->

Below are CIDs, CARs, and short summary of each fixture.

TODO:
1. a CAR with blocks for a small file in DFS order
2. a CAR with blocks for a small file with one block appearing twice

Tests for duplicates use a fixture where a directory contains two files that
are the same. If `dups=n`, then there are no duplicates. If `dups=y`, then the
blocks of the file are sent twice, by the order they show up in the DAG.

The same fixture is used for testing `order=dfs`.
in [#87](https://github.com/ipfs/gateway-conformance/pull/87), and include the below fixture.

- `bafybeihchr7vmgjaasntayyatmp5sv6xza57iy2h4xj7g46bpjij6yhrmy`
([CAR](https://github.com/ipfs/gateway-conformance/raw/v0.3.0/fixtures/trustless_gateway_car/dir-with-duplicate-files.car))
- An UnixFS directory with two files that are the same (same CID).
- If `dups=n`, then there should be no duplicate blocks in the returned CAR.
- If `dups=y`, then the blocks of the file are sent twice.
- The same fixture can be used for testing `order=dfs` and checking if blocks that belong to files arrive in the DFS order.
- It is encouraged to also test DFS order with HAMT fixture such as `bafybeidbclfqleg2uojchspzd4bob56dqetqjsj27gy2cq3klkkgxtpn4i`
([CAR](https://github.com/ipfs/gateway-conformance/raw/v0.3.0/fixtures/trustless_gateway_car/single-layer-hamt-with-multi-block-files.car))

### Copyright

Expand Down

0 comments on commit 0b1d0e2

Please sign in to comment.