Skip to content

Latest commit

 

History

History
484 lines (378 loc) · 20.3 KB

http-headers.adoc

File metadata and controls

484 lines (378 loc) · 20.3 KB
Raw

REST Basics - HTTP headers

In this section we explain a handful of (standard) HTTP headers, which we found most useful in certain situations and that require additional explanation to be used effectively.

Note: we generally discourage the usage of proprietary headers. However, they are sometimes useful to pass generic, service independent, overarching information relevant for our specific application architecture. We consistently define these proprietary headers below.

Whether a service supports a certain header is part of the service contract. Therefore APIs should use the parameters and headers definition of the API endpoint and response to clarify what is supported in a certain context.

Using Standard Header definitions

Usually, you can the standard HTTP request and response header definition provided by the guideline to simplify API by using well recognized patterns. The best practice importing headers providing the highest readability is as follows:

path:
  '/resource'
    get:
      parameters:
        - $ref: '#/components/parameters/ETag

components:
  parameters|headers:
    ETag:
      $ref: 'https://opensource.zalando.com/restful-api-guidelines/models/headers-1.0.0.yaml#/ETag'

  responses:
    Default:
      headers:
        ETag:
          $ref: '#/components/(parameters|headers)/ETag'

Note: It is a question of taste whether headers for responses are defined in #/components/headers or #/components/parameters. Unfortunately, headers in the first section cannot be referenced from a parameters-list, while it is possible to reference the second also from a headers-list.

{MAY} use standard headers

APIs may make use of HTTP headers defined by non-obsolete RFCs (see following list of standard HTTP headers). The supported headers must be explicitly mentioned in the API specification.

{SHOULD} use kebab-case with uppercase separate words for HTTP headers

This convention is followed by (most of) the standard headers e.g. as defined in {RFC-2616}[RFC 2616], {RFC-4229}[RFC 4229], {RFC-9110}[RFC 9110]. Examples:

If-Modified-Since
Accept-Encoding
Content-ID
Language

Note: The HTTP standard defines headers (now called HTTP fields) to be case-insensitive (see {RFC-9110}#section-5.1[RFC 9110 Section 5.1]). However, for the sake of readability and consistency, you should follow the convention when defining standard or proprietary headers. Exceptions are common abbreviations like ID.

{MUST} use Content-* headers correctly

Content or entity headers are headers with a Content- prefix. They describe the content of the body of the message and they can be used in both, HTTP requests and responses. Commonly used content headers include but are not limited to:

  • {Content-Disposition} can indicate that the representation is supposed to be saved as a file, and the proposed file name.

  • {Content-Encoding} indicates compression or encryption algorithms applied to the content.

  • {Content-Length} indicates the length of the content (in bytes).

  • {Content-Language} indicates that the body is meant for people literate in some human language(s).

  • {Content-Location} indicates where the body can be found otherwise ({MAY} use Content-Location header for more details]).

  • {Content-Range} is used in responses to range requests to indicate which part of the requested resource representation is delivered with the body.

  • {Content-Type} indicates the media type of the body content.

{SHOULD} use Location header instead of Content-Location header

As a correct usage of the {Content-Location} response header (see {MAY} use Content-Location header) with respect to caching and its method specific semantics is difficult, we discourage the use of {Content-Location}. In most cases it is sufficient to inform clients about the resource location in create or re-direct responses by using the {Location} header while avoiding the {Content-Location} specific ambiguities and complexities.

For more details see {RFC-9110}#section-10.2.2[RFC 9110 Section 10.2.2 Location], {RFC-9110}#section-8.7[RFC 9110 Section 8.2 Content-Location]

{MAY} use Content-Location header

The {Content-Location} response header is an optional header that can be used in successful write operations ({PUT}, {POST}, or {PATCH}) and read operations ({GET}, {HEAD}) to guide caching and signal a receiver the actual location of the resource transmitted in the response body. This allows clients to identify the resource and to update their local copy when receiving a response with this header.

The {Content-Location} header can be used to support the following use cases:

  • For reading operations {GET} and {HEAD}, a different location than the requested URL can be used to indicate that the returned resource is subject to content negotiations, and that the value provides a more specific identifier of the resource.

  • For writing operations {PUT} and {PATCH}, an identical location to the requested URL can be used to explicitly indicate that the returned resource is the current representation of the newly created or updated resource.

  • For writing operations {POST} and {DELETE}, a content location can be used to indicate that the body contains a status report resource in response to the requested action, which is available at provided location.

Note: When using the {Content-Location} header, the {Content-Type} header has to be set as well. For example:

GET /products/123/images HTTP/1.1

HTTP/1.1 200 OK
Content-Type: image/png
Content-Location: /products/123/images?format=raw

See also [227].

{MAY} consider to support Prefer header to handle processing preferences

The {Prefer} header defined in {RFC-7240}[RFC 7240] allows clients to request processing behaviors from servers. It pre-defines a number of preferences and is extensible, to allow others to be defined. Support for the {Prefer} header is entirely optional and at the discretion of API designers, but as an existing Internet Standard, is recommended over defining proprietary "X-" headers for processing directives.

The {Prefer} header can be defined in the API specification as follows:

link:../includes/prefer.yaml[role=include]

Note: Please, copy only the behaviors into your {Prefer} header specification that are supported by your API endpoint. If necessary, specify different {Prefer} headers for each supported use case.

Supporting APIs may return the {Preference-Applied} header also defined in {RFC-7240}[RFC 7240] to indicate whether a preference has been applied.

{MAY} consider to support ETag together with If-Match/If-None-Match header

When creating or updating resources it may be necessary to expose conflicts and to prevent the lost update or initially created problem. Following {RFC-9110}[RFC 9110 Section 13 "Conditional Requests"] this can be best accomplished by supporting the {ETag} header together with the {If-Match} or {If-None-Match} conditional header. The contents of an ETag: <entity-tag> header is either (a) a hash of the response body, (b) a hash of the last modified field of the entity, or (c) a version number or identifier of the entity version.

To expose conflicts between concurrent update operations via {PUT}, {POST}, or {PATCH}, the If-Match: <entity-tag> header can be used to force the server to check whether the version of the updated entity is conforming to the requested {entity-tag}. If no matching entity is found, the operation is supposed a to respond with status code {412} - precondition failed.

Beside other use cases, If-None-Match: * can be used in a similar way to expose conflicts in resource creation. If any matching entity is found, the operation is supposed a to respond with status code {412} - precondition failed.

The {ETag}, {If-Match}, and {If-None-Match} headers can be defined as follows in the API specification (see also the default definition below):

components:
  parameters|headers:
    ETag:
      $ref: 'https://opensource.zalando.com/restful-api-guidelines/models/headers-1.0.0.yaml#/ETag'
    If-Match:
      $ref: 'https://opensource.zalando.com/restful-api-guidelines/models/headers-1.0.0.yaml#/If-Match'
    If-None-Match:
      $ref: 'https://opensource.zalando.com/restful-api-guidelines/models/headers-1.0.0.yaml#/If-None-Match'
link:../includes/etag.yaml[role=include]
link:../includes/if-match.yaml[role=include]
link:../includes/if-none-match.yaml[role=include]

Please see [optimistic-locking] for a detailed discussion as well as [cache-support-patterns] for additional use cases.

{MAY} consider to support Idempotency-Key header

When creating or updating resources it can be helpful or necessary to ensure a strong [idempotent] behavior comprising same responses, to prevent duplicate execution in case of retries after timeout and network outages. Generally, this can be achieved by sending a client specific unique request key – that is not part of the resource – via {Idempotency-Key} header.

The unique request key is stored temporarily, e.g. for 24 hours, together with the response and the request hash (optionally) of the first request in a key cache, regardless of whether it succeeded or failed. The service can now look up the unique request key in the key cache and serve the response from the key cache, instead of re-executing the request, to ensure [idempotent] behavior. Optionally, it can check the request hash for consistency before serving the response. If the key is not in the key store, the request is executed as usual and the response is stored in the key cache.

This allows clients to safely retry requests after timeouts, network outages, etc. while receive the same response multiple times. Note: The request retry in this context requires to send the exact same request, i.e. updates of the request that would change the result are off-limits. The request hash in the key cache can protection against this misbehavior. The service is recommended to reject such a request using status code {400}.

Important: To grant a reliable [idempotent] execution semantic, the resource and the key cache have to be updated with hard transaction semantics – considering all potential pitfalls of failures, timeouts, and concurrent requests in a distributed systems. This makes a correct implementation exceeding the local context very hard.

The {Idempotency-Key} header must be defined as follows, but you are free to choose your expiration time:

link:../includes/idempotency-key.yaml[role=include]

If you do not want to change the expiration time, you can also use the standard definition provided by the guideline:

components:
  parameters|headers:
    Idempotency-Key:
      $ref: 'https://opensource.zalando.com/restful-api-guidelines/models/headers-1.0.0.yaml#/Idempotency-Key'

Hint: The key cache is not intended as request log, and therefore should have a limited lifetime, else it could easily exceed the data resource in size.

Note: The {Idempotency-Key} header unlike other headers in this section is not standardized in an RFC. Our only reference are the usage in the Stripe API. However, we do not want to change the header name and semantic, and do not name it like the proprietary headers below. The header addresses a generic REST concern and is different from the Zalando landscape specific proprietary headers.

{SHOULD} use only the specified proprietary Zalando headers

As a general rule, proprietary HTTP headers should be avoided. In addition from a conceptual point of view, the business semantics and intent of an operation should always be expressed via the path and query parameters, the method, and the content, but not via proprietary headers.

Headers are typically used to implement protocol processing aspects, such as flow control, content negotiation, and authentication, and represent business agnostic request modifiers that provide generic context information ({RFC-9110}#section-10[RFC 9110 Section 10 "Message Context"]).

However, the exceptional usage of proprietary headers is still helpful when domain-specific generic context information

  1. needs to be passed end-to-end along the service call chain (even if not all called services use it as input for steering service behavior, e.g. {X-Sales-Channel} header), and/or

  2. is provided by specific gateway components, for instance, our Fashion Shop API or Merchant API gateway.

Below, we explicitly define the list of proprietary headers usable for all services for passing through generic context information of our fashion domain (use case 1).

Per convention, non standardized, proprietary header names are prefixed with X- and use the dash (-) as separator (dash-case). (Due to backward compatibility, we do not follow the recommendation of Internet Engineering Task Force in {RFC-6648}[RFC 6648] to deprecate the usage of `X-`headers.) Remember that HTTP header field names are not case-sensitive:

Header field name Type Description Header field value example

{X-Flow-ID}

String

For more information see {MUST} support {X-Flow-ID}.

GKY7oDhpSiKY_gAAAABZ_A

{X-Tenant-ID}

String

Identifies the tenant initiated the request to the multi tenant Zalando Platform. The {X-Tenant-ID} must be set according to the Business Partner ID extracted from the OAuth token when a request from a Business Partner hits the Zalando Platform.

9f8b3ca3-4be5-436c-a847-9cd55460c495

{X-Sales-Channel}

String

Sales channels are owned by retailers and represent a specific consumer segment being addressed with a specific product assortment that is offered via CFA retailer catalogs to consumers (see {glossary}[fashion platform glossary (internal link)]).

52b96501-0f8d-43e7-82aa-8a96fab134d7

{X-Frontend-Type}

String

Consumer facing applications (CFAs) provide business experience to their customers via different frontend application types, for instance, mobile app or browser. Info should be passed-through as generic aspect — there are diverse concerns, e.g. pushing mobiles with specific coupons, that make use of it. Current range is mobile-app, browser, facebook-app, chat-app, email.

mobile-app

{X-Device-Type}

String

There are also use cases for steering customer experience (incl. features and content) depending on device type. Via this header info should be passed-through as generic aspect. Current range is smartphone, tablet, desktop, other.

tablet

{X-Device-OS}

String

On top of device type above, we even want to differ between device platform, e.g. smartphone Android vs. iOS. Via this header info should be passed-through as generic aspect. Current range is iOS, Android, Windows, Linux, MacOS.

Android

{X-Mobile-Advertising-Id}

String

It is either the IDFA (Apple Identifier for mobile Advertising) for iOS, or the GAID (Google mobile Advertising Identifier) for Android. It is a unique, customer-resettable identifier provided by mobile device’s operating system to facilitate personalized advertising, and usually passed by mobile apps via HTTP header when calling backend services. Called services should be ready to pass this parameter through when calling other services. It is not sent if the customer disables it in the settings for respective mobile platform.

b89fadce-1f42-46aa-9c83-b7bc49e76e1f

Exception: The only exception to this guideline are the conventional hop-by-hop X-RateLimit- headers which can be used as defined in [153].

As part of the guidelines we provide the default definition of all proprietary headers, so you can simply reference them when defining the API endpoint. For details see Using Standard Header definitions.

Hint: This guideline does not standardize proprietary headers for our specific gateway components (2. use case above). This include, for instance, non pass-through headers X-Zalando-Customer, X-Zalando-Client-ID, X-Zalando-Request-Host, X-Zalando-Request-URI defined by Fashion Shop API (RKeep), or X-Consumer, X-Consumer-Signature, X-Consumer-Key-ID defined by Merchant API gateway. All these proprietary headers are allowlisted in the API Linter (Zally) checking this rule.

{MUST} propagate proprietary headers

All Zalando’s proprietary headers defined in {SHOULD} use only the specified proprietary Zalando headers are end-to-end headers [1]) defines two types of headers: end-to-end and hop-by-hop headers. End-to-end headers must be transmitted to the ultimate recipient of a request or response. Hop-by-hop headers, on the contrary, are meaningful for a single connection only.] and must be propagated to the services down the call chain. The header names and values must remain unchanged.

For example, the values of the custom headers like {X-Device-Type} can affect the results of queries by using device type information to influence recommendation results. Besides, the values of the custom headers can influence the results of the queries (e.g. the device type information influences the recommendation results).

Sometimes the value of a proprietary header will be used as part of the entity in a subsequent request. In such cases, the proprietary headers must still be propagated as headers with the subsequent request, despite the duplication of information.

{MUST} support {X-Flow-ID}

The {Flow-ID} is a generic parameter to be passed through service APIs and events and written into log files and traces. A consequent usage of the {Flow-ID} facilitates the tracking of call flows through our system and allows the correlation of service activities initiated by a specific call. This is extremely helpful for operational troubleshooting and log analysis. Main use case of {Flow-ID} is to track service calls of our SaaS fashion commerce platform and initiated internal processing flows (executed synchronously via APIs or asynchronously via published events).

Data Definition

The {Flow-ID} must be passed through:

The following formats are allowed:

  • UUID ({RFC-4122}[RFC-4122])

  • base64 ({RFC-4648}[RFC-4648])

  • base64url ({RFC-4648}#section-5[RFC-4648 Section 5])

  • Random unique string restricted to the character set [a-zA-Z0-9/+_-=] maximal of 128 characters.

Note: If a legacy subsystem can only process Flow-IDs with a specific format or length, it must define this restriction in its API specification, and be generous and remove invalid characters or cut the length to the supported limit.

Hint: In case distributed tracing is supported by {SRE-open-tracing}[OpenTracing (internal link)] you should ensure that created spans are tagged using flow_id — see {SRE-open-tracing}/blob/master/wg-semantic-conventions/best-practices/flowid.md[How to Connect Log Output with OpenTracing Using Flow-IDs (internal link)] or {SRE-open-tracing}/blob/master/wg-semantic-conventions/best-practices.md[Best practices (internal link)].

Service Guidance

  • Services must support {Flow-ID} as generic input, i.e.

    • RESTful API endpoints must support {X-Flow-ID} header in requests

    • Event listeners must support the metadata flow-id from events.

Note: API-Clients must provide {Flow-ID} when calling a service or producing events. If no {Flow-ID} is provided in a request or event, the service must create a new {Flow-ID}.

  • Services must propagate {Flow-ID}, i.e. use {Flow-ID} received with API calls or consumed events as…​

    • input for all API called and events published during processing

    • data field written for logging and tracing

Hint: This rule also applies to application internal interfaces and events not published via Nakadi (but e.g. via AWS SQS, Kinesis or service specific DB solutions).

1. HTTP/1.1 standard ({RFC-9110}#section-7.6.1[RFC 9110 Section 7.6.1