Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

disableCompression: Expose configuration to toggle Envoy GZIP compression on the responses #6546

Open
wants to merge 35 commits into
base: main
Choose a base branch
from

Conversation

chaosbox
Copy link

@chaosbox chaosbox commented Jul 9, 2024

For #6511

This PR adds,

  • Listener configuration that exposes a boolean flag to disable compression, by default compression is enabled. This also provides us a way to disable if the users prefer to trade network for CPU, especially when teams want to run lean Envoy instances and rely on horizontal scalability.
  • We will run the test build for a while in our cluster to show the actual cost benefit.

Related #310, there had been mentions about disabling compression, the ticket we had raised shows the reason where disabling compression can bring cost benefits.

@chaosbox chaosbox requested a review from a team as a code owner July 9, 2024 09:04
@chaosbox chaosbox requested review from skriss and sunjayBhatia and removed request for a team July 9, 2024 09:04
@sunjayBhatia sunjayBhatia requested review from a team, rajatvig and davinci26 and removed request for a team July 9, 2024 09:18
Copy link

github-actions bot commented Jul 9, 2024

Hi @chaosbox! Welcome to our community and thank you for opening your first Pull Request. Someone will review it soon. Thank you for committing to making Contour better. You can also join us on our mailing list and in our channel in the Kubernetes Slack Workspace

dependabot bot and others added 2 commits July 9, 2024 10:42
…jectcontour#6543)

Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3.3.0 to 3.4.0.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](docker/setup-buildx-action@d70bba7...4fd8129)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: chaosbox <[email protected]>
Copy link

The Contour project currently lacks enough contributors to adequately respond to all PRs.

This bot triages PRs according to the following rules:

  • After 14d of inactivity, lifecycle/stale is applied
  • After 30d of inactivity since lifecycle/stale was applied, the PR is closed

You can:

  • Ensure your PR is passing all CI checks. PRs that are fully green are more likely to be reviewed. If you are having trouble with CI checks, reach out to the #contour channel in the Kubernetes Slack workspace.
  • Mark this PR as fresh by commenting or pushing a commit
  • Close this PR
  • Offer to help out with triage

Please send feedback to the #contour channel in the Kubernetes Slack

@github-actions github-actions bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 25, 2024
@chaosbox
Copy link
Author

Ping

@github-actions github-actions bot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jul 29, 2024
@tsaarni tsaarni added the release-note/small A small change that needs one line of explanation in the release notes. label Aug 2, 2024
# Conflicts:
#	.github/workflows/build_main.yaml
#	.github/workflows/build_tag.yaml
#	.github/workflows/prbuild.yaml
Copy link
Member

@tsaarni tsaarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've included suggestions inline for some linter nags (link).

Please add the new disableCompression option in "Configuration File" chapter in site/content/docs/main/configuration.md.

Please add also a changelog file.

internal/featuretests/v3/compression_test.go Outdated Show resolved Hide resolved
internal/featuretests/v3/compression_test.go Outdated Show resolved Hide resolved
test/e2e/httpproxy/envoy_compression_test.go Outdated Show resolved Hide resolved
test/e2e/httpproxy/envoy_compression_test.go Outdated Show resolved Hide resolved
test/e2e/httpproxy/envoy_compression_test.go Outdated Show resolved Hide resolved
internal/envoy/v3/listener.go Show resolved Hide resolved
@geomacy
Copy link

geomacy commented Aug 19, 2024

Thanks very much for the feedback @tsaarni 🙇 we'll have a look at this and try to get back with updates as soon as possible.

@geomacy
Copy link

geomacy commented Aug 21, 2024

Many thanks for taking the time to provide the lint fixes. We'll update the docs and add the changelog as soon as possible.

Copy link

codecov bot commented Aug 21, 2024

Codecov Report

Attention: Patch coverage is 70.12987% with 23 lines in your changes missing coverage. Please review.

Project coverage is 80.97%. Comparing base (54ceade) to head (45ea946).
Report is 58 commits behind head on main.

Files with missing lines Patch % Lines
cmd/contour/servecontext.go 12.50% 13 Missing and 1 partial ⚠️
pkg/config/parameters.go 0.00% 7 Missing and 1 partial ⚠️
cmd/contour/serve.go 50.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #6546      +/-   ##
==========================================
- Coverage   81.76%   80.97%   -0.79%     
==========================================
  Files         133      133              
  Lines       15944    20052    +4108     
==========================================
+ Hits        13037    16238    +3201     
- Misses       2614     3519     +905     
- Partials      293      295       +2     
Files with missing lines Coverage Δ
internal/envoy/v3/listener.go 98.54% <100.00%> (+0.08%) ⬆️
internal/xdscache/v3/listener.go 92.05% <100.00%> (-0.07%) ⬇️
cmd/contour/serve.go 21.81% <50.00%> (-0.86%) ⬇️
pkg/config/parameters.go 86.00% <0.00%> (-2.03%) ⬇️
cmd/contour/servecontext.go 82.05% <12.50%> (-3.90%) ⬇️

... and 122 files with indirect coverage changes

---- 🚨 Try these New Features:

@geomacy geomacy force-pushed the main branch 3 times, most recently from 619ac16 to 16f72d7 Compare August 22, 2024 16:38
Geoff Macartney and others added 5 commits August 22, 2024 17:46
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Copy link
Contributor

@davinci26 davinci26 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I have one api comment that I left open but I think the code overall lgtm and useful.

@@ -0,0 +1 @@
Add "compression" flag to set/disable the compression type applied in the default HTTP filter chain.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might need update as well based on the new api

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

included in 5dcdc33

Geoff Macartney added 3 commits October 23, 2024 17:59
Signed-off-by: Geoff Macartney <[email protected]>
Avoid NPE in parameters.Validate

also tidy flag handling in registerServe, don't set compression struct unless parameter is supplied.

Signed-off-by: Geoff Macartney <[email protected]>
@geomacy
Copy link

geomacy commented Oct 25, 2024

Have added one more fix of a test, but I still need to finish testing on-cluster, will update the ticket when I get that completed.

Geoff Macartney added 5 commits October 28, 2024 10:10
Signed-off-by: Geoff Macartney <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
@geomacy
Copy link

geomacy commented Oct 29, 2024

Hello @davinci26 @tsaarni, I have updated the code per the proposal to support better API extensibility by wrapping the settings in a struct. I believe this PR is ready for review. Some results below from testing on a cluster:

default

out of the box behaviour

curl -v -H "Accept-Encoding: gzip,deflate"  $URL | gzip -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:36:04 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: gzip
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

response was gzip.

parameter disabled

added to container args:

        - --compression=disabled

curl:

$ curl -v -H "Accept-Encoding: gzip,deflate"  $URL
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:40:55 GMT
< content-length: 353
< x-envoy-upstream-service-time: 1
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

No compression applied, it has been disabled.

parameter gzip

set container args to have

        - --compression=gzip

curl

$ curl -v -H "Accept-Encoding: gzip,deflate"  $URL | gzip -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:44:05 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: gzip
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

parameter brotli

set container args to have

        - --compression=brotli

curl

$ curl -v -H "Accept-Encoding: br,deflate"  $URL | brotli -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:46:35 GMT
< x-envoy-upstream-service-time: 2
< content-encoding: br
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

brotli encoding applied

parameter zstd

set container args with

        - --compression=zstd

curl

$ curl -v -H "Accept-Encoding: zstd,deflate"  $URL | zstd -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:49:23 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: zstd
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

zstd encoding applied

config disabled

Remove compression flag from container args and set config map to have:

apiVersion: v1
data:
  contour.yaml: |-
    compression:
      algorithm: disabled
    ...

Restart contour pods. Do curl requesting gzip

$ curl -v -H "Accept-Encoding: gzip,deflate"  $URL
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:54:23 GMT
< content-length: 353
< x-envoy-upstream-service-time: 1
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",
  "version": "6.0.0",

compression is disabled.

config gzip

edit config

apiVersion: v1
data:
  contour.yaml: |-
    compression:
      algorithm: gzip
    ...

Restart contour and curl:

$ curl -v -H "Accept-Encoding: gzip,deflate"  $URL | gzip -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:56:37 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: gzip
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

response encoded with gzip

config brotli

edit config to

apiVersion: v1
data:
  contour.yaml: |-
    compression:
      algorithm: brotli

Restart contour and curl:

$ curl -v -H "Accept-Encoding: br,deflate"  $URL | brotli -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 09:59:48 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: br
< vary: Accept-Encoding
< server: envoy
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

config zstd

edit config to

apiVersion: v1
data:
  contour.yaml: |-
    compression:
      algorithm: zstd

curl:

$ curl -v -H "Accept-Encoding: zstd,deflate"  $URL | zstd -d
...
< HTTP/2 200
< content-type: application/json; charset=utf-8
< x-content-type-options: nosniff
< date: Tue, 29 Oct 2024 10:29:20 GMT
< x-envoy-upstream-service-time: 1
< content-encoding: zstd
< vary: Accept-Encoding
< server: envoy
<
{
  "hostname": "podinfo-5bd5b49f6d-n847k",

zstd compression applied.

@geomacy
Copy link

geomacy commented Nov 5, 2024

Hello @tsaarni, @davinci26, would you be able to review this? Thanks.

@davinci26
Copy link
Contributor

@geomacy will take a look later today.

@geomacy
Copy link

geomacy commented Nov 13, 2024

hi @davinci26 @tsaarni did you get a chance to look at this?

Copy link
Member

@tsaarni tsaarni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the long delay! I've finally had a chance to review and added comments inline.

The PR title and description are slightly outdated after the latest changes, and could use an update.

Thank you for your efforts and patience!

serve.Flag("accesslog-format", "Format for Envoy access logs.").PlaceHolder("<envoy|json>").StringVar((*string)(&ctx.Config.AccessLogFormat))

serve.Flag("compression", "Set or disable compression type in default Listener filters.").PlaceHolder("<gzip|brotli|zstd|disabled>").StringVar((*string)(&ctx.Config.Compression.Algorithm))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove the command line option and make configuration available only through config file and ContourConfiguration CRD.

The command line options currently are a bit of a mess, so we previously agreed to avoid introducing new options there unless absolutely necessary.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See 95d38af

Comment on lines 32 to 39
func (a CompressionAlgorithm) Validate() error {
switch a {
case BrotliCompression, DisabledCompression, GzipCompression, ZstdCompression:
return nil
default:
return fmt.Errorf("invalid compression type: %q", a)
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be cleaner to treat an empty string as a valid value here, rather than completely bypassing validation in Parameters.Validate()? While algorithm is the only field currently, that might change.

Because +optionalstring in Go becomes an empty string when user did not set the field, it might be best to consider it as valid algorithm name in the Go side (CRD validation for the field can still reject it). It is already taken into account elsewhere in this change: inhttpConnectionManagerBuilder.DefaultFilters() you've added switch-default branch which will use gzip for empty algorithm string.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See 366f48d

internal/featuretests/v3/compression_test.go Outdated Show resolved Hide resolved
"application/grpc-web-text", "application/grpc-web-text+proto", "application/grpc-web-text+thrift",
var compressor proto.Message = &envoy_compression_gzip_compressor_v3.Gzip{}
compressorName := string(contour_v1alpha1.GzipCompression)
if b.compression != nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems that DefaultFilters() function used to set up a set of fixed filters - I don't see any of the other builder method impacting the default set except the new Compression() method, which now also makes it important in which order the builder methods are called - Now the configuration would be different depending if Compression() is called before or after DefaultFilters().

Maybe it would be better to (1) remove compressor from the default set completely and only add it when the new Compression() builder method is called? Or maybe some other alternative: (2) instantiate gzip by default and modify it later according to chosen algorithm when Get() is called or (3) delay adding compressor until Get() but add it by default at that point.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's true that the way the code is at present means the order does matter. At the same time the intent of the PR is to allow a way of changing the compression algorithm in the default filter chain (as opposed to something more generic), so it kind of makes sense that DefaultFilters() no longer sets up a fixed set. Maybe Compression() is a bad name for that method, perhaps it should be something like OverrideDefaultCompression(), and documented more clearly to explain the behaviour including how it needs to be called before DefaultFilters() if it is to do anything.

I've been trying to avoid changing the code in any way that would risk changing behaviour for users who don't opt in to the new method. At a basic level this certainly means we can't change the signatures say of HTTPConnectionManagerBuilder() or DefaultFilters() itself, but I would also want to avoid changing the behaviour of DefaultFilters() in the absence of using the new compression method. So in thinking about these options:

About 1. Remove compressor from the default set entirely.
I wouldn't be keen to do this as the behaviour of adding compression is effectively already part of the contract of what DefaultFilters() does. Changing this could change the behaviour of downstream users' code.

Option 3 is similar, in that you're changing the order of filters (currently default filters put compression filter first). Less dramatic than option 1 but still a change in behaviour.

Option 2 might be possible - I guess Get() could scan the filter chain looking for one named "envoy.filters.http.compressor" (value of CompressorFilterName as set in DefaultFilters() at present) to update. Would that be good enough? Can it update the first such filter it finds, on the assumption that there will be only one?

I'd like to be sure about any changes before I make them here, but my feeling is that perhaps no change is needed - other than renaming the Compression() func and providing better documentation. What do you think?

| debug | boolean | `false` | Enables debug logging. |
| default-http-versions | string array | <code style="white-space:nowrap">HTTP/1.1</code> <br> <code style="white-space:nowrap">HTTP/2</code> | This array specifies the HTTP versions that Contour should program Envoy to serve. HTTP versions are specified as strings of the form "HTTP/x", where "x" represents the version number. |
| disableAllowChunkedLength | boolean | `false` | If this field is true, Contour will disable the RFC-compliant Envoy behavior to strip the `Content-Length` header if `Transfer-Encoding: chunked` is also set. This is an emergency off-switch to revert back to Envoy's default behavior in case of failures. |
| compression | string | `gzip` | Sets the compression type applied in the compression HTTP filter of the default Listener filters. Setting this to `disabled` will make Envoy skip "Accept-Encoding: gzip,deflate" request header and always return uncompressed response. Values:`gzip` (default), `brotli`, `zstd`, `disabled`. |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compression here is still string while it should be CompressionAlgorithm. The new type needs a new table with algorithm field with the content described here.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see 1a6465f

changelogs/unreleased/6546-chaosbox-small.md Outdated Show resolved Hide resolved
@geomacy
Copy link

geomacy commented Nov 22, 2024

Many thanks for the comments @tsaarni will have a look through and apply changes as soon as possible.

Geoff Macartney and others added 4 commits November 25, 2024 15:04
Signed-off-by: Geoff Macartney <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Co-authored-by: Tero Saarni <[email protected]>
Signed-off-by: Geoff Macartney <[email protected]>
Geoff Macartney added 4 commits November 26, 2024 16:39
from review comment:

Let's remove the command line option and make configuration available only through config file and ContourConfiguration CRD.

The command line options currently are a bit of a mess, so we previously agreed to avoid introducing new options there unless absolutely necessary.

Signed-off-by: Geoff Macartney <[email protected]>
From review:

Would it be cleaner to treat an empty string as a valid value here, rather than completely bypassing validation in Parameters.Validate()? While algorithm is the only field currently, that might change.

Signed-off-by: Geoff Macartney <[email protected]>
@geomacy
Copy link

geomacy commented Nov 26, 2024

hi @tsaarni, thanks for the comments and suggestions. I have a branch in preparation to do these and will merge it here when I see tests pass.

What do you think about the notes here on the question about the builder methods?

Signed-off-by: Geoff Macartney <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note/small A small change that needs one line of explanation in the release notes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants