Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supports configurable compression level #3567

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/modules/ROOT/pages/http-server.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,16 @@ You can configure the `HTTP` server to send a compressed response, depending on
* `compress(int)`: The compression is performed once the response size exceeds the given value (in bytes).
* `compress(BiPredicate<HttpServerRequest, HttpServerResponse>)`: The compression is performed if
the predicate returns `true`.
* `compressSettings(Consumer<HttpCompressionSettingsSpec.Builder> compressionSettings`: Specifies the compression level for GZIP, DEFLATE, and ZSTD.
+
[NOTE]
====
Supported Compression Levels

* gzip : only the range 0 to 9 is allowed. (default: 6)
* deflate : only the range 0 to 9 is allowed. (default: 6)
* zstd : only the range -7 to 22 is allowed. (default: 3)
====

The following example uses the `compress` method (set to `true`) to enable compression:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@
final class Http2StreamBridgeServerHandler extends ChannelDuplexHandler {

final BiPredicate<HttpServerRequest, HttpServerResponse> compress;
final HttpCompressionSettingsSpec compressionSettings;
final ServerCookieDecoder cookieDecoder;
final ServerCookieEncoder cookieEncoder;
final HttpServerFormDecoderProvider formDecoderProvider;
Expand All @@ -84,6 +85,7 @@ final class Http2StreamBridgeServerHandler extends ChannelDuplexHandler {

Http2StreamBridgeServerHandler(
@Nullable BiPredicate<HttpServerRequest, HttpServerResponse> compress,
HttpCompressionSettingsSpec compressionSettings,
ServerCookieDecoder decoder,
ServerCookieEncoder encoder,
HttpServerFormDecoderProvider formDecoderProvider,
Expand All @@ -94,6 +96,7 @@ final class Http2StreamBridgeServerHandler extends ChannelDuplexHandler {
@Nullable Duration readTimeout,
@Nullable Duration requestTimeout) {
this.compress = compress;
this.compressionSettings = compressionSettings;
this.cookieDecoder = decoder;
this.cookieEncoder = encoder;
this.formDecoderProvider = formDecoderProvider;
Expand Down Expand Up @@ -140,6 +143,7 @@ public void channelRead(ChannelHandlerContext ctx, Object msg) {
listener,
request,
compress,
compressionSettings,
connectionInfo,
cookieDecoder,
cookieEncoder,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ final class Http3Codec extends ChannelInitializer<QuicStreamChannel> {
final Function<String, String> methodTagValue;
final ChannelMetricsRecorder metricsRecorder;
final int minCompressionSize;
final HttpCompressionSettingsSpec compressionSettings;
final ChannelOperations.OnSetup opsFactory;
final Duration readTimeout;
final Duration requestTimeout;
Expand All @@ -83,6 +84,7 @@ final class Http3Codec extends ChannelInitializer<QuicStreamChannel> {
@Nullable Function<String, String> methodTagValue,
@Nullable ChannelMetricsRecorder metricsRecorder,
int minCompressionSize,
HttpCompressionSettingsSpec compressionSettings,
ChannelOperations.OnSetup opsFactory,
@Nullable Duration readTimeout,
@Nullable Duration requestTimeout,
Expand All @@ -101,6 +103,7 @@ final class Http3Codec extends ChannelInitializer<QuicStreamChannel> {
this.methodTagValue = methodTagValue;
this.metricsRecorder = metricsRecorder;
this.minCompressionSize = minCompressionSize;
this.compressionSettings = compressionSettings;
this.opsFactory = opsFactory;
this.readTimeout = readTimeout;
this.requestTimeout = requestTimeout;
Expand All @@ -118,13 +121,13 @@ protected void initChannel(QuicStreamChannel channel) {

p.addLast(NettyPipeline.H3ToHttp11Codec, new Http3FrameToHttpObjectCodec(true, validate))
.addLast(NettyPipeline.HttpTrafficHandler,
new Http3StreamBridgeServerHandler(compressPredicate, cookieDecoder, cookieEncoder, formDecoderProvider,
new Http3StreamBridgeServerHandler(compressPredicate, compressionSettings, cookieDecoder, cookieEncoder, formDecoderProvider,
forwardedHeaderHandler, httpMessageLogFactory, listener, mapHandle, readTimeout, requestTimeout));

boolean alwaysCompress = compressPredicate == null && minCompressionSize == 0;

if (alwaysCompress) {
p.addLast(NettyPipeline.CompressionHandler, new SimpleCompressionHandler());
p.addLast(NettyPipeline.CompressionHandler, SimpleCompressionHandler.create(compressionSettings));
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apply the configured compression settings.

}

ChannelOperations.addReactiveBridge(channel, opsFactory, listener);
Expand Down Expand Up @@ -166,14 +169,15 @@ static ChannelHandler newHttp3ServerConnectionHandler(
@Nullable Function<String, String> methodTagValue,
@Nullable ChannelMetricsRecorder metricsRecorder,
int minCompressionSize,
HttpCompressionSettingsSpec compressionSettings,
ChannelOperations.OnSetup opsFactory,
@Nullable Duration readTimeout,
@Nullable Duration requestTimeout,
@Nullable Function<String, String> uriTagValue,
boolean validate) {
return new Http3ServerConnectionHandler(
new Http3Codec(accessLogEnabled, accessLog, compressPredicate, decoder, encoder, formDecoderProvider, forwardedHeaderHandler,
httpMessageLogFactory, listener, mapHandle, methodTagValue, metricsRecorder, minCompressionSize,
httpMessageLogFactory, listener, mapHandle, methodTagValue, metricsRecorder, minCompressionSize, compressionSettings,
opsFactory, readTimeout, requestTimeout, uriTagValue, validate));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ final class Http3ServerOperations extends HttpServerOperations {
ConnectionObserver listener,
HttpRequest nettyRequest,
@Nullable BiPredicate<HttpServerRequest, HttpServerResponse> compressionPredicate,
HttpCompressionSettingsSpec compressionSettings,
ConnectionInfo connectionInfo,
ServerCookieDecoder decoder,
ServerCookieEncoder encoder,
Expand All @@ -54,7 +55,7 @@ final class Http3ServerOperations extends HttpServerOperations {
@Nullable Duration requestTimeout,
boolean secured,
ZonedDateTime timestamp) {
super(c, listener, nettyRequest, compressionPredicate, connectionInfo, decoder, encoder, formDecoderProvider,
super(c, listener, nettyRequest, compressionPredicate, compressionSettings, connectionInfo, decoder, encoder, formDecoderProvider,
httpMessageLogFactory, isHttp2, mapHandle, readTimeout, requestTimeout, secured, timestamp, true);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@

final class Http3StreamBridgeServerHandler extends ChannelDuplexHandler {
final BiPredicate<HttpServerRequest, HttpServerResponse> compress;
final HttpCompressionSettingsSpec compressionSettings;
final ServerCookieDecoder cookieDecoder;
final ServerCookieEncoder cookieEncoder;
final HttpServerFormDecoderProvider formDecoderProvider;
Expand All @@ -74,6 +75,7 @@ final class Http3StreamBridgeServerHandler extends ChannelDuplexHandler {

Http3StreamBridgeServerHandler(
@Nullable BiPredicate<HttpServerRequest, HttpServerResponse> compress,
HttpCompressionSettingsSpec compressionSettings,
ServerCookieDecoder decoder,
ServerCookieEncoder encoder,
HttpServerFormDecoderProvider formDecoderProvider,
Expand All @@ -84,6 +86,7 @@ final class Http3StreamBridgeServerHandler extends ChannelDuplexHandler {
@Nullable Duration readTimeout,
@Nullable Duration requestTimeout) {
this.compress = compress;
this.compressionSettings = compressionSettings;
this.cookieDecoder = decoder;
this.cookieEncoder = encoder;
this.formDecoderProvider = formDecoderProvider;
Expand Down Expand Up @@ -131,6 +134,7 @@ public void channelRead(ChannelHandlerContext ctx, Object msg) {
listener,
request,
compress,
compressionSettings,
connectionInfo,
cookieDecoder,
cookieEncoder,
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,178 @@
/*
* Copyright (c) 2020-2023 VMware, Inc. or its affiliates, All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* https://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package reactor.netty.http.server;

import java.util.ArrayList;
import java.util.List;

import io.netty.handler.codec.compression.Brotli;
import io.netty.handler.codec.compression.BrotliOptions;
import io.netty.handler.codec.compression.CompressionOptions;
import io.netty.handler.codec.compression.DeflateOptions;
import io.netty.handler.codec.compression.GzipOptions;
import io.netty.handler.codec.compression.SnappyOptions;
import io.netty.handler.codec.compression.StandardCompressionOptions;
import io.netty.handler.codec.compression.Zstd;
import io.netty.handler.codec.compression.ZstdOptions;
import io.netty.util.internal.ObjectUtil;

/**
* HTTP Compression configuration builder for the {@link SimpleCompressionHandler}.
*
* @author raccoonback
*/
public final class HttpCompressionSettingsSpec {

private final GzipOptions gzipOptions;
private final DeflateOptions deflateOptions;
private final SnappyOptions snappyOptions;
private BrotliOptions brotliOptions;
private ZstdOptions zstdOptions;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compression settings supported by Netty.


private HttpCompressionSettingsSpec() {
gzipOptions = StandardCompressionOptions.gzip();
deflateOptions = StandardCompressionOptions.deflate();
snappyOptions = StandardCompressionOptions.snappy();

if (Brotli.isAvailable()) {
brotliOptions = StandardCompressionOptions.brotli();
}

if (Zstd.isAvailable()) {
zstdOptions = StandardCompressionOptions.zstd();
}
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default constructor sets the available compression to default settings.


private HttpCompressionSettingsSpec(Build build) {
gzipOptions = build.gzipOptions;
deflateOptions = build.gzipOptions;
snappyOptions = StandardCompressionOptions.snappy();

if (Brotli.isAvailable()) {
brotliOptions = StandardCompressionOptions.brotli();
}

if (Zstd.isAvailable() && build.zstdOptions != null) {
zstdOptions = build.zstdOptions;
}
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initializes to each compression setting entered by the Builder.


/**
* Creates a builder for {@link HttpCompressionSettingsSpec}.
*
* @return a new {@link HttpCompressionSettingsSpec.Builder}
*/
public static Builder builder() {
return new Build();
}

static HttpCompressionSettingsSpec provideDefault() {
return new HttpCompressionSettingsSpec();
}

CompressionOptions[] adaptToOptions() {
List<CompressionOptions> options = new ArrayList<>();
options.add(this.gzipOptions);
options.add(this.deflateOptions);
options.add(this.snappyOptions);

if (brotliOptions != null) {
options.add(this.brotliOptions);
}

if (zstdOptions != null) {
options.add(this.zstdOptions);
}

return options.toArray(new CompressionOptions[0]);
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adapts to the type to be provided to the constructor argument of Netty's HttpContentCompressor.


public interface Builder {

/**
* Build a new {@link HttpCompressionSettingsSpec}.
*
* @return a new {@link HttpCompressionSettingsSpec}
*/
HttpCompressionSettingsSpec build();

/**
* Sets the gzip compression level.
*
* @return a new {@link HttpCompressionSettingsSpec.Builder}
*/
Builder gzip(int compressionLevel);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@raccoonback If in the future you need to extend the configuration API with window Bits, memory level etc. How will we introduce them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@violetagg
Thank you for the great question!

The reason why windowBits and memoryLevel are not currently configurable is that their default settings in GZIP are optimized for most use cases. Exposing these options could lead to unnecessary increases in memory usage or make configuration overly complex.

For now, we’ve chosen to rely on default values without exposing these settings directly to users. However, we’ve considered the need for extensibility in future requirements.

If it becomes necessary to make windowBits and memoryLevel configurable, the API can be extended using with method overloading:

Builder gzip(int compressionLevel);

Builder gzip(int compressionLevel, int windowBits);

Builder gzip(int compressionLevel, int windowBits, int memoryLevel);

// ...

Additionally, if required, we can extend the API to provide these as optional settings while maintaining the default values. Let me know your thoughts, and I’ll adjust the design accordingly.

Copy link
Member

@violetagg violetagg Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mmm ... method overloading is not something that I would like to see in the API. Can we use a similar approach to Netty's CompressionOptions. Thus the API will be "please provide CompressionOptions[]" and the API will stay stable and we can touch only particular CompressionOption if needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@violetagg
Thank you for your feedback!
I will revise the implementation to follow a structure similar to Netty's CompressionOptions as you suggested.


/**
* Sets the deflate compression level.
*
* @return a new {@link HttpCompressionSettingsSpec.Builder}
*/
Builder deflate(int compressionLevel);

/**
* Sets the zstd compression level.
*
* @return a new {@link HttpCompressionSettingsSpec.Builder}
*/
Builder zstd(int compressionLevel);
}

private static final class Build implements Builder {

GzipOptions gzipOptions = StandardCompressionOptions.gzip();
DeflateOptions deflateOptions = StandardCompressionOptions.deflate();
ZstdOptions zstdOptions;

private static final int DEFLATE_DEFAULT_WINDOW_BITS = 15;
private static final int DEFLATE_DEFAULT_MEMORY_LEVEL = 8;
private static final int ZSTD_DEFAULT_COMPRESSION_LEVEL = 3;
private static final int ZSTD_DEFAULT_BLOCK_SIZE = 65536;
private static final int ZSTD_MAX_BLOCK_SIZE = 1 << ZSTD_DEFAULT_COMPRESSION_LEVEL + 7 + 15;

@Override
public HttpCompressionSettingsSpec build() {
return new HttpCompressionSettingsSpec(this);
}

@Override
public Builder gzip(int compressionLevel) {
ObjectUtil.checkInRange(compressionLevel, 0, 9, "compressionLevel");

gzipOptions = StandardCompressionOptions.gzip(compressionLevel, DEFLATE_DEFAULT_WINDOW_BITS, DEFLATE_DEFAULT_MEMORY_LEVEL);
return this;
}

@Override
public Builder deflate(int compressionLevel) {
ObjectUtil.checkInRange(compressionLevel, 0, 9, "compressionLevel");

this.deflateOptions = StandardCompressionOptions.deflate(compressionLevel, DEFLATE_DEFAULT_WINDOW_BITS, DEFLATE_DEFAULT_MEMORY_LEVEL);
return this;
}

@Override
public Builder zstd(int compressionLevel) {
if (!Zstd.isAvailable()) {
throw new IllegalStateException("Unable to set compression level on zstd.");
}
ObjectUtil.checkInRange(compressionLevel, -7, 22, "compressionLevel");

this.zstdOptions = StandardCompressionOptions.zstd(compressionLevel, ZSTD_DEFAULT_BLOCK_SIZE, ZSTD_MAX_BLOCK_SIZE);
return this;
}
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A builder that takes each compression setting as input from the user.

}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defines a spec for configuring HTTP compression.

Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,7 @@ public final HttpServer compress(BiPredicate<HttpServerRequest, HttpServerRespon
/**
* Specifies whether GZip response compression is enabled if the client request
* presents accept encoding.
* default compression level is 6.
*
* @param compressionEnabled if true GZip response compression
* is enabled if the client request presents accept encoding, otherwise disabled.
Expand Down Expand Up @@ -336,6 +337,35 @@ public final HttpServer compress(int minResponseSize) {
return dup;
}

/**
* Specifies GZIP, DEFLATE, ZSTD Compression Level.
*
* @param compressionSettings configures {@link HttpCompressionSettingsSpec} after enable compress.
* <pre>
* {@code
* HttpServer.create()
* .compress(true)
* .compressOptions(
* builder -> builder.gzip(6)
* .deflate(6)
* .zstd(3)
* )
* .bindNow();
* }
* </pre>
* @return a new {@link HttpServer}
*/
public final HttpServer compressSettings(Consumer<HttpCompressionSettingsSpec.Builder> compressionSettings) {
Objects.requireNonNull(compressionSettings, "compressionSettings");

HttpCompressionSettingsSpec.Builder builder = HttpCompressionSettingsSpec.builder();
compressionSettings.accept(builder);

HttpServer dup = duplicate();
dup.configuration().compressionSettings = builder.build();
return dup;
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method takes compression settings from the user.


/**
* Configure the
* {@link ServerCookieEncoder}; {@link ServerCookieDecoder} will be
Expand Down
Loading