1.0.6-rc0

What's Changed

• fix(issues1222): fix #openStream resp nextOffset by @superhx in #1226
• chore: switch to dev mode by @Chillax-0v0 in #1230
• fix: compatible with s3-Stream by @warr99 in #1251
• fix(issues1248): fix partition reassignment deadlock by @superhx in #1250
• feat(metadata): fix data race when close and reopen coordinator partition by @ShadowySpirits in #1255
• fix(metadata): ensure that the consumer offset is deleted along with the deletion of the topic by @ShadowySpirits in #1263
• fix(metadata): clean up the consumption offset for the deleted partitions by @ShadowySpirits in #1265
• fix(issues1276): group the batch request by nodeId and apiKey (#1277) by @superhx in #1278
• ci: bump version to 1.0.6-rc0 by @Chillax-0v0 in #1289

Full Changelog: 1.0.5...1.0.6-rc0
S3Stream Changelog: 1.0.5-s3stream...1.0.6-s3stream-rc0

1.0.5(End of Support)

What's Changed

• fix(issues1078): fix RequestChannel shutdown blocking by @superhx in #1079
• build(issues1033): switch log appender from DailyRollingFileAppender to RollingFileAppender #1092 by @warr99 in #1111
• feat(auto_balancer): support client auth config by @SCNieh in #1138
• feat: bump docker version to 1.0.5-rc0 by @superhx in #1139
• feat(s3stream): switch to branch main by @superhx in #1145
• feat: 1.0 use the old block cache impl by @superhx in #1150
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1152
• Revert "fix(issues1140): fix S3ObjectControlManager NPE" by @superhx in #1156
• feat(config): change s3.object.delete.retention.minutes default to 5min by @superhx in #1160
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1161
• feat(issues1171): improve objects deletion performance by @superhx in #1173
• feat(release): 1.0.5-rc1 by @superhx in #1175
• fix(core): fix leader fast ABA reassign cause epoch outdated by @superhx in #1187
• chore(controller): more logs by @Chillax-0v0 in #1202
• feat(MM2): allow customizing MM2 offset-syncs/checkpoints/heartbeats topic easily by @aaron-ai in #1221
• ci: bump version to 1.0.5 by @Chillax-0v0 in #1223

Full Changelog: 1.0.4...1.0.5
S3Stream Changelog: 1.0.4-s3stream...1.0.5-s3stream

1.0.5-rc1

What's Changed

• feat(s3stream): switch to branch main by @superhx in #1145
• feat: 1.0 use the old block cache impl by @superhx in #1150
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1152
• Revert "fix(issues1140): fix S3ObjectControlManager NPE" by @superhx in #1156
• feat(config): change s3.object.delete.retention.minutes default to 5min by @superhx in #1160
• fix(issues1140): fix S3ObjectControlManager NPE by @superhx in #1161
• feat(issues1171): improve objects deletion performance by @superhx in #1173
• feat(release): 1.0.5-rc1 by @superhx in #1175

Full Changelog: 1.0.5-rc0...1.0.5-rc1
S3Stream Changelog: 1.0.4-s3stream...1.0.5-s3stream-rc0

1.0.5-rc0

What's Changed

• fix(issues1078): fix RequestChannel shutdown blocking by @superhx in #1079
• build(issues1033): switch log appender from DailyRollingFileAppender to RollingFileAppender #1092 by @warr99 in #1111
• feat(auto_balancer): support client auth config by @SCNieh in #1138
• feat: bump docker version to 1.0.5-rc0 by @superhx in #1139

Full Changelog: 1.0.4...1.0.5-rc0

1.1.0-rc0

Highlights

• Upgrade Apache Kafka from version 3.4.0 to 3.7.0

What's Changed

• chore: move s3stream module back to automq kafka repo by @daniel-y in #929
• feat(others): migrate other files by @superhx in #930
• fix(README): add README image by @superhx in #931
• feat(s3stream): add s3stream into gradle build system by @ShadowySpirits in #932
• fix(core): fix optimize condition check by @SCNieh in #934
• fix(core): fix configuration for test cluster by @ShadowySpirits in #936
• feat(s3stream): policies of allocating memory by @Chillax-0v0 in #938
• fix(core): fix ElasticReplicaManagerTest by @ShadowySpirits in #940
• chore(s3stream): remove allocator policy "UNPOOLED_HEAP" by @Chillax-0v0 in #942
• chore: optimize README file by @KaimingWan in #945
• feat(metadata): add automq metadata to metadata shell by @superhx in #947
• chore(workflow): ignore tags like "1.0.2-s3stream" by @Chillax-0v0 in #955
• perf(ReplicaManager): release unused permits after reads by @Chillax-0v0 in #935
• feat: tagging S3 objects by @Chillax-0v0 in #956
• feat(config): auto memory adaptation by @Chillax-0v0 in #949
• fix(s3stream): set tagging only when not null by @Chillax-0v0 in #957
• fix(controller): fix partition migration fail after broker fence by @superhx in #958
• fix(core): fix adjusting s3 cache related configuration by @ShadowySpirits in #959
• fix(controller): fix reassignment fail by @superhx in #960
• fix(core): fix ElasticLog#formatStreamKey by @ShadowySpirits in #961
• fix(core): fix repeatedly opening the partition by @ShadowySpirits in #962
• fix(core): implement parallelism in the creating log process by @ShadowySpirits in #963
• feat: add hook point by @superhx in #964
• refactor(config): adjust configs in KafkaConfig rather than Kafka by @Chillax-0v0 in #970
• fix(bin): fix logs variable by @SCNieh in #972
• feat(metrics): support kraft related metrics by @SCNieh in #974
• fix(tools): controllerOnlyMode is false no extra commands will be output by @funky-eyes in #967
• feat(tool): next node id by @superhx in #976
• fix(test): fix new group coordinator load by @superhx in #977
• fix(test): disable E2E zookeeper related test by @superhx in #980
• fix(test): update E2E base image by @superhx in #981
• fix(common): get cause NPE by @superhx in #984
• fix(network): adapt the callback queue by @superhx in #985
• chore(E2E): split main E2E test to 3 tasks by @superhx in #986
• fix(issues987): fix StreamMetadataManager NPE by @superhx in #988
• fix(metadata): avoid getObjects return invalid when endOffset is -1 by @SCNieh in #991
• fix(s3stream): fix potential concurrent modification on size during tryRealFree by @SCNieh in #990
• fix(s3stream/wal): fail fast if the path does not exist in recovery mode by @Chillax-0v0 in #993
• fix(test): increase heap size to 1GiB by @superhx in #994
• fix(issues996): fix fetch NPE by @superhx in #997
• chore(test): move transactions_test to seperate test suite by @superhx in #998
• fix(s3stream): fix cf combination by @SCNieh in #1000
• refactor(s3stream): rename tagging key from "clusterId" to "automq:ClusterID" by @Chillax-0v0 in #999
• refactor(s3stream): rename tagging key from "automq:ClusterID" to "s3stream:namespace" by @Chillax-0v0 in #1001
• test(s3stream/wal): check file existence on startup rather than init by @Chillax-0v0 in #1004
• chore(workflow): run s3stream tests by @Chillax-0v0 in #1003
• feat(core): optimize block cache performance for high-spec node by @SCNieh in #1005
• fix(s3stream): fix pending fetches modification order by @SCNieh in #1008
• fix(s3stream): temporarily disable throttle for slow read by @SCNieh in #1009
• perf(s3stream): reuse ObjectMapper by @lifepuzzlefun in #1012
• chore(core): upgrade "netty-tcnative-boringssl-static" from 2.0.48 to… by @Chillax-0v0 in #1010
• revert(s3stream): revert bypass slow read throttle by @SCNieh in #1015
• fix: order the transaction requests in request pipeline to adapt KIP-890 by @superhx in #1016
• feat(auto_balancer): use nw bandwidth capacity as score limit by @SCNieh in #1018
• feat(s3stream): provide extra tokens to prevent throttle starvation by @SCNieh in #1020
• fix(test): disable useless E2E test case by @superhx in #1019
• feat(auto_balancer): log expected dist bound on each detection by @SCNieh in #1022
• chore(all): checkstyle by @superhx in #1026
• chore(all): double check the merge by @superhx in #1029
• feat: merge apache kafka trunk by @superhx in #1030
• fix(test): E2E add consumer rebalance protocols by @superhx in #1032
• perf(s3stream): only read the first record from log cache before reading block cache by @Chillax-0v0 in #1011
• fix(s3stream): use thread safe data structures to prevent race condition by @SCNieh in #1036
• fix(core): enhancement AnomalyDetector and LoadRetriever by @Wzy19930507 in #1035
• revert(s3stream): "perf(s3stream): only read the first record from log cache before reading block cache" by @Chillax-0v0 in #1038
• chore(action): trigger build for any changes by @superhx in #1040
• feat(s3stream): add S3 API compatibility for BOS DeleteObjects by @lifepuzzlefun in #1028
• fix(metrics): fix jmx rules by @SCNieh in #1041
• fix(stream): fix compaction interval check by @superhx in #1046
• fix(dashboard): present latency avg to timed range by @SCNieh in #1045
• fix(jmx_rule): remove unused rules that could lead to confusion by @SCNieh in #1048
• chore(s3stream/wal): upgrade metrics level of wal_write and wal_await by @Chillax-0v0 in #1047
• fix(s3stream): limit the expired objects delete count by @superhx in #1050
• fix(config): add elasticstream.replication.factor for compability by @superhx in #1051
• fix(issues1060): only allow one cleanup task infight by @superhx in #1062
• fix(controller): check continuity after redundancy in commitStreamSetObject by @Chillax-0v0 in #1056
• fix(test): disable new consumer coordinator unstable consumer protoco… by @superhx in #1066
• test(controller): commit the same stream set object or stream object by @Chillax-0v0 in #1064
• fix(core): ensure partition closed before metadata listener close by @superhx in #1068
• fix(issues1069): stream client shutdown await opening stream close by @superhx in #1071
• fix(test): ignore unstable new consumer coordinator e2e test by @superhx in #1073
• chore(workflow): ignore tags like "1.0.3-s3stream" by @Chillax-0v0 in #1043
• feat(release): bump version to 1.1.0-rc0 by @superhx in #1077

New Contributors

• @lifepuzzlefun made their first contribution in #1012
• @Wzy19930507 made their first contribution in #1035

Full Changelog: 1.0.4...1.1.0-rc0

1.0.4(End of Support)

What's Changed

• ci: bump version to 1.0.3 by @Chillax-0v0 in #1042
• chore: switch to dev mode by @Chillax-0v0 in #1044
• fix(jmx_rule): remove unused rules that could lead to confusion by @SCNieh in #1049
• ci: bump version to 1.0.4-rc0 by @Chillax-0v0 in #1052
• chore: switch to dev mode by @Chillax-0v0 in #1053
• chore(release): tag in the main branch manually rather than the PR branch by @Chillax-0v0 in #1054
• fix(issues1058): clear offline mark before reopen by @superhx in #1059
• fix(issues1060): only allow one cleanup task infight by @superhx in #1061
• fix(controller): check continuity after redundancy in commitStreamSetObject by @Chillax-0v0 in #1057
• test(controller): commit the same stream set object or stream object by @Chillax-0v0 in #1065
• fix(core): ensure partition closed before metadata listener close by @superhx in #1067
• ci: bump version to 1.0.4 by @superhx in #1075

Full Changelog: 1.0.3...1.0.4
S3Stream Changelog: 1.0.3-s3stream...1.0.4-s3stream

1.0.4-rc0

What's Changed

• ci: bump version to 1.0.3 by @Chillax-0v0 in #1042
• chore: switch to dev mode by @Chillax-0v0 in #1044
• fix(jmx_rule): remove unused rules that could lead to confusion by @SCNieh in #1049
• ci: bump version to 1.0.4-rc0 by @Chillax-0v0 in #1052

Full Changelog: 1.0.3...1.0.4-rc0
S3Stream Changelog:1.0.3-s3stream...1.0.4-s3stream-rc0

1.0.3(End of Support)

Highlights

Block Cache Performance Optimization: This update includes an adaptive adjustment of inflight data size and thread pool size, which are now automatically tuned according to the configured cache size and CPU cores. We've also introduced a LRU cache in this release to accelerate the speed of stream set object lookups. This optimization is expected to enhance the overall performance and efficiency of catch up reads.
Network Throttle Optimization: The network throttle strategy has been optimized to prevent starvation during stream openning.

What's Changed

• ci: bump version to 1.0.3-rc0 by @SCNieh in #1024
• chore: switch to dev mode by @SCNieh in #1025

Full Changelog: 1.0.2...1.0.3
S3Stream Changelog:1.0.2-s3stream...1.0.3-s3stream

1.0.3-rc0

Highlights

Block Cache Performance Optimization: This update includes an adaptive adjustment of inflight data size and thread pool size, which are now automatically tuned according to the configured cache size and CPU cores. We've also introduced a LRU cache in this release to accelerate the speed of stream set object lookups. This optimization is expected to enhance the overall performance and efficiency of catch up reads.

What's Changed

• chore: switch to dev mode by @Chillax-0v0 in #979
• fix(metadata): avoid getObjects return invalid when endOffset is -1 by @SCNieh in #992
• feat(core): optimize block cache performance for high-spec node by @SCNieh in #1006
• chore(core): upgrade "netty-tcnative-boringssl-static" by @SCNieh in #1014
• feat(auto_balancer): use nw bandwidth capacity as score limit by @SCNieh in #1017
• feat(auto_balancer): log expected dist bound on each detection by @SCNieh in #1021
• ci: bump version to 1.0.3-rc0 by @SCNieh in #1023

Full Changelog: 1.0.2...1.0.3-rc0
S3Stream Changelog:1.0.2-s3stream...1.0.3-s3stream-rc0

1.0.2(End of Support)

Important Update

Adaptive Memory-Related Configurations: This new feature simplifies your setup process, enabling AutoMQ Kafka to run on machines of any memory size without requiring manual configuration adjustments.
With this update, AutoMQ Kafka intelligently adjusts heap sizes and various cache sizes based on the available machine memory by default. This advancement is designed to significantly simplify your setup process, enabling optimal performance without manual intervention and helping you get started with AutoMQ Kafka more efficiently.

What's Changed

• feat(release): update s3stream version by @Chillax-0v0 in #908
• fix(core): fix move out action break condition & extract common code by @SCNieh in #916
• feat(telemetry): support gzip compression for grpc otlp exporter by @SCNieh in #921
• feat(core): refactor AbstractGoal for better scalability by @SCNieh in #924
• chore(log): remove useless error log by @Chillax-0v0 in #926
• fix(core): fix optimize condition check by @SCNieh in #933
• chore: change dependency of s3stream from automq-for-rocketmq to automq-for-kafka by @Chillax-0v0 in #939
• feat(config): add config s3.stream.allocator.policy by @Chillax-0v0 in #941
• feat(config): add config s3.stream.allocator.policy by @Chillax-0v0 in #943
• chore(gradle): adapt to Gradle 8.6 by @Chillax-0v0 in #948
• feat(config): auto memory adaptation by @Chillax-0v0 in #946
• chore(release): update the release script by @Chillax-0v0 in #950
• fix(release): update main branch in the release script by @Chillax-0v0 in #951
• ci: bump version to 1.0.2 by @Chillax-0v0 in #952
• chore: switch to develop mode by @Chillax-0v0 in #953
• chore(workflow): ignore tags like "1.0.2-s3stream" by @Chillax-0v0 in #954
• feat(config): add a switch on tagging by @Chillax-0v0 in #969
• fix(config): adjust configs in KafkaConfig rather than Kafka by @Chillax-0v0 in #971

Full Changelog: 1.0.1...1.0.2
S3Stream Changelog: 1.0.1-s3stream...1.0.2-s3stream