Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge 4.18.0 #307

Merged
merged 36 commits into from
Jun 17, 2024
Merged

Merge 4.18.0 #307

merged 36 commits into from
Jun 17, 2024

Conversation

Bouncheck
Copy link
Collaborator

@Bouncheck Bouncheck commented Jun 10, 2024

Adding most of the changes from last merge up to 4.18.0 with slight modifications to resolve conflicts.

absurdfarce and others added 30 commits July 12, 2023 17:34
…voked: schemaListener1.onSessionReady (apache#1670)

ListenersIT.java:
- Add 500ms wait on SchemaListener#onSessionReady call verification
- Add latch to wait for MySchemaChangeListener#onSessionReady
- Add some comments around which listeners/methods need synchronizations and why/not
jackson-jaxrs-json-provider to 2.13.4 to address recent CVEs

Additional:
- Remove unused maven property legacy-jackson.version
…se @BackendRequirement (apache#1667)

Refactor @BackendRequirement test skipping logic into new rule, BackendRequirementRule.
…ust not send frame with WARNING flag for native protocol version < 4 (apache#1669)

NullSavingStrategyIT.java:
- Remove V3 protocol configuration from class SessionRule
- Create new session configured with V3 protocol in @BeforeClass method to use in tests
…ts with specified jvm (apache#1719)

Additionally:
- Set --jvm_version=8 when running dse 6.8.19+ with graph workloads (DSP-23501)
- Update commons-configuration2 to 2.9.0 + deps in BundleOptions to support java17
- Update felix framework version to 7.0.1 for java17 (FELIX-6287)
- Pick up newer bndlib for ArrayIndexOutOfBounds error printed with OsgiGraphIT (bndtools/bnd issue#3405)
  - Update pax-url-wrap to 2.6.4 (and bring pax-url-reference up to the same version)
  - Force newer tinybundles version 3.0.0 (default 2.1.1 version required older bndlib)
apache#1689)

* add support for publishing percentile time series for the histogram metrics cql-requests, cql-messages and throttling delay.

Motivation:

Histogram metrics is generating too many metrics overloading the promethous servers. if application has 500 Vms
and 1000 cassandra nodes, The histogram metrics generates 100*500*1000 = 50,000,000 time series every 30 seconds.
This is just too much metrics.  Let us say we can generate percentile 95 timeseries for for every cassandra nodes,
then we only have 1*500 = 500 metrics and in applciation side, we can ignore the _bucket time series. This way there
will be very less metrics.

Modifications:
add configurable pre-defined percentiles to Micrometer Timer.Builder.publishPercentiles. This change is being added to
cql-requests, cql-messages and throttling delay.

Result:
Based on the configuration, we will see additonal quantile time series for cql-requests, cql-messages and throttling delay
histogram metrics.

* add support for publishing percentile time series for the histogram metrics cql-requests, cql-messages and throttling delay.

Motivation:

Histogram metrics is generating too many metrics overloading the promethous servers. if application has 500 Vms
and 1000 cassandra nodes, The histogram metrics generates 100*500*1000 = 50,000,000 time series every 30 seconds.
This is just too much metrics.  Let us say we can generate percentile 95 timeseries for for every cassandra nodes,
then we only have 1*500 = 500 metrics and in applciation side, we can ignore the _bucket time series. This way there
will be very less metrics.

Modifications:
add configurable pre-defined percentiles to Micrometer Timer.Builder.publishPercentiles. This change is being added to
cql-requests, cql-messages and throttling delay.

Result:
Based on the configuration, we will see additonal quantile time series for cql-requests, cql-messages and throttling delay
histogram metrics.

* using helper method as suggested in review

* fixes as per review comments

* add configuration option which switches aggregable histogram generation on/off for all metric flavors [default=on]

* updating java doc

* rename method to publishPercentilesIfDefined

* renmae method

---------

Co-authored-by: Nagappa Paraddi <[email protected]>
…ndra-java-driver.html )

 patch by Mick Semb Wever; reviewed by Henry Hughes for CASSANDRA-18611
patch by Henry Hughes; reviewed by Mick Semb Wever for CASSANDRA-18969
patch by Claude Warren; reviewed by Henry Hughes, Mick Semb Wever for CASSANDRA-18969
patch by Claude Warren; reviewed by Henry Hughes, Mick Semb Wever for CASSANDRA-18969
LICENSE + NOTICE.txt is added to source jars, LICENSE + NOTICE_binary.txt
is added to regular jars. Make parent project inherit from apache pom.
Updated NOTICE wording to "developed at ..." per latest instructions.

patch by Henry Hughes; reviewed by Mick Semb Wever for CASSANDRA-18969
…alls

* Source files missing from sources jars due to maven-source-plugin include rule
* New submodule to generate distribution source tarball
* Binary/source tarball artifacts should be prefixed with apache-cassandra
* Change groupId to org.apache.cassandra
* Remove javadoc jars (javadoc plugin is still used for leak detections)
* Create binary versions for LICENSE and NOTICE, with licenses and entries for asm, HdrHistogram, jnr-posix, jnr-x86asm, reactive-streams, slf4j-api
* Add checksums to distribution tarballs, and clean toplevel readme a little

 patch by Henry Hughes; reviewed by Mick Semb Wever for CASSANDRA-18969
@Bouncheck
Copy link
Collaborator Author

Bouncheck commented Jun 10, 2024

There are 2 new modules that I've removed after conflict resolution, but maybe I should've just renamed their groupId to com.scylladb? Specifically distribution-tests and distribution-source.

This is very likely not ready for merge because I could've merged in something that did not really conflict with anything but should not be merged in anyway.

From what I've seen most of the conflicts were due to:

  • Changes to LICENSE files. They were renamed, modified and added as resources in several pom.xml
  • Changes to license headers, because upstream is now apache instead of datastax
  • Changes in README files. Mostly differences in maven groupIds but also some small changes like new instructions how to exclude/include JARs for annotations.
  • Different plugin versions and sets of plugins used. Most of upstream additions were discarded. Changes to scope of some plugins.
  • Unification of some version requirement annotations into BackendRequirement annotation and subsequent changes to tests and test rules.

@Bouncheck
Copy link
Collaborator Author

On another look I don't see anything besides .asf.yaml that should be deleted. Rest of the files seem to be already present in scylla-4.x branch

@Bouncheck Bouncheck requested a review from avelanarius June 10, 2024 15:52
@Bouncheck
Copy link
Collaborator Author

Some of the tests are failing, I still need to look into that.

Bouncheck and others added 4 commits June 10, 2024 17:59
…tion tests)

 patch by Mick Semb Wever; reviewed by Wei Deng for CASSANDRA-18970

(cherry picked from commit 8d5849c)
Adds ScyllaSkip annotation to PreparedStatementIT.handle_id_changes_on_reprepare()
This is a test method that comes with the most recent merge with 4.18.0

It checks if in case of performing `USE <keyspace>` the server side will not
reprepare the statement under different ID if keyspace was already defined
in query string. It seems it works this way in select versions of Cassandra
up to 4.0.1 and every higher than that.
This test fails due to negotiating protocol version v4, while this feature
requires protocol version v5 which seems to not be fully supported by Scylla.
Adding ScyllaSkip annotation to the test. It can be re-enabled later.
@Bouncheck
Copy link
Collaborator Author

I think this is ready to merge, but needs to be reviewed.
Is version 4.18.0.0-SNAPSHOT correct here?

@roydahan
Copy link
Collaborator

@avelanarius please give high priority for review

@Bouncheck Bouncheck merged commit 87ba8b6 into scylladb:scylla-4.x Jun 17, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants