[Discuss] Changed default settings for 3.0

[andrross]

Is your feature request related to a problem? Please describe

Coming from part of @Bukhtawar's comment here, the 3.0 release is an opportunity to change default settings or behavior. We often introduce new features or guardrails that are generally helpful but are considered breaking behavior to automatically enable by default. This issue is to discuss any such changes that should be made and get them added to the list for the 3.0 release.

Describe the solution you'd like

List changes to defaults that should be made for the 3.0 release.

Proposed Changes:

• [Feature Request] Make segment replication the default for 3.0 #17162
• Increase maxMergeAtOnce parameter in OpenSearch 3.0 #17051

[andrross]

I'm intentionally spamming the component labels to get visibility here across all components :)

[msfroh]

Ooh, ooh! Pick me!

• Increase maxMergeAtOnce parameter in OpenSearch 3.0 #17051 -- We can get a head-start on a new default planned for Lucene 11.
• I would also change a few TieredMergePolicy settings to get more consistent segment sizes:
  • floorSegmentMB should be much higher than the default 2MB -- say 50 MB
  • segmentsPerTier should probably be lower -- I've had good experience with 5 (down from 10)
  • IMO we should add a setting to cap maxSegmentSize at a fraction of total shard size (e.g. no segment should be larger than 1/5 of the shard size) to see benefits from concurrent segment search even for smaller shards.

[HenryTheSir]

Hi,

I‘m unsure if I have to follow guidelines to suggest changes. If this is wrong just delete my comment.

My two suggestions would be to change the default codec to zstd, docs say read and write are both increased with this codec.
Second suggestion would be to enable cluster.allocator.existing_shards_allocator.batch_enabled by default to lower pressure on clusters on startups.

best regards
Henry

[kkewwei]

My two suggestions:
1.enable cluster.routing.allocation.balance.prefer_primary and cluster.routing.allocation.rebalance.primary.enable
2. increase cluster.max_shards_per_node, 1000 is too small, how about 3000?