[DataGridPremium] Improve aggregation performance for multiple columns

[cherniavskii]

Extracted from #9877

Why

Aggregation is one of the core features for pivoting. It often involves aggregating dozens of columns.
This PR improves aggregation calculation performance with row grouping and multiple aggregation rules.

How

I've changed the way we calculate aggregation value for each row group.

Before

1. Loop: Iterate over aggregation rules (getGroupAggregatedValue). For each aggregation field:
   2. Get the child rowIds for the row group (getRowGroupChildren). Requires traversing the group tree.
   3. Loop: Iterate over child rowIds and calculate the aggregated value

Optimizations:

• Point 2 can be moved up to do 1 x row_group, as opposed to 1 x row_group x aggregation_rule.
• In the inner loop (point 3) we return early for some rows, so the outer loop iterations are unnecessary for some rows.

After

1. Get the child rowIds for the row group (getRowGroupChildren). Requires traversing the group tree.
2. Loop: Iterate over child rowIds. For each row:
   3. Loop: Iterate over aggregation rules and store row values.
3. Loop: Iterate over the aggregation rules and calculate the aggregated value.

Results

Setup	Before (median)	After (median)	Difference (%)
rows: 10_000 (Commodities) row grouping: by commodity aggregation rules: 1	121ms	121ms	0%
rows: 10_000 (Commodities) row grouping: none aggregation rules: 1	91ms	91ms	0%
rows: 10_000 (Commodities) row grouping: by commodity aggregation rules: 6	193ms	162ms	-16%
rows: 10_000 (Commodities) row grouping: by commodity aggregation rules: 10	300ms	190ms	-36%
rows: 10_000 (Commodities) row grouping: by commodity aggregation rules: 17	363ms	261ms	-28%
rows: 10_000 (Column virtualization demo) row grouping: by price1M aggregation rules: 50	1017ms	548ms	-46%

Detailed results

rows: 10_000 (Commodities)
row grouping: by commodity
aggregation:
  {
      quantity: 'sum',
    }
scripting time:
  before:
    measurements: 120ms, 121ms, 128ms, 119ms, 122ms
    median: 121ms
  after:
    measurements: 121ms, 121ms, 123ms, 120ms, 120ms
    median: 121ms
  difference: 0%

rows: 10_000 (Commodities)
row grouping: None
aggregation:
  {
    quantity: 'sum',
  }
scripting time:
  before:
    measurements: 98ms, 86ms, 92ms, 88ms, 90ms, 93ms
    median: 91ms
  after:
    measurements: 84ms, 94ms, 92ms, 89ms, 90ms, 95ms
    median: 91ms
  difference: 0%

rows: 10_000 (Commodities)
row grouping: by commodity
aggregation:
  {
    quantity: 'sum',
    filledQuantity: 'size',
    isFilled: 'size',
    status: 'size',
    unitPrice: 'sum',
    subTotal: 'sum',
  }
scripting time:
  before:
    measurements: 193ms, 193ms, 194ms, 192ms, 194ms
    median: 193ms
  after:
    measurements: 162ms, 162ms, 162ms, 166ms, 164ms
    median: 162ms
  difference: -16%

rows: 10_000 (Commodities)
row grouping: by commodity
aggregation:
  {
    quantity: 'sum',
    filledQuantity: 'size',
    isFilled: 'size',
    status: 'size',
    unitPrice: 'sum',
    subTotal: 'sum',
    unitPriceCurrency: 'size',
    feeRate: 'sum',
    feeAmount: 'sum',
    incoTerm: 'size',
  }
scripting time:
  before:
    measurements: 363ms, 250ms, 266ms, 251ms, 346ms, 327ms, 300ms
    median: 300ms
  after:
    measurements: 186ms, 194ms, 184ms, 248ms, 213ms, 188ms, 190ms
    median: 190ms
  difference: -36%

rows: 10_000 (Commodities)
row grouping: by commodity
aggregation:
  {
    quantity: 'sum',
    filledQuantity: 'size',
    isFilled: 'size',
    status: 'size',
    unitPrice: 'sum',
    subTotal: 'sum',
    unitPriceCurrency: 'size',
    feeRate: 'sum',
    feeAmount: 'sum',
    incoTerm: 'size',
    desk: 'size',
    traderName: 'size',
    traderEmail: 'size',
    totalPrice: 'sum',
    pnl: 'sum',
    maturityDate: 'size',
    tradeDate: 'size',
  }
scripting time:
  before:
    measurements: 361ms, 365ms, 363ms, 399ms, 399ms, 331ms, 329ms
    median: 363ms
  after:
    measurements: 303ms, 236ms, 211ms, 223ms, 297ms, 261ms, 262ms
    median: 261ms
  difference: -28%

rows: 10_000 (Column virtualization demo (mui.com/x/react-data-grid/virtualization/#column-virtualization))
row grouping: by price1M
aggregation: 50 columns
  {
    price1M: 'size',
    price2M: 'size',
    price3M: 'size',
    price4M: 'size',
    price5M: 'size',
    price6M: 'size',
    price7M: 'size',
    price8M: 'size',
    price9M: 'size',
    price10M: 'size',
    price11M: 'size',
    price12M: 'size',
    price13M: 'size',
    price14M: 'size',
    price15M: 'size',
    price16M: 'size',
    price17M: 'size',
    price18M: 'size',
    price19M: 'size',
    price20M: 'size',
    price21M: 'size',
    price22M: 'size',
    price23M: 'size',
    price24M: 'size',
    price25M: 'size',
    price26M: 'size',
    price27M: 'size',
    price28M: 'size',
    price29M: 'size',
    price30M: 'size',
    price31M: 'size',
    price32M: 'size',
    price33M: 'size',
    price34M: 'size',
    price35M: 'size',
    price36M: 'size',
    price37M: 'size',
    price38M: 'size',
    price39M: 'size',
    price40M: 'size',
    price41M: 'size',
    price42M: 'size',
    price43M: 'size',
    price44M: 'size',
    price45M: 'size',
    price46M: 'size',
    price47M: 'size',
    price48M: 'size',
    price49M: 'size',
    price50M: 'size',
  }
scripting time:
  before:
    measurements: 1062ms, 1007ms, 1062ms, 1012ms, 1018ms, 994ms, 1071ms, 1037ms, 1017ms, 1010ms
    median: 1017ms
  after:
    measurements: 548ms, 548ms, 613ms, 556ms, 543ms, 536ms, 596ms, 536ms, 548ms, 573ms
    median: 548ms
  difference: -46%

Demos

Before: https://codesandbox.io/p/sandbox/wispy-leaf-r2kpzm
After: https://codesandbox.io/p/sandbox/wispy-architecture-8xpsg8

[mui-bot]

Deploy preview: https://deploy-preview-16097--material-ui-x.netlify.app/

Generated by 🚫 dangerJS against 6eeff6c

[romgrk]

Nice change.

Unrelated optimization but we could speed this further by replacing the object params with regular params.

[cherniavskii]

Unrelated optimization but we could speed this further by replacing the object params with regular params.

Thanks for the suggestion, I'll give it a try!

[cherniavskii]

Changing to regular parameters didn't bring significant improvements (547ms median compared to 548ms before the change). EDIT: it might have a bigger impact on memory footprint, but I didn't track it for this PR.
I'll keep it anyway. I also spotted a couple of potential small optimizations around that part of the code.