[Bug]: CN crashed by oom when running Insert on dup with 1000000 basic data and 1000 thread for about 20 minutes

[aressu1985]

Is there an existing issue for the same bug?

• I have checked the existing issues.

Branch Name

2.0-dev

Commit ID

6c37ff0

Other Environment Information

- Hardware parameters:
3*CN: 16C 64G
1*DN: 16C 64G
3*LOG: 4C 16G
3*PROXY: 3C 7G
- OS type:
- Others:

Actual Behavior

sql:
insert into sbtest6 values(?,3214,'83868641912-28773972837-60736120486-75162659906-27563526494-20381887404-41576422241-93426793964-56405065102-33518432330','67847967377-48000963322-62604785301-91415491898-96926520291') on duplicate key update c='aaa';

ddl:
mysql> show create table sbtest1;
+---------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+---------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| sbtest1 | CREATE TABLE sbtest1 (
id int NOT NULL,
k int NOT NULL DEFAULT 0,
c char(120) NOT NULL DEFAULT '',
pad char(60) NOT NULL DEFAULT '',
PRIMARY KEY (id),
KEY k_1 (k)
) |
+---------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

conflict rate is 100%

heap :
https://grafana.ci.matrixorigin.cn/explore?panes=%7B%22OeX%22:%7B%22datasource%22:%22pyroscope%22,%22queries%22:%5B%7B%22groupBy%22:%5B%5D,%22labelSelector%22:%22%7Bpod%20%3D%20%5C%22nightly-regression-dis-tp-cn-788hd%5C%22%7D%22,%22queryType%22:%22both%22,%22refId%22:%22A%22,%22datasource%22:%7B%22type%22:%22grafana-pyroscope-datasource%22,%22uid%22:%22pyroscope%22%7D,%22profileTypeId%22:%22memory:inuse_space:bytes:space:bytes%22%7D%5D,%22range%22:%7B%22from%22:%221733644908363%22,%22to%22:%221733645073929%22%7D%7D%7D&schemaVersion=1&orgId=1

Expected Behavior

No response

Steps to Reproduce

1. git clone mo-sysbench and mo-load
2. prepare data:
   cd mo-sysbench 
   sysbench  --mysql-host=xxxxx --mysql-port=6001 --mysql-user=dump --mysql-password=111 oltp_common.lua --mysql-db=sysbench_db --tables=10 --table_size=1000000 --threads=100 --time=30 --report-interval=10    cleanup;
sysbench  --mysql-host=xxxxxx --mysql-port=6001 --mysql-user=dump --mysql-password=111 oltp_common.lua --mysql-db=sysbench_db --tables=10 --table_size=1000000 --threads=100 --time=30 --report-interval=10  --range_selects=off --point_selects=1 --auto-inc=off  prepare;

3. run test
   cd mo-load
   ./start.sh -c cases/sysbench/simple_insert_duplicate_10_1000000_prepare -h xxxxxx -t 1000 -d 50 -g -b sysbench_db

Additional information

No response

[heni02]

单机insert duplicate并发1000，主键冲突100%测试30分钟左右oom
2.0-dev：
oom时间：Dec 9 10:54:53

Dec 9 10:53 heap/malloc/cpu文件：
hn_download (1).zip

复现步骤同上

[aunjgr]

这问题是一直都有的，还是最近才出现的？

[aunjgr]

not working on it today

[ouyuanning]

刚接手。明天看

[ouyuanning]

从profile看。heap占用如下：

堆外占用如下：

[ouyuanning]

我在该commit id和最新的main下，好像是会panic的

[ouyuanning]

panic的先在这里处理 #20879

[ouyuanning]

未投入

[ouyuanning]

未投入

[ouyuanning]

未投入