[Feature Request]: blob支持最大存储67,108,864字节(64MB)数据

[yangj1211]

Is there an existing issue for the same feature request?

• I have checked the existing issues.

Is your feature request related to a problem?

目前，MO 中的 BLOB 类型最大支持存储 64 KB 的数据，不足以处理较大的文件。

https://github.com/matrixorigin/matrixone/issues/20666

Describe the feature you'd like

更改为blob最大支持存储 67,108,864 字节（即 64 MB）的数据。

Describe implementation you've considered

No response

Documentation, Adoption, Use Case, Migration Strategy

No response

Additional information

No response

[allengaoo]

目前前线有一些项目需要保存非结构化数据，建议这个feature 能够在2.0.2 版本提供。

[qingxinhome]

This function has been implemented, please verify and test it @aressu1985 @yangj1211 @allengaoo @sukki37

[heni02]

2.0-dev commit：06354304c7daa5e4c104c84292559f431eaad942

1. insert 56.9MB大小压缩文件，测试结果不符合预期 [Bug]: blob length size less than max_allowed_packet ，but select blob reported ERROR Got packet bigger than 'max_allowed_packet' bytes #20880

mysql> create table table_blob(c1 int primary key,c2 blob);
Query OK, 0 rows affected (0.03 sec)

mysql> insert into table_blob values(834,LOAD_FILE('/Users/heni/Downloads/blob_test_data/blob04.tar.gz'));
Query OK, 1 row affected (0.66 sec)

mysql> select length(c2) from table_blob;
+------------+
| length(c2) |
+------------+
|   56874074 |
+------------+
1 row in set (0.12 sec)

mysql> select * from table_blob;
ERROR 2020 (HY000): Got packet bigger than 'max_allowed_packet' bytes

mysql> SHOW VARIABLES like "%max_allowed_packet%";
+--------------------+----------+
| Variable_name      | Value    |
+--------------------+----------+
| max_allowed_packet | 67108864 |
+--------------------+----------+
1 row in set (0.00 sec)

此处有疑问，max_allowed_packet为64M，c2 blob列长度没有超过该值，select还是报错

mysql：验证max_allowed_packet也是64M，但是select blob和lenth都返回NULL值，没有报错

insert 4M的文件，也是显示NULL

2. insert 4.1MB jpeg图片文件,select blob列返回值

mysql> insert into table_blob values(1,LOAD_FILE('/Users/heni/Downloads/blob_test_data/blob01.jpg'));
Query OK, 1 row affected (0.66 sec)

mysql> select * from table_blob where c1=1;
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)

3.insert 大于64M的文件（99M），insert报错，符合预期

mysql> insert into table_blob values(3,LOAD_FILE('/Users/heni/test_data/100w20col.tar.gz'));
ERROR 20101 (HY000): internal error: Data too long for blob

4.insert 9.7M音频文件，select查询返回值

mysql> insert into table_blob values(1,LOAD_FILE('/Users/heni/Downloads/blob_test_data/blob3.ncm'));
Query OK, 1 row affected (0.27 sec)

mysql> select * from table_blob;
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.03 sec)

mysql> select length(c2) from table_blob;
+------------+
| length(c2) |
+------------+
|    9678155 |
+------------+
1 row in set (0.01 sec)

5.插入10300行9.7M blob数据，select 300行显示符合预期

mysql> select count(*) from table_blob;
+----------+
| count(*) |
+----------+
|    10300 |
+----------+
1 row in set (0.01 sec)

mysql> show create table table_blob;
+------------+------------------------------------------------------------------------------------------------------------------+
| Table      | Create Table                                                                                                     |
+------------+------------------------------------------------------------------------------------------------------------------+
| table_blob | CREATE TABLE `table_blob` (
  `c1` int NOT NULL AUTO_INCREMENT,
  `c2` blob DEFAULT NULL,
  PRIMARY KEY (`c1`)
) |
+------------+------------------------------------------------------------------------------------------------------------------+
1 row in set (0.01 sec)
mysql> select length(c2) from table_blob limit 3;
+------------+
| length(c2) |
+------------+
|    9678155 |
|    9678155 |
|    9678155 |
+------------+
3 rows in set (0.21 sec)

[heni02]

#20800 已解决，test done