Skip to content

Latest commit

 

History

History
206 lines (164 loc) · 29.8 KB

release-8.1.1.md

File metadata and controls

206 lines (164 loc) · 29.8 KB
Raw
title summary
TiDB 8.1.1 Release Notes
Learn about the compatibility changes, improvements, and bug fixes in TiDB 8.1.1.

TiDB 8.1.1 Release Notes

Release date: August 27, 2024

TiDB version: 8.1.1

Quick access: Quick start | Production deployment

Compatibility changes

  • When using TiDB Lightning to import a CSV file, if you set strict-format = true to split a large CSV file into multiple small CSV files to improve concurrency and import performance, you need to explicitly specify terminator. The values can be \r, \n or \r\n. Failure to specify a line terminator might result in an exception when parsing the CSV file data. #37338 @lance6716
  • When using IMPORT INTO to import a CSV file, if you specify the SPLIT_FILE parameter to split a large CSV file into multiple small CSV files to improve concurrency and import performance, you need to explicitly specify the line terminator LINES_TERMINATED_BY. The values can be \r, \n or \r\n. Failure to specify a line terminator might result in an exception when parsing the CSV file data. #37338 @lance6716
  • Change the default value of the tidb_enable_parallel_hashagg_spill variable from ON to OFF to avoid incorrect query results caused by disk spill during parallel calculation. For clusters upgraded to v8.1.1 from v8.0.0 or v8.1.0, this variable will remain at its previous default value of ON after the upgrade, and it is recommended to manually change it to OFF. #55290 @xzhangxian1008

Offline package changes

In v8.1.1, arbiter is removed from the TiDB-community-toolkit binary package.

Improvements

  • TiDB

    • By batch deleting TiFlash placement rules, improve the processing speed of data GC after performing the TRUNCATE or DROP operation on partitioned tables #54068 @Lloyd-Pottiger
    • Remove stores without Regions during MPP load balancing #52313 @xzhangxian1008
    • Temporarily adjust the priority of statistics synchronously loading tasks to high to avoid widespread timeouts during TiKV high loads, as these timeouts might result in statistics not being loaded #50332 @winoros
    • The EXPLAIN statement supports applying the tidb_redact_log setting and further optimizes the log handling logic.
    • Support applying the tidb_redact_log setting to the output of EXPLAIN statements and further optimize the logic in processing logs #54565 @hawkingrei

  • PD

  • TiFlash

    • Mitigate the issue that TiFlash might panic due to updating certificates after TLS is enabled #8535 @windtalker
    • Reduce lock conflicts under highly concurrent data read operations and optimize short query performance #9125 @JinheLin

  • Tools

    • Backup & Restore (BR)

      • Support encryption of temporary files generated during log backup #15083 @YuJuncen
      • Except for the br log restore subcommand, all other br log subcommands support skipping the loading of the TiDB domain data structure to reduce memory consumption #52088 @Leavrth
      • Support setting Alibaba Cloud access credentials through environment variables #45551 @RidRisR
      • Support checking whether the disk space in TiKV is sufficient before TiKV downloads each SST file. If the space is insufficient, BR terminates the restore and returns an error #17224 @RidRisR

    • TiCDC

      • Support sending BOOTSTRAP messages of all tables to the downstream in one go when a changefeed using Simple Protocol starts #11315 @asddongmen
      • Support directly outputting raw events when the downstream is a Message Queue (MQ) or cloud storage #11211 @CharlesCheung96

Bug fixes

  • TiDB

    • Fix the issue that the disk spill for the HashAgg operator causes incorrect query results during parallel calculation #55290 @xzhangxian1008
    • Fix the issue that INDEX_HASH_JOIN cannot exit properly when SQL is abnormally interrupted #54688 @wshwsh12
    • Fix the issue that RANGE partitioned tables that are not strictly self-incrementing can be created #54829 @Defined2014
    • Fix the issue that PointGet execution plans for _tidb_rowid can be generated #54583 @Defined2014
    • Fix the issue that internal SQL statements in the slow log are redacted to null by default #54190 #52743 #53264 @lcwangchao
    • Fix the issue that the UPDATE operation can cause TiDB OOM in multi-table scenarios #53742 @hawkingrei
    • Fix the issue that the Window function might panic when there is a related subquery in it #42734 @hi-rustin
    • Fix the issue that the LENGTH() condition is unexpectedly removed when the collation is utf8_bin or utf8mb4_bin #53730 @elsa0520
    • Fix the issue that after a statement within a transaction is killed by OOM, if TiDB continues to execute the next statement within the same transaction, you might get an error Trying to start aggressive locking while it's already started and a panic occurs #53540 @MyonKeminta
    • Fix the issue that PREPARE/EXECUTE statements with the CONV expression containing a ? argument might result in incorrect query results when executed multiple times #53505 @qw4990
    • Fix the issue that the recursive CTE operator incorrectly tracks memory usage #54181 @guo-shaoge
    • Fix the issue that using SHOW WARNINGS; to obtain warnings might cause a panic #48756 @xhebox
    • Fix the issue that the TopN operator might be pushed down incorrectly #37986 @qw4990
    • Fix the issue that TiDB panics when executing the SHOW ERRORS statement with a predicate that is always true #46962 @elsa0520
    • Fix the issue that the STATE field in the INFORMATION_SCHEMA.TIDB_TRX table is empty due to the size of the STATE field not being defined #53026 @cfzjywxk
    • Fix the issue that executing the SELECT DISTINCT CAST(col AS DECIMAL), CAST(col AS SIGNED) FROM ... query might return incorrect results #53726 @hawkingrei
    • Fix the issue that DDL statements incorrectly use etcd and cause tasks to queue up #52335 @wjhuang2016
    • Fix the issue that the Distinct_count information in GlobalStats might be incorrect #53752 @hawkingrei
    • Fix the issue that the tidb_enable_async_merge_global_stats and tidb_analyze_partition_concurrency system variables do not take effect during automatic statistics collection #53972 @hi-rustin
    • Fix the issue that the TIMESTAMPADD() function goes into an infinite loop when the first argument is month and the second argument is negative #54908 @xzhangxian1008
    • Fix the issue that the Connection Count monitoring metric in Grafana is incorrect when some connections exit before the handshake is complete #54428 @YangKeao
    • Fix the issue that the Connection Count of each resource group is incorrect when using TiProxy and resource groups #54545 @YangKeao
    • Fix the issue that using a view does not work in recursive CTE #49721 @hawkingrei
    • Fix the issue that the final AggMode and the non-final AggMode cannot coexist in Massively Parallel Processing (MPP) #51362 @AilinKid
    • Fix the issue of incorrect WARNINGS information when using Optimizer Hints #53767 @hawkingrei
    • Fix the issue of abnormally high memory usage caused by memTracker not being detached when the HashJoin or IndexLookUp operator is the driven side sub-node of the Apply operator #54005 @XuHuaiyu
    • Fix the issue that the illegal column type DECIMAL(0,0) can be created in some cases #53779 @tangenta
    • Fix the issue of potential data races during the execution of (*PointGetPlan).StatsInfo() #49803 #43339 @qw4990
    • Fix the issue that improper use of metadata locks might lead to writing anomalous data when using the plan cache under certain circumstances #53634 @zimulala
    • Fix the issue that JSON-related functions return errors inconsistent with MySQL in some cases #53799 @dveeden
    • Fix the issue that TiDB does not create corresponding statistics metadata (stats_meta) when creating a table with foreign keys #53652 @hawkingrei
    • Fix the issue that the memory_quota hint might not work in subqueries #53834 @qw4990
    • Fix the issue that TiDB might report an error due to GC when loading statistics at startup #53592 @you06
    • Fix the issue that executing CREATE OR REPLACE VIEW concurrently might result in the table doesn't exist error #53673 @tangenta
    • Fix the issue that the query latency of stale reads increases, caused by information schema cache misses #53428 @crazycs520
    • Fix the issue that SELECT INTO OUTFILE does not work when clustered indexes are used as predicates #42093 @qw4990
    • Fix the issue that comparing a column of YEAR type with an unsigned integer that is out of range causes incorrect results #50235 @qw4990
    • Fix the issue that TiDB might return incorrect query results when you query tables with virtual columns in transactions that involve data modification operations #53951 @qw4990
    • Fix the issue that TiDB fails to reject unauthenticated user connections in some cases when using the auth_socket authentication plugin #54031 @lcwangchao
    • Fix the issue that when queries contain non-correlated subqueries and LIMIT clauses, column pruning might be incomplete, resulting in a less optimal plan #54213 @qw4990
    • Fix the issue that non-BIGINT unsigned integers might produce incorrect results when compared with strings/decimals #41736 @LittleFall
    • Fix the issue that setting max-index-length causes TiDB to panic when adding indexes using the Distributed eXecution Framework (DXF) #53281 @zimulala
    • Fix the issue that certain filter conditions in queries might cause the planner module to report an invalid memory address or nil pointer dereference error #53582 #53580 #53594 #53603 @YangKeao
    • Fix the issue that recursive CTE queries might result in invalid pointers #54449 @hawkingrei
    • Fix the overflow issue of the Longlong type in predicates #45783 @hawkingrei
    • Fix the issue that indirect placeholder ? references in a GROUP BY statement cannot find columns #53872 @qw4990
    • Fix the issue that the memory used by transactions might be tracked multiple times #53984 @ekexium
    • Fix the issue that using CURRENT_DATE() as the default value for a column results in incorrect query results #53746 @tangenta
    • Fix the issue that the performance is unstable when adding indexes using Global Sort #54147 @tangenta
    • Fix the issue that SHOW IMPORT JOBS reports an error Unknown column 'summary' after upgrading from v7.1 #54241 @tangenta
    • Fix the issue that root user cannot query tidb_mdl_view #53292 @tangenta
    • Fix the issue that the network partition during adding indexes using the Distributed eXecution Framework (DXF) might cause inconsistent data indexes #54897 @tangenta
    • Fix the issue that the error occurred during initialization of the TiDB Lightning physical import mode might cause resource leaks #53659 @D3Hunter
    • Fix the issue that obtaining the column information using information_schema.columns returns warning 1356 when a subquery is used as a column definition in a view definition #54343 @lance6716
    • Fix the issue that using index acceleration to add a unique index might cause a Duplicate entry error when the owner is switched #49233 @lance6716
    • Fix the unclear error message when setting global.tidb_cloud_storage_uri #54096 @lance6716
    • Fix the issue that the Sync Load QPS monitoring metric is incorrect #53558 @hawkingrei
    • Fix the issue that some statistics information might be missed when loading initial statistics concurrently #53607 @hawkingrei
    • Fix the issue of reusing wrong point get plans for SELECT ... FOR UPDATE #54652 @qw4990

  • TiKV

    • Fix the issue that CDC and log-backup do not limit the timeout of check_leader using the advance-ts-interval configuration, causing the resolved_ts lag to be too large when TiKV restarts normally in some cases #17107 @MyonKeminta
    • Fix the issue that setting the gRPC message compression method via grpc-compression-type does not take effect on messages sent from TiKV to TiDB #17176 @ekexium
    • Fix the failure of make docker and make docker_test #17075 @shunki-fujita
    • Fix the issue that the gRPC request sources duration metric is displayed incorrectly in the monitoring dashboard #17133 @King-Dylan
    • Fix the issue that the output of the raft region command in tikv-ctl does not include the Region status information #17037 @glorv
    • Fix the issue that changing the raftstore.periodic-full-compact-start-times configuration item online might cause TiKV to panic #17066 @SpadeA-Tang
    • Fix the issue that TiKV might repeatedly panic when applying a corrupted Raft data snapshot #15292 @LykxSassinator
    • Fix the issue that releasing cache entries before they are persisted causes TiKV to panic #17040 @glorv

  • PD

    • Fix the issue that an incorrect PD API is called when you retrieve table attributes #55188 @JmPotato
    • Fix the issue that the time data type in the INFORMATION_SCHEMA.RUNAWAY_WATCHES table is incorrect #54770 @HuSharp
    • Fix the issue that some logs are not redacted #8419 @rleungx
    • Fix the issue of missing data in the Filter monitoring metric #8098 @nolouch
    • Fix the issue that the HTTP client might panic when TLS is enabled #8237 @okJiang
    • Fix the issue that the encryption manager is not initialized before use #8384 @rleungx
    • Fix the issue that resource groups could not effectively limit resource usage under high concurrency #8435 @nolouch
    • Fix the data race issue related to store limit #8253 @lhy1024
    • Fix the issue that the scaling progress is displayed incorrectly after the scheduling microservice is enabled #8331 @rleungx
    • Fix the issue that the TSO node is not dynamically updated after the tso microservice is enabled #8154 @rleungx
    • Fix the data race issue of resource groups #8267 @HuSharp
    • Fix the issue that a resource group encounters quota limits when requesting tokens for more than 500 ms #8349 @nolouch
    • Fix the issue that manually transferring the PD leader might fail #8225 @HuSharp
    • Fix the issue that deleted nodes still appear in the candidate connection list in etcd client #8286 @JmPotato
    • Fix the issue that ALTER PLACEMENT POLICY cannot modify the placement policy #52257 #51712 @jiyfhust
    • Fix the issue that the scheduling of write hotspots might break placement policy constraints #7848 @lhy1024
    • Fix the issue that down peers might not recover when using Placement Rules #7808 @rleungx
    • Fix the issue that a large number of retries occur when canceling resource groups queries #8217 @nolouch
    • Fix the data race issue that PD encounters during operator checks #8263 @lhy1024
    • Fix the issue that no error is reported when binding a role to a resource group #54417 @JmPotato
    • Fix the issue that setting the TiKV configuration item coprocessor.region-split-size to a value less than 1 MiB causes PD panic #8323 @JmPotato

  • TiFlash

    • Fix the issue that a network partition (network disconnection) between TiFlash and any PD might cause read request timeout errors #9243 @Lloyd-Pottiger
    • Fix the issue that the SUBSTRING_INDEX() function might cause TiFlash to crash in some corner cases #9116 @wshwsh12
    • Fix the issue that a large number of duplicate rows might be read in FastScan mode after importing data via BR or TiDB Lightning #9118 @JinheLin
    • Fix the issue that TiFlash might panic when a database is deleted shortly after creation #9266 @JaySon-Huang
    • Fix the issue that setting the SSL certificate configuration to an empty string in TiFlash incorrectly enables TLS and causes TiFlash to fail to start #9235 @JaySon-Huang
    • Fix the issue that in the disaggregated storage and compute architecture, null values might be incorrectly returned in queries after adding non-null columns in DDL operations #9084 @Lloyd-Pottiger
    • Fix the issue that TiFlash might panic after executing RENAME TABLE ... TO ... on a partitioned table with empty partitions across databases #9132 @JaySon-Huang
    • Fix the issue of query timeout when executing queries on partitioned tables that contain empty partitions #9024 @JinheLin
    • Fix the issue that some queries might report a column type mismatch error after late materialization is enabled #9175 @JinheLin
    • Fix the issue that queries with virtual generated columns might return incorrect results after late materialization is enabled #9188 @JinheLin

  • Tools

    • Backup & Restore (BR)

      • Fix the issue that the backup performance during checkpoint backups is affected due to interruptions in seeking Region leaders #17168 @Leavrth
      • Fix the inefficiency issue in scanning DDL jobs during incremental backups #54139 @3pointer
      • Fix the issue that BR fails to correctly identify errors due to multiple nested retries during the restore process #54053 @RidRisR
      • Fix the issue that BR fails to restore a transactional KV cluster due to an empty EndKey #52574 @3pointer
      • Fix the issue that log backup might be paused after the advancer owner migration #53561 @RidRisR
      • Fix the issue that DDLs requiring backfilling, such as ADD INDEX and MODIFY COLUMN, might not be correctly recovered during incremental restore #54426 @3pointer
      • Fix the issue that a PD connection failure could cause the TiDB instance where the log backup advancer owner is located to panic #52597 @YuJuncen

    • TiCDC

      • Fix the issue that Region changes cause downstream panic #17233 @hicqu
      • Fix the issue that TiCDC fails to decode primary keys in clustered index tables correctly when the new collation is disabled in the upstream #11371 @lidezhu
      • Fix the issue that the checksum is not correctly set to 0 after splitting UPDATE events #11402 @3AceShowHand
      • Fix the issue that data inconsistency might occur when restarting Changefeed repeatedly when performing a large number of UPDATE operations in a multi-node environment #11219 @lidezhu
      • Fix the issue that the Processor module might get stuck when the downstream Kafka is inaccessible #11340 @asddongmen

    • TiDB Data Migration (DM)

      • Fix the issue that SET statements cause DM to panic during the migration of MariaDB data #10206 @dveeden
      • Fix the connection blocking issue by upgrading go-mysql #11041 @D3Hunter
      • Fix the issue that data replication is interrupted when the index length exceeds the default value of max-index-length #11459 @michaelmdeng
      • Fix the issue that schema tracker incorrectly handles LIST partition tables, causing DM errors #11408 @lance6716

    • TiDB Lightning

      • Fix the issue that TiDB Lightning outputs a confusing WARN log when it fails to obtain the keyspace name #54232 @kennytm

    • Dumpling

      • Fix the issue that Dumpling reports an error when exporting tables and views at the same time #53682 @tangenta

    • TiDB Binlog

      • Fix the issue that deleting rows during the execution of ADD COLUMN might report an error data and columnID count not match when TiDB Binlog is enabled #53133 @tangenta