Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pika中存在较多无意义的binlog记录 #2417

Open
u6th9d opened this issue Feb 26, 2024 · 10 comments
Open

pika中存在较多无意义的binlog记录 #2417

u6th9d opened this issue Feb 26, 2024 · 10 comments
Assignees
Labels
3.5.4 ☢️ Bug Something isn't working

Comments

@u6th9d
Copy link
Contributor

u6th9d commented Feb 26, 2024

Is this a regression?

Yes

Description

pika中存在较多无意义的binlog记录,比如对一个不存在的key执行lpop,del一些不存在的key,也会写binlog然后同步给从库

Please provide a link to a minimal reproduction of the bug

No response

Screenshots or videos

No response

Please provide the version you discovered this bug in (check about page for version information)

No response

Anything else?

No response

@u6th9d u6th9d added the ☢️ Bug Something isn't working label Feb 26, 2024
@chenbt-hz
Copy link
Collaborator

梳理了并测试了以下命令,都会存在这个情况:

列表(List)

  • LPOP key:移除并返回列表首元素。
  • RPOP key:移除并返回列表尾元素。
  • LREM key count value:根据参数count的值,移除与参数value相等的元素。

集合(Set)

  • SREM key member [member ...]:移除集合中一个或多个成员。

有序集合(Sorted Set)

  • ZREM key member [member ...]:移除有序集合中的一个或多个成员。

哈希(Hash)

  • HDEL key field [field ...]:删除一个或多个哈希表字段。

键(Key)

  • DEL key [key ...]:删除一个或多个键。
  • EXPIRE key seconds、PEXPIRE key milliseconds:设置键的过期时间(即使键当前不存在)。

HyperLogLog

  • PFADD key element [element ...]:向HyperLogLog添加元素,即使该键最终未被修改(例如,当添加的元素已经存在时),尝试添加的操作可能还是会被记录。

地理空间(Geo)

  • GEOADD key longitude latitude member [longitude latitude member ...]:向指定的key中添加一个或多个地理空间位置信息,即使这些操作没有改变已有数据,也可能被记录。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


After sorting out and testing the following commands, this situation will always exist:

List

  • LPOP key: Remove and return the first element of the list.
  • RPOP key: Remove and return the tail element of the list.
  • LREM key count value: Based on the value of parameter count, remove elements equal to parameter value.

Set

  • SREM key member [member ...]: Remove one or more members from the collection.

Sorted Set

  • ZREM key member [member ...]: Remove one or more members from an ordered set.

Hash

  • HDEL key field [field ...]: Delete one or more hash table fields.

Key

  • DEL key [key ...]: Delete one or more keys.
  • EXPIRE key seconds, PEXPIRE key milliseconds: Set the expiration time of the key (even if the key does not currently exist).

HyperLogLog

  • PFADD key element [element ...]: Add an element to HyperLogLog. Even if the key is not ultimately modified (for example, when the added element already exists), the attempted addition may still be logged.

Geospatial (Geo)

  • GEOADD key longitude latitude member [longitude latitude member ...]: Add one or more geospatial location information to the specified key. Even if these operations do not change existing data, they may be recorded.

@chenbt-hz
Copy link
Collaborator

我有疑问:是否有必要去将这个机制修改掉?
收益: 一般我们执行操作,大部分命令还是正常的,上面这些可能无用binlog是否会带了很多的资源浪费?
影响:对于主从,对于不存在的命令,slave重新执行这些命令,不会导致不一致。但是删除后,是否会存在某个情况,反而导致主从不一致?例如: master和slave存在时间差。master中key A value为5,A过期的下一瞬间,执行了自增的命令,此时master的A=1,而slave尚未过期,因此slave只执行了自增的命令,slave的A=6

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I have a question: Is it necessary to modify this mechanism?
Benefits: Generally, when we perform operations, most of the commands are normal. The above may be useless. Will the binlog bring a lot of waste of resources?
Impact: For master-slave, for commands that do not exist, slave will re-execute these commands without causing inconsistency. But after deletion, will there be a situation that will lead to master-slave inconsistency? For example: There is a time difference between master and slave. The key A value in the master is 5. The next moment A expires, the auto-increment command is executed. At this time, the master's A=1, but the slave has not expired, so the slave only executes the auto-increment command, and the slave's A=6.

@chenbt-hz
Copy link
Collaborator

综合上述,建议关闭这个issue

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Based on the above, it is recommended to close this issue

@AlexStocks
Copy link
Collaborator

把 case 梳理好,该删掉的无效日志还是删掉吧。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Sort out the cases and delete the invalid logs that should be deleted.

@chenbt-hz
Copy link
Collaborator

感觉可以把修改值命令的无意义binlog去掉,删除命令的binlog保留

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I feel like we can remove the meaningless binlog of the value modification command and keep the binlog of the delete command.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.5.4 ☢️ Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants