20240727 Weekly

[Tangruilin]

需 review issue && PR

issue

遗留 issue:

• 如何同步 pika 的增量数据到 hbase 或其它nosql系统？ pikiwidb#356
  Do these memory optimization apply to pika? pikiwidb#329

PR

• restore the default behavior to 8M Block Cache per CF pikiwidb#361 [待 review @hotjump @Tangruilin ]
  bugfix when the configuration file does not have a write-binlog item,… pikiwidb#349 (网络库重构)
  support slowlog command pikiwidb#338 [@dingxiaoshuai123 @panlei-coder ]

Summer Code

题目	负责人	进度
PikiwiDB 缓存层实现冷热数据分离	沈梦菊	1. redis cache 依赖引入完成, redis cache 类封装完成；2. 下周同步 PCache 进度
PikiwiDB 实现集群模式	@longfar-ncy	配置文件可能需要支持嵌套。任务描述清晰以后 @longfar-ncy 发送到微信群 @Tangruilin 整理为 issue
实现 Pika 高性能流量转发代理	@黄恺一	Task 模块进行中

core

Item	Priority	Issue/Pr	Owner	Description
feat: add client command	P0	OpenAtomFoundation/pikiwidb#361		@长治 Review
feat: add sort commands	P0	OpenAtomFoundation/pikiwidb#357		@长治 Review
refactor:Replacement network library	P0	OpenAtomFoundation/pikiwidb#349		@dxs Review
feat: enable wal for rocksdb in the single-node scenario	P0	OpenAtomFoundation/pikiwidb#340	trl
add info section: server stats cpu commandstats	P1	OpenAtomFoundation/pikiwidb#338	smj	@dxs @LanaO Review
feat: support read consistency	P1	OpenAtomFoundation/pikiwidb#336	pl
loading pikiwidb.conf overuses atomic variables	P1	OpenAtomFoundation/pikiwidb#329	lihuan
add infoCmd section:server stats cpu commandstats	P1	OpenAtomFoundation/pikiwidb#326	smj
distinguish between normal node startup and snapshot loading	P1	OpenAtomFoundation/pikiwidb#319	pl
Check fd number	P1	OpenAtomFoundation/pikiwidb#315	lyc
将存储引擎中直接写 Rocksdb 的部分换成调用封装的 Batch 接口	P1	OpenAtomFoundation/pikiwidb#280
add command object pool	P2	OpenAtomFoundation/pikiwidb#260	lqx
feat: Add online profiling functionality	P1	OpenAtomFoundation/pikiwidb#256
modify readme	P1	OpenAtomFoundation/pikiwidb#230
主从与 raft 的切换	P1	OpenAtomFoundation/pikiwidb#342	pl
Pika 主从复制一些经验教训	P1	OpenAtomFoundation/pikiwidb#176
command develop tasks	P1	OpenAtomFoundation/pikiwidb#30
降低 Pika 碎片化监控，提升 compact 性能	P1	OpenAtomFoundation/pikiwidb#2255	gukaijie

todo:

• 1 binlog for master-slave; raft; slot
• 2 master -> slave replication： (1) using redis command SYNC&PSYNC; (2) using RocksDB WAL instead of pika binlog by RocksDB API GetUpdateSince()
• 3 redis cache
• 4 info command
• 5 hello command

存储模型 floyd

• issue bugfix config set maxclients don't work pikiwidb#269

Item	Priority	Issue/Pr	Owner	Description
添加关闭 RocksDB wal 的接口	P0		dxs	done
先把 floyd 移植过来	P0		dxs	done
一个 db 内一个 key 只能有一个 类型	P0

raft 集群进度

• issue bugfix SRandmemberCmd::Do() pikiwidb#277

State	Item	Priority	Issue/Pr	Owner	Description
已合并	引入 braft	p0	OpenAtomFoundation/pikiwidb#130	@panlei-coder, @longfar-ncy , @dingxiaoshuai123
已合并	redis raft 指令	p0	OpenAtomFoundation/pikiwidb#136	@panlei-coder, @KKorpse, @hotjump
已合并	braft log index 映射	p0	OpenAtomFoundation/pikiwidb#246	@longfar-ncy, @hotjump
已合并	pikiwidb checkpoint	p0	OpenAtomFoundation/pikiwidb#226	@dingxiaoshuai123, @hotjump
已合并	apply 支持	p0	OpenAtomFoundation/pikiwidb#213	@longfar-ncy, @hotjump
merged	braft 快照自定义日志截断点	p0	arana-db/braft#2	@panlei-coder, @hotjump
merged	raft 快照支持	p0	OpenAtomFoundation/pikiwidb#238, OpenAtomFoundation/pikiwidb#245	@dingxiaoshuai123, @panlei-coder, @hotjump	第二个 pr 包含第一个 pr
merged	将 import-braft 合并到 unstable	p0	OpenAtomFoundation/pikiwidb#300
	braft 主从模式	p1	arana-db/braft#1	@KKorpse
	pikiwidb 主从命令	p1		@panlei-coder	依赖于 braft 主从代码
	优雅关闭 pikiwidb	p1		@dingxiaoshuai123	和 snapshot 相关
	logindex 维护的多队列优化	p1		@longfar-ncy	思路见OpenAtomFoundation/pikiwidb#265
	flush event 时机	p1
	单独 cf 保存其它 cf flush 位点	p1
	braft 新增接口，判断两个 logindex 之间数据量	p1
	on_snapshot_load 区分自己安装快照 vs 自己正在启动	p0	OpenAtomFoundation/pikiwidb#319	@panlei-coder	非优化，正常功能
	on_snapshot_load中设置自定义的快照回放点	p1	OpenAtomFoundation/pikiwidb#319	@panlei-coder	优化
	raft 集群操作文档	p1
	raft 测试	p1			参考 redis-raft https://github.com/RedisLabs/redisraft/tree/master/tests

Pika binlog + braft -> 1 经典模式； 2 raft 模式。

可观测

Item	Priority	Issue/Pr	Owner	Description

工具集

Item	Priority	Issue/Pr	Owner	Description

云原生

Item	Priority	Issue/Pr	Owner	Description

质量建设

Item	Priority	Issue/Pr	Owner	Description
feature: add go test for pikiwidb from pika	P0	OpenAtomFoundation/pikiwidb#75
TCL 测试错误汇总	P0	OpenAtomFoundation/pikiwidb#266