Chunk pool format asyn #2775
Chunk pool format asyn #2775
Conversation
|
cicheck |
Vigor-jpg
force-pushed
the
format
branch
3 times, most recently
from
830ebcf
to
4541895
Compare
|
cicheck |
|
cicheck |
|
cicheck |
|
cicheck |
2 similar comments
|
cicheck |
|
cicheck |
conf/chunkserver.conf.example
Outdated
| @@ -199,6 +199,10 @@ chunkfilepool.clean.enable=true | |||
| chunkfilepool.clean.bytes_per_write=4096 | |||
| # The throttle iops for cleaning chunk (4KB/IO) | |||
| chunkfilepool.clean.throttle_iops=500 | |||
| # The size of chunkfilepool | |||
| chunkfilepool.chunk_file_pool_size=0 | |||
There was a problem hiding this comment.
consistent with chunkserver.conf
proto/heartbeat.proto
Outdated
| @@ -89,6 +89,8 @@ message ChunkServerStatisticInfo { | |||
| required uint64 chunkSizeTrashedBytes = 7; | |||
| // chunkfilepool的大小 | |||
| optional uint64 chunkFilepoolSize = 8; | |||
| // chunkfilepool格式化进度 | |||
| } | ||
|
|
||
| formatStat_.preAllocateNum = needSpace / bytesPerPage; | ||
| formatStat_.allocateChunkNum = 0; |
There was a problem hiding this comment.
There's no problem initializing it to zero. If we need to create a filepool which chunk number is 10, and there are already 3 chunks in this chunkfile pool, then the preAllocateNum is 7, allocateChunkNum is 0.
| @@ -192,6 +195,14 @@ int FilePoolHelper::DecodeMetaInfoFromMetaFile( | |||
| break; | |||
| } | |||
|
|
|||
| if (!value[kChunkNum].isNull()) { | |||
There was a problem hiding this comment.
How to be compatible with meta files of older versions so that they can be upgraded
|
cicheck |
1 similar comment
|
cicheck |
|
cicheck |
1 similar comment
|
cicheck |
|
cicheck |
|
cicheck |
|
cicheck |
1 similar comment
|
cicheck |
|
CI failed, http://59.111.91.248:8080/job/curve_untest_job/7785/console you can build and run test locally |
|
cicheck |
3 similar comments
|
cicheck |
|
cicheck |
|
cicheck |
|
cicheck |
|
cicheck |
|
cicheck |
|
cicheck |
1 similar comment
|
cicheck |
|
cicheck |
|
cicheck |
1 similar comment
|
cicheck |
Signed-off-by: yyyyufeng <[email protected]>
|
cicheck |
1 similar comment
|
cicheck |
This problem is caused by insufficient disk space when we use percentage to allocate chunks in the test environment. It has been fixed now. |
What problem does this PR solve?
Issue Number: #2261
Problem Summary:
The time for format chunkfilepool is too long.
What is changed and how it works?
What's Changed:
Make it in backgroud.
How it Works:
Now we start a backgroud thread to format chunkfilepool while we startting chunkserver
Side effects(Breaking backward compatibility? Performance regression?):
Check List