Skip to content

Commit

Permalink
[GLUTEN-5731][CORE] Fix the logic to calculate shuffle write time in …
Browse files Browse the repository at this point in the history
…RssPartitionWriter (#5742)

Co-authored-by: yibing <[email protected]>
  • Loading branch information
hahazyb201 and yibing authored May 15, 2024
1 parent a53ecc4 commit 2984bd7
Showing 1 changed file with 1 addition and 2 deletions.
3 changes: 1 addition & 2 deletions cpp/core/shuffle/rss/RssPartitionWriter.cc
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,6 @@ arrow::Status RssPartitionWriter::evict(
bool reuseBuffers,
bool hasComplexType) {
rawPartitionLengths_[partitionId] += inMemoryPayload->getBufferSize();

ScopedTimer timer(&spillTime_);
auto payloadType = (codec_ && inMemoryPayload->numRows() >= options_.compressionThreshold)
? Payload::Type::kCompressed
: Payload::Type::kUncompressed;
Expand All @@ -69,6 +67,7 @@ arrow::Status RssPartitionWriter::evict(
payload = nullptr; // Invalidate payload immediately.

// Push.
ScopedTimer timer(&spillTime_);
ARROW_ASSIGN_OR_RAISE(auto buffer, rssBufferOs->Finish());
bytesEvicted_[partitionId] += rssClient_->pushPartitionData(
partitionId, reinterpret_cast<char*>(const_cast<uint8_t*>(buffer->data())), buffer->size());
Expand Down

0 comments on commit 2984bd7

Please sign in to comment.