Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[blockchain] Add database file size metric #4185

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
+29 −7
Open

[blockchain] Add database file size metric #4185

Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5a67668
add dbfile size metric
envestcc Mar 14, 2024
4eb0144
move db size metric into boltdb
envestcc Mar 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 44 additions & 1 deletion blockchain/blockchain.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ package blockchain

import (
"context"
"os"
"strconv"
"sync"
"sync/atomic"
Expand All @@ -28,12 +29,15 @@ import (
"github.com/iotexproject/iotex-core/pkg/lifecycle"
"github.com/iotexproject/iotex-core/pkg/log"
"github.com/iotexproject/iotex-core/pkg/prometheustimer"
"github.com/iotexproject/iotex-core/pkg/routine"
)

// const
const (
SigP256k1 = "secp256k1"
SigP256sm2 = "p256sm2"

fileSizeCollectInterval = 30 * time.Second
envestcc marked this conversation as resolved.
Show resolved Hide resolved
)

var (
Expand All @@ -45,6 +49,14 @@ var (
},
[]string{"type"},
)
fileSizeMtc = prometheus.NewGaugeVec(
envestcc marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three ways to monitor the file size:

  1. Export the metrics directly from the cloud service.
  2. Run a daemon service or process to report the metrics.
  3. Monitor the metrics in db_bolt.go.

The last one is the least preferred if the first two can't work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for 1, it should only be able to monitor at the mount level and cannot monitor specific files.
for 2, it may be a heavy changes for this small requirement.
I think 3 is a good approach that can also address the monitoring of chain-000000xx.db files mentioned above.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for 2, there have been existing tools to use, like https://github.com/prometheus/node_exporter

prometheus.GaugeOpts{
Name: "iotex_file_size_metrics",
Help: "File size metrics.",
},
[]string{"path"},
)

// ErrInvalidTipHeight is the error returned when the block height is not valid
ErrInvalidTipHeight = errors.New("invalid tip height")
// ErrInvalidBlock is the error returned when the block is not valid
Expand All @@ -59,6 +71,7 @@ var (

func init() {
prometheus.MustRegister(_blockMtc)
prometheus.MustRegister(fileSizeMtc)
}

type (
Expand Down Expand Up @@ -190,7 +203,29 @@ func NewBlockchain(cfg Config, g genesis.Genesis, dao blockdao.BlockDAO, bbf Blo
}
chain.lifecycle.Add(chain.dao)
chain.lifecycle.Add(chain.pubSubManager)

// add file size collect task
files := []string{
cfg.ChainDBPath,
envestcc marked this conversation as resolved.
Show resolved Hide resolved
cfg.TrieDBPath,
cfg.TrieDBPatchFile,
cfg.IndexDBPath,
cfg.BloomfilterIndexDBPath,
cfg.CandidateIndexDBPath,
cfg.StakingIndexDBPath,
cfg.SGDIndexDBPath,
cfg.ContractStakingIndexDBPath,
}
collectTask := routine.NewRecurringTask(func() {
for _, file := range files {
size, err := fileSize(file)
if err != nil {
log.L().Warn("Failed to get file size.", zap.Error(err))
continue
}
fileSizeMtc.WithLabelValues(file).Set(float64(size))
}
}, fileSizeCollectInterval)
chain.lifecycle.Add(collectTask)
return chain
}

Expand Down Expand Up @@ -489,3 +524,11 @@ func (bc *blockchain) emitToSubscribers(blk *block.Block) {
}
bc.pubSubManager.SendBlockToSubscribers(blk)
}

func fileSize(path string) (int64, error) {
info, err := os.Stat(path)
if err != nil {
return 0, err
}
return info.Size(), nil
}
Loading