diff --git a/CHANGES.md b/CHANGES.md index a7eb6ba972..e7f6a31c7a 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -2,11 +2,219 @@ This file details changes made to TrueBlocks over time. All version prior to version 3.0.0 are now no longer supported. -## v4.0.0 - Getting Serious - Indiana (2024/09/01) +## v3.5.0 - MultiChain Scraper - Jefferson (2024/09/22) **Summary** -- Bumped version to 4.0.0 +- Bumps version to 3.5.0 for all repos (that is all submodules are tagged with v3.5.0) +- Adds indexManager dev_tool (currently only checks disc usage). +- Improves "headless" modes for both chifra scrape and chifra init to produce much better log messages. +- Improves greatly `trueblocks-node` to allow for scraping mulitple chains at the same time in headless mode. +- Improves chifra chunks index --pin to better recover from errors during pinning (in fact, it didn't work previously). +- Small updates to documentation for SDK, node, and examples. +- Added dev_tool called indexManager which currently only reports on disc usage, but will eventually report on index status. + +## SDK + +- Minor changes to improve DX. + +## Changes to the Unchained Index Specification + +- No changes. + +## Breaking Changes + +- Added trueblocks-node which runs the scraper as a separate process. This hints at a future where `chifra scrape` is deprecated. It is not yet deprecated, but it will be one day. The same applies to chifra monitors --watch which will also eventually be part of trueblocks-node. (Not a breaking change, just hinting at a future breaking change.) + +## System Wide Changes + +- Better headless mode for chifra init and chifra scrape + +## Changes to Data Models + +### Modified data models + +| model | description | +| ----- | ----------- | +| None | | + +### New data models + +| model | description | +| ----- | ----------- | +| None | | + +### Removed data models + +| model | description | +| ----- | ----------- | +| None | | + +### Renamed data models + +| model | description | +| ----- | ----------- | +| None | | + +## Tool Specific Changes + +**chifra list** + +- No changes. + +**chifra export** + +- No changes. + +**chifra monitors** + +- No changes. + +**chifra names** + +- No changes. + +**chifra abis** + +- No changes. + +**chifra blocks** + +- No changes. + +**chifra transactions** + +- No changes. + +**chifra receipts** + +- No changes. + +**chifra logs** + +- No changes. + +**chifra traces** + +- No changes. + +**chifra when** + +- No changes. + +**chifra state** + +- No changes. + +**chifra tokens** + +- No changes. + +**chifra config** + +- No changes. + +**chifra status** + +- No changes. + +**chifra daemon** + +- No changes. + +**chifra scrape** + +- Adds TB_NODE_HEADLESS environment variable to chifra scrape to allow for headless operation of the init command. +- Greatly improves logging in chifra scrape to make it easier to run under headless mode. + +**chifra chunks** + +- Improves chifra chunks manifest --pin to correctly recover from errors generated during pinning. Previously, an error would cause the manifest to contain empty CIDs for the missed pin. Now, the process will not continue until the error is resolved. +- Improves chifra chunks index --check to check for empty CIDs in the manifest. Previously, the check would not catch empty CIDs. + +**chifra init** + +- Adds TB_NODE_HEADLESS environment variable to chifra init to allow for headless operation of the init command. + +**chifra explore** + +- No changes. + +**chifra slurp** + +- No changes. + +**makeClass** + +- No changes. + +**testRunner** + +- No changes. + +## Pull Requests (33) + + + +- #3888 Various changes to run scraper and init in headlessMode +- #3886 Adds a devtool to check disc usage on various computers +- #3885 Updates go.mods to latest release +- #3884 Develop +- #3882 Updates node +- #3880 Updates a few names +- #3878 Feature/better manifests +- #3876 Submodualarizatooor - moves a lot of code into submodules for MUCH easier mangement +- #3875 Version 3.3.0 +- #3874 Tries to start using SemiVers for our go.mod files +- #3871 Starts to improve chifra init +- #3870 Cleans up name editing +- #3869 Extends goMaker to generate sorting code for various types +- #3866 Adds --staged to chifra monitors +- #3865 Adds --count to chifra monitors +- #3863 Adds hasConstructor and hasFallback to Abi type; fixes tests +- #3862 Feature/better sorting +- #3860 Feature/chifra abis count +- #3858 Feature/better chunk data +- #3857 Feature/easier name editing in sdk +- #3856 Fixes a bug related to turning off loggers +- #3853 Fix/better use of filepath +- #3851 Assigns part type to names +- #3850 Feature/chifra chunks counts +- #3848 Adds name to monitor type and makes marshable +- #3847 Moves Parts from names package to types package because we need it in… +- #3846 Feature/improve chifra abis list +- #3845 Adds chifra abis --list for use in miniDapp +- #3844 Cleans up some caching code including allowing for filename caches (such as miniDapp files) +- #3842 Fix/better auto gen 2 +- #3840 Small improvement to auto-code-gen +- #3839 chore: fix some comments +- #3838 Updates go-mods and adds some names + +## Issues Closed (0) + + + +- None + +## Issues Opened (3) + + + +- #3883 Add this to every .gitignore at the top of all repos +- #3881 Minimum cmake version required is 3.20 - should be better documented (or better yet - automatically fail the build) +- #3879 chifra init + +## v3.4.0 - Getting Serious - Indiana (2024/09/01) + +**Summary** + +- Bumped version to 3.4.0 - Much better support for SDK - First working version of Browse (separate repo) - First semi-working version of TrueBlocks-Node (separate repo) diff --git a/VERSION b/VERSION index 15a2799817..1545d96657 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -3.3.0 +3.5.0 diff --git a/bin/.gitignore b/bin/.gitignore index fe7b337702..7a6b3667f1 100644 --- a/bin/.gitignore +++ b/bin/.gitignore @@ -4,6 +4,7 @@ SymbolCache.noindex chifra* goMaker +indexManager sdkFuzzer makeClass testRunner diff --git a/docs b/docs index 88f9be3c9f..744361bd5c 160000 --- a/docs +++ b/docs @@ -1 +1 @@ -Subproject commit 88f9be3c9fe7166555d0c979e24ad30d0236ae99 +Subproject commit 744361bd5c69b3d483170de3df2d9a023b603155 diff --git a/examples b/examples index f1eb168419..16ff71ce9d 160000 --- a/examples +++ b/examples @@ -1 +1 @@ -Subproject commit f1eb16841993d633aed45422305107b57840652b +Subproject commit 16ff71ce9d36bcbbfa842442a9517f12c952ba55 diff --git a/node b/node index 10d58595b4..c759d84234 160000 --- a/node +++ b/node @@ -1 +1 @@ -Subproject commit 10d58595b4f7cd54acacb50076641565a839c8b0 +Subproject commit c759d84234904ca3c27bb3712706bc5f8d0a7b60 diff --git a/scripts/go-mod-tidy.bat b/scripts/go-mod-tidy.bat index 8f90017500..f26178e264 100644 --- a/scripts/go-mod-tidy.bat +++ b/scripts/go-mod-tidy.bat @@ -10,17 +10,18 @@ set SCRIPT_DIR=%script_dir_backslash:~0,-1% set ROOT_DIR=%SCRIPT_DIR%\.. :: Find all go.mod files and tidy them -cd /d %ROOT_DIR%\examples\balanceChart && go mod tidy -cd /d %ROOT_DIR%\examples\cancelContext && go mod tidy -cd /d %ROOT_DIR%\examples\findFirst && go mod tidy -cd /d %ROOT_DIR%\examples\nameManager && go mod tidy -cd /d %ROOT_DIR%\examples\simple && go mod tidy -cd /d %ROOT_DIR%\examples\withStreaming && go mod tidy -cd /d %ROOT_DIR%\src\apps\chifra && go mod tidy -cd /d %ROOT_DIR%\src\node && go mod tidy -cd /d %ROOT_DIR%\src\dev_tools\goMaker && go mod tidy -cd /d %ROOT_DIR%\src\dev_tools\sdkFuzzer && go mod tidy -cd /d %ROOT_DIR%\src\dev_tools\testRunner && go mod tidy +cd /d %ROOT_DIR%\examples\balanceChart && go mod tidy +cd /d %ROOT_DIR%\examples\cancelContext && go mod tidy +cd /d %ROOT_DIR%\examples\findFirst && go mod tidy +cd /d %ROOT_DIR%\examples\nameManager && go mod tidy +cd /d %ROOT_DIR%\examples\simple && go mod tidy +cd /d %ROOT_DIR%\examples\withStreaming && go mod tidy +cd /d %ROOT_DIR%\src\apps\chifra && go mod tidy +cd /d %ROOT_DIR%\src\node && go mod tidy +cd /d %ROOT_DIR%\src\dev_tools\goMaker && go mod tidy +cd /d %ROOT_DIR%\src\dev_tools\indexManager && go mod tidy +cd /d %ROOT_DIR%\src\dev_tools\sdkFuzzer && go mod tidy +cd /d %ROOT_DIR%\src\dev_tools\testRunner && go mod tidy :: Return to the original directory and exit exit /b diff --git a/scripts/go-show-trueblocks.sh b/scripts/go-show-trueblocks.sh index d381f7b27b..5566e0fa72 100755 --- a/scripts/go-show-trueblocks.sh +++ b/scripts/go-show-trueblocks.sh @@ -1,10 +1,12 @@ #!/usr/bin/env bash # This script lists all uses of either chifra or the sdk in the go.mod files. Used for debugging. -find . -type f -name 'go.mod' -exec grep -His "TrueBlocks" {} ';' | \ - grep "2024" | \ - cut -f2 -d: | \ - sed 's/ \/\/ indirect//' | \ - sed 's/require //' | \ - tr '\t' '|' | sed 's/|//' | sed 's/^/ /' | \ - sort -u +find . -type f -name "go.mod" -exec grep -His TrueBlocks {} ';' | \ + grep -v module | \ + sed 's/ \/\/ indirect//' | \ + sed 's/require //' | \ + grep -v replace | \ + tr '\t' '|' | \ + sed 's/|//' | \ + cut -f2 -d: | \ + sort -u diff --git a/scripts/go-work-sync.bat b/scripts/go-work-sync.bat index 1ccb9e2d7b..1407f09984 100644 --- a/scripts/go-work-sync.bat +++ b/scripts/go-work-sync.bat @@ -24,6 +24,7 @@ cd %ROOT_DIR% && go work use .\examples\withStreaming cd %ROOT_DIR% && go work use .\src\apps\chifra cd %ROOT_DIR% && go work use .\src\node cd %ROOT_DIR% && go work use .\src\dev_tools\goMaker +cd %ROOT_DIR% && go work use .\src\dev_tools\indexManager cd %ROOT_DIR% && go work use .\src\dev_tools\sdkFuzzer cd %ROOT_DIR% && go work use .\src\dev_tools\testRunner diff --git a/scripts/go-work-sync.sh b/scripts/go-work-sync.sh index ab802bd86b..c0f1dcc76e 100755 --- a/scripts/go-work-sync.sh +++ b/scripts/go-work-sync.sh @@ -43,7 +43,7 @@ find . -type f -name 'go.mod' | while read -r modfile; do fi if ! $isSdk && ! $isChifra && ! $isGoMaker; then - go get github.com/TrueBlocks/trueblocks-core/sdk/v3@latest + go get github.com/TrueBlocks/trueblocks-sdk/v3@latest fi if ! $isSimple && ! $isChifra; then go get github.com/TrueBlocks/trueblocks-core/src/apps/chifra@latest diff --git a/sdk b/sdk index 79b2de1c30..9fa9e99861 160000 --- a/sdk +++ b/sdk @@ -1 +1 @@ -Subproject commit 79b2de1c302370225430f153a1fad6be985efe6a +Subproject commit 9fa9e99861deb30c77dbefef7456d607c62a1f55 diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 4741c0ac36..d23e1319c3 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -87,11 +87,13 @@ add_subdirectory(other/install) ADD_GO_INSTALLABLE_PROGRAM(chifra ${CMAKE_SOURCE_DIR}/apps/chifra ${BIN_DIR}) ADD_GO_INSTALLABLE_PROGRAM(goMaker ${CMAKE_SOURCE_DIR}/dev_tools/goMaker ${BIN_DIR}) +ADD_GO_INSTALLABLE_PROGRAM(indexManager ${CMAKE_SOURCE_DIR}/dev_tools/indexManager ${BIN_DIR}) if (NOT WIN32) ADD_GO_INSTALLABLE_PROGRAM(testRunner ${CMAKE_SOURCE_DIR}/dev_tools/testRunner ${BIN_DIR}) ADD_GO_INSTALLABLE_PROGRAM(sdkFuzzer ${CMAKE_SOURCE_DIR}/dev_tools/sdkFuzzer ${BIN_DIR}) add_dependencies(testRunner goMaker) add_dependencies(sdkFuzzer goMaker) + add_dependencies(indexManager goMaker) add_dependencies(chifra testRunner) endif() diff --git a/src/apps/chifra/internal/chunks/handle_check.go b/src/apps/chifra/internal/chunks/handle_check.go index ff6f5f32fd..e63279473b 100644 --- a/src/apps/chifra/internal/chunks/handle_check.go +++ b/src/apps/chifra/internal/chunks/handle_check.go @@ -155,6 +155,20 @@ func (opts *ChunksOptions) check(rCtx *output.RenderCtx, blockNums []base.Blknum } reports = append(reports, sizes) + // are all the hashes present? + contentCheck := types.ReportCheck{} + contentCheck.Reason = "Remote manifest contents" + if err := opts.CheckManContents(remoteManifest, &contentCheck); err != nil { + return err, false + } + reports = append(reports, contentCheck) + + contentCheck.Reason = "Local manifest contents" + if err := opts.CheckManContents(remoteManifest, &contentCheck); err != nil { + return err, false + } + reports = append(reports, contentCheck) + // compare with çached manifest with files on disc d2c := types.ReportCheck{Reason: "Disc files to cached manifest"} if err := opts.CheckManifest(fnArray, cacheArray, &d2c); err != nil { diff --git a/src/apps/chifra/internal/chunks/handle_check_manifest.go b/src/apps/chifra/internal/chunks/handle_check_manifest.go index 5508c4a6f3..14eeb56631 100644 --- a/src/apps/chifra/internal/chunks/handle_check_manifest.go +++ b/src/apps/chifra/internal/chunks/handle_check_manifest.go @@ -8,7 +8,10 @@ import ( "fmt" "strings" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/config" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/manifest" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/types" + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/version" ) type CompareState struct { @@ -22,9 +25,9 @@ type CompareState struct { // failB int } -// CheckManifest takes two arrays (either onDisc vs. LocalManifest, onDisc vs. RemoteManifest, or LocalManifest -// vs. RemoteManifest) and compares them for equality. If everything is up to date, all three arrays should -// be identical. Only the block ranges are in the arrays. +// CheckManifest takes two arrays (either onDisc vs. LocalManifest, onDisc vs. RemoteManifest, +// or LocalManifest vs. RemoteManifest) and compares them for equality. If everything is up +// to date, all three arrays should be identical. Only the block ranges are in the arrays. func (opts *ChunksOptions) CheckManifest(arrayA, arrayB []string, report *types.ReportCheck) error { comp := CompareState{ testMode: opts.Globals.TestMode, @@ -46,7 +49,6 @@ func (opts *ChunksOptions) CheckManifest(arrayA, arrayB []string, report *types. return comp.checkArrays(report) } -// TODO: Can this be made concurrent? func (comp *CompareState) checkArrays(report *types.ReportCheck) error { marker := "" if comp.testMode { @@ -96,3 +98,70 @@ func (comp *CompareState) checkArrays(report *types.ReportCheck) error { return nil } + +// CheckManContents spins through the manifest and makes sure all the +// bloom and index CIDs are present. It does not check that the CIDs are available. +func (opts *ChunksOptions) CheckManContents(man *manifest.Manifest, report *types.ReportCheck) error { + errs := []string{} + + report.VisitedCnt += 3 + report.CheckedCnt += 3 + + if !version.IsValidVersion(man.Version) { + errs = append(errs, "invalid version string "+opts.Tag) + } else { + report.PassedCnt++ + } + + if opts.Globals.Chain != man.Chain { + errs = append(errs, "different chains: globals("+opts.Globals.Chain+") manifest ("+man.Chain+")") + } else { + report.PassedCnt++ + } + + expectedSpec := config.SpecTags["trueblocks-core@v2.0.0-release"] + if expectedSpec != man.Specification.String() { + errs = append(errs, "invalid spec "+man.Specification.String()) + } else { + report.PassedCnt++ + } + + for _, chunk := range man.Chunks { + report.VisitedCnt++ + report.CheckedCnt++ + if len(chunk.Range) == 0 { + errs = append(errs, "empty range") + } else { + report.PassedCnt++ + } + + report.VisitedCnt++ + report.CheckedCnt++ + hasBloom := len(chunk.BloomHash) > 0 + hasIndex := len(chunk.IndexHash) > 0 + bloomSize := chunk.BloomSize > 0 + indexSize := chunk.IndexSize > 0 + if hasBloom && hasIndex && bloomSize && indexSize { + report.PassedCnt++ + } else { + if !hasBloom { + errs = append(errs, chunk.Range+" - missing bloom hash") + } + if !hasIndex { + errs = append(errs, chunk.Range+" - missing index hash") + } + if !bloomSize { + errs = append(errs, chunk.Range+" - missing bloom size") + } + if !indexSize { + errs = append(errs, chunk.Range+" - missing index size") + } + } + } + + if len(errs) > 0 { + report.MsgStrings = append(report.MsgStrings, errs...) + } + + return nil +} diff --git a/src/apps/chifra/internal/chunks/handle_pin.go b/src/apps/chifra/internal/chunks/handle_pin.go index 7418209fc0..58d65f009d 100644 --- a/src/apps/chifra/internal/chunks/handle_pin.go +++ b/src/apps/chifra/internal/chunks/handle_pin.go @@ -96,40 +96,49 @@ func (opts *ChunksOptions) HandlePin(rCtx *output.RenderCtx, blockNums []base.Bl return rng1.First < rng2.First }) - for _, path := range fileList { + failCnt := 1.0 + for i := 0; i < len(fileList); i++ { + sleep := opts.Sleep + + path := fileList[i] if opts.Globals.Verbose { logger.Info("pinning path:", path) } + local, remote, err := pinning.PinOneChunk(chain, path, opts.Remote) if err != nil { errorChan <- err logger.Error("Pin failed:", path, err) - } - - blMatches, idxMatches := matches(&local, &remote) - opts.matchReport(blMatches, local.BloomHash, remote.BloomHash) - opts.matchReport(idxMatches, local.IndexHash, remote.IndexHash) + failCnt *= 2. + sleep = failCnt + i-- // try again after sleeping for a bit + logger.Info(colors.Yellow, "Sleeping for", sleep, "seconds then trying again.", colors.Off) - if opts.Remote { - man.Chunks = append(man.Chunks, remote) } else { - man.Chunks = append(man.Chunks, local) - } - _ = man.SaveManifest(chain, outPath) + blMatches, idxMatches := matches(&local, &remote) + opts.matchReport(blMatches, local.BloomHash, remote.BloomHash) + opts.matchReport(idxMatches, local.IndexHash, remote.IndexHash) - if opts.Globals.Verbose { if opts.Remote { - fmt.Println("result.Remote:", remote.String()) + man.Chunks = append(man.Chunks, remote) } else { - fmt.Println("result.Local:", local.String()) + man.Chunks = append(man.Chunks, local) + } + _ = man.SaveManifest(chain, outPath) + + if opts.Globals.Verbose { + if opts.Remote { + fmt.Println("result.Remote:", remote.String()) + } else { + fmt.Println("result.Local:", local.String()) + } } } - sleep := opts.Sleep if sleep > 0 { ms := time.Duration(sleep*1000) * time.Millisecond if !opts.Globals.TestMode { - logger.Info(fmt.Sprintf("Sleeping for %g seconds", sleep)) + logger.Info("Sleeping for", sleep, "seconds") } time.Sleep(ms) } diff --git a/src/apps/chifra/internal/init/handle_init.go b/src/apps/chifra/internal/init/handle_init.go index 4a67298aaf..cf586d7dd6 100644 --- a/src/apps/chifra/internal/init/handle_init.go +++ b/src/apps/chifra/internal/init/handle_init.go @@ -6,6 +6,7 @@ package initPkg import ( "fmt" + "os" "path/filepath" "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/base" @@ -27,7 +28,12 @@ func (opts *InitOptions) HandleInit(rCtx *output.RenderCtx) error { // TODO: BOGUS - IF THE SCRAPER IS RUNNING, THIS WILL CAUSE PROBLEMS // Make sure that the temporary scraper folders are empty, so that, when the // scraper starts, it starts on the correct block. - _ = file.CleanFolder(chain, config.PathToIndex(chain), []string{"ripe", "unripe", "maps", "staging"}) + cleanList := []string{"ripe", "unripe", "maps", "staging"} + isHeadless := os.Getenv("TB_NODE_HEADLESS") == "true" + if isHeadless { + cleanList = []string{"ripe", "unripe"} + } + _ = file.CleanFolder(chain, config.PathToIndex(chain), cleanList) existing, err := manifest.LoadManifest(chain, opts.PublisherAddr, manifest.LocalCache) if err != nil { diff --git a/src/apps/chifra/internal/init/handle_init_download.go b/src/apps/chifra/internal/init/handle_init_download.go index bf3deebbb0..d20ef7a5e5 100644 --- a/src/apps/chifra/internal/init/handle_init_download.go +++ b/src/apps/chifra/internal/init/handle_init_download.go @@ -105,7 +105,7 @@ func (opts *InitOptions) downloadAndReportProgress(chunks []types.ChunkRecord, c if sleep > .25 { sleepStr = fmt.Sprintf(" [sleep: %0.2fms]", sleep*1000) } - msg := fmt.Sprintf("Finished download of %s%s%s %s%s%s (% 4d of %4d %0.1f%%%s)", col, event.Message, colors.Off, col, rng, colors.Off, nProcessed, nTotal, pct, sleepStr) + msg := fmt.Sprintf("Finished download of %s%s%s %s%s%s (% 4d of %4d %0.1f%%%s) chain: %s", col, event.Message, colors.Off, col, rng, colors.Off, nProcessed, nTotal, pct, sleepStr, opts.Globals.Chain) logger.Info(msg, spaces) if successCount%10 == 0 { sleep = base.Max(.0125, sleep/1.2) diff --git a/src/apps/chifra/internal/scrape/handle_show.go b/src/apps/chifra/internal/scrape/handle_show.go index 64748bc5fb..e9a6894726 100644 --- a/src/apps/chifra/internal/scrape/handle_show.go +++ b/src/apps/chifra/internal/scrape/handle_show.go @@ -54,6 +54,8 @@ func (opts *ScrapeOptions) HandleScrape(rCtx *output.RenderCtx) error { provider = "--rpc-provider--" } + isHeadless := os.Getenv("TB_NODE_HEADLESS") == "true" + msg1 := fmt.Sprintf("Scraping %s", chain) msg2 := fmt.Sprintf(" Rpc %s", provider) msg3 := fmt.Sprintf(" Path %s", path) @@ -67,10 +69,12 @@ func (opts *ScrapeOptions) HandleScrape(rCtx *output.RenderCtx) error { } return output.StreamMany(rCtx, fetchData, opts.Globals.OutputOpts()) } else { - logger.Info(msg1) - logger.Info(msg2) - logger.Info(msg3) - logger.Info(msg4) + if !isHeadless { + logger.Info(msg1) + logger.Info(msg2) + logger.Info(msg3) + logger.Info(msg4) + } } // Handle Ctr-C, docker stop and docker compose down (provided they @@ -143,6 +147,7 @@ func (opts *ScrapeOptions) HandleScrape(rCtx *output.RenderCtx) error { processedMap: make(map[base.Blknum]bool, opts.BlockCnt), meta: bm.meta, nChannels: int(opts.Settings.ChannelCount), + isHeadless: isHeadless, } // Order dependant, be careful! @@ -194,7 +199,9 @@ func (opts *ScrapeOptions) HandleScrape(rCtx *output.RenderCtx) error { } if bm.nRipe == 0 { - logger.Info(colors.Green+"no ripe files to consolidate", spaces, colors.Off) + if !bm.isHeadless { + logger.Info(colors.Green+"no ripe files to consolidate on chain", chain, strings.Repeat(" ", 40), colors.Off) + } goto PAUSE } else { @@ -215,7 +222,9 @@ func (opts *ScrapeOptions) HandleScrape(rCtx *output.RenderCtx) error { runCount++ if opts.RunCount != 0 && runCount >= opts.RunCount { // No reason to clean up here. Next round will do so and user can use these files in the meantime. - logger.Info("run count reached") + if !isHeadless { + logger.Info("run count reached") + } break } diff --git a/src/apps/chifra/internal/scrape/save_timestamps.go b/src/apps/chifra/internal/scrape/save_timestamps.go index 558cd58668..53bb6eb3ed 100644 --- a/src/apps/chifra/internal/scrape/save_timestamps.go +++ b/src/apps/chifra/internal/scrape/save_timestamps.go @@ -81,16 +81,20 @@ func (bm *BlazeManager) WriteTimestamps(ctx context.Context, blocks []base.Blknu } ts := bm.timestamps[block] - logProgressTs("Updating timestamps ", block, blocks[len(blocks)-1]) + if !bm.isHeadless { + logProgressTs("Updating timestamps ", block, blocks[len(blocks)-1]) + } if err := binary.Write(fp, binary.LittleEndian, &ts); err != nil { return err } } - logger.Progress(true, fmt.Sprintf("Finished writing timestamps to block %-04d"+spaces, - blocks[len(blocks)-1], - )) + if !bm.isHeadless { + logger.Progress(true, fmt.Sprintf("Finished writing timestamps to block %-04d"+spaces, + blocks[len(blocks)-1], + )) + } return nil } diff --git a/src/apps/chifra/internal/scrape/scrape_blaze.go b/src/apps/chifra/internal/scrape/scrape_blaze.go index 4cf1736fae..362912db8a 100644 --- a/src/apps/chifra/internal/scrape/scrape_blaze.go +++ b/src/apps/chifra/internal/scrape/scrape_blaze.go @@ -186,9 +186,9 @@ func (bm *BlazeManager) WriteAppearances(bn base.Blknum, addrMap uniq.AddressBoo sort.Strings(appearanceArray) blockNumStr := utils.PadNum(int(bn), 9) - fileName := filepath.Join(ripePath, blockNumStr + ".txt") + fileName := filepath.Join(ripePath, blockNumStr+".txt") if bn > bm.ripeBlock { - fileName = filepath.Join(unripePath, blockNumStr + ".txt") + fileName = filepath.Join(unripePath, blockNumStr+".txt") } toWrite := []byte(strings.Join(appearanceArray[:], "\n") + "\n") @@ -243,12 +243,17 @@ func (bm *BlazeManager) syncedReporting(bn base.Blknum, force bool) { if bm.ripeBlock > bn { dist = (bm.ripeBlock - bn) } - msg := fmt.Sprintf("Scraping %-04d of %-04d at block %d of %d (%d blocks from head)", + if bm.isHeadless && dist < 100 { + return + } + msg := fmt.Sprintf("Scraping %-04d of %-04d at block %d of %d (%d blocks from %s head)", bm.nProcessed(), bm.BlockCount(), bn, bm.ripeBlock, - dist) + dist, + bm.chain, + ) logger.Progress(true, msg) } } diff --git a/src/apps/chifra/internal/scrape/scrape_manager.go b/src/apps/chifra/internal/scrape/scrape_manager.go index a80eba3ec9..ba805c873b 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager.go +++ b/src/apps/chifra/internal/scrape/scrape_manager.go @@ -26,6 +26,7 @@ type BlazeManager struct { nTimestamps int nChannels int errors []scrapeError + isHeadless bool } type scrapeError struct { diff --git a/src/apps/chifra/internal/scrape/scrape_manager_utils.go b/src/apps/chifra/internal/scrape/scrape_manager_utils.go index 3ef032905b..9c45c5b808 100644 --- a/src/apps/chifra/internal/scrape/scrape_manager_utils.go +++ b/src/apps/chifra/internal/scrape/scrape_manager_utils.go @@ -17,18 +17,28 @@ func (bm *BlazeManager) report(nBlocks, perChunk, nChunks, nAppsNow, nAppsFound, nNeeded := perChunk - base.Min(perChunk, nAppsNow) appsPerAddr := float64(nAppsFound) / float64(nAddrsFound) pctFull := float64(nAppsNow) / float64(perChunk) + pctStr := fmt.Sprintf("%0.1f%%", pctFull*100) + if len(pctStr) < 5 { + pctStr = " " + pctStr + } + chunksStr := "" + if nChunks > 0 { + chunksStr = fmt.Sprintf(" created {%d} chunk(s)", nChunks) + } else { + chunksStr = fmt.Sprintf(" %s", bm.chain) + } - report := `%s #{%d}, found {%6d} apps and {%5d} addrs ({%0.1f apps/addr}). created {%d} chunks, staged {%5d} apps (@%0.1f%%}), need {%5d}.` + report := `%7.7s @#% 9d}: {% 8d}/{% 8d} ({%0.1f apps/addr}) stage {% 8d} need {% 8d} (@%5.5s})%s` msg := fmt.Sprintf(report, bm.chain, bm.EndBlock()-1, nAppsFound, nAddrsFound, appsPerAddr, - nChunks, nAppsNow, - pctFull*100, nNeeded, + pctStr, + chunksStr, ) logger.Info(colors.Colored(msg)) } diff --git a/src/apps/chifra/pkg/version/string.go b/src/apps/chifra/pkg/version/string.go index 873f5922fa..75e3e5b666 100644 --- a/src/apps/chifra/pkg/version/string.go +++ b/src/apps/chifra/pkg/version/string.go @@ -7,4 +7,4 @@ package version -const LibraryVersion = "GHC-TrueBlocks//3.3.0-release" +const LibraryVersion = "GHC-TrueBlocks//3.5.0-release" diff --git a/src/dev_tools/goMaker/go.mod b/src/dev_tools/goMaker/go.mod index 4828f69c01..c3950bb55c 100644 --- a/src/dev_tools/goMaker/go.mod +++ b/src/dev_tools/goMaker/go.mod @@ -4,7 +4,7 @@ module github.com/TrueBlocks/trueblocks-core/goMaker go 1.22 require ( - github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240901212707-3b0228642466 + github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de github.com/gocarina/gocsv v0.0.0-20230123225133-763e25b40669 golang.org/x/text v0.16.0 ) diff --git a/src/dev_tools/goMaker/go.sum b/src/dev_tools/goMaker/go.sum index fe815b4ac0..387f679c58 100644 --- a/src/dev_tools/goMaker/go.sum +++ b/src/dev_tools/goMaker/go.sum @@ -1,5 +1,5 @@ -github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240901212707-3b0228642466 h1:DNH7mYWEEehKLRiFjBoiMoZJMdpuVNiLnFvsstQW2To= -github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240901212707-3b0228642466/go.mod h1:YKuaek8RSyh681TFMK6Ua8MsHtuY+c1VJWi78Y35+J8= +github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de h1:9OmsWEdGlHfhSMeBMtSSbSz61IZm9CxoTedXI2oQocc= +github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de/go.mod h1:YKuaek8RSyh681TFMK6Ua8MsHtuY+c1VJWi78Y35+J8= github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A= github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/btcsuite/btcd/btcec/v2 v2.2.0 h1:fzn1qaOt32TuLjFlkzYSsBC35Q3KUjT1SwPxiMSCF5k= diff --git a/src/dev_tools/indexManager/.gitignore b/src/dev_tools/indexManager/.gitignore new file mode 100644 index 0000000000..2478c7c77d --- /dev/null +++ b/src/dev_tools/indexManager/.gitignore @@ -0,0 +1 @@ +disc_usage_report.md diff --git a/src/dev_tools/indexManager/README.md b/src/dev_tools/indexManager/README.md new file mode 100644 index 0000000000..efb3c481f4 --- /dev/null +++ b/src/dev_tools/indexManager/README.md @@ -0,0 +1,3 @@ +# indexManager + +The `indexManager` tool is for internal use only. diff --git a/src/dev_tools/indexManager/diskUsage.go b/src/dev_tools/indexManager/diskUsage.go new file mode 100644 index 0000000000..effb14da7c --- /dev/null +++ b/src/dev_tools/indexManager/diskUsage.go @@ -0,0 +1,162 @@ +package main + +import ( + "fmt" + "sort" + "strings" + "sync" +) + +// DiskUsage represents the disk usage information for a single partition +type DiskUsage struct { + Alias string + Machine12 string + Partition string + Size string + Used string + Available string + Usage string + SizeValue int64 + UsedValue int64 +} + +// getDiscUsage gathers disk usage information from multiple machines concurrently +// It returns a slice of DiskUsage structs and an error if any occurred during the process +func getDiscUsage(configPath string) ([]DiskUsage, error) { + config, err := loadConfig(configPath) + if err != nil { + return nil, fmt.Errorf("error loading configuration: %w", err) + } + + var mu sync.Mutex + var errors []error + var wg sync.WaitGroup + resultChan := make(chan []DiskUsage, len(config.Machines)) + for _, machine := range config.Machines { + wg.Add(1) + go func(mach string) { + defer wg.Done() + data, err := gatherDiskUsage(mach) + if err != nil { + mu.Lock() + errors = append(errors, fmt.Errorf("error gathering data from %s: %w", mach, err)) + mu.Unlock() + return + } + resultChan <- processMachineData(mach, data, config.Aliases[mach]) + }(machine) + } + + go func() { + wg.Wait() + close(resultChan) + }() + + var allUsages []DiskUsage + for result := range resultChan { + allUsages = append(allUsages, result...) + } + + sort.SliceStable(allUsages, func(i, j int) bool { + if allUsages[i].Alias == allUsages[j].Alias { + return allUsages[i].SizeValue < allUsages[j].SizeValue + } + return allUsages[i].Alias < allUsages[j].Alias + }) + + if len(errors) > 0 { + return allUsages, fmt.Errorf("encountered %d errors while gathering disk usage: %v", len(errors), errors) + } + + return allUsages, nil +} + +// processMachineData parses the raw disk usage data for a single machine +// It returns a slice of DiskUsage structs for the given machine +func processMachineData(machine, data, alias string) []DiskUsage { + var usages []DiskUsage + uniqueRecords := make(map[string]DiskUsage) + + lines := strings.Split(strings.TrimSpace(data), "\n") + for _, line := range lines[1:] { + fields := strings.Fields(line) + if len(fields) < 5 { + continue + } + usageValue := parsePercentage(fields[4]) + + // Skip rows with invalid or zero usage + if usageValue == 0 { + continue + } + + // Only process rows with more than 5% usage + if usageValue > 5 { + if len(fields) >= 6 && strings.Contains(fields[0], "/dev/") { + key := fmt.Sprintf("%s|%s", machine, fields[1]) + usedValue := parseSize(fields[2]) // Used value in kilobytes + if existing, found := uniqueRecords[key]; found { + // Keep the record with the highest used value + if usedValue > existing.UsedValue { + uniqueRecords[key] = DiskUsage{ + Alias: alias, + Machine12: machine, + Partition: fields[0], + Size: fields[1], + Used: fields[2], + Available: fields[3], + Usage: fields[4], + SizeValue: parseSize(fields[1]), + UsedValue: usedValue, + } + } + } else { + uniqueRecords[key] = DiskUsage{ + Alias: alias, + Machine12: machine, + Partition: fields[0], + Size: fields[1], + Used: fields[2], + Available: fields[3], + Usage: fields[4], + SizeValue: parseSize(fields[1]), + UsedValue: usedValue, + } + } + } + } + } + + for _, usage := range uniqueRecords { + usages = append(usages, usage) + } + + return usages +} + +type Config struct { + Machines []string + Aliases map[string]string +} + +// loadConfig returns a map of machine names to their aliases +func loadConfig(path string) (Config, error) { + var config Config + config.Machines = []string{ + "wildmolasses.local", + "linux.local", + "laptop.local", + "desktop.local", + "unchainedindex.io", + "192.34.63.136", + } + config.Aliases = map[string]string{ + "laptop.local": "1.0-Laptop", + "desktop.local": "2.0-Desktop", + "wildmolasses.local": "4.0-WildMolasses", + "linux.local": "3.0-Linux", + "unchainedindex.io": "5.0-UnchainedIndex", + "192.34.63.136": "6.0-Testing", + } + return config, nil +} diff --git a/src/dev_tools/indexManager/go.mod b/src/dev_tools/indexManager/go.mod new file mode 100644 index 0000000000..3e9b74c7a9 --- /dev/null +++ b/src/dev_tools/indexManager/go.mod @@ -0,0 +1,17 @@ +module github.com/TrueBlocks/trueblocks-core/indexManager + +// Go Version +go 1.22 + +replace github.com/TrueBlocks/trueblocks-core/sdk => ../../../sdk + +require github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de + +require ( + github.com/ethereum/go-ethereum v1.13.15 // indirect + github.com/holiman/uint256 v1.2.4 // indirect + github.com/inconshreveable/mousetrap v1.1.0 // indirect + github.com/spf13/cobra v1.7.0 // indirect + github.com/spf13/pflag v1.0.5 // indirect + golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa // indirect +) diff --git a/src/dev_tools/indexManager/go.sum b/src/dev_tools/indexManager/go.sum new file mode 100644 index 0000000000..0cd3b85ffb --- /dev/null +++ b/src/dev_tools/indexManager/go.sum @@ -0,0 +1,18 @@ +github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de h1:9OmsWEdGlHfhSMeBMtSSbSz61IZm9CxoTedXI2oQocc= +github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de/go.mod h1:YKuaek8RSyh681TFMK6Ua8MsHtuY+c1VJWi78Y35+J8= +github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o= +github.com/ethereum/go-ethereum v1.13.15 h1:U7sSGYGo4SPjP6iNIifNoyIAiNjrmQkz6EwQG+/EZWo= +github.com/ethereum/go-ethereum v1.13.15/go.mod h1:TN8ZiHrdJwSe8Cb6x+p0hs5CxhJZPbqB7hHkaUXcmIU= +github.com/holiman/uint256 v1.2.4 h1:jUc4Nk8fm9jZabQuqr2JzednajVmBpC+oiTiXZJEApU= +github.com/holiman/uint256 v1.2.4/go.mod h1:EOMSn4q6Nyt9P6efbI3bueV4e1b3dGlUCXeiRV4ng7E= +github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8= +github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw= +github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I= +github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0= +github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA= +github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg= +golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa h1:FRnLl4eNAQl8hwxVVC17teOw8kdjVDVAiFMtgUdTSRQ= +golang.org/x/exp v0.0.0-20231110203233-9a3e6036ecaa/go.mod h1:zk2irFbV9DP96SEBUUAy67IdHUaZuSnrz1n472HUCLE= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/src/dev_tools/indexManager/main.go b/src/dev_tools/indexManager/main.go new file mode 100644 index 0000000000..f47f970305 --- /dev/null +++ b/src/dev_tools/indexManager/main.go @@ -0,0 +1,41 @@ +package main + +import ( + "fmt" + "os" + "time" + + "github.com/TrueBlocks/trueblocks-core/src/apps/chifra/pkg/file" +) + +func main() { + allUsages, err := getDiscUsage("./config.json") + if err != nil { + fmt.Printf("Error returned from getDiscUsage: %v\n", err) + return + } + + screen := getTitle() + screen += headerRow() + var lastMachine string + for _, usage := range allUsages { + line, machine := dataRow(&usage, lastMachine) + screen += line + lastMachine = machine + } + screen += blankRow() + screen += legendRow() + + outputFile := "disc_usage.html" + file.StringToAsciiFile(outputFile, screen) + + now := time.Now() + ts := now.Format("2006-01-02 15:04:05") + if msg, err := copyToRemote(outputFile, "docs", "unchainedindex.io", "/var/www/unchainedindex.io/html/"); err != nil { + fmt.Printf("%s: Failed to copy file to remote host: %v\n", ts, err) + } else { + fmt.Printf("%s: %s", ts, msg) + } + os.Remove(outputFile) + fmt.Printf("%s: Report generated.\n", ts) +} diff --git a/src/dev_tools/indexManager/table.go b/src/dev_tools/indexManager/table.go new file mode 100644 index 0000000000..a2cb3f716c --- /dev/null +++ b/src/dev_tools/indexManager/table.go @@ -0,0 +1,99 @@ +package main + +import ( + "fmt" + "strings" + "time" +) + +func getTitle() string { + fmtStr := `

Disk Usage Report

+ +Report generated on %s +

+ +` + return fmt.Sprintf(fmtStr, time.Now().Format(time.RFC1123)) +} + +func headerRow() string { + return ` + + + + + + + + + +` +} + +// cnt := 0 + +func dataRow(usage *DiskUsage, lastMachine string) (string, string) { + var ret string + + machineName := " " + systemName := " " + if usage.Alias != lastMachine { + parts := strings.Split(usage.Alias, "-") + machineName = fmt.Sprintf("%s", parts[1]) + systemName = usage.Machine12 + if lastMachine != "" { + ret += blankRow() + } + lastMachine = usage.Alias + } + + usageValue := parsePercentage(usage.Usage) + usageColor := "" + if usageValue < 50 { + usageColor = "background-color: #ccffcc;" // Light green + } else if usageValue > 85 { + usageColor = "background-color: #ffcccc;" // Light red (pink) + } else { + usageColor = "background-color: #ffffcc;" // Pale yellow + } + + fmtStr := ` + + + + + + + + + +` + + ret += fmt.Sprintf(fmtStr, + machineName, + systemName, + usage.Partition, + usage.Size, usage.Used, + usage.Available, + usageColor, + usage.Usage) + + return ret, lastMachine +} + +func blankRow() string { + return ` + + +` +} + +func legendRow() string { + return `
MachineSystemPartitionSizeUsedAvailablePct
%s%s%s%s%s%s%s
 
+

+ 50% disc usage or less + okay + 85% disc usage or more +
+` +} diff --git a/src/dev_tools/indexManager/utils.go b/src/dev_tools/indexManager/utils.go new file mode 100644 index 0000000000..b2739c611d --- /dev/null +++ b/src/dev_tools/indexManager/utils.go @@ -0,0 +1,82 @@ +package main + +import ( + "fmt" + "log" + "os" + "os/exec" + "strconv" + "strings" +) + +// copyToRemote copies a file to a remote server using SCP +func copyToRemote(localFile, remoteUser, remoteHost, remotePath string) (string, error) { + scpCmd := fmt.Sprintf("scp -p %s %s@%s:%s", localFile, remoteUser, remoteHost, remotePath) + cmd := exec.Command("sh", "-c", scpCmd) + cmd.Stdout = os.Stdout + cmd.Stderr = os.Stderr + err := cmd.Run() + msg := "" + if err == nil { + msg = fmt.Sprintf("File copied to %s@%s:%s\n", remoteUser, remoteHost, remotePath) + } + return msg, err +} + +// gatherDiskUsage connects to the given machine via SSH and runs the df -h | grep -v snap command +func gatherDiskUsage(machine string) (string, error) { + cmd := exec.Command("ssh", machine, "df -h | grep -v snap") + output, err := cmd.CombinedOutput() + return string(output), err +} + +// parseSize converts a size string (e.g., "7.3Ti", "500Mi") to an integer value in kilobytes for sorting +func parseSize(sizeStr string) int64 { + multiplier := int64(1) + switch { + case strings.HasSuffix(sizeStr, "Ti"): + multiplier = 1024 * 1024 * 1024 + sizeStr = strings.TrimSuffix(sizeStr, "Ti") + case strings.HasSuffix(sizeStr, "Gi"): + multiplier = 1024 * 1024 + sizeStr = strings.TrimSuffix(sizeStr, "Gi") + case strings.HasSuffix(sizeStr, "Mi"): + multiplier = 1024 + sizeStr = strings.TrimSuffix(sizeStr, "Mi") + case strings.HasSuffix(sizeStr, "Ki"): + sizeStr = strings.TrimSuffix(sizeStr, "Ki") + case strings.HasSuffix(sizeStr, "T"): + multiplier = 1024 * 1024 * 1024 + sizeStr = strings.TrimSuffix(sizeStr, "T") + case strings.HasSuffix(sizeStr, "G"): + multiplier = 1024 * 1024 + sizeStr = strings.TrimSuffix(sizeStr, "G") + case strings.HasSuffix(sizeStr, "M"): + multiplier = 1024 + sizeStr = strings.TrimSuffix(sizeStr, "M") + case strings.HasSuffix(sizeStr, "K"): + sizeStr = strings.TrimSuffix(sizeStr, "K") + } + value, err := strconv.ParseFloat(sizeStr, 64) + if err != nil { + log.Printf("Error parsing size: %v", err) + return 0 + } + return int64(value * float64(multiplier)) +} + +// parsePercentage converts a percentage string (e.g., "39%") to an integer value for sorting +func parsePercentage(pctStr string) int64 { + // Check if the string ends with '%' + if !strings.HasSuffix(pctStr, "%") { + return 0 + } + + // Remove the '%' suffix and parse the number + pctStr = strings.TrimSuffix(pctStr, "%") + value, err := strconv.ParseInt(pctStr, 10, 64) + if err != nil { + return 0 + } + return value +} diff --git a/src/dev_tools/sdkFuzzer/go.mod b/src/dev_tools/sdkFuzzer/go.mod index bc78905e70..b0d03b02b1 100644 --- a/src/dev_tools/sdkFuzzer/go.mod +++ b/src/dev_tools/sdkFuzzer/go.mod @@ -4,8 +4,8 @@ module github.com/TrueBlocks/trueblocks-core/sdkFuzzer go 1.22 require ( - github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240901212707-3b0228642466 - github.com/TrueBlocks/trueblocks-sdk/v3 v3.3.2 + github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de + github.com/TrueBlocks/trueblocks-sdk/v3 v3.4.0 ) require ( diff --git a/src/dev_tools/sdkFuzzer/go.sum b/src/dev_tools/sdkFuzzer/go.sum index ba3ca7a7da..6cfa1b3403 100644 --- a/src/dev_tools/sdkFuzzer/go.sum +++ b/src/dev_tools/sdkFuzzer/go.sum @@ -40,10 +40,10 @@ github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migc github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5+sAH+4kjUM= github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= github.com/StackExchange/wmi v0.0.0-20180116203802-5d049714c4a6/go.mod h1:3eOhrUMpNV+6aFIbp5/iudMxNCF27Vw2OZgy4xEx0Fg= -github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240901212707-3b0228642466 h1:DNH7mYWEEehKLRiFjBoiMoZJMdpuVNiLnFvsstQW2To= -github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240901212707-3b0228642466/go.mod h1:YKuaek8RSyh681TFMK6Ua8MsHtuY+c1VJWi78Y35+J8= -github.com/TrueBlocks/trueblocks-sdk/v3 v3.3.2 h1:SVemhtXxeYuOV/dQlQ/ECr66RiIPBS38rZ+WCqrXDSQ= -github.com/TrueBlocks/trueblocks-sdk/v3 v3.3.2/go.mod h1:GFP6A7O1tNgun1f/CrrhJkro9cbqu5t1prt5aboKqJU= +github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de h1:9OmsWEdGlHfhSMeBMtSSbSz61IZm9CxoTedXI2oQocc= +github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de/go.mod h1:YKuaek8RSyh681TFMK6Ua8MsHtuY+c1VJWi78Y35+J8= +github.com/TrueBlocks/trueblocks-sdk/v3 v3.4.0 h1:lKnvOxsFdWvrNOEOY2Qin9QQWv+3vpKIROhitl8+1lo= +github.com/TrueBlocks/trueblocks-sdk/v3 v3.4.0/go.mod h1:GFP6A7O1tNgun1f/CrrhJkro9cbqu5t1prt5aboKqJU= github.com/VictoriaMetrics/fastcache v1.6.0/go.mod h1:0qHz5QP0GMX4pfmMA/zt5RgfNuXJrTP0zS7DqpHGGTw= github.com/VictoriaMetrics/fastcache v1.12.1 h1:i0mICQuojGDL3KblA7wUNlY5lOK6a4bwt3uRKnkZU40= github.com/VictoriaMetrics/fastcache v1.12.1/go.mod h1:tX04vaqcNoQeGLD+ra5pU5sWkuxnzWhEzLwhP9w653o= diff --git a/src/dev_tools/testRunner/go.mod b/src/dev_tools/testRunner/go.mod index 2c641d4149..a255c90ce7 100644 --- a/src/dev_tools/testRunner/go.mod +++ b/src/dev_tools/testRunner/go.mod @@ -6,8 +6,8 @@ go 1.22 replace github.com/TrueBlocks/trueblocks-core/sdk => ../../../sdk require ( - github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240901212707-3b0228642466 - github.com/TrueBlocks/trueblocks-sdk/v3 v3.3.2 + github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de + github.com/TrueBlocks/trueblocks-sdk/v3 v3.4.0 ) require ( diff --git a/src/dev_tools/testRunner/go.sum b/src/dev_tools/testRunner/go.sum index ba3ca7a7da..6cfa1b3403 100644 --- a/src/dev_tools/testRunner/go.sum +++ b/src/dev_tools/testRunner/go.sum @@ -40,10 +40,10 @@ github.com/Microsoft/go-winio v0.6.1 h1:9/kr64B9VUZrLm5YYwbGtUJnMgqWVOdUAXu6Migc github.com/Microsoft/go-winio v0.6.1/go.mod h1:LRdKpFKfdobln8UmuiYcKPot9D2v6svN5+sAH+4kjUM= github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU= github.com/StackExchange/wmi v0.0.0-20180116203802-5d049714c4a6/go.mod h1:3eOhrUMpNV+6aFIbp5/iudMxNCF27Vw2OZgy4xEx0Fg= -github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240901212707-3b0228642466 h1:DNH7mYWEEehKLRiFjBoiMoZJMdpuVNiLnFvsstQW2To= -github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240901212707-3b0228642466/go.mod h1:YKuaek8RSyh681TFMK6Ua8MsHtuY+c1VJWi78Y35+J8= -github.com/TrueBlocks/trueblocks-sdk/v3 v3.3.2 h1:SVemhtXxeYuOV/dQlQ/ECr66RiIPBS38rZ+WCqrXDSQ= -github.com/TrueBlocks/trueblocks-sdk/v3 v3.3.2/go.mod h1:GFP6A7O1tNgun1f/CrrhJkro9cbqu5t1prt5aboKqJU= +github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de h1:9OmsWEdGlHfhSMeBMtSSbSz61IZm9CxoTedXI2oQocc= +github.com/TrueBlocks/trueblocks-core/src/apps/chifra v0.0.0-20240914021502-857c9b6f37de/go.mod h1:YKuaek8RSyh681TFMK6Ua8MsHtuY+c1VJWi78Y35+J8= +github.com/TrueBlocks/trueblocks-sdk/v3 v3.4.0 h1:lKnvOxsFdWvrNOEOY2Qin9QQWv+3vpKIROhitl8+1lo= +github.com/TrueBlocks/trueblocks-sdk/v3 v3.4.0/go.mod h1:GFP6A7O1tNgun1f/CrrhJkro9cbqu5t1prt5aboKqJU= github.com/VictoriaMetrics/fastcache v1.6.0/go.mod h1:0qHz5QP0GMX4pfmMA/zt5RgfNuXJrTP0zS7DqpHGGTw= github.com/VictoriaMetrics/fastcache v1.12.1 h1:i0mICQuojGDL3KblA7wUNlY5lOK6a4bwt3uRKnkZU40= github.com/VictoriaMetrics/fastcache v1.12.1/go.mod h1:tX04vaqcNoQeGLD+ra5pU5sWkuxnzWhEzLwhP9w653o= diff --git a/tests b/tests index 21d5dfb15d..d381819eba 160000 --- a/tests +++ b/tests @@ -1 +1 @@ -Subproject commit 21d5dfb15d6aabe8cc58b46418858398d75e2d15 +Subproject commit d381819eba3bfdb9e16b47685c41cb4857ed63a4