Can't start indexer

[ivnsch]

Subject of the issue

Trying to delete everything and restart (to see if it helps with #118):

docker rmi sandbox_algod -f
docker rmi sandbox_indexer -f
docker rmi postgres -f

sandbox up dev -v

The indexer doesn't start.
Output:

[...]
Building indexer
[+] Building 128.8s (12/12) FINISHED                                                                                                                             
 => [internal] load build definition from Dockerfile                                                                                                        0.0s
 => => transferring dockerfile: 616B                                                                                                                        0.0s
 => [internal] load .dockerignore                                                                                                                           0.0s
 => => transferring context: 34B                                                                                                                            0.0s
 => [internal] load metadata for docker.io/library/golang:1.17.5-alpine                                                                                     1.8s
 => [internal] load build context                                                                                                                           0.0s
 => => transferring context: 196B                                                                                                                           0.0s
 => [1/7] FROM docker.io/library/golang:1.17.5-alpine@sha256:4918412049183afe42f1ecaf8f5c2a88917c2eab153ce5ecf4bf2d55c1507b74                               0.1s
 => => resolve docker.io/library/golang:1.17.5-alpine@sha256:4918412049183afe42f1ecaf8f5c2a88917c2eab153ce5ecf4bf2d55c1507b74                               0.0s
 => => sha256:4918412049183afe42f1ecaf8f5c2a88917c2eab153ce5ecf4bf2d55c1507b74 1.65kB / 1.65kB                                                              0.0s
 => => sha256:4e6ba1f23bdaa946184b3a88627b69f1c55436b4fc6f42eb80a1e3cfa92e53c9 1.36kB / 1.36kB                                                              0.0s
 => => sha256:939b513d7ff818b7dbf6602c8c82e3616a5c322d170ec228aef1b7b2d4036f00 5.21kB / 5.21kB                                                              0.0s
 => [2/7] WORKDIR /opt/indexer                                                                                                                              0.0s
 => [3/7] RUN apk add --no-cache git bzip2 make bash libtool boost-dev autoconf automake g++                                                               11.7s
 => [4/7] COPY images/indexer/disabled.go /tmp/disabled.go                                                                                                  0.0s
 => [5/7] COPY images/indexer/start.sh /tmp/start.sh                                                                                                        0.0s
 => [6/7] COPY images/indexer/install.sh /tmp/install.sh                                                                                                    0.0s
 => [7/7] RUN /tmp/install.sh                                                                                                                             108.0s
 => exporting to image                                                                                                                                      7.1s 
 => => exporting layers                                                                                                                                     7.1s 
 => => writing image sha256:e6f629c3a0cc27774c1e2c83d847544d6b7695ff002b4e841870276c1039e771                                                                0.0s 
 => => naming to docker.io/library/sandbox_indexer                                                                                                          0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
WARNING: Image for service indexer was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
algorand-sandbox-postgres is up-to-date
Recreating algorand-sandbox-algod ... done
Recreating algorand-sandbox-indexer ... done
* waiting for services to initialize.
the following did not start:
* indexer node
One or more services failed to start.

Your environment

• Software version: algod -v
  3.5.1.stable [rel/stable] (commit #aa2fb0ee)
• sandbox version

* algod version
12885295106
3.6.2.stable [rel/stable] (commit #5e6bc6fc)
go-algorand is licensed with AGPLv3.0
source code available at https://github.com/algorand/go-algorand

Indexer version
Error response from daemon: Container c883eacb5d509b2022afcc5454a191cfaf721e0826c70090aa9c496cc5a32eb8 is restarting, wait until the container is running --> seems to never finish?

• Git:

$ git log -1
commit 306e1f5dbef23f28e77ebc4f4889ff6e7f2a0166 (HEAD -> master, origin/master, origin/HEAD)
Author: Will Winder <[email protected]>
Date:   Tue May 17 17:17:27 2022 -0400

• Node status if applicable: sandbox goal node status

Last committed block: 0
Time since last block: 0.0s
Sync Time: 0.0s
Last consensus protocol: future
Next consensus protocol: future
Round for next consensus protocol: 1
Next consensus protocol supported: true
Last Catchpoint: 
Genesis ID: sandnet-v1
Genesis hash: VfUJIswubibx7f07C7aiGjVhXXzPVokK3EavGm7q/b4=

• Operating System details.
  MacOS 12.1 (21C52)
  Apple M1

Steps to reproduce

1. Remove images
2. sandbox up dev -v

Expected behaviour

The indexer starts

Actual behaviour

The indexer doesn't start

Logs

sandbox logs
Watching file: /opt/data/node.log...
could not decode line from JSON: ++++++++++++++++++++++++++++++++++++++++
could not decode line from JSON: Logging Starting
could not decode line from JSON: Telemetry Disabled
could not decode line from JSON: ++++++++++++++++++++++++++++++++++++++++
15:41:50.657 : RoundConcluded - |
15:41:50.693 : RoundConcluded - |

Docker

Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Starting indexer against algod.
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved
Connecting to algod:4001 (172.18.0.3:4001)
saving to 'genesis.json'
genesis.json         100% |********************************|  2054  0:00:00 ETA
'genesis.json' saved

[winder]

Does ./sandbox clean help? How about the stable configuration?

[bmdelacruz]

I also cannot start the indexer.

I inspected the indexer docker container's /tmp/indexer-log.txt and found this: {"error":"genesis hash not matching","level":"error","msg":"importer.EnsureInitialImport() error","time":"2022-12-01T08:25:33Z"}

I'm using config.release: indexer's version is 2.15.0-dev.unknown compiled at 2022-12-01T07:12:28+0000 from git hash 300b69fa095d31edac1c6ac70901d99a59b3f7a6 (modified) and goal's version is 3.12.2.stable [rel/stable] (commit #181490e3)

[bmdelacruz]

I ran ./sandbox reset and the indexer started.