Indexer crash?

Nodes Indexers
1

image
image1088×345 73.8 KB

@vjrj, your indexer seems crashed, can you show me the logs? I really hope it’s not again #16 because it’s an upstream problem and I do not know if they’re going to fix it.

2

Here they are:

How do you query the height? because with telemetry I see different values, or maybe it’s just slow:
imagen
I think that I saw something similar some days ago and it recovered fast after a restart, let me check. Exactly, after a restart, in seconds:
imagen
Maybe we have to monitor the resources use.

3

Telemetry is only for Duniter, not Duniter-Squid. I simply subscribe to latest block : src/squid/meta.gql · master · Hugo Trentesaux / duniter-vue · GitLab

The logs are mixed, it’s easier to only look at the squid-processor logs. However I found this:

squid-processor-1  | {"level":2,"time":1732104300360,"ns":"sqd:processor","msg":"3983762 / 3983762, rate: 0 blocks/sec, mapping: 73 blocks/sec, 219 items/sec, eta: 0s"}
squid-processor-1  | {"level":5,"time":1732104644705,"ns":"sqd:processor","err":{"stack":"Error: Connection terminated unexpectedly\n    at Connection.<anonymous> (/squid/node_modules/.pnpm/pg@8.13.1/node_modules/pg/lib/client.js:131:73)\n    at Object.onceWrapper (node:events:632:28)\n    at Connection.emit (node:events:518:28)\n    at Socket.<anonymous> (/squid/node_modules/.pnpm/pg@8.13.1/node_modules/pg/lib/connection.js:62:12)\n    at Socket.emit (node:events:518:28)\n    at TCP.<anonymous> (node:net:337:12)"}}
duniter-archive-1  | 2024-11-19 17:38:18 🏆 Imported #3972709 (0x5d3b…d712 → 0xc042…a821)    
                   ⬇️
"stack":"Error: Connection terminated unexpectedly
    at Connection.<anonymous> (/squid/node_modules/.pnpm/pg@8.13.1/node_modules/pg/lib/client.js:131:73)
    at Object.onceWrapper (node:events:632:28)
    at Connection.emit (node:events:518:28)
    at Socket.<anonymous> (/squid/node_modules/.pnpm/pg@8.13.1/node_modules/pg/lib/connection.js:62:12)
    at Socket.emit (node:events:518:28)
    at TCP.<anonymous> (node:net:337:12)"

It seems that your postgres database was cut at some point. I dug more and found this:

hasura-squid-1     | {"detail":{"message":{"host":"Unknown or invalid host","message":"Postgres connection failed","retry_attempt":0}},"level":"warn","timestamp":"2024-11-27T09:10:43.499+0000","type":"pg-client"}
squid-processor-1  | {"level":2,"time":1732100802208,"ns":"sqd:processor","msg":"3983180 / 3983180, rate: 0 blocks/sec, mapping: 149 blocks/sec, 448 items/sec, eta: 0s"}
hasura-squid-1     | {"detail":{"message":{"host":"Unknown or invalid host","message":"Postgres connection failed","retry_attempt":1}},"level":"warn","timestamp":"2024-11-27T09:11:04.171+0000","type":"pg-client"}
hasura-squid-1     | {"detail":{"info":{"error":{"info":{"code":"postgres-error","error":"connection error","internal":"could not translate host name \"squid-db\" to address: Temporary failure in name resolution\n","path":"$"},"type":"query_error"}},"thread_type":"listener"},"level":"error","timestamp":"2024-11-27T09:11:04.171+0000","type":"schema-sync"}

I do not know how squid behaves if the database access becomes unavailable, but it should not happen. Are you sure that your database is healthy?

1 Like
4

I found the network scan page in the duniter panel:
https://duniter--vue-coinduf-eu.ipns.pagu.re/#/settings/network-scan
that is what I was looking for. Thanks. And yes, I use the lastBlock too.

For some reason the db container stop responding well at some point. This server has mainly g1 stuff, but probably needs some extra tuning.

5

Are you using the same database for multiple services? Or is this db container dedicated to duniter-squid?

6

Only for duniter-squid