g1-squid.asycn.io et gtest-squid.asycn.io à jour.
Oui a mon avis on a soucis de spam de query graphql quelque part. Peut-être la vue réseau (et l’icône d’état) de Cesium desktop, qui scan toutes les 10 secondes - cc @cgeek ?
@poka est-ce possible de configurer le rate limiter sur certaines querkes/fields uniquement ? Par exemple celles utilisées pour l’état du réseau ?
Bonne question, il faut voir ce qu’on peut faire côté PostGraphile, le plugin rate-limiter, mais sachant que Graphile est déjà pensé comme modulable et que j’ai déjà dû faire 2 plugins pour squid, au pire des cas ça devrait pouvoir se coder si le plugin rate-limiter de base ne le permet pas. A creuser 
Je n’arrive pas à lancer mon squid gtest 0.5.11 il m’affiche un message d’erreur pour chaque bloc :
{"level":2,"time":1774040778509,"ns":"sqd:processor:mapping","msg":"Genesis flushed"}
{"level":2,"time":1774040778534,"ns":"sqd:processor:mapping","msg":"====================="}
{"level":2,"time":1774040778534,"ns":"sqd:processor:mapping","msg":"Starting blockchain indexing with 7 smiths, 7778 members and 43108 accounts!"}
{"level":3,"time":1774040778793,"ns":"sqd:processor:mapping","msg":"Validator g1PH3gQ4n53EqpcLVLtG77jRRo71tLUjvDYcm87MENkAa1NPz not found for block 1, skipping forge count"}
{"level":3,"time":1774040778796,"ns":"sqd:processor:mapping","msg":"Validator g1PH3gQ4n53EqpcLVLtG77jRRo71tLUjvDYcm87MENkAa1NPz not found for block 2, skipping forge count"}
{"level":3,"time":1774040778798,"ns":"sqd:processor:mapping","msg":"Validator g1PH3gQ4n53EqpcLVLtG77jRRo71tLUjvDYcm87MENkAa1NPz not found for block 3, skipping forge count"}
{"level":3,"time":1774040778799,"ns":"sqd:processor:mapping","msg":"Validator g1PH3gQ4n53EqpcLVLtG77jRRo71tLUjvDYcm87MENkAa1NPz not found for block 4, skipping forge count"}
{"level":3,"time":1774040778800,"ns":"sqd:processor:mapping","msg":"Validator g1PH3gQ4n53EqpcLVLtG77jRRo71tLUjvDYcm87MENkAa1NPz not found for block 5, skipping forge count"}
puis à partir du bloc 48783 il redémarre en boucle sur la même erreur :
{"level":3,"time":1774040983302,"ns":"sqd:processor:mapping","msg":"Validator g1PH3gQ4n53EqpcLVLtG77jRRo71tLUjvDYcm87MENkAa1NPz not found for block 48772, skipping forge count"}
{"level":3,"time":1774040983303,"ns":"sqd:processor:mapping","msg":"Validator g1PH3gQ4n53EqpcLVLtG77jRRo71tLUjvDYcm87MENkAa1NPz not found for block 48773, skipping forge count"}
{"level":3,"time":1774040983307,"ns":"sqd:processor:mapping","msg":"Validator g1PH3gQ4n53EqpcLVLtG77jRRo71tLUjvDYcm87MENkAa1NPz not found for block 48778, skipping forge count"}
{"level":2,"time":1774040983318,"ns":"sqd:processor:mapping","msg":"48783 SmithMembers.SmithCertAdded"}
{"level":5,"time":1774040983362,"ns":"sqd:processor","err":{"stack":"EntityNotFoundError: Could not find any entity of type \"Smith\" matching: {\n \"index\": 12483\n}\n at /squid/node_modules/.pnpm/typeorm@0.3.26_ioredis@5.7.0_pg@8.16.3_reflect-metadata@0.2.2/node_modules/typeorm/entity-manager/EntityManager.js:682:39\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async StoreWithCache.findOneByOrFail (/squid/node_modules/.pnpm/@belopash+typeorm-store@1.5.0_@subsquid+typeorm-config@4.1.1_typeorm@0.3.26_ioredis@5.7_0562d5ed8c2d447272b97a9e909772f4/node_modules/@belopash/typeorm-store/lib/store.js:153:21)\n at async DataHandler.processNewData (/squid/lib/data_handler.js:412:32)\n at async /squid/lib/main.js:91:9\n at async TypeormDatabaseWithCache.performUpdates (/squid/node_modules/.pnpm/@belopash+typeorm-store@1.5.0_@subsquid+typeorm-config@4.1.1_typeorm@0.3.26_ioredis@5.7_0562d5ed8c2d447272b97a9e909772f4/node_modules/@belopash/typeorm-store/lib/database.js:45:13)\n at async /squid/node_modules/.pnpm/@subsquid+typeorm-store@1.5.1_@subsquid+big-decimal@1.0.0_typeorm@0.3.26_ioredis@5.7.0_pg@8.16.3_reflect-metadata@0.2.2_/node_modules/@subsquid/typeorm-store/lib/database.js:84:13\n at async EntityManager.transaction (/squid/node_modules/.pnpm/typeorm@0.3.26_ioredis@5.7.0_pg@8.16.3_reflect-metadata@0.2.2/node_modules/typeorm/entity-manager/EntityManager.js:75:28)\n at async TypeormDatabaseWithCache.submit (/squid/node_modules/.pnpm/@subsquid+typeorm-store@1.5.1_@subsquid+big-decimal@1.0.0_typeorm@0.3.26_ioredis@5.7.0_pg@8.16.3_reflect-metadata@0.2.2_/node_modules/@subsquid/typeorm-store/lib/database.js:164:24)\n at async Runner.withProgressMetrics (/squid/node_modules/.pnpm/@subsquid+util-internal-processor-tools@4.2.1/node_modules/@subsquid/util-internal-processor-tools/lib/runner.js:217:22)","criteria":{"index":12483},"message":"Could not find any entity of type \"Smith\" matching: {\n \"index\": 12483\n}"}}
{"level":2,"time":1774040985556,"ns":"sqd:commands","msg":"MIGRATION:COPY-CUSTOM"}
{"level":2,"time":1774040985585,"ns":"sqd:commands","msg":"MIGRATION:APPLY"}
query: SELECT version()
query: SELECT * FROM current_schema()
query: SELECT * FROM "information_schema"."tables" WHERE "table_schema" = 'public' AND "table_name" = 'migrations'
query: SELECT * FROM "migrations" "migrations" ORDER BY "id" DESC
No migrations are pending
{"level":2,"time":1774040987381,"ns":"sqd:commands","msg":"PROCESS:PROD"}
[rpc-check] Connecting to ws://v2s:9944 ...
[rpc-check] DNS OK: v2s <E2><86><92> 172.26.0.4
[rpc-check] TCP OK: 172.26.0.4:9944
[rpc-check] RPC OK: connected to chain "<C4><9E>Test"
[rpc-check] All checks passed
{"level":2,"time":1774040989418,"ns":"sqd:processor","msg":"last processed final block was 48779"}
{"level":2,"time":1774040989419,"ns":"sqd:processor","msg":"processing blocks from 48780"}
{"level":2,"time":1774040989419,"ns":"sqd:processor","msg":"using chain RPC data source"}
{"level":2,"time":1774040989449,"ns":"sqd:processor","msg":"prometheus metrics are served at port 43877"}
<F0><9F><94><91> SS58 prefix from chain: 4450
{"level":2,"time":1774040989701,"ns":"sqd:processor:mapping","msg":"48783 SmithMembers.SmithCertAdded"}
{"level":5,"time":1774040989819,"ns":"sqd:processor","err":{"stack":"EntityNotFoundError: Could not find any entity of type \"Smith\" matching: {\n \"index\": 12483\n}\n at /squid/node_modules/.pnpm/typeorm@0.3.26_ioredis@5.7.0_pg@8.16.3_reflect-metadata@0.2.2/node_modules/typeorm/entity-manager/EntityManager.js:682:39\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async StoreWithCache.findOneByOrFail (/squid/node_modules/.pnpm/@belopash+typeorm-store@1.5.0_@subsquid+typeorm-config@4.1.1_typeorm@0.3.26_ioredis@5.7_0562d5ed8c2d447272b97a9e909772f4/node_modules/@belopash/typeorm-store/lib/store.js:153:21)\n at async DataHandler.processNewData (/squid/lib/data_handler.js:412:32)\n at async /squid/lib/main.js:91:9\n at async TypeormDatabaseWithCache.performUpdates (/squid/node_modules/.pnpm/@belopash+typeorm-store@1.5.0_@subsquid+typeorm-config@4.1.1_typeorm@0.3.26_ioredis@5.7_0562d5ed8c2d447272b97a9e909772f4/node_modules/@belopash/typeorm-store/lib/database.js:45:13)\n at async /squid/node_modules/.pnpm/@subsquid+typeorm-store@1.5.1_@subsquid+big-decimal@1.0.0_typeorm@0.3.26_ioredis@5.7.0_pg@8.16.3_reflect-metadata@0.2.2_/node_modules/@subsquid/typeorm-store/lib/database.js:84:13\n at async EntityManager.transaction (/squid/node_modules/.pnpm/typeorm@0.3.26_ioredis@5.7.0_pg@8.16.3_reflect-metadata@0.2.2/node_modules/typeorm/entity-manager/EntityManager.js:75:28)\n at async TypeormDatabaseWithCache.submit (/squid/node_modules/.pnpm/@subsquid+typeorm-store@1.5.1_@subsquid+big-decimal@1.0.0_typeorm@0.3.26_ioredis@5.7.0_pg@8.16.3_reflect-metadata@0.2.2_/node_modules/@subsquid/typeorm-store/lib/database.js:164:24)\n at async Runner.withProgressMetrics (/squid/node_modules/.pnpm/@subsquid+util-internal-processor-tools@4.2.1/node_modules/@subsquid/util-internal-processor-tools/lib/runner.js:217:22)","criteria":{"index":12483},"message":"Could not find any entity of type \"Smith\" matching: {\n \"index\": 12483\n}"}}
{"level":2,"time":1774040990702,"ns":"sqd:commands","msg":"MIGRATION:COPY-CUSTOM"}
{"level":2,"time":1774040990727,"ns":"sqd:commands","msg":"MIGRATION:APPLY"}
query: SELECT version()
query: SELECT * FROM current_schema()
query: SELECT * FROM "information_schema"."tables" WHERE "table_schema" = 'public' AND "table_name" = 'migrations'
query: SELECT * FROM "migrations" "migrations" ORDER BY "id" DESC
No migrations are pending
{"level":2,"time":1774040991949,"ns":"sqd:commands","msg":"PROCESS:PROD"}
[rpc-check] Connecting to ws://v2s:9944 ...
[rpc-check] DNS OK: v2s <E2><86><92> 172.26.0.4
[rpc-check] TCP OK: 172.26.0.4:9944
[rpc-check] RPC OK: connected to chain "<C4><9E>Test"
[rpc-check] All checks passed
J’ai essayé de supprimer le volume et de relancer mais ca recommence à nouveau.
Oui, Squid n’est pas fonctionnel sur la ĞTest pour un lancement from scratch. C’est connu et pas corrigé.
Arf ok merci pour le pointeur je vais regarder ca.
Squid en cours d’indexation ! J’ai un peu galéré avec la mémoire, ça a planté plusieurs fois avant d’aboutir… J’ai fini par arrêter tous les docker de la machine pour ne lancer que l’archive et le squid pour qu’ils se calent sans cramer toute la mémoire.
Je reviens dans la forge
@poka je pense que cela bloque les squids en GTest non ?
A moins que l’on ait également une version 0.5.13 pour la GTest ? (je n’ai plus d’indexer en GTest donc je ne peux pas vérifier)
En effet, j’ai installé Squid en Gtest avec la dernière version disponible qui est la 0.5.11 donc elle n’est pas accepté par Gecko en Gtest 
Oui, c’est cassé pour la gtest, j’ai décrit ce qu’il faudrait faire ici pour faire une nouvelle image fonctionnelle :
Je viens de mettre à jour le .env pour mon indexeur, et j’ai redémarré les services avec docker compose up -d. Mais aucun conteneur n’a été mis à jour par cette action.
J’en conclus qu’il faut en plus citer explicitement ces variables dans le docker-compose.yml pour qu’elles soient prises en compte. Mais pour quel service ? Le server avec l’image duniter/squid-graphile-g1 ?
EDIT: Sachant qu’on retrouve rate_limit.js dans les images Docker squid-app et squid-graphile j’ai ajouté les variables d’environnement RATE_LIMIT_* aux services processor et server.