# Need to fix GitLab CI for Duniter-v2s
**URL:** https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164
**Category:** Duniter-v2s
**Created:** [13 December 2022 19:25 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164 "2022-12-13T19:25:28Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [13 December 2022 19:25 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/1 "2022-12-13T19:25:28Z")
The [Duniter-v2s repo](https://git.duniter.org/nodes/rust/duniter-v2s) has a heavy CI allowing to build Duniter with embeded chainspec and publish an image to [Docker Hub](https://hub.docker.com/r/duniter/duniter-v2s/tags) (made by @pini). Elois stopped providing a server for this runner so @poka [moved it to Axiom Team server](https://forum.duniter.org/t/migration-du-runner-ci-de-elois-vers-infra-axiom-team/10081). But seemingly due to obscure tag practices, the CI is broken. @1000i100 told us how to fix it during [11-13 November meeting in Bordeaux](https://forum.duniter.org/t/rencontre-technique-a-bordeaux-du-11-au-13-novembre-2022/9803/26) but it still has to be done. @Moul you seem to be working on it? Do you need help from @immae for example?
---
### Author:  [@Moul](https://forum.duniter.org/u/Moul)
#### Post date: [13 December 2022 20:59 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/2 "2022-12-13T20:59:46Z")
The only runner with the `dind` label (`axiom-team-ci-privileged-dind`) was apparently not available.
Once I added `dind` label to redshift runner, the job started.
---
### Author:  [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [13 December 2022 21:12 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/3 "2022-12-13T21:12:12Z")
Can you explain me why we are using tags? As I can read on the [GitLab CI documentation](https://docs.gitlab.com/ee/ci/yaml/index.html), tags allow to select runners. But in our case, we have only three or four runners with similar capabilities (I don’t know) and no need to keep any of them available for an urgent task, so any task can go in any runner, no?
---
### Author:  [@Moul](https://forum.duniter.org/u/Moul)
#### Post date: [13 December 2022 21:16 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/4 "2022-12-13T21:16:45Z")
Right, but we have some runners which can and other can’t build containers, for instance. That’s the purpose of the `docker` label.
Some runners might be dedicated to a project, with secrets in, or you could dedicate a runner to a project for performance purposes.
---
### Author:  [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [13 December 2022 21:30 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/5 "2022-12-13T21:30:26Z")
> [@Moul](#):
>
> some runners […] can’t build containers
I know nothing about gitlab runners but in my mind, they are only kind of virtual machines so I could not imagine that they would have different capabilities other than the physical limitation of ram and cpu of their host machine.
---
### Author:  [@Moul](https://forum.duniter.org/u/Moul)
#### Post date: [13 December 2022 21:34 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/6 "2022-12-13T21:34:13Z")
There is special configuration to be done on the runner to be able to build containers.
I am not sure anymore, but it might be `dind` Docker in Docker.
---
### Author:  [@1000i100](https://forum.duniter.org/u/1000i100)
#### Post date: [14 December 2022 00:16 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/7 "2022-12-14T00:16:31Z")
Yep dind (docker in docker) need more privilege (running in a docker called by root) so there is more security breach usable for these runners if malicious build is stated in it. if i remember well, i configure it to run only on protected branch and tag. Perhaps that’s what lock somewhere in git flow.
---
### Author:  [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [14 December 2022 13:28 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/8 "2022-12-14T13:28:14Z")
Ok, now that the CI is running, it fails for a reason of password: [deploy\_docker\_release\_sha (#76660) · Jobs · nodes / rust / Duniter v2S · GitLab](https://git.duniter.org/nodes/rust/duniter-v2s/-/jobs/76660)
Does someone know how to fix it?
---
### Author:  [@Moul](https://forum.duniter.org/u/Moul)
#### Post date: [14 December 2022 14:03 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/9 "2022-12-14T14:03:24Z")
[This §Variables](https://git.duniter.org/nodes/rust/duniter-v2s/-/settings/ci_cd) is only available on [§ Protected branches or tags](https://git.duniter.org/nodes/rust/duniter-v2s/-/settings/repository). The branch should be named `release/*` for the variable to be available in the CI.
---
### Author:  [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [14 December 2022 14:11 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/10 "2022-12-14T14:11:37Z")
I’m not confortable with these kind of limitations. As @poka told me, the CI should be here to make the developer’s work easier, but as elois implemented it, they actually make it a nightmare (see [Bootstraper une ĞDev](https://forum.duniter.org/t/bootstraper-une-gdev/9757) and [Difficultés avec l'outillage](https://forum.duniter.org/t/difficultes-avec-loutillage/9945) for examples of what an undocumented CI can make me feel link). If nobody is here to manage this part, I’m in favor of removing all the limitations so that @poka and I can publish docker images for smith to use and for developers to test (like duniter-indexer, gecko…).
---
### Author:  [@Moul](https://forum.duniter.org/u/Moul)
#### Post date: [14 December 2022 14:39 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/11 "2022-12-14T14:39:28Z")
Without this feature, Docker Hub can be displayed in the CI and stolen by anyone having the right to push on a branch on this repository. I would not take the risk to remove this security feature.
One option could be to replace Docker Hub publication with [GitLab container registry](https://git.duniter.org/nodes/rust/duniter-v2s/container_registry). The latter does not require a CI/CD variable to be protected, since the authentication is integrated.
I am not the one to take this decision. Publishing on Docker Hub was made on purpose to make the images available on the main platform. This is part of the vision as having duniter-v2s repository on GitHub for more visibility.
---
### Author:  [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [14 December 2022 14:57 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/12 "2022-12-14T14:57:12Z")
Ok, thanks for the explanations. So @poka only needs to rename his branch `release/*` for the pipeline to pass?
---
### Author:  [@immae](https://forum.duniter.org/u/immae)
#### Post date: [17 December 2022 07:27 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/13 "2022-12-17T07:27:12Z")
It looks like I arrived after the battle, but if you still need help I’m around.
I confirm that release/ and master are the only branches that can access to the dockerhub’s password, and those branches are reserved to “high privileged” (“maintainer”) users for good reason, since we don’t want to limit contribution from random persons but we don’t want them to do an “echo $PASSWORD” either.
---
### Author:  [@tuxmain](https://forum.duniter.org/u/tuxmain)
#### Post date: [1 February 2023 23:04 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/14 "2023-02-01T23:04:14Z")
It seems the current CI is highly inefficient: each step installs Rust, downloads all the dependencies, rebuilds them from scratch…
I think the cache should be kept between non-concurrent pipelines. Cargo is smart enough to rebuild what needs to be rebuilt, and Substrate is already long enough to build.
Is this possible?
Edit: La CI est très bien et très utile, c’est juste le côté “pour voir ce que ça donne si on déplace l’interrupteur du 15e étage, on va tout détruire et reconstruire l’immeuble” qui me gêne, avec la latence et le gâchis d’énergie et d’occupation et d’usure du serveur qui vont avec.
---
### Author:  [@Moul](https://forum.duniter.org/u/Moul)
#### Post date: [14 February 2023 20:38 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/15 "2023-02-14T20:38:41Z")
I gave it a try with [these changes](https://git.duniter.org/nodes/rust/duniter-v2s/-/merge_requests/new?merge_request%5Bsource_branch%5D=ci_cache), but it seems to take more time because of the size of the cache (around 4GB) to be uploaded and then downloaded. I had jobs taken up to one hour and more.
---
### Author:  [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [15 February 2023 11:27 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/16 "2023-02-15T11:27:18Z")
I do not know a lot about CI in general and for Rust in particular, but I wonder the relevance of the virtual machine approach to run this CI because:
- building Duniter from scratch is very long (~30 min - 1h)
- building Rust is very CPU-intensive, one powerful container is preferable than multiple limited ones
- the target folder gets huge with debug build (20 Go - 30 Go)
- we do not really need to run multiple CI in parallel
- Rust should not be that bad for reproducible compilation and can manage cache himself
So, is it possible to set up one container to run all Duniter pipelines, keep everything between each run (except output files of end2end tests for instance), and only run one pipeline at a time?
---
### Author:  [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [6 March 2023 14:39 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/17 "2023-03-06T14:39:23Z")
Duniter CI still has an issue: [Pipeline · nodes / rust / Duniter v2S · GitLab](https://git.duniter.org/nodes/rust/duniter-v2s/-/pipelines/18464)
This step failed: [deploy\_docker\_release\_sha (#78591) · Jobs · nodes / rust / Duniter v2S · GitLab](https://git.duniter.org/nodes/rust/duniter-v2s/-/jobs/78591)
with the error
```auto
Error: Cannot perform an interactive login from a non TTY device
```
Can somebody work to fix it as it prevents publishing a new official Docker image which is the easiest option available to new smith?
---
### Author:  [@Moul](https://forum.duniter.org/u/Moul)
#### Post date: [6 March 2023 20:35 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/18 "2023-03-06T20:35:34Z")
> **[Error: Cannot perform an interactive login from a non TTY device](https://forum.gitlab.com/t/error-cannot-perform-an-interactive-login-from-a-non-tty-device/59936)**
>
> Hello, I am trying to push a docker image in the gitlab registry but at login I receive the error : Error: Cannot perform an interactive login from a non TTY device My code is following as much as could be the gitlab official documentation from...
Same [as above](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/9) …
---
### Author:  [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [6 March 2023 20:37 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/19 "2023-03-06T20:37:37Z")
Ok, but this time, the branch is named `release/*` so it should have access to protected variables.
> [@Moul](#):
>
> The branch should be named `release/*` for the variable to be available in the CI.
Branch name is `release/poka-chainspec-gdev5-pini-docker`
---
### Author:  [@Moul](https://forum.duniter.org/u/Moul)
#### Post date: [7 March 2023 07:46 UTC](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164/20 "2023-03-07T07:46:09Z")
These variables `REPO_DOCKER_{USER,PASS}` are simply not defined:
[https://git.duniter.org/nodes/rust/duniter-v2s/-/settings/ci\_cd](https://git.duniter.org/nodes/rust/duniter-v2s/-/settings/ci_cd)
[This commit](https://git.duniter.org/nodes/rust/duniter-v2s/-/commit/6c8d7b8018b1293a73ef02bfd74e06985c3b28f7) can be dropped in the meantime.
[Next page](https://forum.duniter.org/t/need-to-fix-gitlab-ci-for-duniter-v2s/10164.md?page=2)