Is there any howto about building the runtime with srtool? I’ve read docs/dev/release-new-runtime.md but I don’t think it is relevant in this context.
1 Like
This is the actual howto:
3 Likes
Oops. I didn’t notice the link.Thanks.
It works, but there is a caveat: the srtool container runs with internal user builder (uid = 1001). If your local user doesn’t have the same uid you have to chmod 777 runtime/gdev or the build fails with error: failed to create directory '/build/runtime/gdev/target'.
I’ve also tested the build using srtool-cli (same caveat as above).
4 Likes
Cgeek avait déjà eu le même problème :
et elois a automatisé ça dans le code, c’est pour ça que ça ne m’a pas gêné : xtask/src/release_runtime.rs · master · nodes / rust / Duniter v2S · GitLab
2 Likes
I am having an issue with the permissions:
duniter-v2s ((runtime-401))> podman run \
-i \
--rm \
-e PACKAGE=gdev-runtime \
-e RUNTIME_DIR=runtime/gdev \
-v $PWD:/build \
paritytech/srtool:1.66.1 build --app --json -cM
🧰 Substrate Runtime Toolbox - srtool v0.9.25 🧰
- by Chevdor -
info: using existing install for '1.66.1-x86_64-unknown-linux-gnu'
info: override toolchain for '/build' set to '1.66.1-x86_64-unknown-linux-gnu'
1.66.1-x86_64-unknown-linux-gnu unchanged - rustc 1.66.1 (90743e729 2023-01-10)
🏗 Building gdev-runtime as release using rustc 1.66.1 (90743e729 2023-01-10)
⏳ That can take a little while, be patient... subsequent builds will be faster.
Since you have to wait a little, you may want to learn more about Substrate runtimes:
https://docs.substrate.io/v3/getting-started/architecture/
Error: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }
!!! The RUNTIME_DIR 'runtime/gdev' does not look like a Cargo project. Is it pointing to the repo of your runtime?
I’ll continue digging tomorrow.
chmod 777 runtime/gdev from your host.
1 Like
Doesn’t help:
chmod 777 runtime/gdev/
drwxrwxrwx@ - moul 16 Mar 08:56 gdev
Always give me:
Error: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }
I also tried with docker and chmod -R 777 runtime/gdev/
Tried solutions in this thread.
Then try on runtime/gdev/target.
This folder does not exists, I created it, changed the permissions, but same issue.
I think the issue starts when trying to read runtime/gdev/Cargo.toml
Same with srtools-cli:
duniter-v2s ((runtime-401))> srtool build --package gdev-runtime -e podman -r runtime/gtest/
Found 1.66.1, we will be using paritytech/srtool:1.66.1 for the build
🧰 Substrate Runtime Toolbox - srtool v0.9.25 🧰
- by Chevdor -
info: using existing install for '1.66.1-x86_64-unknown-linux-gnu'
info: override toolchain for '/build' set to '1.66.1-x86_64-unknown-linux-gnu'
1.66.1-x86_64-unknown-linux-gnu unchanged - rustc 1.66.1 (90743e729 2023-01-10)
🏗 Building gdev-runtime as release using rustc 1.66.1 (90743e729 2023-01-10)
⏳ That can take a little while, be patient... subsequent builds will be faster.
Since you have to wait a little, you may want to learn more about Substrate runtimes:
https://docs.substrate.io/v3/getting-started/architecture/
Error: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }
!!! The RUNTIME_DIR 'runtime/gtest/' does not look like a Cargo project. Is it pointing to the repo of your runtime?
I give up in the meantime regarding srtool requiring a container. Containers software have their issues with root permissions on mounted volumes:
- Rootless + updates by chevdor · Pull Request #44 · paritytech/srtool · GitHub
- Change permissions of target directory by zjb0807 · Pull Request #35 · paritytech/srtool · GitHub
- Regression: new rootless version doesn't work for regular users · Issue #46 · paritytech/srtool · GitHub
- https://github.com/moby/moby/issues/2259
1 Like
I also don’t like use and abuse of containers. Since the only objective is to provide a reproducible way to build the runtime, we might want to search for an other way of doing it. I did not try to build the runtime using my host machine rustc, but maybe by chance it can produce the same bytecode 
1 Like
By default rustc does not produce reproductible builds, at least because:
- it links dynamically to the host’s shared libraries. This is stupid and I can’t see any valid reason, but there is no easy workaround afaik (or you have to install a complete cross-compilation suite). This is also why releases must be built on the oldest targeted OS.
However we should not have this problem with WASM. - runtime errors and debug data contain file paths from the host’s filesystem. There is an option to change this but it may not be enough.
2 Likes
Here is a recipe with a clean clone of the V2S repo:
git clone git@git.duniter.org:nodes/rust/duniter-v2s.git /tmp/duniter-v2s
cd /tmp/duniter-v2s
git checkout release/runtime-401
mkdir runtime/gdev/target -p
chmod 777 runtime/gdev/target
docker run \
-i \
--rm \
-e PACKAGE=gdev-runtime \
-e RUNTIME_DIR=runtime/gdev \
-v $PWD:/build \
paritytech/srtool:1.66.1 build --app --json -cM
Thanks for the nice sequence of commands, but I get the same permission errors.
What do you get if you run these commands:
docker run \
-it \
--rm \
-e PACKAGE=gdev-runtime \
-e RUNTIME_DIR=runtime/gdev \
-v $PWD:/build \
paritytech/srtool:1.66.1 bash
builder@15de4839a731:/srtool$ ls -ld /build/
drwx------. 1 root root 740 Mar 17 11:34 /build/
builder@15de4839a731:/srtool$ ./build --app --json -cM
./build: line 12: cd: /build: Permission denied
I am wondering why are you all able to overcome this issue with a simple chmod on the host. I am wondering whether this is related to Fedora/podman.
I think we should change strategy and use containers as they are intended to be used: not link volume to local folder but instead let docker/podman manage the volume himself and fetch the output binary in it. I don’t have access to my computer soon but if someone finds how to do it easily we will use this option in the documentation.
2 Likes
Tell me if you need help on this topic.
1 Like
I cloned duniter-v2s repository and moved its content into /build/.
So, I share it in case someone else might need it (myself in the future? ):
podman run -it --rm -e PACKAGE=gdev-runtime -e RUNTIME_DIR=runtime/gdev paritytech/srtool:1.77.0 bash
cd /build
git clone https://git.duniter.org/nodes/rust/duniter-v2s -b network/gdev-XXX .
./srtool/build
The compressed release hashes are identical to the ones released as well as on Polkadot proposal
Summary generated with srtool v0.9.25 using the docker image paritytech/srtool:1.66.1:
Package : gdev-runtime v3.0.0
GIT commit :
GIT tag :
GIT branch :
Rustc : rustc 1.66.1 (90743e729 2023-01-10)
Time : 2023-03-17T13:38:19Z
== Compact
Version : gdev-401 (duniter-gdev-1.tx1.au1)
Metadata : V14
Size : 2.01 MB (2110367 bytes)
setCode : 0xf647517499d13e38131e43117ae98324b587723eefd327b393c7317522cf128b
authorizeUpgrade : 0xa793724ec460e40a8f89b58ef63f42977b48067c4ac8e161486e23c1fa0d07e2
IPFS : QmP1hkckzzHRJQ547ovJ3xMoST6WjhwzN72pNqh4f615Fd
BLAKE2_256 : 0xd4695ef1f512748598a07b93d2d50120f264a4abfac10272b2ffd8f7c17479a0
Wasm : runtime/gdev/target/srtool/release/wbuild/gdev-runtime/gdev_runtime.compact.wasm
== Compressed
Version : gdev-401 (duniter-gdev-1.tx1.au1)
Metadata : V14
Size : 446.71 KB (457433 bytes)
Compression : 78.33%
setCode : 0xe377706f91aea6f20911f2bb750de9c267fd1019227424dbd64465e732c85efb
authorizeUpgrade : 0x0e365bfe8727770fe950978cae277bebb7fbd2f69b723949a0282e0857a2b542
IPFS : QmWxT6F2Jwdo2GY4bLBy7CWcJfFPGnwk8XK1gNfXwLruhG
BLAKE2_256 : 0x59e84456b6e8ae8e8ddba9e4617e673e1569773d359989e689ada34206d0f8d3
Wasm : runtime/gdev/target/srtool/release/wbuild/gdev-runtime/gdev_runtime.compact.compressed.wasm
2 Likes
Thanks for the hint for not using the volume feature. I thought about a git clone
I am not sure what you mean exactly, but I think the runtime should be built in this reproducible environment as explained by tuxmain.
I’m not sure either how to achieve it. As I understand it, the -v option (
-v, --volume list Bind mount a volume
)
used as
-v $PWD:/build
means that a folder from the host machine is mounted on the container file system, which can cause permission problems. My idea is to avoid doing this, and let the container manage its file system. Once the build is complete, we have to find a way to look what is inside the container volume. I do not already know how to do it, so this proposition is welcome:
If I remember well, it is still possible to vote after the threshold is reached (even change your vote), but the interface does not display it anymore. Anybody can close the proposal, even somebody not in the technical committee I think. If nobody does, the proposal will be closed automatically at the end of its life, and will be applied or rejected depending of the total of votes at this moment.
Nice! I can see your vote at block 1335020. It seems the old proposals were closed by poka at blocks 1349118 and 1349121
Here is a POC proposal:
$ cat scripts/srtool.sh
#!/bin/bash
docker run --rm -i -e PACKAGE=gdev-runtime -e RUNTIME_DIR=runtime/gdev -v $PWD:/build -v $PWD/srtool-out:/out -v srtool-cache:/home/builder/cargo -u root paritytech/srtool:1.66.1 sh -c "
sed -i '/builder:/s/1001/'$UID'/' /etc/passwd
chown builder /home/builder
chown builder /out
exec su builder -c '
PATH=.:\$PATH
$*
'
"
$ scripts/srtool.sh build --app --json -cM
🧰 Substrate Runtime Toolbox - srtool v0.9.25 🧰
- by Chevdor -
info: using existing install for '1.66.1-x86_64-unknown-linux-gnu'
info: override toolchain for '/build' set to '1.66.1-x86_64-unknown-linux-gnu'
1.66.1-x86_64-unknown-linux-gnu unchanged - rustc 1.66.1 (90743e729 2023-01-10)
🏗 Building gdev-runtime as release using rustc 1.66.1 (90743e729 2023-01-10)
⏳ That can take a little while, be patient... subsequent builds will be faster.
Since you have to wait a little, you may want to learn more about Substrate runtimes:
https://docs.substrate.io/v3/getting-started/architecture/
Updating crates.io index
Updating git repository `https://github.com/duniter/substrate`
Updating git repository `https://github.com/duniter/subxt.git`
Downloading crates ...
Downloaded cargo-platform v0.1.2
...
Compiling gdev-runtime v3.0.0 (/build/runtime/gdev)
Finished release [optimized] target(s) in 1m 50s
✨ Your Substrate WASM Runtime is ready! ✨
{"gen":"srtool v0.9.25","src":"git","version":"3.0.0","commit":"29d2c0fdac255c5fa9ce5064327779a9159be17d","tag":"v0.4.0","branch":"master","rustc":"rustc 1.66.1 (90743e729 2023-01-10)","pkg":"gdev-runtime","tmsp":"2023-03-20T21:06:43Z","size":"457427","prop":"0x11da79206c2884228c0b1cfc34a30618577700f78880c2df53369703368d6bc9","authorize_upgrade_prop":"0xf4738be9bfd111bb7cb73fdc470c9b6a0cf887b9fd3378202807be657a451e1b","ipfs":"QmTZZFFZeKLZaPsNNQ6fCQwgcvQyFGdWEZcrLeQZEoQ8YU","sha256":"0x2fcb7d8f66c5da69c0cf31f9b85c90aa43f825b691d633b93bb9b305924562f0","wasm":"runtime/gdev/target/srtool/release/wbuild/gdev-runtime/gdev_runtime.compact.compressed.wasm","info":{"generator":{"name":"srtool","version":"0.9.25"},"src":"git","version":"3.0.0","git":{"commit":"29d2c0fdac255c5fa9ce5064327779a9159be17d","tag":"","branch":"master"},"rustc":"rustc 1.66.1 (90743e729 2023-01-10)","pkg":"gdev-runtime","profile":"release"},"context":{"package":"gdev-runtime","runtime_dir":"runtime/gdev","docker":{"image":"paritytech/srtool","tag":"1.66.1"},"profile":"release"},"runtimes":{"compact":{"tmsp":"2023-03-20T21:06:26Z","size":"2110367","prop":"0x95e5b6e574c23ccdb62d9a75cc51d3b18adf9ca2e7e8d4837b134ebdd9c50add","authorize_upgrade_prop":"0x2f1c594280c7de5d097b5de8b678582f0c5fdb33f3eea73aa60e6ba9ca7bf8d1","blake2_256":"0xa981498a33c27c43ace1fe4800aa807d602040aac226d1a90f4f8801045c0a48","ipfs":"QmaDpCKFGqZXnFzJMjWFpAuDB5KJVBJppJtwpmSrLVFkZa","sha256":"0x2b042fd2bceab19cd7cf4d91a8a9f318f934854c95be110d376e6055daa40d87","wasm":"runtime/gdev/target/srtool/release/wbuild/gdev-runtime/gdev_runtime.compact.wasm","subwasm":{"size":2110367,"compression":{"size_compressed":2110367,"size_decompressed":2110367,"compressed":false},"reserved_meta":[109,101,116,97],"reserved_meta_valid":true,"metadata_version":14,"core_version":{"specName":"gdev","implName":"duniter-gdev","authoringVersion":1,"specVersion":400,"implVersion":1,"apis":[["0x687ad44ad37f03c2",1],["0xcbca25e39f142387",2],["0xdf6acb689907609b",4],["0x37e397fc7c91f5e4",1],["0x40fe3ad401f8959a",6],["0xd2bc9897eed08f15",3],["0xf78b278be53f454c",2],["0xab3c0572291feb8b",1],["0xed99c5acb25eedf5",3],["0xbc9d89904f5b923f",1],["0x37c8bb1350a9a2a8",2]],"transactionVersion":1},"proposal_hash":"0x95e5b6e574c23ccdb62d9a75cc51d3b18adf9ca2e7e8d4837b134ebdd9c50add","parachain_authorize_upgrade_hash":"0x2f1c594280c7de5d097b5de8b678582f0c5fdb33f3eea73aa60e6ba9ca7bf8d1","ipfs_hash":"QmaDpCKFGqZXnFzJMjWFpAuDB5KJVBJppJtwpmSrLVFkZa","blake2_256":"0xa981498a33c27c43ace1fe4800aa807d602040aac226d1a90f4f8801045c0a48"}},"compressed":{"tmsp":"2023-03-20T21:06:18Z","size":"457427","prop":"0x11da79206c2884228c0b1cfc34a30618577700f78880c2df53369703368d6bc9","authorize_upgrade_prop":"0xf4738be9bfd111bb7cb73fdc470c9b6a0cf887b9fd3378202807be657a451e1b","blake2_256":"0x2b9fac8f14362e9d59a6912d66e94c9e0b0b7e40e7bbde49bfcb5288b1c77edf","ipfs":"QmTZZFFZeKLZaPsNNQ6fCQwgcvQyFGdWEZcrLeQZEoQ8YU","sha256":"0x2fcb7d8f66c5da69c0cf31f9b85c90aa43f825b691d633b93bb9b305924562f0","wasm":"runtime/gdev/target/srtool/release/wbuild/gdev-runtime/gdev_runtime.compact.compressed.wasm","subwasm":{"size":457427,"compression":{"size_compressed":457427,"size_decompressed":2110367,"compressed":true},"reserved_meta":[109,101,116,97],"reserved_meta_valid":true,"metadata_version":14,"core_version":{"specName":"gdev","implName":"duniter-gdev","authoringVersion":1,"specVersion":400,"implVersion":1,"apis":[["0x687ad44ad37f03c2",1],["0xcbca25e39f142387",2],["0xdf6acb689907609b",4],["0x37e397fc7c91f5e4",1],["0x40fe3ad401f8959a",6],["0xd2bc9897eed08f15",3],["0xf78b278be53f454c",2],["0xab3c0572291feb8b",1],["0xed99c5acb25eedf5",3],["0xbc9d89904f5b923f",1],["0x37c8bb1350a9a2a8",2]],"transactionVersion":1},"proposal_hash":"0x11da79206c2884228c0b1cfc34a30618577700f78880c2df53369703368d6bc9","parachain_authorize_upgrade_hash":"0xf4738be9bfd111bb7cb73fdc470c9b6a0cf887b9fd3378202807be657a451e1b","ipfs_hash":"QmTZZFFZeKLZaPsNNQ6fCQwgcvQyFGdWEZcrLeQZEoQ8YU","blake2_256":"0x2b9fac8f14362e9d59a6912d66e94c9e0b0b7e40e7bbde49bfcb5288b1c77edf"}}}}
1 Like