# Request for review - Adding db persistence for all SecretFormat of vault keys as well as supporting derivations

**URL:** https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829
**Category:** Ğ1cli
**Tags:** derivation, gcli
**Created:** [29 December 2024 10:23 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829 "2024-12-29T10:23:55Z")
**Posts on this page:** 20
**Page:** 1

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [29 December 2024 10:23 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/1 "2024-12-29T10:23:55Z")

</div>

Post écrit en anglais comme je ne sais pas trop quel développeur parle quelle langue (n’hésitez pas à répondre en français si vous préférez 😀)

I made a proof of concept merge request for GCli which adds (sqlite) db persistence for the vault keys of the different SecretFormat types (Seed, Substrate, Cesium, (Predefined)) and also supports derivations for all of those.

Small reminder, I’m still new to the rust language so don’t hesitate to give _constructive_ remarks 😀

> **[Adding db persistence for all SecretFormat of vault keys as well as...](https://git.duniter.org/clients/rust/gcli-v2s/-/merge_requests/41)**
>
> Added "/.idea" exclusion in .gitignore (for when using JetBrains IDEs) Added dialoguer dependency for easier user input handling (see in inputs.rs) Added sea-orm dependency...

  
In more details:
## DB persistence

I picked up crate [sea-orm](https://crates.io/crates/sea-orm) (first time using db with rust so there might be a better dependency - but I wanted support for entity mapping without having to manually create the tables).

DB Tables:

 ![db_schema](https://forum.duniter.org/uploads/default/original/2X/5/5bc7d141cc0c10b765df35ca325a8a246b3a598c.png)

### vault\_account

One `vault_account` is created per **root** key and it contains the encrypted [substrate uri](https://polkadot.js.org/docs/keyring/start/suri) for that key.

Examples (you can find source of those cases in the [derivations tests](https://git.duniter.org/clients/rust/gcli-v2s/-/blob/vault-db-persistence-and-derivation-support/src/keys.rs?ref_type=heads#L299-664) I added):

```auto
crypto_type: Sr25519Mnemonic
encrypted_private_key(decrypted): "bottom drive obey lake curtain smoke basket hold race lonely fit walk"

crypto_type: Sr25519Seed
encrypted_private_key(decrypted): "0xf813535799c3a15b8e419a06964e87fabd3f265caebcbb38c935a1acdbe05253"

crypto_type: Ed25519Seed
encrypted_private_key(decrypted): "0x2101d2bc68de9ad149c06293bfe489c8608de576c88927aa5439a81be17aae84"

```

This _substrate uri_ format allows to easily compose the derivation by just adding the **derivation path** after it. In thoses cases here, I believe the **seed** is also called **mini-secret**.

fields:

- `address` _String_: The SS58 Address of the **root** key
- `crypto_type` _String_: “Sr25519Mnemonic” / “Sr25519Seed” / “Ed25519Seed”
  - The `Ed25519Seed` is used for **Cesium V1** keys; see [it’s derivation test](https://git.duniter.org/clients/rust/gcli-v2s/-/blob/vault-db-persistence-and-derivation-support/src/keys.rs?ref_type=heads#L541-663)  
  
**Maybe** using this kind of Ed25519 seed constructed using `nacl::sign::Keypair` which we construct with _cesium v1_ `id` and `secret` **is not a good idea for security reasons** ❓

- `encrypted_private_key` _bytes_: encrypted _substrate uri_ using the **same encryption** that was used for the **vault files** ( **might need to be adapted​❓** )

### vault\_derivation

For each of those ( **root** ) `vault_account` one `vault_derivation` is created with `path` at **None** and both `address` and `root_address` having the same value.

Then to add a derivation, we only need one extra row in `vault_derivation` that have a valid non-null `path` value and refers to the correct `root_address`

fields:

- `address` _String_: The SS58 Address of this specific derivation (if `path` is None, then it’s the **root** address)
- `name` _Option\<String\>_: allows to have a _vault name_ associated with any of the vault key derivation; it has a **unique constraint** to avoid issues
- `path` _Option\<String\>_: allows to provide the derivation path which should be `None` or a string starting with `'/'`
- `root_address` _String_: The SS58 Address of the root for this derivation. It is also a _foreign key_ to the `vault_account` table.

## Adapted/Extra `vault` commands

I ~~added [dialoguer](https://crates.io/crates/dialoguer)~~ updated `inquire` crate dependency because of a [bug](https://github.com/mikaelmello/inquire/issues/149) for the password confirmation in the version 0.6.2 we were using.

Using it to manage extra input (see in [inputs.rs](https://git.duniter.org/clients/rust/gcli-v2s/-/blob/vault-db-persistence-and-derivation-support/src/inputs.rs?ref_type=heads))

(We might want to also convert remaining usages of `rpassword` crate and remove that dependency ❓)

### `list [all|root]` command

I adapted the list command into 2 sub commands.

```auto
gcli vault list --help   
List available keys

Usage: gcli vault list <COMMAND>

Commands:
  all List all keys and derivations in the vault
  root List only root keys
  help Print this message or the help of the given subcommand(s)

Options:
  -h, --help Print help

```

`list all`: that will list all the keys and their derivations in order; also computing a derivation name if it doesn’t have a name but it’s root key does.

Values inside ‘\<…\>’ are computed.

example output:

```auto
gcli vault list all         
available keys:
┌──────────────────────────────────────────────────────────────────────────────────────────┐
│ SS58 Address Path Vault Name │
╞══════════════════════════════════════════════════════════════════════════════════════════╡
│ 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV <Root> mySubstrateKey │
│ 5D34dL5prEUaGNQtPPZ3yN5Y6BnkfXunKXXz6fo7ZJbLwRRH //0 <mySubstrateKey//0> │
│ 5GBNeWRhZc2jXu7D55rBimKYDk8PGk8itRYFTPfC8RJLKG5o //1 substrateDeriv1CustomName │
│ 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY //Alice <mySubstrateKey//Alice> │
│ 5ET2jhgJFoNQUpgfdSkdwftK8DKWdqZ1FKm5GKWdPfMWhPr4 <Root> myCesiumKey │
│ 5Ca1HrNxQ4hiekd92Z99fzhfdSAqPy2rUkLBmwLsgLCjeSQf //0 <myCesiumKey//0> │
│ 5GjAp6kkbsDjxABGouMvu1Mxbr7PaFqbMrprEoZ466vPF2Vt <Root> mySeedKey │
│ 5HBwy19piXNWg7bfShuNgWsBCWRzkG99M8eRGB6PHkxeAKAV //0 <mySeedKey//0> │
└──────────────────────────────────────────────────────────────────────────────────────────┘

```

`list root`: only shows root keys for easier selections for the `derivation` command

example:

```auto
gcli vault list root
available root keys:
┌────────────────────────────────────────────────────────────────────────────┐
│ SS58 Address Path Vault Name │
╞════════════════════════════════════════════════════════════════════════════╡
│ 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV <Root> mySubstrateKey │
│ 5ET2jhgJFoNQUpgfdSkdwftK8DKWdqZ1FKm5GKWdPfMWhPr4 <Root> myCesiumKey │
│ 5GjAp6kkbsDjxABGouMvu1Mxbr7PaFqbMrprEoZ466vPF2Vt <Root> mySeedKey │
└────────────────────────────────────────────────────────────────────────────┘

```

### Setting / Using `vault name`

I adapted the base argument `-a <ADDRESS>` to support one of `-a <ADDRESS>` or `-v <NAME>`.

`-v <NAME>`: Vault name for a key

This same choice of _Address_ / _VaultName_ is also allowed for several of the `vault` commands

A **computed** _VaultName_ can be provided as well (example `mySubstrateKey//Alice` from the list above)

This _VaultName_ can be **set** when `import`ing a key, or with new `vault rename` command.

### `use` command

```auto
gcli vault use --help
Use specific vault key (changes the config address)

Usage: gcli vault use <-a <ADDRESS>|-v <NAME>>

Options:
  -a <ADDRESS> key SS58 Address
  -v <NAME> Vault name for a key
  -h, --help Print help

```

### `import` command

Adapted to allow providing the _Secret key format_ but still takes `substrate` as default if not provided

```auto
gcli vault import --help   
Import key from (substrate)mnemonic or other format with interactive prompt

Usage: gcli vault import [OPTIONS]

Options:
  -S, --secret-format <SECRET_FORMAT> Secret key format (substrate, seed, cesium) [default: substrate]
  -h, --help Print help

```

### `derivation` command

```auto
gcli vault derivation --help
Add a derivation to an existing (root) vault key

Usage: gcli vault derivation <-a <ADDRESS>|-v <NAME>>

Options:
  -a <ADDRESS> key SS58 Address
  -v <NAME> Vault name for a key
  -h, --help Print help

```

also has aliases `deriv` and `derive`

### `rename` command

```auto
gcli vault rename --help
Give a meaningful vault name to a vault key or derivation

Usage: gcli vault rename <ADDRESS>

Arguments:
  <ADDRESS> key SS58 Address

Options:
  -h, --help Print help

```

### `remove` command

```auto
gcli vault remove --help
Remove a vault key (and potential derivations if it's a root key)

Usage: gcli vault remove <-a <ADDRESS>|-v <NAME>>

Options:
  -a <ADDRESS> key SS58 Address
  -v <NAME> Vault name for a key
  -h, --help Print help

```

### `list-files` command

```auto
gcli vault list-files --help   
(deprecated)List available key files (needs to be migrated with command "vault migrate" in order to use them)

Usage: gcli vault list-files

Options:
  -h, --help Print help

```

### `migrate` command

```auto
gcli vault migrate --help   
(deprecated)Migrate old key files into db (will have to provide password for each key)

Usage: gcli vault migrate

Options:
  -h, --help Print help

```

---

<div class="post-metadata">

### Author: ![HugoTrentesaux](https://forum.duniter.org/user_avatar/forum.duniter.org/hugotrentesaux/32/6396_2.png) [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [30 December 2024 16:26 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/2 "2024-12-30T16:26:50Z")

</div>

Some quick general feedback before a code review:

> **about english**
>
> > [@Nicolas80](#):
> >
> > écrit en anglais
> 
> On a plusieurs fois essayé de passer à l’anglais sur ce forum
> 
> - [Discussions in English?](https://forum.duniter.org/t/discussions-in-english/9399)
> - [Rendre plus facile le changement de langue / Make language change more accessible - #5 by HugoTrentesaux](https://forum.duniter.org/t/rendre-plus-facile-le-changement-de-langue-make-language-change-more-accessible/9905/5)
> 
> Mon avis est qu’il vaut mieux écrire anglais quand ce n’est pas trop un frein pour faciliter l’intégration de nouveaux contributeurs non francophones, sachant que la plupart des devs comprennent plutôt bien l’anglais.

> [@Nicolas80](#):
>
> I picked up crate [sea-orm](https://crates.io/crates/sea-orm)

I knew about Diesel ([https://diesel.rs/](https://diesel.rs/)), not SeaORM. Did you do some comparison to guide your choice?

> [@Nicolas80](#):
>
> `crypto_type: Sr25519Mnemonic`

The mnemonic is kind of a human-readable version of a seed and can be used to derive any kind of key using the appropriate [KDF](https://en.wikipedia.org/wiki/Key_derivation_function). The naming appears a bit confusing to me.

> [@Nicolas80](#):
>
> Maybe using this kind of Ed25519 seed […] is not a good idea for security reasons ❓

The seed is not “ed25519”, it is used to derive a key pair. And the problem is mainly the low entropy of the seed if a human is asked as entropy source. See these steps:

```auto
/// get keypair from Cesium id/pwd
pub fn pair_from_cesium(id: String, pwd: String) -> nacl::sign::Keypair {
	let params = scrypt::Params::new(12u8, 16u32, 1u32, 32).unwrap();
	let seed = &mut [0u8; 32];
	scrypt::scrypt(&pwd.into_bytes(), &id.into_bytes(), &params, seed).unwrap();
	nacl::sign::generate_keypair(seed)
}

```

The scheme is:

human entropy (id, passwd)  
⬇ (scrypt)  
seed  
⬇ (nacl)  
keypair

The security problem lies in the way we are using human “entropy” and scrypt to build the seed, not the way we are using nacl KDF to derive the key.

> [@Nicolas80](#):
>
> `encrypted_private_key` _bytes_: encrypted _substrate uri_

You should not call this “private key” when it is only a SURI, this is confusing.

> [@Nicolas80](#):
>
> should be `None` or a string starting with `'/'`

Then `None` and `/` have the same meaning 😉

> [@Nicolas80](#):
>
> (We might want to also convert remaining usages of `rpassword` crate and remove that dependency ❓)

Always a good idea to reduce the number of dependencies!

I’ll be happy to have @vit feedback on the functional part since a lot of these features are covered in Tikka!

---

<div class="post-metadata">

### Author: ![vit](https://forum.duniter.org/user_avatar/forum.duniter.org/vit/32/2775_2.png) [@vit](https://forum.duniter.org/u/vit)
#### Post date: [30 December 2024 17:11 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/3 "2024-12-30T17:11:12Z")

</div>

Great work !

Same comment about the names of crypto types :

`crypto_type` _String_: “BIP39Mnemonic” / “EntropyKDFSeed” / “G1V1Seed”

are the names that come in my mind, as SRxxx or EDxxx refer only to keypairs.

I see just a little problem in your docs that confuse me a little. Better use the term “account key” instead of “vault key” where the behavior will change if the key refers to a “root account key” (vault) or a “derivation account key” (derivation).

```auto
-v <NAME> Vault name for a key

```

As well, “vault name” is a term only valid for root accounts, not derivation accounts. _Account name for a key_ should be better. Imho.

I am missing a command to list the derivation of only one vault, giving a “root account” key or name.

[EDIT]

- If I create a derivation from the 2 first crypto\_type, I get the same result ? Or the BIP39 seed from the words and the substrate seed from the entropy ?
- Normally it is not possible to make derivation from ED25519 (I saw it somewhere in the substrate documentation). If gcli can do that, may be it’ll be the only one, if nobody else add it in the other API clients…

---

<div class="post-metadata">

### Author: ![poka](https://forum.duniter.org/user_avatar/forum.duniter.org/poka/32/6827_2.png) [@poka](https://forum.duniter.org/u/poka)
#### Post date: [30 December 2024 17:38 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/4 "2024-12-30T17:38:18Z")

</div>

We should not add features to legacy Cesium salt/password login.

---

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [31 December 2024 10:45 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/5 "2024-12-31T10:45:50Z")

</div>

Thank you for your responses 🙂; I’ll try to cover all the points.

> [@HugoTrentesaux](#):
>
> I knew about Diesel ([https://diesel.rs/](https://diesel.rs/)), not SeaORM. Did you do some comparison to guide your choice?

I first tried `rusqlite` but it was quite limited, then I looked at several posts and tried to find one crate with a minimum of “stars” on github that seemed good 🙂 (and diesel was also mentionned)

I only saw later that we can see an amount of downloads of a crate on [https://crates.io](https://crates.io) (so indeed diesel is with a bit more usages).

If diesel also supports having proper entity mappings (it think it does) we could check to change to that one if you think it would be better in the long run ❓

* * *

> [@HugoTrentesaux](#):
>
> The mnemonic is kind of a human-readable version of a seed and can be used to derive any kind of key using the appropriate [KDF](https://en.wikipedia.org/wiki/Key_derivation_function). The naming appears a bit confusing to me.

> [@vit](#):
>
> Same comment about the names of crypto types :
> 
> `crypto_type` _String_: “BIP39Mnemonic” / “EntropyKDFSeed” / “G1V1Seed”

> [@vit](#):
>
> If I create a derivation from the 2 first crypto\_type, I get the same result ? Or the BIP39 seed from the words and the substrate seed from the entropy ?

I agree the naming could be improved. In fact, since I only save the secret in “substrate URI” format the 2 `crypto_type` **Sr25519Mnemonic & Sr25519Seed** are mostly the same; I create the KeyPair for them the same way (`sp_core::sr25519::Pair::from_string(&deriv_suri, None).unwrap();`).

And to answer @vit last point; in the case 2 different root keys were created, one with a BIP39 and one with a “mini-secret”/seed; the secret part stored will be as in my example:

```auto
BIP39:
"bottom drive obey lake curtain smoke basket hold race lonely fit walk"

mini-secret/seed:
"0xf813535799c3a15b8e419a06964e87fabd3f265caebcbb38c935a1acdbe05253"

```

And when creating the “//0” derivation for those; the “substrate uri” used will be:

```auto
BIP39:
"bottom drive obey lake curtain smoke basket hold race lonely fit walk//0"

mini-secret/seed:
"0xf813535799c3a15b8e419a06964e87fabd3f265caebcbb38c935a1acdbe05253//0"

```

So what would be best for those `crypto_type` values and should I still keep 2 different ones for the BIP39 | mini-secret/seed ❓  
(I’m fine with the proposition of @vit : “BIP39Mnemonic” / “EntropyKDFSeed” / “G1V1Seed”)

* * *

> [@HugoTrentesaux](#):
>
> The seed is not “ed25519”, it is used to derive a key pair. And the problem is mainly the low entropy of the seed if a human is asked as entropy source. See these steps:

> [@poka](#):
>
> We should not add features to legacy Cesium salt/password login.

I understand that the human provided cesium id & secret are not really safe; but we still need to support them in order for people to migrate the identity and perform some more operations.

So indeed, I guess we should **not allow creating derivations for those** ; although it is technically possible as shown in my test and from what I read in [this page](https://polkadot.js.org/docs/keyring/start/suri) the hard derviations `//` are supported for all crypto; but soft derivation `/` are only supported by sr25519 pairs.

So the question that remains for G1v1 keys is do we still allow adding it to the vault ❓

If yes, we should show in the `vault list` somehow that no derivation can be made on those; any idea ❓

- An extra column “derivable” with values yes/no or a ‘\*’ to say yes…
- Or I should just also display the `crypto_type` (we need to come up with good names indeed) and mention that derivations are only allowed for the non G1v1 ones. Maybe I should reuse the `SecretFormat` values (Substrate, Seed, Cesium) to display to the user since the user has to provide a specific SecretFormat when importing something else than BIP39 ? (I adapted the `vault list all` provided below with that idea)

Last question regarding G1v1 keys; is it ok if I continue persisting the “seed” suri of the key instead of the double id & secret (which is a bit annoying to save into 1 db field) ❓

* * *

> [@HugoTrentesaux](#):
>
> > [@Nicolas80](#):
> >
> > `encrypted_private_key` _bytes_: encrypted _substrate uri_
> 
> You should not call this “private key” when it is only a SURI, this is confusing.

Indeed, I took the name from `tikka` db mappings in the first place.

So 2 questions for this one:

- Is it ok to continue using the same `encrypt` function we used for key files (`age::Encryptor::with_user_passphrase(Secret::new(passphrase));`) ❓
- What name would be best; `encrypted_suri` ❓ (in which case it would only be correct if we keep persisting G1v1 seed as a `suri` as well)

* * *

> [@HugoTrentesaux](#):
>
> > [@Nicolas80](#):
> >
> > should be `None` or a string starting with `'/'`
> 
> Then `None` and `/` have the same meaning 😉

I didn’t get what you meant in this case. The user input for “derivation path” (requested using `inquire`) is allowed to be empty (=\> None in Rust and null in DB) or any string that must start with at least one ‘/’ character since all derivations path must have at least one of those.

We may want to add extra checks for the _user input_ path; but it might be enough to check we don’t already have the same SS58 `address` persisted in db when computing the keypair with the “root” suri + the derivation path.

* * *

> [@vit](#):
>
> I see just a little problem in your docs that confuse me a little. Better use the term “account key” instead of “vault key” where the behavior will change if the key refers to a “root account key” (vault) or a “derivation account key” (derivation).
> 
> ```auto
> -v <NAME> Vault name for a key
> 
> ```
> 
> As well, “vault name” is a term only valid for root accounts, not derivation accounts. _Account name for a key_ should be better. Imho.

Indeed, I might not be consistent in my namings everywhere and if we can come up with better names, I’m all for it 🙂

#### About “vault name”

Since the `name` db property is in `vault_derivation` db table; we can have a name for any derivation (`<Root>` or otherwise).

In my example of `list all` above, “`substrateDeriv1CustomName`” was a specific name given to a derivation with `path` “//1”.

And since most of the commands should allow us to select a specific `SS58 Address` or it’s **name** this is why I decided on “vault name”; I also think “account name” is risky and might be confused with the **“Identity”** name linked to the account.

Maybe it would be better to mention “ **derivation name** ” ❓ (I gave examples below using that - to be seen if it’s clear/good enough)

It could be something like

```auto
Options:
  -a <ADDRESS> SS58 Address
  -d <NAME> Derivation name for a SS58 Address (can be a <Root> derivation name)

```

And in the `vault list all`

```auto
gcli vault list all
available derivations:
┌──────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ SS58 Address Format Path Derivation Name │
╞══════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV substrate <Root> mySubstrateKey │
│ 5D34dL5prEUaGNQtPPZ3yN5Y6BnkfXunKXXz6fo7ZJbLwRRH //0 <mySubstrateKey//0> │
│ 5GBNeWRhZc2jXu7D55rBimKYDk8PGk8itRYFTPfC8RJLKG5o //1 substrateDeriv1CustomName │
│ 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY //Alice <mySubstrateKey//Alice> │
│ 5ET2jhgJFoNQUpgfdSkdwftK8DKWdqZ1FKm5GKWdPfMWhPr4 cesium <Root> myCesiumKey │
│ 5GjAp6kkbsDjxABGouMvu1Mxbr7PaFqbMrprEoZ466vPF2Vt seed <Root> mySeedKey │
│ 5HBwy19piXNWg7bfShuNgWsBCWRzkG99M8eRGB6PHkxeAKAV //0 <mySeedKey//0> │
│ 5HdRvcHctMZG4KVFZd1qLwgKyLhvKWTAGYpXsu2JRkUp3u5h //2 deleteme │
└──────────────────────────────────────────────────────────────────────────────────────────────────────┘

```

#### About “vault key”

Maybe also using “ **derivation** ” for that instead ❓

Example of adated commands:

```auto
gcli vault derivation --help
Add a derivation to an existing <Root> derivation

Usage: gcli vault derivation <-a <ADDRESS>|-d <NAME>>

Options:
  -a <ADDRESS> SS58 Address
  -d <NAME> Derivation name for a SS58 Address (can be a <Root> derivation name)
  -h, --help Print help

```

For that one, _if needed_ we can adapt to use a different `clap` command “argument” that would have a slightly modified explanation for `-d <NAME>` : `derivation name for a <Root> SS58 Address`

```auto
gcli vault rename --help
Give a meaningful derivation name to a key derivation (can be for a <Root> derivation)

Usage: gcli vault rename <ADDRESS>

Arguments:
  <ADDRESS> SS58 Address

Options:
  -h, --help Print help

```

```auto
gcli vault remove --help
Remove a derivation (and all linked derivations along with the key if a <Root> derivation is given)

Usage: gcli vault remove <-a <ADDRESS>|-d <NAME>>

Options:
  -a <ADDRESS> SS58 Address
  -d <NAME> Derivation name for a SS58 Address (can be a <Root> derivation name)
  -h, --help Print help

```

```auto
gcli vault use --help
Use specific derivation (changes the config Address)

Usage: gcli vault use <-a <ADDRESS>|-d <NAME>>

Options:
  -a <ADDRESS> SS58 Address
  -d <NAME> Derivation name for a SS58 Address (can be a <Root> derivation name)
  -h, --help Print help

```

* * *

> [@vit](#):
>
> I am missing a command to list the derivation of only one vault, giving a “root account” key or name.

I can add another sub-command for `vault list`.

Maybe `vault list for` where we could provide `SS58 Address` or `name` of a (\<Root\>) derivation

Something like:

```auto
gcli vault list --help   
List available derivations

Usage: gcli vault list <COMMAND>

Commands:
  all List all <Root> derivations and their linked derivations in the vault
  for List all derivations linked to the one selected
  root List only <Root> derivations in the vault
  help Print this message or the help of the given subcommand(s)

Options:
  -h, --help Print help

```

```auto
gcli vault list for --help
List all derivations linked to the one selected

Usage: gcli vault list for <-a <ADDRESS>|-d <NAME>>

Options:
  -a <ADDRESS> SS58 Address
  -d <NAME> Derivation name for a SS58 Address (can be a <Root> derivation name)
  -h, --help Print help

```

---

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [31 December 2024 12:51 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/6 "2024-12-31T12:51:14Z")

</div>

I adapted the code to reflect the suggested changes in descriptions above; and I see it’s also possible with `clap` to provide a `long_about` more detailed description when requesting `--help` on a specific command; I did that for `import` and `derivation`.

(I also added the `vault list for` command as suggested above)

```auto
gcli vault --help                  
Key management (import, generate, list...)

Usage: gcli vault <COMMAND>

Commands:
  list List available derivations
  use Use specific derivation (changes the config Address)
  generate Generate a mnemonic
  import Import key from (substrate)mnemonic or other format with interactive prompt
  derivation Add a derivation to an existing <Root> derivation
  rename Give a meaningful derivation name to a key derivation (can be for a <Root> derivation)
  remove Remove a derivation (and all linked derivations along with the key if a <Root> derivation is given)
  list-files (deprecated)List available key files (needs to be migrated with command `vault migrate` in order to use them)
  migrate (deprecated)Migrate old key files into db (will have to provide password for each key)
  where Show where vault db (or old keys) is stored
  help Print this message or the help of the given subcommand(s)

Options:
  -h, --help Print help

```

```auto
gcli vault import --help   
Import key from (substrate)mnemonic or other format with interactive prompt

If a (substrate)mnemonic is provided with a derivation path, it will ensure the <Root> derivation
and associated key exists before creating the new derivation; but please use command 
`vault derivation|derive|deriv` to add a derivation to an existing <Root> derivation instead.

Usage: gcli vault import [OPTIONS]

Options:
  -S, --secret-format <SECRET_FORMAT>
          Secret key format (substrate, seed, cesium)
          
          [default: substrate]

  -h, --help
          Print help (see a summary with '-h')

```

```auto
gcli vault derivation --help   
Add a derivation to an existing <Root> derivation

Only "substrate" and "seed" format are supported for derivations
Use command `vault list root` to see available <Root> derivations and their format

Usage: gcli vault derivation <-a <ADDRESS>|-d <NAME>>

Options:
  -a <ADDRESS>
          SS58 Address

  -d <NAME>
          Derivation name for a SS58 Address (can be a <Root> derivation name)

  -h, --help
          Print help (see a summary with '-h')

```

---

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [1 January 2025 17:42 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/7 "2025-01-01T17:42:17Z")

</div>

Specifically for Cesium V1; I added a [test](https://git.duniter.org/clients/rust/gcli-v2s/-/blame/vault-db-persistence-and-derivation-support/src/keys.rs?ref_type=heads#L674-726) to verify we don’t need the `nacl::sign::Keypair` (we can keep using `scrypt` to retrieve the seed and then use `sp_core::ed25519::Pair`)

This means we could remove some code linked to that (`CesiumSigner` in src/commands/cesium.rs) and possibly the `nacl` dependency ❓

---

<div class="post-metadata">

### Author: ![vit](https://forum.duniter.org/user_avatar/forum.duniter.org/vit/32/2775_2.png) [@vit](https://forum.duniter.org/u/vit)
#### Post date: [2 January 2025 16:50 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/8 "2025-01-02T16:50:43Z")

</div>

> [@Nicolas80](#):
>
> Maybe it would be better to mention “ **derivation name** ” ❓ (I gave examples below using that - to be seen if it’s clear/good enough)

I do not understand why you use a specific subcategory of account/keypair to use with the “name” statement, like “vault” or “derivation”.

Because, basically, you just give a _name_ to an “account” or “address” or “keypair”. So you can use those three designation that are referring to the same thing. As long as you must add an explanation with parenthesis ( `(can be a <Root> derivation name`), it is a sign to use another designation.

Besides, there is no such thing as “a root derivation”. A root account has no derivation. It is just your db model that use these trick, but you should not tell about it in the cli user interface. Imho.

---

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [2 January 2025 22:26 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/9 "2025-01-02T22:26:04Z")

</div>

Thanks for the remark. I adapted to simply “name” and mention primarily “SS58 Address” in most places now (maybe I should drop the “SS58” part ?)

Sorry for the spam 🙂; but this would give those descriptions/outputs:

```auto
gcli -h            
A command-line interface for Duniter v2s uses

Usage: gcli [OPTIONS] <COMMAND>

Commands:
  account Account (balance, transfer...)
  identity Identity (get, create, confirm, revoke...)
  smith Smith (certify, go-online, go-offline...)
  tech Tech (list members, proposals, vote...)
  ud Universal Dividend (claim...)
  oneshot Oneshot account (balance, create, consume...)
  blockchain Blockchain (current block, runtime info...)
  indexer Indexer (check, latest block)
  config Config (show, save...)
  vault Key management (import, generate, list...)
  help Print this message or the help of the given subcommand(s)

Options:
  -i, --indexer <INDEXER> Overwrite indexer endpoint
      --no-indexer Do not use indexer
  -s, --secret <SECRET> Secret key or BIP39 mnemonic (only used when secret format is compatible) (eventually followed by derivation path)
  -S, --secret-format <SECRET_FORMAT> Secret key format (seed, substrate, cesium)
  -a <ADDRESS> SS58 Address
  -v <NAME> Name of an SS58 Address in the vault
  -u, --url <URL> Overwrite duniter websocket RPC endpoint
  -n, --network <NETWORK> Target network (local, gdev, gtest...)
      --no-wait prevent waiting for extrinsic completion
  -o, --output-format <OUTPUT_FORMAT> Output format (human, json, ...) [default: human]
  -h, --help Print help
  -V, --version Print version

```

```auto
gcli vault -h
Key management (import, generate, list...)

Usage: gcli vault <COMMAND>

Commands:
  list List available SS58 Addresses in the vault
  use Use specific SS58 Address (changes the config Address)
  generate Generate a mnemonic
  import Import key from (substrate)mnemonic or other format with interactive prompt
  derivation Add a derivation to an existing <Account>
  rename Give a meaningful name to an SS58 Address in the vault
  remove Remove an SS58 Address from the vault
  list-files (deprecated)List available key files (needs to be migrated with command `vault migrate` in order to use them)
  migrate (deprecated)Migrate old key files into db (will have to provide password for each key)
  where Show where vault db (or old keys) is stored
  help Print this message or the help of the given subcommand(s)

Options:
  -h, --help Print help

```

```auto
gcli vault list -h   
List available SS58 Addresses in the vault

Usage: gcli vault list <COMMAND>

Commands:
  all List all <Account> and their linked derivations SS58 Addresses in the vault
  for List <Account> and derivations SS58 Addresses linked to the selected one
  account List all <Account> SS58 Addresses in the vault
  help Print this message or the help of the given subcommand(s)

Options:
  -h, --help Print help

```

```auto
gcli vault list all                
available SS58 Addresses:
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ SS58 Address Format Account/Path Name │
╞═══════════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV substrate <Account> mySubstrateKey │
│ 5D34dL5prEUaGNQtPPZ3yN5Y6BnkfXunKXXz6fo7ZJbLwRRH //0 <mySubstrateKey//0> │
│ 5GBNeWRhZc2jXu7D55rBimKYDk8PGk8itRYFTPfC8RJLKG5o //1 substrateDeriv1CustomName │
│ 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY //Alice <mySubstrateKey//Alice> │
│ 5ET2jhgJFoNQUpgfdSkdwftK8DKWdqZ1FKm5GKWdPfMWhPr4 cesium <Account> myCesiumKey │
│ 5GjAp6kkbsDjxABGouMvu1Mxbr7PaFqbMrprEoZ466vPF2Vt seed <Account> mySeedKey │
│ 5HBwy19piXNWg7bfShuNgWsBCWRzkG99M8eRGB6PHkxeAKAV //0 <mySeedKey//0> │
│ 5HdRvcHctMZG4KVFZd1qLwgKyLhvKWTAGYpXsu2JRkUp3u5h //2 deleteme │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────┘

```

```auto
gcli vault list for -v substrateDeriv1CustomName
available SS58 Addresses linked to Derivation[address:"5GBNeWRhZc2jXu7D55rBimKYDk8PGk8itRYFTPfC8RJLKG5o", name:Some("substrateDeriv1CustomName"), path:Some("//1"), account_address:"5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV"]:
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ SS58 Address Format Account/Path Name │
╞═══════════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV substrate <Account> mySubstrateKey │
│ 5D34dL5prEUaGNQtPPZ3yN5Y6BnkfXunKXXz6fo7ZJbLwRRH //0 <mySubstrateKey//0> │
│ 5GBNeWRhZc2jXu7D55rBimKYDk8PGk8itRYFTPfC8RJLKG5o //1 substrateDeriv1CustomName │
│ 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY //Alice <mySubstrateKey//Alice> │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────┘

```

```auto
gcli vault list for -v mySubstrateKey                         
available SS58 Addresses linked to Account[address:"5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV", name:Some("mySubstrateKey")]:
┌───────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ SS58 Address Format Account/Path Name │
╞═══════════════════════════════════════════════════════════════════════════════════════════════════════════╡
│ 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV substrate <Account> mySubstrateKey │
│ 5D34dL5prEUaGNQtPPZ3yN5Y6BnkfXunKXXz6fo7ZJbLwRRH //0 <mySubstrateKey//0> │
│ 5GBNeWRhZc2jXu7D55rBimKYDk8PGk8itRYFTPfC8RJLKG5o //1 substrateDeriv1CustomName │
│ 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY //Alice <mySubstrateKey//Alice> │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────┘

```

```auto
gcli vault list account
available <Account> SS58 Addresses:
┌──────────────────────────────────────────────────────────────────────────────────────────────┐
│ SS58 Address Format Account/Path Name │
╞══════════════════════════════════════════════════════════════════════════════════════════════╡
│ 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV substrate <Account> mySubstrateKey │
│ 5ET2jhgJFoNQUpgfdSkdwftK8DKWdqZ1FKm5GKWdPfMWhPr4 cesium <Account> myCesiumKey │
│ 5GjAp6kkbsDjxABGouMvu1Mxbr7PaFqbMrprEoZ466vPF2Vt seed <Account> mySeedKey │
└──────────────────────────────────────────────────────────────────────────────────────────────┘

```

```auto
gcli vault import --help
Import key from (substrate)mnemonic or other format with interactive prompt

If a (substrate)mnemonic is provided with a derivation path, it will ensure the base <Account>
and associated SS58 Address exists before creating the derivation; but please use command 
`vault derivation|derive|deriv` to add a derivation to an existing <Account> instead.

Usage: gcli vault import [OPTIONS]

Options:
  -S, --secret-format <SECRET_FORMAT>
          Secret key format (substrate, seed, cesium)
          
          [default: substrate]

  -h, --help
          Print help (see a summary with '-h')

```

```auto
gcli vault derivation --help   
Add a derivation to an existing <Account>

Only "substrate" and "seed" format are supported for derivations
Use command `vault list account` to see available <Account> and their format

Usage: gcli vault derivation <-a <ADDRESS>|-v <NAME>>

Options:
  -a <ADDRESS>
          SS58 Address

  -v <NAME>
          Name of an SS58 Address in the vault

  -h, --help
          Print help (see a summary with '-h')

```

```auto
gcli vault remove --help
Remove an SS58 Address from the vault

If an <Account> SS58 Address is given it will also remove all linked derivations

Usage: gcli vault remove <-a <ADDRESS>|-v <NAME>>

Options:
  -a <ADDRESS>
          SS58 Address

  -v <NAME>
          Name of an SS58 Address in the vault

  -h, --help
          Print help (see a summary with '-h')

```

---

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [3 January 2025 23:28 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/10 "2025-01-03T23:28:19Z")

</div>

@HugoTrentesaux if you have a bit of time, could you verify if it’s ok to remove `nacl` dependency and directly use `sp_core::ed25519::Pair` instead of `nacl::sign::Keypair` ❓

I did that change in my [last commit](https://git.duniter.org/clients/rust/gcli-v2s/-/merge_requests/41/diffs?commit_id=4eafec2ad988b115a5bc13ffd268c5c490b12816#b246f00e3a068a310b6548833b6a1b52230cd5ad) and everything seems to work properly.  
And I can still perform `account transfer` from a _cesium_ account using `sp_core::ed25519::Pair` and it’s signature instead of the ones of `nacl` (but maybe there was another reason I don’t know for using `nacl` in the first place ?)

---

<div class="post-metadata">

### Author: ![HugoTrentesaux](https://forum.duniter.org/user_avatar/forum.duniter.org/hugotrentesaux/32/6396_2.png) [@HugoTrentesaux](https://forum.duniter.org/u/HugoTrentesaux)
#### Post date: [6 January 2025 11:56 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/11 "2025-01-06T11:56:55Z")

</div>

To test your branch, I migrated my keys and got:

```auto
> ./target/debug/gcli vault migrate
[...]
Error while fetching vault data for address 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY: Input("Excessive work parameter for passphrase.\nDecryption would take around \u{2068}64\u{2069} seconds.")

```

I seems ok to not use nacl. I used it to mimic previous behavior, but only the scrypt part is relevant to generate the seed. Then the key derivation is done the same way in the two implementations (nacl and ed25519-zebra).

> [@Nicolas80](#):
>
> we could check to change to that one if you think it would be better in the long run ❓

I do not have opinion between Diesel / SeaORM. Did not use any of them. See [https://www.sea-ql.org/SeaORM/docs/internal-design/diesel/](https://www.sea-ql.org/SeaORM/docs/internal-design/diesel/) for more info. There are pros and cons for both. @tuxmain did you use one?

> [@Nicolas80](#):
>
> So the question that remains for G1v1 keys is do we still allow adding it to the vault ❓

If we allow to add a _seed_ / mini-secret to the vault, we can not prevent it to be generated by an unsecured technique or even made public on the internet. So the answer is: yes, we allow to add them to the vault. But we will only handle the seed and not the id/pasword used as scrypt input to generate this seed. No need to prevent derivation either.

> [@Nicolas80](#):
>
> is it ok if I continue persisting the “seed” suri of the key instead of the double id & secret

Yes it is ok to store the seed. Remember the definition of SURI ([Substrate Uri | polkadot{.js}](https://polkadot.js.org/docs/keyring/start/suri/))

`<mnemonic or mini-secret>[//hard-derivation][/soft-derivation][///password]`

You can save the mnemonic / mini-secret / “seed” and the [//hard-derivation][/soft-derivation] separately if you want, this will allow to build the SURI. By the way we do not use the ///password component since the UX is more complex.

> [@Nicolas80](#):
>
> Is it ok to continue using the same `encrypt` function we used for key files (`age::Encryptor::with_user_passphrase(Secret::new(passphrase));`) ❓

I used age because it’s a rather standard encryption scheme with a lot of existing implementations. See [my comment](https://git.duniter.org/clients/rust/gcli-v2s/-/merge_requests/40#note_46136):

> [@](#):
>
> The reason I wanted age-encrypted secrets is for portability. Example using rage ([https://github.com/str4d/rage](https://github.com/str4d/rage)):
> 
> ```auto
> > gcli vault where
> /home/hugo/.local/share/gcli
> > rage --decrypt ~/.local/share/gcli/5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
> bottom drive obey lake curtain smoke basket hold race lonely fit walk//Alice
> 
> ```

This is not a strict constraint however.

> [@Nicolas80](#):
>
> I didn’t get what you meant in this case.

I meant that we have multiple representations of the same thing. Usually we want to have a canonical form to prevent these duplicates. See this example:

```auto
> subkey inspect "bottom drive obey lake curtain smoke basket hold race lonely fit walk"
Secret phrase: bottom drive obey lake curtain smoke basket hold race lonely fit walk
  Secret seed: 0xfac7959dbfe72f052e5a0c3c8d6530f202b02fd8f9f5ca3580ec8deb7797479e
  Public key (hex): 0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a
  SS58 Address: 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV
> subkey inspect "bottom drive obey lake curtain smoke basket hold race lonely fit walk///"
Secret Key URI `bottom drive obey lake curtain smoke basket hold race lonely fit walk///` is account:
  Secret seed: 0xfac7959dbfe72f052e5a0c3c8d6530f202b02fd8f9f5ca3580ec8deb7797479e
  Public key (hex): 0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a
  SS58 Address: 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV

```

We get the same result with different inputs. No matter if it is “secret phrase” or “secret key URI”. Having these two stored in different ways in the database would be confusing.

---

<div class="post-metadata">

### Author: ![tuxmain](https://forum.duniter.org/user_avatar/forum.duniter.org/tuxmain/32/6423_2.png) [@tuxmain](https://forum.duniter.org/u/tuxmain)
#### Post date: [6 January 2025 12:06 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/12 "2025-01-06T12:06:53Z")

</div>

> [@HugoTrentesaux](#):
>
> There are pros and cons for both. @tuxmain did you use one?

No, I know that lots of rustaceans use Diesel but I personally dislike SQL databases.

I don’t know what it’s about but I really doubt Ğcli would need a relational database, or even a key-value database. Why not just JSON, so it could be modified by hand? Or something else lighter than a database?

---

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [6 January 2025 13:53 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/13 "2025-01-06T13:53:57Z")

</div>

> [@HugoTrentesaux](#):
>
> To test your branch, I migrated my keys and got:
> 
> ```auto
> > ./target/debug/gcli vault migrate
> [...]
> Error while fetching vault data for address 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY: Input("Excessive work parameter for passphrase.\nDecryption would take around \u{2068}64\u{2069} seconds.")
> 
> ```

I don’t get the reason of your error; I see this is the key of “Alice”; I guess you imported it originally with the mnemonic in the code + “//Alice” suri.

Also, the exception is thrown by the same decrypt function which I didn’t adapt 🤔

When I try to do the same import from key file locally; it works.  
I tried with a key file protected by long password and by empty passord; both end up doing this when I migrate:

```auto
gcli vault migrate
Migrating existing key files to db
available key files to possibly migrate:
┌──────────────────────────────────────────────────┐
│ Key file │
╞══════════════════════════════════════════════════╡
│ 5Funkng8dL397H9ZEJ9sXDjFhStCqRYUj1a62ngcrEMy7ivn │
│ 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY │
│ 5HE6gH87fVuGj6akXneNceBtgEiaUZua43TJbVBRQTT77gp6 │
└──────────────────────────────────────────────────┘

Trying to migrate key 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
Enter password to unlock account 5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY
Password: 
Created vault account [address:"5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV"]
Created derivation Derivation[address:"5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY", name:None, path:Some("//Alice"), account_address:"5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV"]
Import done: Derivation[address:"5GrwvaEF5zXb26Fz9rcQpDWS57CtERHpNehXCPcNoHGKutQY", name:None, path:Some("//Alice"), account_address:"5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV"]
Migration done

```

* * *

> [@HugoTrentesaux](#):
>
> ```auto
> > subkey inspect "bottom drive obey lake curtain smoke basket hold race lonely fit walk///"
> Secret Key URI `bottom drive obey lake curtain smoke basket hold race lonely fit walk///` is account:
> Secret seed: 0xfac7959dbfe72f052e5a0c3c8d6530f202b02fd8f9f5ca3580ec8deb7797479e
> Public key (hex): 0x46ebddef8cd9bb167dc30878d7113b7e168e6f0646beffd77d69d39bad76b47a
> SS58 Address: 5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV
> 
> ```
> 
> We get the same result with different inputs. No matter if it is “secret phrase” or “secret key URI”. Having these two stored in different ways in the database would be confusing.

With the way I did it, it should not be possible.

When importing (or migrating old keys) I retrieve the provided input (from user or from key file) and search for any ‘/’ inside it.

If there is any, I take the first part as the suri; and second part as a potential derivation path.

Then I’ll make sure the \<Account\> is stored in DB: vault\_account for the address + `encrypted_suri` together with the linked vault\_derivation for the same (\<Account\>) address.

If there was a derivation path; then I create the keypair for the whole derivation suri and check if it’s SS58 address is already present in DB or not.

So in the case someone would try to import **“bottom drive obey lake curtain smoke basket hold race lonely fit walk///”** which is a valid suri with empty `password`; it will first create (if not yet existing) vault\_account and vault\_derivation for **“bottom drive obey lake curtain smoke basket hold race lonely fit walk”** suri; then it will check if the derivation suri already has an address; which it does =\> will not import it.

_Actually_, I had to make a small fix for when directly importing such a suri directly without having a db entry for the base suri yet (it was giving a db `UNIQUE constraint failed: vault_derivation.address`).

Example with the small code fix when trying to import **“bottom drive obey lake curtain smoke basket hold race lonely fit walk///”** directly:

```auto
gcli vault import  
Mnemonic: 
Trying to import for SS58 address :'5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV'
Enter password to protect the key
> Password ********
(Optional) Enter a name for the vault entry
> Name: direct_import_with_empty_suri_password
Created vault account [address:"5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV"]
Derivation path provided:'///' linked to the same SS58 Address than the base suri without derivation
For that reason only the base suri was imported
Created: Account[address:"5DfhGyQdFobKM8NsWvEeAKk5EQQgYe9AydgJ7rMB6E1EqRzV", name:Some("direct_import_with_empty_suri_password")]
Import done

```

* * *

@tuxmain, for the fact of using a DB I checked the execution time overhead (of initializing the db schema when starting without a sqlite file present) and it was usually below 1ms.

And the really good thing in my opinion is the ease of use and the protection we get with the DB constraints that were added (on the unique PK `address` and on the unique `name`).

Also, in the case someone would `vault import` directly a suri with a valid derivation path; I can create all the entries in the same transaction and all is rollbacked in case of error (I just had that issue with proper rollback when testing the case of Hugo above).

---

<div class="post-metadata">

### Author: ![poka](https://forum.duniter.org/user_avatar/forum.duniter.org/poka/32/6827_2.png) [@poka](https://forum.duniter.org/u/poka)
#### Post date: [6 January 2025 17:57 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/14 "2025-01-06T17:57:35Z")

</div>

I agree gcli doesn’t need relational db, but I disagree using json. A database is safer about data race and management.  
Not sure it would be useful to manipulate data’s by hand in a json.  
For gecko I swiped JSON files to key value db (Hive, today deprecated but that’s another story…) long time ago for instance. Clearly easier to use and safer.

> [@Nicolas80](#):
>
> it was usually below 1ms

> [@Nicolas80](#):
>
> ease of use

That the 2 points for me, plus popularity to guarantee maintenability on long term.

---

<div class="post-metadata">

### Author: ![Moul](https://forum.duniter.org/user_avatar/forum.duniter.org/moul/32/9145_2.png) [@Moul](https://forum.duniter.org/u/Moul)
#### Post date: [16 February 2025 13:59 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/15 "2025-02-16T13:59:53Z")

</div>

v0.4.0 seems ready for release.  
@HugoTrentesaux, would you let @Nicolas80 with moderator rights on the repository so he can release it?

---

<div class="post-metadata">

### Author: ![poka](https://forum.duniter.org/user_avatar/forum.duniter.org/poka/32/6827_2.png) [@poka](https://forum.duniter.org/u/poka)
#### Post date: [16 February 2025 14:31 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/16 "2025-02-16T14:31:19Z")

</div>

There is a command `gcli publish` you can use to faciliate release deployment.

---

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [16 February 2025 18:35 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/17 "2025-02-16T18:35:47Z")

</div>

Any documentation somewhere of how / when to use that ?

And if done manually, the proper way to create Tag & release (I never made releases through gitlab before 🙂)

As for rights, it seems I have rights to create Tags and edit releases - so probably create releases as well ?

 ![image](https://forum.duniter.org/uploads/default/original/2X/4/4fbd139616a5ef48292c651cbdc0fb5c57821e54.png)

Edit: I checked that `gcli publish` was basically juste creating the proper git Tag and did it with the master branch up to date locally.

Now I see the tag, and I guess I can just “create release” when build is done and ok

 ![image](https://forum.duniter.org/uploads/default/original/2X/3/3d2bcec992291b094b2b4d751002d831c60f4897.png)

---

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [16 February 2025 18:46 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/18 "2025-02-16T18:46:46Z")

</div>

@Moul Tests & Builds passed, but the release part failed:

 ![image](https://forum.duniter.org/uploads/default/original/2X/9/9b36e0690af8fb09a0f409823db4d2355cabff01.png)

Error: `error="failed to create release: API Error Response status_code: 401 message: error: invalid_token" version=0.21.0`

> **[Pipeline · clients / Rust / Ğcli-v2s · GitLab](https://git.duniter.org/clients/rust/gcli-v2s/-/pipelines/39914)**
>
> Rust written command line interface client for v2 ecosystem, targeting advanced users

---

<div class="post-metadata">

### Author: ![poka](https://forum.duniter.org/user_avatar/forum.duniter.org/poka/32/6827_2.png) [@poka](https://forum.duniter.org/u/poka)
#### Post date: [16 February 2025 19:09 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/19 "2025-02-16T19:09:29Z")

</div>

> [@Nicolas80](#):
>
> Now I see the tag, and I guess I can just “create release” when build is done and ok

I just remember, publish jsut create a tag an push it. Then a pipeline make the build and publish de release, so you don’t have to create the release manualy.

I don’t know for this error.

* * *

edit: The [gitlab release image](https://git.duniter.org/clients/rust/gcli-v2s/-/blame/master/.gitlab-ci.yml?ref_type=heads#L71) I used to automaticly publish release of gcli is deprecated : [GitLab.org / release-cli · GitLab](https://gitlab.com/gitlab-org/release-cli)

Now we should use release command of the glab CLI tool to publish gcli: [docs/source/release · main · GitLab.org / cli · GitLab](https://gitlab.com/gitlab-org/cli/-/tree/main/docs/source/release)

The error is because my token which was used to publish release is expired I guess, we have to renew token.  
For now you can just publish release manualy.

* * *

edit 2: It’s ok, the variable key name changed on latest gitlab CLI tool version ( 🤷‍♂️), I updated it and run the job again, and release has been published: [Releases · clients / Rust / Ğcli-v2s · GitLab](https://git.duniter.org/clients/rust/gcli-v2s/-/releases)

For admins, the variable is set here: [https://git.duniter.org/clients/rust/gcli-v2s/-/settings/ci\_cd#js-cicd-variables-settings](https://git.duniter.org/clients/rust/gcli-v2s/-/settings/ci_cd#js-cicd-variables-settings)

Ans the documentation is here: [docs/index.md · master · GitLab.org / release-cli · GitLab](https://gitlab.com/gitlab-org/release-cli/-/blob/master/docs/index.md#usage)

---

<div class="post-metadata">

### Author: ![Nicolas80](https://forum.duniter.org/user_avatar/forum.duniter.org/nicolas80/32/11019_2.png) [@Nicolas80](https://forum.duniter.org/u/Nicolas80)
#### Post date: [17 February 2025 14:56 UTC](https://forum.duniter.org/t/request-for-review-adding-db-persistence-for-all-secretformat-of-vault-keys-as-well-as-supporting-derivations/12829/20 "2025-02-17T14:56:57Z")

</div>

I close this thread by giving the link to the release post:

> [@Ğcli version 0.4.0](https://forum.duniter.org/t/gcli-version-0-4-0/12937):
>
> A new Ğcli…
