Fun with the Wot in Neo4j

F2F ??
EDIT : Display the shortest path between a specific nodes and all other nodes ?

F2F -> friend of friend ! See the original post of greyzlii :slight_smile:

@inso ok, tks ! :slight_smile:

What about to create a new API, named “WOT_GRAPH_API” ?
Could you:

  • Add your project into duniter repo, ( as subproject “duniter-neo4j” ?)
  • Add an endpoint to your Duniter node. So we could display this extended API into network view.

After that, we will be able to use it in Cesium+, to suggest friend. But maybe we will need to add features, such as sort (first suggestion should be link that could reduce the WoT size").

(sorry for my approximate English again)

1 « J'aime »

It seems that @yannlefranco reach 94% of sentries with stepmax at 3.

MATCH (n {sentry : 1} )
WHERE NOT n.uid = "yannlefranco"
WITH count(n) as count_n
MATCH p=ShortestPath((member {uid:"yannlefranco"}) <-[*..3]- (r_sentry {sentry : 1}))
RETURN member.uid,count_n as nb_sentries, count(r_sentry) as reachable_sentries, 100.0 * count(r_sentry) / count_n AS percent

╒══════════════╤═════════════╤════════════════════╤═════════════════╕
│"member.uid"  │"nb_sentries"│"reachable_sentries"│"percent"        │
╞══════════════╪═════════════╪════════════════════╪═════════════════╡
│"yannlefranco"│18           │17                  │94.44444444444444│
└──────────────┴─────────────┴────────────────────┴─────────────────┘

This number reduces at 55 when 2 steps only are authorized :

╒══════════════╤═════════════╤════════════════════╤═════════════════╕
│"member.uid" │"nb_sentries"│"reachable_sentries"│"percent" │
╞══════════════╪═════════════╪════════════════════╪═════════════════╡
│"yannlefranco"│18 │10 │55.55555555555556│
└──────────────┴─────────────┴────────────────────┴─────────────────┘

cgeek reaches 100% of sentries at step2 :

╒════════════╤═════════════╤═══════╤════════════════════╤═════════════════╕
│"member.uid"│"nb_sentries"│"steps"│"reachable_sentries"│"percent"        │
╞════════════╪═════════════╪═══════╪════════════════════╪═════════════════╡
│"cgeek"     │18           │1      │13                  │72.22222222222223│
├────────────┼─────────────┼───────┼────────────────────┼─────────────────┤
│"cgeek"     │18           │2      │5                   │27.77777777777778│
└────────────┴─────────────┴───────┴────────────────────┴─────────────────┘
4 « J'aime »

I see 19 sentries right now, do you agree with this statement?

I agree with this statement.

That’s pretty cool you have such a tool! :slight_smile:

Merci beaucoup greyzlii pour ces nouvelles API,

J’ai migré le dépot vers l’organisation duniter car les fonctionnalités sont vraiment prometteuses ! https://github.com/duniter/duniter-neo4j

1 « J'aime »

Oui, moi aussi je suis content de ton retour @greyzlii :slight_smile:

On avance vite avec tout ca

My database is not up to date.
I tried to sync but I have an error. Maybe someone know why.
If not, I will search for myself (no problem, I like it :slight_smile: )

Progress:

Download: [||||||||||||||||||||] 100 %
Apply:    [                    ] 0 %

Status: This URI has reached its maximum usage quota. Please retry later.
All done.

EDIT : solved !

EDIT2 :

I agree !

MATCH (n {sentry:1}) RETURN count(n) as nb_sentries

╒═════════════╕
│"nb_sentries"│
╞═════════════╡
│19           │
└─────────────┘

The request for all members :

MATCH (n)
WITH collect(n.uid) as collect_uid
UNWIND collect_uid as uid
MATCH (n {sentry : 1} )
WHERE NOT n.uid = uid
WITH count(n) as count_n, uid
MATCH p=ShortestPath((member {uid:uid}) <-[*]-(r_sentry {sentry : 1}))
WITH member, count_n, r_sentry,p
WHERE length(p) <= 3
RETURN member.uid,count_n as nb_sentries, count(r_sentry) as reachable_sentries, 100.0 * count(r_sentry) / count_n AS percent
╒════════════════╤═════════════╤════════════════════╤═════════════════╕
│"member.uid"    │"nb_sentries"│"reachable_sentries"│"percent"        │
╞════════════════╪═════════════╪════════════════════╪═════════════════╡
│"kimamila"      │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Yann-Lug"      │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"vincentux"     │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"filb"          │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"rimek"         │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"RavanH"        │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"CandideSK8"    │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"gnu-tux"       │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"b3j0f"         │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"yannlefranco"  │19           │18                  │94.73684210526316│
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Caizohan"      │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"DebOrah"       │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"pafzedog"      │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"gpsqueeek"     │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"cuckooland"    │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"elois"         │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"PierreYves"    │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"FugazziPL"     │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Fiatouxxx"     │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"JeanFerreira"  │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Thatoo"        │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"vit"           │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"MichelDuchemin"│19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"nay4"          │19           │18                  │94.73684210526316│
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"gerard94"      │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"carotte46"     │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Alfybe"        │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Mententon"     │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Tortue"        │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Paulart"       │19           │18                  │94.73684210526316│
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Gat"           │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"urodelus"      │18           │17                  │94.44444444444444│
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"inso"          │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"cgeek"         │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"moul"          │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Nadou"         │19           │18                  │94.73684210526316│
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Darunya"       │19           │18                  │94.73684210526316│
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"Galuel"        │18           │18                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"matiou"        │19           │19                  │100              │
├────────────────┼─────────────┼────────────────────┼─────────────────┤
│"stanlog"       │18           │18                  │100              │
└────────────────┴─────────────┴────────────────────┴─────────────────┘
2 « J'aime »

There must be an issue somewhere, the number of sentries should be the same for all users: in the table, we can see “nb_sentries” sometimes with value 18, sometimes with value 19.

It’s normal.
When you are sentry yourself, you do not have to calculate if you can reach yourself. :slight_smile:

2 « J'aime »

Indeed, that’s what is done in Duniter itself. You can count 18 :slight_smile:

1 « J'aime »

I updated duniter-neo4j with an API wich give the mean length path to reach sentries for a psecific user.

Here is the request for all members at once :

MATCH (n)
WITH collect(n.uid) as collect_uid
UNWIND collect_uid as uid
MATCH (n {sentry : 1} )
WHERE NOT n.uid = uid
WITH count(n) as count_n, uid
MATCH p=ShortestPath((member {uid:uid}) <-[*]-(r_sentry {sentry : 1}))
RETURN uid, 1.0 * SUM(length§) / count§

╒════════════════╤══════════════════════════════╕
│"uid"           │"1.0 * SUM(length(p)) / count(│
│                │p)"                           │
╞════════════════╪══════════════════════════════╡
│"Galuel"        │1.6666666666666667            │
├────────────────┼──────────────────────────────┤
│"carotte46"     │2.0454545454545454            │
├────────────────┼──────────────────────────────┤
│"Sivmatt"       │2.590909090909091             │
├────────────────┼──────────────────────────────┤
│"DebOrah"       │1.8571428571428572            │
├────────────────┼──────────────────────────────┤
│"moul"          │1.7142857142857142            │
├────────────────┼──────────────────────────────┤
│"pafzedog"      │1.8095238095238095            │
├────────────────┼──────────────────────────────┤
│"FugazziPL"     │2.3636363636363638            │
├────────────────┼──────────────────────────────┤
│"elois"         │1.6666666666666667            │
├────────────────┼──────────────────────────────┤
│"Darunya"       │2.272727272727273             │
├────────────────┼──────────────────────────────┤
│"inso"          │1.5714285714285714            │
├────────────────┼──────────────────────────────┤
│"stanlog"       │1.8571428571428572            │
├────────────────┼──────────────────────────────┤
│"filb"          │2.090909090909091             │
├────────────────┼──────────────────────────────┤
│"MarcRutkowski" │2.1818181818181817            │
├────────────────┼──────────────────────────────┤
│"gnu-tux"       │1.9523809523809523            │
├────────────────┼──────────────────────────────┤
│"cuckooland"    │2                             │
├────────────────┼──────────────────────────────┤
│"b3j0f"         │2.3181818181818183            │
├────────────────┼──────────────────────────────┤
│"Yann-Lug"      │2.238095238095238             │
├────────────────┼──────────────────────────────┤
│"gpsqueeek"     │1.9523809523809523            │
├────────────────┼──────────────────────────────┤
│"jytou"         │2.227272727272727             │
├────────────────┼──────────────────────────────┤
│"matiou"        │2.1363636363636362            │
├────────────────┼──────────────────────────────┤
│"Fiatouxxx"     │1.9545454545454546            │
├────────────────┼──────────────────────────────┤
│"RavanH"        │2.227272727272727             │
├────────────────┼──────────────────────────────┤
│"Nadou"         │2.5                           │
├────────────────┼──────────────────────────────┤
│"Paulart"       │2.380952380952381             │
├────────────────┼──────────────────────────────┤
│"Thatoo"        │1.9047619047619047            │
├────────────────┼──────────────────────────────┤
│"nay4"          │2.1818181818181817            │
├────────────────┼──────────────────────────────┤
│"vincentux"     │1.8095238095238095            │
├────────────────┼──────────────────────────────┤
│"Gat"           │2.1818181818181817            │
├────────────────┼──────────────────────────────┤
│"vit"           │1.9545454545454546            │
├────────────────┼──────────────────────────────┤
│"MichelDuchemin"│2.1363636363636362            │
├────────────────┼──────────────────────────────┤
│"yannlefranco"  │2.3636363636363638            │
├────────────────┼──────────────────────────────┤
│"Caizohan"      │2.0454545454545454            │
├────────────────┼──────────────────────────────┤
│"CandideSK8"    │1.7619047619047619            │
├────────────────┼──────────────────────────────┤
│"Tortue"        │2.090909090909091             │
├────────────────┼──────────────────────────────┤
│"JeanFerreira"  │1.619047619047619             │
├────────────────┼──────────────────────────────┤
│"cgeek"         │1.380952380952381             │
├────────────────┼──────────────────────────────┤
│"Mententon"     │1.8095238095238095            │
├────────────────┼──────────────────────────────┤
│"PierreYves"    │1.8095238095238095            │
├────────────────┼──────────────────────────────┤
│"Alfybe"        │2.272727272727273             │
├────────────────┼──────────────────────────────┤
│"urodelus"      │1.8571428571428572            │
├────────────────┼──────────────────────────────┤
│"gerard94"      │2                             │
├────────────────┼──────────────────────────────┤
│"rimek"         │2.3181818181818183            │
├────────────────┼──────────────────────────────┤
│"kimamila"      │1.7142857142857142            │
└────────────────┴──────────────────────────────┘

If you have ideas about what informations we should extract from the wot, please post. :slight_smile:

3 « J'aime »

“Which groups of 3 users can certify, and so make come in the WoT a newcomer ?”

1 « J'aime »

I think I got it.
I do it by checking all possible combinations of groups (members who issued all their sigs are excluded).

MATCH (a), (b), (c)
WHERE id(a) > id(b) > id(c) AND NOT a.nb_issued_sig = 15 AND NOT b.nb_issued_sig = 15 AND NOT c.nb_issued_sig = 15
RETURN a.uid, b.uid, c.uid

Returned 11480 records in 2464 ms

I found 11480 possible groups.

For each group and for each sentry, I check that the sentry is reachable at stepmax - 1 by at least one of member. If 90% or more of sentries are reachable with this method, I consider that the group can make come a newcomer.

Here is the request :

MATCH (a), (b), (c)
WHERE id(a) > id(b) > id(c) AND NOT a.nb_issued_sig = 15 AND NOT b.nb_issued_sig = 15 AND NOT c.nb_issued_sig = 15

WITH a, b, c
MATCH (sentry {sentry:1})
WHERE sentry <> a AND sentry <> b AND sentry <> c
WITH a, b, c, count(sentry) as count_sentry

OPTIONAL MATCH p1=ShortestPath((a) <-[*..3]- (sentry {sentry:1})), 
p2=ShortestPath((b) <-[*..3]- (sentry {sentry:1})), 
p3=ShortestPath((c) <-[*..3]- (sentry {sentry:1}))
WHERE sentry <> a AND sentry <> b AND sentry <> c

WITH a, b, c, p1, p2, p3, sentry, count_sentry, 3 as stepmax
WHERE length(p1) < stepmax OR length(p2) < stepmax OR length (p3) < stepmax 

WITH a.uid as a_uid, b.uid as b_uid, c.uid as c_uid, stepmax, count(sentry) as reachable_sentries, count_sentry as total_sentries, 100.0 * count(sentry) / count_sentry as percent
WHERE percent > 90
RETURN a_uid, b_uid, c_uid, stepmax, reachable_sentries, total_sentries, percent

Returned 9316 records in 93018 ms, displaying first 1000 rows.

It returns 9316 possible groups. (It took 93s to compute on my small 1 core/512Mo Ram VM).
The complete result is here.

export_groups_of_three.csv (383.0 KB)

Don’t know what to think about the result, but I learned a lot about making request in Neo4j. :slight_smile:

2 « J'aime »

It is nice. Then now it would be usefull to know the 3 ones, when 1 is yet chosen (I’m new, I know 1 member, he is ok to certify me, what are the other 2 possibilies for me to be certifed by, in order to become a member !?).

  • imagine a specialised node with Gephi where you could click on one member, and the good 2 possible others are in green, then you choose the 2nd one, and the 3rd ones appear in green…

Would be very usefull to help the WoT to be very efficient.

5 « J'aime »

I tinkered an API in duniter-neo4j that could help to build such tool.

Let’s say I know a member in the Wot. For example @Moul
I want to check which members at step 1 from Moul are good potential signers.

To know it, I check which percentage of sentries is reachable if @Moul + the potential signer give me a certification.

http://151.80.155.53:10500/neo4j/recommendations/signers/moul/1/none

{
    referring_member: "moul",
    optionnal_second_member: null,
    recommended_member: "CandideSK8",
    percent_sentries: 100
},
{
    referring_member: "moul",
    optionnal_second_member: null,
    recommended_member: "Thatoo",
    percent_sentries: 100
},

I added a “second member” option.
It allows to check which member is recommended in the case, I already have two signers ( @Moul, the referring member, a second member). The recommendation will return me best members at step 1 of Moul.

http://151.80.155.53:10500/neo4j/recommendations/signers/moul/1/vit

This implementation is for now just a proof of concept and would need much more work to be efficient but it gives an idea about how this tool could be built.

4 « J'aime »

great !

@greyzlii Ce serait top que tu puisses nous faire un atelier lors des RML9 au Havre, avec @inso.

Voici les attentes qui pourraient être les miennes :

  • savoir installer les outils de développement pour contribuer à duniter-neo4j
  • savoir requeter et debugger un requete

Qu’en dis-tu ?

1 « J'aime »

Disons que je pourrais vous livrer un retour d’expérience sur l’utilisation d’une base de type graph…
Et donner les clés pour faire les premières requêtes.

1 « J'aime »

Extra ! @Paidge si tu nous écoutes :wink: