Produits Entr'ouvert » Passerelle

Invisible dans le patch, la modification du update_streets_test.gz : mise au norme de l'id (selon ce qui existe en vrai désormais dans les fichiers BAN), ajout de numéro de rue non-numérique ("bis"), bref de quoi couvrir le code du patch.

--- update_streets_test.orig    2021-08-07 02:19:28.965926649 +0200
+++ update_streets_test    2021-08-07 02:19:21.213939183 +0200
@@ -1,7 +1,7 @@
-{"id":"73001_0004_45e80d","type":"street","name":"Chemin de la Vie, LA GRANGE DU TRIEU","postcode":["73610"],"citycode":["73001", 73123],"lon":5.818476,"lat": 45.539846,"city":["Aiguebelette-le-Lac"],"context":"73, Savoie, Rhône-Alpes","importance":0.0019,"housenumbers":{"5180":{"lat":45.539555,"lon":5.818317,"id":"ADRNIVX_0000000279029403"},"5218 F":{"lat":45.540136,"lon":5.818635,"id":"ADRNIVX_0000000279029404"}}}
-{"id":"73001_0004B060_f16e6d","type":"street","name":"Chemin de la Vie, LE PILLIOT","postcode":"73610","citycode":"73001","lon":5.818343,"lat": 45.539391,"city":["Aiguebelette-le-Lac"],"context":"73, Savoie, Rhône-Alpes","importance":0.0028,"housenumbers":{"5079":{"lat":45.538942,"lon":5.818094,"id":"ADRNIVX_0000000279029397"},"5080":{"lat":45.539366,"lon":5.818327,"id":"ADRNIVX_0000000279029398"},"5081":{"lat":45.539887,"lon":5.818616,"id":"ADRNIVX_0000000279029399"},"5167":{"lat":45.539137,"lon":5.818198,"id":"ADRNIVX_0000000279029400"},"5195":{"lat":45.539622,"lon":5.818481,"id":"ADRNIVX_0000000279029402"}}}
-{"id":"73001_0004_d2ea0c","type":"street","name":"Chemin de la Vie","postcode":["73610"],"citycode":"73001","lon":5.817947,"lat": 45.538726,"city":["Aiguebelette-le-Lac"],"context":"73, Savoie, Rhône-Alpes","importance":0.0019,"housenumbers":{"5174":{"lat":45.538281,"lon":5.817800,"id":"ADRNIVX_0000000279029409"},"5234":{"lat":45.539170,"lon":5.818095,"id":"ADRNIVX_0000000326665769"}}}
-{"id":"73001_A015_c95d7e","type":"locality","name":"LES CHALETS DU LAC","postcode":"73610","citycode":"73001","lon":5.817341,"lat": 45.539502,"city":"Aiguebelette-le-Lac","context":"73, Savoie, Rhône-Alpes","importance":0.0013,"housenumbers":{"5179":{"lat":45.539502,"lon":5.817341,"id":"ADRNIVX_0000000326665772"}}}
-{"id":"91001_B008_4b7615","type":"street","name":["La Cave"] ,"postcode":"91150","citycode": ["91001"],"lon":2.174828,"lat": 48.340897,"x":638856.0,"y":6804828.7,"city":["Abbéville-la-Rivière"],"context":"91, Essonne, Île-de-France","importance":0.0019,"housenumbers":{"5050":{"lat":48.340897,"lon":2.174828,"id":"ADRNIVX_0000000271599105","x":638856,"y":6804828.7}}}
-{"id":"97401_B008_674a69","type":"street","name":["Le Brule"] ,"postcode":["97425"],"citycode": ["97401"],"lon":55.338655,"lat": -21.250773,"x":327610.5,"y":7649192.2,"city":"Les Avirons","context":"974, La Réunion","importance":0.0425,"housenumbers":{"32":{"lat":-21.250773,"lon":55.338655,"id":"ADRNIVX_0000001402354816","x":327610.5,"y":7649192.2}}}
-{"id":"97401_B022_128951","type":"street","name":["Ravine Seche"] ,"postcode":"97425","citycode": ["97401"],"lon":55.350468,"lat": -21.242886,"x":328827.4,"y":7650078.2,"city":["Les Avirons"],"context":"974, La Réunion","importance":0.0425,"housenumbers":{"11 B":{"lat":-21.242886,"lon":55.350468,"id":"ADRNIVX_0000001402620432","x":328827.4,"y":7650078.2}}}
+{"id":"73001_0004","type":"street","name":"Chemin de la Vie, LA GRANGE DU TRIEU","postcode":["73610"],"citycode":["73001", 73123],"lon":5.818476,"lat": 45.539846,"city":["Aiguebelette-le-Lac"],"context":"73, Savoie, Rhône-Alpes","importance":0.0019,"housenumbers":{"5180":{"lat":45.539555,"lon":5.818317,"id":"ADRNIVX_0000000279029403"},"5218 F":{"lat":45.540136,"lon":5.818635,"id":"ADRNIVX_0000000279029404"}}}
+{"id":"73001_0005","type":"street","name":"Chemin de la Vie, LE PILLIOT","postcode":"73610","citycode":"73001","lon":5.818343,"lat": 45.539391,"city":["Aiguebelette-le-Lac"],"context":"73, Savoie, Rhône-Alpes","importance":0.0028,"housenumbers":{"5079":{"lat":45.538942,"lon":5.818094,"id":"ADRNIVX_0000000279029397"},"5080":{"lat":45.539366,"lon":5.818327,"id":"ADRNIVX_0000000279029398"},"5081":{"lat":45.539887,"lon":5.818616,"id":"ADRNIVX_0000000279029399"},"5167":{"lat":45.539137,"lon":5.818198,"id":"ADRNIVX_0000000279029400"},"5195":{"lat":45.539622,"lon":5.818481,"id":"ADRNIVX_0000000279029402"}}}
+{"id":"73001_0006","type":"street","name":"Chemin de la Vie","postcode":["73610"],"citycode":"73001","lon":5.817947,"lat": 45.538726,"city":["Aiguebelette-le-Lac"],"context":"73, Savoie, Rhône-Alpes","importance":0.0019,"housenumbers":{"5174":{"lat":45.538281,"lon":5.817800,"id":"ADRNIVX_0000000279029409"},"5234":{"lat":45.539170,"lon":5.818095,"id":"ADRNIVX_0000000326665769"},"5234bis":{"lat":45.539170,"lon":5.818095}}}
+{"id":"73001_A015","type":"locality","name":"LES CHALETS DU LAC","postcode":"73610","citycode":"73001","lon":5.817341,"lat": 45.539502,"city":"Aiguebelette-le-Lac","context":"73, Savoie, Rhône-Alpes","importance":0.0013,"housenumbers":{"5179":{"lat":45.539502,"lon":5.817341,"id":"ADRNIVX_0000000326665772"}}}
+{"id":"91001_B008","type":"street","name":["La Cave"] ,"postcode":"91150","citycode": ["91001"],"lon":2.174828,"lat": 48.340897,"x":638856.0,"y":6804828.7,"city":["Abbéville-la-Rivière"],"context":"91, Essonne, Île-de-France","importance":0.0019,"housenumbers":{"5050":{"lat":48.340897,"lon":2.174828,"id":"ADRNIVX_0000000271599105","x":638856,"y":6804828.7}}}
+{"id":"97401_B009","type":"street","name":["Le Brule"] ,"postcode":["97425"],"citycode": ["97401"],"lon":55.338655,"lat": -21.250773,"x":327610.5,"y":7649192.2,"city":"Les Avirons","context":"974, La Réunion","importance":0.0425,"housenumbers":{"32":{"lat":-21.250773,"lon":55.338655,"id":"ADRNIVX_0000001402354816","x":327610.5,"y":7649192.2}}}
+{"id":"97401_B022","type":"street","name":["Ravine Seche"] ,"postcode":"97425","citycode": ["97401"],"lon":55.350468,"lat": -21.242886,"x":328827.4,"y":7650078.2,"city":["Les Avirons"],"context":"974, La Réunion","importance":0.0425,"housenumbers":{"11 B":{"lat":-21.242886,"lon":55.350468,"id":"ADRNIVX_0000001402620432","x":328827.4,"y":7650078.2}}}

Auto-relecture, je me prends la tête pour rien sur min et max, je vais retirer ça pour l'instant, ça ne sert à rien.

(S'il fallait revoir la chose, plutôt de stocker explicitement tous les numéros possibles d'une rue dans un ArrayField... mais je n'en devine pas encore pas l'usage)

Voici une version sans ce min/max qui ne sert à rien.

J'ajoute cependant un contrôle du code INSEE (citycode, code de la ville qui contient la rue). En effet certains JSON des fichiers BAN proposent des rues qui ont plusieurs codes INSEE dans la clé "citycode". Dans ce cas, on ne retient que le citycode qui est le début du code BAN (qui a le format "code INSEE + code rue"). Et on en profite pour ne pas retenir les rues qui pourraient avoir une incohérence à ce niveau. (Tout ça parce que le format des fichiers BAN a évolué et semble aujourd'hui mieux cadré : profitons-en).

C'est vraiment possible que ban_id ne soit pas défini ? Parce que là pas de ban_id, alors pas de citycode défini et boom (ou alors on aura le citycode de l'itération d'avant), je dirai que pas de ban_id on saute la ligne ou alors on décide que c'est impossible et le fichier est mal formé.

Benjamin Dauvergne a écrit :

C'est vraiment possible que ban_id ne soit pas défini ? Parce que là pas de ban_id, alors pas de citycode défini et boom (ou alors on aura le citycode de l'itération d'avant), je dirai que pas de ban_id on saute la ligne ou alors on décide que c'est impossible et le fichier est mal formé.

Tu as raison, voici une nouvelle version où on ne charge plus les lignes où ban_id n'est pas de la forme «citycode_quelquechose».

On permet cependant toujours des requêtes sur id=<id-en-base> pour les éventuelles requêtes sur des id enregistrés avant cette modif, ou avant le rechargement d'une nouvelle version des fichiers BAN (c'est un cron, donc le code doit fonctionner même si la table de rue n'a pas encore de valeurs dans la colonne ban_id).

Go.

commit 28576589a82ddc37d5a4c52e137339330c84e1d1
Author: Thomas NOËL <tnoel@entrouvert.com>
Date:   Fri Aug 6 21:13:09 2021 +0200

    base_adresse: store BAN identifier in streets (#56035)