Page MenuHomeDevCentral

Parse FANTOIR database
Needs ReviewPublic

Authored by dereckson on Jun 9 2021, 23:25.
Tags
None
Referenced Files
F12376073: D2379.id5984.diff
Fri, Oct 31, 01:13
F12376070: D2379.id5983.diff
Fri, Oct 31, 01:13
F12373781: D2379.diff
Thu, Oct 30, 18:32
Unknown Object (File)
Sat, Oct 25, 18:35
Unknown Object (File)
Sat, Oct 25, 07:30
Unknown Object (File)
Sat, Oct 25, 07:30
Unknown Object (File)
Sat, Oct 25, 07:29
Unknown Object (File)
Fri, Oct 24, 14:47
Subscribers
None

Details

Reviewers
dereckson
Summary

The FANTOIR database is provided in a fixed width record format.

This change transforms it into a JSON document, ready to be parsed
by any tool for database import.

Test Plan

make clean all

Diff Detail

Repository
rDBVOIES Nasqueron Databases :: Datasources :: Voies
Lint
Lint Passed
Unit
No Test Coverage
Branch
parse-fantoir
Build Status
Buildable 3692
Build 3942: arc lint + arc unit

Event Timeline

dereckson created this revision.

Tweak Git ignored files and Makefile

https://about.staging.gitlab.com/blog/2016/03/18/fast-search-using-postgresql-trigram-indexes/ makes me think a Postgresql table with with those GIN trigram indexes would be a better idea than json documents 🤔

Draft implementation: rFANTDB59128e4bfd41 (needs to add all FANTOIR field).

Plan is to build a DAG "check nouvelle version > télécharger db > unzip > extraire version et lancer fantoir2db > modifier config > notifier qu'on a une nouvelle version déployée"