Page MenuHomeDevCentral

Fetch last FANTOIR file
ClosedPublic

Authored by dereckson on Jan 15 2023, 05:57.
Tags
None
Referenced Files
F2745126: D2738.diff
Thu, Mar 28, 21:40
F2744642: D2738.id6951.diff
Thu, Mar 28, 18:48
F2743937: D2738.id6965.diff
Thu, Mar 28, 12:32
Unknown Object (File)
Sun, Mar 24, 00:48
Unknown Object (File)
Sat, Mar 23, 14:43
Unknown Object (File)
Thu, Mar 21, 21:22
Unknown Object (File)
Thu, Mar 21, 15:18
Unknown Object (File)
Thu, Mar 21, 10:08
Subscribers
None

Details

Summary

Query the data.economie.gouv.fr API to get metadata about last FANTOIR file.

Download it if not already existing in the working directory.

Print FANTOIR_FILE=<download and extracted file> on success.

The code download itself the file, but call unzip to handle unzip operation.

Ref T1750.

Test Plan

Downloaded file when and when not existing.

Diff Detail

Repository
rDS Nasqueron Datasources
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

dereckson created this revision.

Blocked by publication of opendatasoft-explore-api library.

Plan is to switch to a manyrepo, ie a monorepo with script to export to fantoir-datasource, opendatasoft-explore-api, and the future ones.

src/commands/fetch/mod.rs
28 ↗(On Diff #6951)

Conflict with unzip exit code

Change and update doc at https://agora.nasqueron.org/Fantoir-datasource

Switch to monorepo, rebase against monorepo state and against D2745

We're blocked by D2745, but this code works fine and download, extract correctly the file.

We instantiate unzip because the ZIP libraries are a real pain, and we can't bring a lot of added value to the tool.

We also need to document unzip is needed in the README for fetch command.

fantoir-datasource/Cargo.toml
14

How do we plan to actually manage this?

It makes sense for development, but not great for packaging:
cargo install fantoir-datasource should be able to fetch and compile opendatasoft-explore-api

fantoir-datasource/src/commands/fetch/fantoir_file.rs
108

Let's add the protocol, so nobody will be confused "where is this file for the test".

Detect unzip, fix exit codes to avoid most conflicts with unzip

Use v0.1.0 of opendatasoft-explore-api, published on crates.io

This revision is now accepted and ready to land.Jan 18 2023, 19:12
This revision was automatically updated to reflect the committed changes.