Page MenuHomeDevCentral

Fetch last FANTOIR file
ClosedPublic

Authored by dereckson on Jan 15 2023, 05:57.
Tags
None
Referenced Files
Unknown Object (File)
Fri, Dec 27, 23:39
Unknown Object (File)
Fri, Dec 27, 19:38
Unknown Object (File)
Thu, Dec 26, 03:21
Unknown Object (File)
Sat, Dec 21, 05:02
Unknown Object (File)
Fri, Dec 20, 23:27
Unknown Object (File)
Fri, Dec 20, 22:59
Unknown Object (File)
Fri, Dec 20, 17:39
Unknown Object (File)
Fri, Dec 20, 05:52
Subscribers
None

Details

Summary

Query the data.economie.gouv.fr API to get metadata about last FANTOIR file.

Download it if not already existing in the working directory.

Print FANTOIR_FILE=<download and extracted file> on success.

The code download itself the file, but call unzip to handle unzip operation.

Ref T1750.

Test Plan

Downloaded file when and when not existing.

Diff Detail

Repository
rDS Nasqueron Datasources
Lint
No Lint Coverage
Unit
No Test Coverage
Branch
fetch
Build Status
Buildable 4333
Build 4600: arc lint + arc unit

Event Timeline

dereckson created this revision.

Blocked by publication of opendatasoft-explore-api library.

Plan is to switch to a manyrepo, ie a monorepo with script to export to fantoir-datasource, opendatasoft-explore-api, and the future ones.

src/commands/fetch/mod.rs
28

Conflict with unzip exit code

Change and update doc at https://agora.nasqueron.org/Fantoir-datasource

Switch to monorepo, rebase against monorepo state and against D2745

We're blocked by D2745, but this code works fine and download, extract correctly the file.

We instantiate unzip because the ZIP libraries are a real pain, and we can't bring a lot of added value to the tool.

We also need to document unzip is needed in the README for fetch command.

fantoir-datasource/Cargo.toml
14 ↗(On Diff #6965)

How do we plan to actually manage this?

It makes sense for development, but not great for packaging:
cargo install fantoir-datasource should be able to fetch and compile opendatasoft-explore-api

fantoir-datasource/src/commands/fetch/fantoir_file.rs
107 ↗(On Diff #6965)

Let's add the protocol, so nobody will be confused "where is this file for the test".

Detect unzip, fix exit codes to avoid most conflicts with unzip

Use v0.1.0 of opendatasoft-explore-api, published on crates.io

This revision is now accepted and ready to land.Jan 18 2023, 19:12
This revision was automatically updated to reflect the committed changes.