Details

Reviewers

dereckson

Maniphest Tasks

T1750: Import FANTOIR database

Commits

rDSfbabea18bd71: Fetch last FANTOIR file

Summary

Query the data.economie.gouv.fr API to get metadata about last FANTOIR file.

Download it if not already existing in the working directory.

Print FANTOIR_FILE=<download and extracted file> on success.

The code download itself the file, but call unzip to handle unzip operation.

Ref T1750.

Test Plan

Downloaded file when and when not existing.

Diff Detail

Repository

rDS Nasqueron Datasources

Lint

No Lint Coverage

Unit

No Test Coverage

Branch

fetch

Build Status

Buildable 4342
Build 4609: arc lint + arc unit

Event Timeline

dereckson requested review of this revision.Jan 15 2023, 05:57

dereckson created this revision.

Harbormaster completed remote builds in B4333: Diff 6951.Jan 15 2023, 05:57

Blocked by publication of opendatasoft-explore-api library.

Plan is to switch to a manyrepo, ie a monorepo with script to export to fantoir-datasource, opendatasoft-explore-api, and the future ones.

dereckson added inline comments.Jan 15 2023, 06:12

src/commands/fetch/mod.rs
28 ↗	(On Diff #6951)	Conflict with unzip exit code Change and update doc at https://agora.nasqueron.org/Fantoir-datasource

dereckson mentioned this in D2745: Consume Opendatasoft Explore API.Jan 17 2023, 21:03

Switch to monorepo, rebase against monorepo state and against D2745

Harbormaster completed remote builds in B4341: Diff 6965.Jan 17 2023, 21:06

dereckson added a parent revision: D2745: Consume Opendatasoft Explore API.Jan 17 2023, 21:06

We're blocked by D2745, but this code works fine and download, extract correctly the file.

We instantiate unzip because the ZIP libraries are a real pain, and we can't bring a lot of added value to the tool.

We also need to document unzip is needed in the README for fetch command.

fantoir-datasource/Cargo.toml
14	How do we plan to actually manage this? It makes sense for development, but not great for packaging: cargo install fantoir-datasource should be able to fetch and compile opendatasoft-explore-api
fantoir-datasource/src/commands/fetch/fantoir_file.rs
108	Let's add the protocol, so nobody will be confused "where is this file for the test".

Detect unzip, fix exit codes to avoid most conflicts with unzip

Harbormaster completed remote builds in B4342: Diff 6966.Jan 18 2023, 02:13

dereckson mentioned this in rDS159b1bdd835c: Consume Opendatasoft Explore API.Jan 18 2023, 06:40

Use v0.1.0 of opendatasoft-explore-api, published on crates.io

Harbormaster completed remote builds in B4351: Diff 6979.Jan 18 2023, 07:25

dereckson accepted this revision.Jan 18 2023, 19:12

This revision is now accepted and ready to land.Jan 18 2023, 19:12

Closed by commit rDSfbabea18bd71: Fetch last FANTOIR file (authored by dereckson). · Explain WhyJan 18 2023, 19:13

This revision was automatically updated to reflect the committed changes.

dereckson added a commit: rDSfbabea18bd71: Fetch last FANTOIR file.

Fetch last FANTOIR file
ClosedPublic
Actions

Details

Diff Detail

Event Timeline

Revision Contents
Changeset List

Diff 6966

.gitignore

fantoir-datasource/Cargo.toml

fantoir-datasource/README.md

fantoir-datasource/src/commands/fetch/fantoir_file.rs

fantoir-datasource/src/commands/fetch/mod.rs

fantoir-datasource/src/commands/fetch/os.rs

fantoir-datasource/src/commands/mod.rs

fantoir-datasource/src/fantoir.rs

fantoir-datasource/src/main.rs

fantoir-datasource/src/services/http_client.rs

Fetch last FANTOIR fileClosedPublicActions

Details

Diff Detail

Event Timeline

Revision ContentsChangeset List

Diff 6966

.gitignore

fantoir-datasource/Cargo.toml

fantoir-datasource/README.md

fantoir-datasource/src/commands/fetch/fantoir_file.rs

fantoir-datasource/src/commands/fetch/mod.rs

fantoir-datasource/src/commands/fetch/os.rs

fantoir-datasource/src/commands/mod.rs

fantoir-datasource/src/fantoir.rs

fantoir-datasource/src/main.rs

fantoir-datasource/src/services/http_client.rs

Fetch last FANTOIR file
ClosedPublic
Actions

Revision Contents
Changeset List