Page MenuHomeDevCentral

Deploy Airflow to Dwellers
ClosedPublic

Authored by dereckson on Mar 29 2023, 19:58.
Tags
None
Referenced Files
F3748265: D2943.diff
Sat, Nov 16, 07:41
F3747122: D2943.id7493.diff
Sat, Nov 16, 00:41
F3746821: D2943.id7494.diff
Fri, Nov 15, 23:20
F3746792: D2943.id7501.diff
Fri, Nov 15, 23:14
F3746716: D2943.id7493.diff
Fri, Nov 15, 22:43
F3746458: D2943.id7493.diff
Fri, Nov 15, 21:24
F3746456: D2943.id7502.diff
Fri, Nov 15, 21:24
F3746314: D2943.id7502.diff
Fri, Nov 15, 20:10
Subscribers
None

Details

Summary

D2754 offers to run fantoir-datasource as a Airflow pipeline.

To do so, we need an Airflow instance, which means:

  • PostgreSQL database on db-A cluster
  • Redis and application containers on the Docker PaaS

Like Sentry, Airflow service is deployed by realm, a realm being
a network and a set of containers for that service, sharing resources
like credentials.

Plan is to deploy it to Dwellers for evaluation purpose,
then move it to docker-002 if we adopt it in production.

Ref T1750

Test Plan

Deploy on Dwellers

Diff Detail

Repository
rOPS Nasqueron Operations
Lint
Lint Passed
Unit
No Test Coverage
Branch
airflow
Build Status
Buildable 4642
Build 4917: arc lint + arc unit

Event Timeline

dereckson created this revision.
dereckson retitled this revision from Deploy Airflow to Deploy Airflow to Dwellers.Mar 29 2023, 21:12
dereckson edited the summary of this revision. (Show Details)

Airflow runs well.

Two changes needed:

  • switch IP to 127.0.0.1
  • nginx (and a DNS record so)

Bind app ports to 127.0.0.1

This revision is now accepted and ready to land.Mar 30 2023, 01:37
This revision was automatically updated to reflect the committed changes.