HomeDevCentral

Deploy Airflow to Dwellers

Description

Deploy Airflow to Dwellers

Summary:
D2754 offers to run fantoir-datasource as a Airflow pipeline.

To do so, we need an Airflow instance, which means:

  • PostgreSQL database on db-A cluster
  • Redis and application containers on the Docker PaaS

Like Sentry, Airflow service is deployed by realm, a realm being
a network and a set of containers for that service, sharing resources
like credentials.

Plan is to deploy it to Dwellers for evaluation purpose,
then move it to docker-002 if we adopt it in production.

Ref T1750

Test Plan: Deploy on Dwellers

Reviewers: dereckson

Reviewed By: dereckson

Maniphest Tasks: T1750

Differential Revision: https://devcentral.nasqueron.org/D2943

Details

Provenance
derecksonAuthored on Mar 29 2023, 19:51
derecksonPushed on Mar 30 2023, 01:37
Reviewer
dereckson
Differential Revision
D2943: Deploy Airflow to Dwellers
Parents
rOPSdf873afbe1d2: Trace every request for Notifications Center to Sentry
Branches
Unknown
Tags
Unknown
Tasks
T1812: Deploy Airflow
T1750: Import FANTOIR database