Page MenuHomeDevCentral
Feed All Stories

Mar 12 2023

dereckson updated the diff for D2881: Deploy Snuba containers for Sentry.

Reconsolidate snuba and snuba_api: that will be less confusing to maintain if snuba is only snuba.

Mar 12 2023, 10:30
dereckson updated the summary of D2881: Deploy Snuba containers for Sentry.
Mar 12 2023, 10:21
dereckson accepted D2881: Deploy Snuba containers for Sentry.

All containers are up:

Mar 12 2023, 10:19
dereckson updated the diff for D2881: Deploy Snuba containers for Sentry.

Use full snuba command with docker exec

Mar 12 2023, 10:06
dereckson updated the diff for D2881: Deploy Snuba containers for Sentry.

Nightly image

Mar 12 2023, 10:01
dereckson planned changes to D2881: Deploy Snuba containers for Sentry.

Not sure the network parameter works, as container _consumer wasn't connected:

Mar 12 2023, 02:08
dereckson added a comment to D2881: Deploy Snuba containers for Sentry.
docker_002
$ snuba_containers="sentry_snuba_api sentry_snuba_consumer sentry_snuba_outcomes_consumer sentry_snuba_sessions_consumer sentry_snuba_transactions_consumer sentry_snuba_replays_consumer sentry_snuba_replacer sentry_snuba_subscription_consumer_events sentry_snuba_subscription_consumer_sessions sentry_snuba_subscription_consumer_transactions"
$ docker stats $snuba_containers
CONTAINER ID   NAME                                              CPU %     MEM USAGE / LIMIT     MEM %     NET I/O           BLOCK I/O   PIDS
d7fb9bf79329   sentry_snuba_api                                  0.00%     0B / 0B               0.00%     0B / 0B           0B / 0B     0
0ca62382910e   sentry_snuba_consumer                             0.45%     92.93MiB / 31.09GiB   0.29%     21.1kB / 10.9kB   0B / 0B     11
b01ba3b46457   sentry_snuba_outcomes_consumer                    0.10%     92.27MiB / 31.09GiB   0.29%     8.08kB / 3.76kB   0B / 0B     5
30fcc4b9374d   sentry_snuba_sessions_consumer                    0.14%     92.26MiB / 31.09GiB   0.29%     10.2kB / 5.66kB   0B / 0B     5
fdabe0103fcf   sentry_snuba_transactions_consumer                0.18%     92.23MiB / 31.09GiB   0.29%     10.2kB / 5.14kB   0B / 0B     8
d67d7260ab3e   sentry_snuba_replays_consumer                     0.11%     92.5MiB / 31.09GiB    0.29%     7.46kB / 3.4kB    0B / 0B     5
7b5920f801d8   sentry_snuba_replacer                             0.10%     92.45MiB / 31.09GiB   0.29%     7.75kB / 3.56kB   0B / 0B     5
8a16768ac8a2   sentry_snuba_subscription_consumer_events         0.28%     93.58MiB / 31.09GiB   0.29%     11.7kB / 5.81kB   0B / 0B     9
0f043aee7ce5   sentry_snuba_subscription_consumer_sessions       0.00%     0B / 0B               0.00%     0B / 0B           0B / 0B     0
1686aea3ef0c   sentry_snuba_subscription_consumer_transactions   0.24%     93.42MiB / 31.09GiB   0.29%     10.3kB / 5.14kB   0B / 0B     9
Mar 12 2023, 01:53
dereckson updated the diff for D2881: Deploy Snuba containers for Sentry.

Remove port bindings

Mar 12 2023, 01:46
dereckson requested review of D2881: Deploy Snuba containers for Sentry.
Mar 12 2023, 01:42
dereckson added a revision to T1791: Refresh Sentry installation: D2881: Deploy Snuba containers for Sentry.
Mar 12 2023, 01:42 · Monitoring and reporting

Mar 11 2023

dereckson closed D2880: Bump pre-commit hook for salt-lint.
Mar 11 2023, 14:15
dereckson committed rOPS8c835c146bce: Bump pre-commit hook for salt-lint (authored by dereckson).
Bump pre-commit hook for salt-lint
Mar 11 2023, 14:15
dereckson accepted D2880: Bump pre-commit hook for salt-lint.
Mar 11 2023, 14:15
dereckson requested review of D2880: Bump pre-commit hook for salt-lint.
Mar 11 2023, 14:15
dereckson closed D2879: Refresh repository structure description.
Mar 11 2023, 14:12
dereckson committed rOPSea531c87ed64: Refresh repository structure description (authored by dereckson).
Refresh repository structure description
Mar 11 2023, 14:12
dereckson accepted D2879: Refresh repository structure description.
Mar 11 2023, 14:12
dereckson requested review of D2879: Refresh repository structure description.
Mar 11 2023, 14:11
dereckson committed rOPS578995933116: Switch references from Freenode to Libera (authored by dereckson).
Switch references from Freenode to Libera
Mar 11 2023, 14:11
dereckson closed D2878: Switch references from Freenode to Libera.
Mar 11 2023, 14:11
dereckson accepted D2878: Switch references from Freenode to Libera.
Mar 11 2023, 14:11
dereckson requested review of D2878: Switch references from Freenode to Libera.
Mar 11 2023, 14:10
dereckson added a revision to T1701: Switch references from Freenode to Libera: D2878: Switch references from Freenode to Libera.
Mar 11 2023, 14:10 · IRC
dereckson added a task to D2479: Update IRC channel to Libera network: T1701: Switch references from Freenode to Libera.
Mar 11 2023, 14:09
dereckson added a revision to T1701: Switch references from Freenode to Libera: D2479: Update IRC channel to Libera network.
Mar 11 2023, 14:09 · IRC
dereckson committed rOPS9f96f0ba42e2: Move scripts to relevant units (authored by dereckson).
Move scripts to relevant units
Mar 11 2023, 13:45
dereckson closed D2877: Move scripts to relevant units.
Mar 11 2023, 13:45
dereckson accepted D2877: Move scripts to relevant units.
Mar 11 2023, 13:45
dereckson updated the summary of D2877: Move scripts to relevant units.
Mar 11 2023, 13:45
dereckson requested review of D2877: Move scripts to relevant units.
Mar 11 2023, 13:44
dereckson closed D2876: Prune config legacy content.
Mar 11 2023, 13:27
dereckson committed rOPSc804d0b20334: Prune config legacy content (authored by dereckson).
Prune config legacy content
Mar 11 2023, 13:27
dereckson accepted D2876: Prune config legacy content.
Mar 11 2023, 13:27
dereckson requested review of D2876: Prune config legacy content.
Mar 11 2023, 13:27
dereckson added a revision to T1794: Provision Discourse through Docker PaaS role: D2876: Prune config legacy content.
Mar 11 2023, 13:27 · Docker images, Forum
dereckson triaged T1794: Provision Discourse through Docker PaaS role as Normal priority.
Mar 11 2023, 13:26 · Docker images, Forum
dereckson created P313 config/forum/forum.yml.tmpl.
Mar 11 2023, 13:22
dereckson added a comment to D2875: Prune geoipupdate container after use.
docker-002
$ geoipupdate
[…]
# STATE: Creating configuration file at /etc/GeoIP.conf
# STATE: Running geoipupdate
$ docker ps -a | grep geoipupdate
[nothing]
Mar 11 2023, 01:09
dereckson closed D2875: Prune geoipupdate container after use.
Mar 11 2023, 00:48
dereckson committed rOPS0751ed08f600: Prune geoipupdate container after use (authored by dereckson).
Prune geoipupdate container after use
Mar 11 2023, 00:48
dereckson accepted D2875: Prune geoipupdate container after use.
Mar 11 2023, 00:48
dereckson requested review of D2875: Prune geoipupdate container after use.
Mar 11 2023, 00:48
dereckson added a revision to T1791: Refresh Sentry installation: D2875: Prune geoipupdate container after use.
Mar 11 2023, 00:48 · Monitoring and reporting
dereckson closed T1793: Renew eglide.org domain as Resolved.

Renewed by @Sandlayth. DNS resolution works again.

Mar 11 2023, 00:10 · Eglide

Mar 10 2023

dereckson triaged T1793: Renew eglide.org domain as High priority.
Mar 10 2023, 20:50 · Eglide
dereckson closed D2874: Fetch GeoIP databases.
Mar 10 2023, 20:11
dereckson committed rOPSe542d69a0933: Fetch GeoIP databases (authored by dereckson).
Fetch GeoIP databases
Mar 10 2023, 20:11
dereckson accepted D2874: Fetch GeoIP databases.
Mar 10 2023, 20:10
dereckson added a comment to D2874: Fetch GeoIP databases.

Wrappers part is omnipotent:

Mar 10 2023, 20:07
dereckson requested review of D2874: Fetch GeoIP databases.
Mar 10 2023, 20:06
dereckson added a revision to T1791: Refresh Sentry installation: D2874: Fetch GeoIP databases.
Mar 10 2023, 20:06 · Monitoring and reporting
dereckson closed D2873: Deploy ClickHouse container for Sentry.
Mar 10 2023, 01:36
dereckson committed rOPSaa82d0a5723c: Deploy ClickHouse container for Sentry (authored by dereckson).
Deploy ClickHouse container for Sentry
Mar 10 2023, 01:36
dereckson accepted D2873: Deploy ClickHouse container for Sentry.
Mar 10 2023, 01:35
dereckson updated the diff for D2873: Deploy ClickHouse container for Sentry.

Linux capabilities (not required by Snuba, but nice to have for ClickHouse features)

Mar 10 2023, 01:30
dereckson added a comment to D2873: Deploy ClickHouse container for Sentry.

2023.03.10 01:21:37.512800 [ 1 ] {} <Information> Application: It looks like the process has no CAP_NET_ADMIN capability, 'taskstats' performance statistics will be disabled. It could happen due to incorrect ClickHouse package installation. You could resolve the problem manually with 'sudo setcap cap_net_admin=+ep /usr/bin/clickhouse'. Note that it will not work on 'nosuid' mounted filesystems. It also doesn't work if you run clickhouse-server inside network namespace as it happens in some containers.

Mar 10 2023, 01:24
dereckson added a revision to T1791: Refresh Sentry installation: D2873: Deploy ClickHouse container for Sentry.
Mar 10 2023, 01:19 · Monitoring and reporting
dereckson requested review of D2873: Deploy ClickHouse container for Sentry.
Mar 10 2023, 01:19
dereckson added a comment to T1791: Refresh Sentry installation.

Current status

Mar 10 2023, 00:01 · Monitoring and reporting

Mar 9 2023

dereckson added a comment to D2872: Fix ZooKeeper data volume path.
HostConfig:
    ----------
    Binds:
        ----------
        new:
            - /srv/zookeeper/sentry_zookeeper/data:/var/lib/zookeeper/data
            - /srv/zookeeper/sentry_zookeeper/log:/var/lib/zookeeper/log
            - /srv/zookeeper/sentry_zookeeper/secrets:/etc/zookeeper/secrets
        old:
            - /srv/zookeeper/sentry_zookeeper/data:/var/lib/zookeper/data
            - /srv/zookeeper/sentry_zookeeper/log:/var/lib/zookeeper/log
            - /srv/zookeeper/sentry_zookeeper/secrets:/etc/zookeeper/secrets
Mar 9 2023, 23:45
dereckson closed D2872: Fix ZooKeeper data volume path.
Mar 9 2023, 23:43
dereckson committed rOPS93d77ac8df79: Fix ZooKeeper data volume path (authored by dereckson).
Fix ZooKeeper data volume path
Mar 9 2023, 23:43
dereckson accepted D2872: Fix ZooKeeper data volume path.
Mar 9 2023, 23:43
dereckson requested review of D2872: Fix ZooKeeper data volume path.
Mar 9 2023, 23:43
dereckson added a revision to T1791: Refresh Sentry installation: D2872: Fix ZooKeeper data volume path.
Mar 9 2023, 23:43 · Monitoring and reporting
dereckson committed rOPS0ce2c4dad220: Provide Kafka and ZooKeeper instances for Sentry (authored by dereckson).
Provide Kafka and ZooKeeper instances for Sentry
Mar 9 2023, 23:19
dereckson closed D2871: Provide Kafka and ZooKeeper instances for Sentry.
Mar 9 2023, 23:19
dereckson accepted D2871: Provide Kafka and ZooKeeper instances for Sentry.

Kafka and zookeeper are up and healthy.

Mar 9 2023, 23:18
dereckson requested review of D2871: Provide Kafka and ZooKeeper instances for Sentry.
Mar 9 2023, 23:14
dereckson added a revision to T1791: Refresh Sentry installation: D2871: Provide Kafka and ZooKeeper instances for Sentry.
Mar 9 2023, 23:14 · Monitoring and reporting
dereckson updated subscribers of T1768: Simplify paas-docker pillar with PillarStack.

We've adopted Salt Tower as a solution.

Mar 9 2023, 23:01 · Salt
dereckson closed D2862: Adopt Salt Tower to merge pillar files.
Mar 9 2023, 20:48
dereckson closed T1768: Simplify paas-docker pillar with PillarStack as Resolved by committing rOPSbbc48f9eb26d: Adopt Salt Tower to merge pillar files.
Mar 9 2023, 20:48 · Salt
dereckson committed rOPSbbc48f9eb26d: Adopt Salt Tower to merge pillar files (authored by dereckson).
Adopt Salt Tower to merge pillar files
Mar 9 2023, 20:48
dereckson accepted D2862: Adopt Salt Tower to merge pillar files.
Mar 9 2023, 20:48
dereckson updated the diff for D2862: Adopt Salt Tower to merge pillar files.

Drop aliases from old main file

Mar 9 2023, 20:46
dereckson removed a revision from T1768: Simplify paas-docker pillar with PillarStack: D2870: Adopt Salt Tower to merge pillar files.
Mar 9 2023, 20:46 · Salt
dereckson removed a task from D2870: Adopt Salt Tower to merge pillar files: T1768: Simplify paas-docker pillar with PillarStack.
Mar 9 2023, 20:46
dereckson abandoned D2870: Adopt Salt Tower to merge pillar files.

Duplicate of D2862.

Mar 9 2023, 20:46
dereckson updated the diff for D2870: Adopt Salt Tower to merge pillar files.

Drop aliases from old main file

Mar 9 2023, 20:45
dereckson updated the summary of D2862: Adopt Salt Tower to merge pillar files.
Mar 9 2023, 20:44
dereckson requested review of D2870: Adopt Salt Tower to merge pillar files.
Mar 9 2023, 20:43
dereckson added a revision to T1768: Simplify paas-docker pillar with PillarStack: D2870: Adopt Salt Tower to merge pillar files.
Mar 9 2023, 20:43 · Salt
dereckson updated the diff for D2862: Adopt Salt Tower to merge pillar files.

Dedup docker_images. Tweaks.

Mar 9 2023, 20:35
dereckson closed D2869: Deploy Salt Tower on salt-primary role.
Mar 9 2023, 19:46
dereckson committed rOPSad12ca153247: Deploy Salt Tower on salt-primary role (authored by dereckson).
Deploy Salt Tower on salt-primary role
Mar 9 2023, 19:46
dereckson accepted D2869: Deploy Salt Tower on salt-primary role.
Mar 9 2023, 19:46
dereckson updated the diff for D2869: Deploy Salt Tower on salt-primary role.

Update Salt config with correct path + header

Mar 9 2023, 19:45
dereckson added a revision to T1768: Simplify paas-docker pillar with PillarStack: D2869: Deploy Salt Tower on salt-primary role.
Mar 9 2023, 19:41 · Salt
dereckson updated the summary of D2869: Deploy Salt Tower on salt-primary role.
Mar 9 2023, 19:41
dereckson requested review of D2869: Deploy Salt Tower on salt-primary role.
Mar 9 2023, 19:40
dereckson closed D2868: Provide Nasqueron FreeBSD packages catalogue.
Mar 9 2023, 19:33
dereckson committed rOPSf284b5e735e8: Provide Nasqueron FreeBSD packages catalogue (authored by dereckson).
Provide Nasqueron FreeBSD packages catalogue
Mar 9 2023, 19:33
dereckson accepted D2868: Provide Nasqueron FreeBSD packages catalogue.
Mar 9 2023, 19:33
dereckson requested review of D2868: Provide Nasqueron FreeBSD packages catalogue.
Mar 9 2023, 19:32
dereckson updated the diff for D2862: Adopt Salt Tower to merge pillar files.

Rebased

Mar 9 2023, 19:09
dereckson closed D2867: Configure client authentication.
Mar 9 2023, 01:58
dereckson committed rDPGSENTRYc7e5a06f8159: Configure client authentication (authored by dereckson).
Configure client authentication
Mar 9 2023, 01:58
dereckson committed rOPS573cff29c5b5: Switch to specialized Sentry PostgreSQL and wal2json Docker image (authored by dereckson).
Switch to specialized Sentry PostgreSQL and wal2json Docker image
Mar 9 2023, 01:58
dereckson closed D2865: Switch to specialized Sentry PostgreSQL and wal2json Docker image.
Mar 9 2023, 01:58