Page MenuHomeDevCentral
Feed All Stories

Jul 27 2024

dereckson committed rOPSe7a3480a4443: Expose RabbitMQ metrics on port 15692 (authored by dereckson).
Expose RabbitMQ metrics on port 15692
Jul 27 2024, 12:15
dereckson closed D3374: Expose RabbitMQ metrics on port 15692.
Jul 27 2024, 12:15
dereckson added a comment to T1633: Collect metrics from RabbitMQ.

Dashboard: https://grafana.nasqueron.org/d/Kn5xm-gZk/rabbitmq-overview?orgId=1&refresh=15s

Jul 27 2024, 12:14 · Operations sprints (Consolidate them all), Servers
dereckson committed rOPSd17f23473772: Scrape RabbitMQ metrics into Prometheus (authored by dereckson).
Scrape RabbitMQ metrics into Prometheus
Jul 27 2024, 12:11
dereckson closed D3376: Scrape RabbitMQ metrics into Prometheus.
Jul 27 2024, 12:11
dereckson closed D3370: Deploy Prometheus on WindRiver.
Jul 27 2024, 12:10
dereckson committed rOPSf2455785f16d: Deploy Prometheus on WindRiver (authored by dereckson).
Deploy Prometheus on WindRiver
Jul 27 2024, 12:10
dereckson accepted D3370: Deploy Prometheus on WindRiver.
Jul 27 2024, 12:10
dereckson published D3370: Deploy Prometheus on WindRiver for review.
Jul 27 2024, 12:09
dereckson closed D3378: Declare libexec to dirs map.
Jul 27 2024, 12:01
dereckson committed rOPSbc4be357c9ec: Declare libexec to dirs map (authored by dereckson).
Declare libexec to dirs map
Jul 27 2024, 12:01
dereckson accepted D3378: Declare libexec to dirs map.
Jul 27 2024, 12:01
dereckson requested review of D3378: Declare libexec to dirs map.
Jul 27 2024, 12:00

Jul 26 2024

dereckson removed a project from T651: Deploy Grafana: Nasqueron Docker deployment squad.

Not deployed to Docker but bare-metal.

Jul 26 2024, 23:09 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson moved T651: Deploy Grafana from Backlog - Monitoring / misc to Working on on the Operations sprints (Ignite Alkane Propulsion) board.
Jul 26 2024, 23:09 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson moved T651: Deploy Grafana from Backlog to Working on on the Servers board.
Jul 26 2024, 23:09 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson closed D3377: Deploy Grafana.
Jul 26 2024, 23:08
dereckson committed rOPS1a46d56b872f: Deploy Grafana (authored by dereckson).
Deploy Grafana
Jul 26 2024, 23:08
dereckson accepted D3377: Deploy Grafana.
Jul 26 2024, 23:08
dereckson updated the diff for D3377: Deploy Grafana.

Fix whitespace issue

Jul 26 2024, 23:06
dereckson accepted D3376: Scrape RabbitMQ metrics into Prometheus.
Jul 26 2024, 23:05
dereckson accepted D3374: Expose RabbitMQ metrics on port 15692.

Still to deploy.

Jul 26 2024, 23:04
dereckson added a comment to T650: Deploy PCP on Docker engines.

T651 has a Grafana ready if we wish to retest this on Dwellers, green light.

Jul 26 2024, 19:39 · Monitoring and reporting, Servers
dereckson added a comment to T651: Deploy Grafana.

Deployed at https://grafana.nasqueron.org/

Jul 26 2024, 19:38 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson updated the summary of D3377: Deploy Grafana.
Jul 26 2024, 19:38
dereckson added a revision to T651: Deploy Grafana: D3377: Deploy Grafana.
Jul 26 2024, 19:38 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson requested review of D3377: Deploy Grafana.
Jul 26 2024, 19:37
dereckson added a revision to T1633: Collect metrics from RabbitMQ: D3376: Scrape RabbitMQ metrics into Prometheus.
Jul 26 2024, 19:34 · Operations sprints (Consolidate them all), Servers
dereckson requested review of D3376: Scrape RabbitMQ metrics into Prometheus.
Jul 26 2024, 19:34
dereckson added a comment to T1496: Deploy StatsD and Graphite.

With a transformation, we can multiply the field by 10 to solve that issue: https://grafana.nasqueron.org/d/fdsy3oogbchs0f/graphite-quux-sandbox?orgId=1&tab=transform&from=1722018733917&to=1722022333917

Jul 26 2024, 19:32 · Operations sprints (Ignite Alkane Propulsion), Nasqueron Docker deployment squad, Monitoring and reporting
dereckson added a comment to T1496: Deploy StatsD and Graphite.

The graphite Docker image provides both pieces of software.

Jul 26 2024, 19:24 · Operations sprints (Ignite Alkane Propulsion), Nasqueron Docker deployment squad, Monitoring and reporting
dereckson renamed T1496: Deploy StatsD and Graphite from Deploy graphite to Deploy StatsD and Graphite.
Jul 26 2024, 19:13 · Operations sprints (Ignite Alkane Propulsion), Nasqueron Docker deployment squad, Monitoring and reporting
dereckson merged task T1495: Deploy StatsD into T1496: Deploy StatsD and Graphite.
Jul 26 2024, 19:13 · Operations sprints (Ignite Alkane Propulsion), Nasqueron Docker deployment squad, Monitoring and reporting
dereckson merged T1495: Deploy StatsD into T1496: Deploy StatsD and Graphite.
Jul 26 2024, 19:13 · Operations sprints (Ignite Alkane Propulsion), Nasqueron Docker deployment squad, Monitoring and reporting

Jul 25 2024

dereckson added a comment to T1505: Automate Let's Encrypt TLS certificates management for every server.

rOPS1e9a54c10365 has worked like a charm on WindRiver to generate grafana.nasqueron.org through DNS.

Jul 25 2024, 20:43 · Servers
dereckson triaged T1505: Automate Let's Encrypt TLS certificates management for every server as Normal priority.
Jul 25 2024, 20:42 · Servers
dereckson committed rOPS1e9a54c10365: Deploy Certbot everywhere (authored by dereckson).
Deploy Certbot everywhere
Jul 25 2024, 20:41
dereckson closed D3248: Deploy Certbot everywhere.
Jul 25 2024, 20:41
dereckson updated the diff for D3248: Deploy Certbot everywhere.

Rebased. Use /usr/local/etc/periodic. Clean certbot_dir.

Jul 25 2024, 20:41
DorianWinty updated the diff for D3242: Install postfix.

add hostname

Jul 25 2024, 18:56
dereckson added a comment to T651: Deploy Grafana.

DNS: grafana. CNAME www-dev.nasqueron.org

Jul 25 2024, 18:49 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson added a comment to T651: Deploy Grafana.

Deployment can be using sqlite3 as long as it's still performant
as we want our monitoring tools to be resiliant.

Jul 25 2024, 18:49 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
DorianWinty updated the diff for D3364: Provisioning Dovecot Config.

follow comments

Jul 25 2024, 18:46
dereckson added a comment to T650: Deploy PCP on Docker engines.

Probably a good part of roles/core/monitoring when grains["os_family"] == "RedHat". Eglide has "Debian" for that grain, but not sure if we've enough RAM there.

Jul 25 2024, 18:17 · Monitoring and reporting, Servers
DorianWinty updated the diff for D3364: Provisioning Dovecot Config.

chmod of files and folder + encryption

Jul 25 2024, 18:14
dereckson added a comment to T652: Install PCP on Dwellers.

Just for reference, this was a test deployment. This is not currently installed on Dwellers, and needs to be in Salt as part of T650.

Jul 25 2024, 18:13 · Servers
dereckson renamed T650: Deploy PCP on Docker engines from Give access to Dwellers key statistics to Deploy PCP on Docker engines.
Jul 25 2024, 18:12 · Monitoring and reporting, Servers
dereckson raised the priority of T651: Deploy Grafana from Low to Normal.
Jul 25 2024, 18:12 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson claimed T651: Deploy Grafana.

This task has been created in 2016 to publish metrics from PCP (Performance Co-Pilot) on RHEL-like servers, especially our Docker engines.

Jul 25 2024, 18:12 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson added a comment to T1623: Deploy Prometheus to gain observability.

RabbitMQ exporters have been added to NetBox under the tag observability -> https://netbox.nasqueron.org/ipam/services/?tag=observability 🔒

Jul 25 2024, 18:04 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson added parent tasks for T1987: Dovecot Metrics: T1931: Dovecot Provisioning, T1623: Deploy Prometheus to gain observability.
Jul 25 2024, 18:03 · Restricted Project, Mail
dereckson added a subtask for T1623: Deploy Prometheus to gain observability: T1987: Dovecot Metrics.
Jul 25 2024, 18:03 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson added a subtask for T1931: Dovecot Provisioning: T1987: Dovecot Metrics.
Jul 25 2024, 18:03 · Mail, Restricted Project, Servers
dereckson added inline comments to D3364: Provisioning Dovecot Config.
Jul 25 2024, 17:47
dereckson requested changes to D3364: Provisioning Dovecot Config.
Jul 25 2024, 17:45
dereckson added a comment to T1932: ViMbAdmin Provisioning.

Next: memcached

Jul 25 2024, 17:39 · Mail, Restricted Project, Servers
dereckson closed D3373: Enable rabbitmq_prometheus plugin.
Jul 25 2024, 17:37
dereckson committed rDRABBITMQ0b92cff35baa: Enable rabbitmq_prometheus plugin (authored by dereckson).
Enable rabbitmq_prometheus plugin
Jul 25 2024, 17:37
dereckson accepted D3373: Enable rabbitmq_prometheus plugin.
Jul 25 2024, 17:36
dereckson added a comment to T1633: Collect metrics from RabbitMQ.

Pending container redeployment with D3374, we can reach metrics set in D3373 with socat:

Jul 25 2024, 17:36 · Operations sprints (Consolidate them all), Servers
DorianWinty published D3364: Provisioning Dovecot Config for review.
Jul 25 2024, 17:09

Jul 24 2024

DorianWinty added a project to T1987: Dovecot Metrics: Restricted Project.
Jul 24 2024, 21:02 · Restricted Project, Mail
DorianWinty triaged T1987: Dovecot Metrics as Normal priority.
Jul 24 2024, 21:01 · Restricted Project, Mail
dereckson added a parent task for T1986: Upgrade Debian version for docker-nginx-php-fpm image: Unknown Object (Maniphest Task).
Jul 24 2024, 19:55 · Docker images
dereckson triaged T1986: Upgrade Debian version for docker-nginx-php-fpm image as High priority.
Jul 24 2024, 19:54 · Docker images
dereckson added a subtask for T1950: Deploy PHP 8.3: Unknown Object (Maniphest Task).
Jul 24 2024, 19:36 · Servers, PHP 8.x support
DorianWinty closed D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:49
DorianWinty committed rOPSfc0d46d845df: Configure pg_HBA for dovecot user (authored by DorianWinty).
Configure pg_HBA for dovecot user
Jul 24 2024, 17:49
dereckson updated the summary of D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:47
dereckson accepted D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:45
DorianWinty added a revision to T1931: Dovecot Provisioning: D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:27 · Mail, Restricted Project, Servers
DorianWinty updated the summary of D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:27
DorianWinty retitled D3375: Configure pg_HBA for dovecot user from configure pg_HBA for dovecot user to Configure pg_HBA for dovecot user.
Jul 24 2024, 17:26
DorianWinty requested review of D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:26
DorianWinty committed rOPS5903a7ce83ce: Upgrade certbot to Python 3.11 (authored by DorianWinty).
Upgrade certbot to Python 3.11
Jul 24 2024, 15:56
DorianWinty closed D3366: Upgrade certbot to Python 3.11.
Jul 24 2024, 15:56
dereckson updated the task description for T1983: Enable telemetry on Vault.
Jul 24 2024, 00:00 · Vault, Monitoring and reporting

Jul 23 2024

dereckson triaged T1983: Enable telemetry on Vault as Low priority.
Jul 23 2024, 23:57 · Vault, Monitoring and reporting
dereckson requested review of D3374: Expose RabbitMQ metrics on port 15692.
Jul 23 2024, 23:34
dereckson added a revision to T1633: Collect metrics from RabbitMQ: D3374: Expose RabbitMQ metrics on port 15692.
Jul 23 2024, 23:34 · Operations sprints (Consolidate them all), Servers
dereckson updated the task description for T1633: Collect metrics from RabbitMQ.
Jul 23 2024, 23:27 · Operations sprints (Consolidate them all), Servers
dereckson updated the diff for D3373: Enable rabbitmq_prometheus plugin.

+port

Jul 23 2024, 23:25
dereckson updated the summary of D3373: Enable rabbitmq_prometheus plugin.
Jul 23 2024, 23:24
dereckson added a revision to T1633: Collect metrics from RabbitMQ: D3373: Enable rabbitmq_prometheus plugin.
Jul 23 2024, 23:24 · Operations sprints (Consolidate them all), Servers
dereckson requested review of D3373: Enable rabbitmq_prometheus plugin.
Jul 23 2024, 23:15
dereckson added a revision to T1623: Deploy Prometheus to gain observability: D3373: Enable rabbitmq_prometheus plugin.
Jul 23 2024, 23:15 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson committed rOPSd98f98eb5e5d: Collect Docker metrics with Prometheus (authored by dereckson).
Collect Docker metrics with Prometheus
Jul 23 2024, 23:08
dereckson closed D3372: Collect Docker metrics with Prometheus.
Jul 23 2024, 23:08
dereckson added a comment to D3372: Collect Docker metrics with Prometheus.

Deployed on both Docker engines, but docker-002 is still to be restarted, probably good idea to sync that with dnf update

Jul 23 2024, 23:08
dereckson accepted D3372: Collect Docker metrics with Prometheus.
Jul 23 2024, 23:07
dereckson added a revision to T1623: Deploy Prometheus to gain observability: D3372: Collect Docker metrics with Prometheus.
Jul 23 2024, 23:05 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson requested review of D3372: Collect Docker metrics with Prometheus.
Jul 23 2024, 23:05
dereckson added a reverting change for D2609: Install Python 3.9 on CentOS/Rocky 8.5 machines: D3371: Revert "Install Python 3.9 on CentOS/Rocky 8.5 machines".
Jul 23 2024, 22:41
dereckson requested review of D3371: Revert "Install Python 3.9 on CentOS/Rocky 8.5 machines".
Jul 23 2024, 22:41
dereckson added a reverting change for rOPSe509029a4712: Install Python 3.9 on CentOS/Rocky 8.5 machines: D3371: Revert "Install Python 3.9 on CentOS/Rocky 8.5 machines".
Jul 23 2024, 22:41
dereckson added a revision to T1982: Upgrade from Python 3.9 to Python 3.11+: D3371: Revert "Install Python 3.9 on CentOS/Rocky 8.5 machines".
Jul 23 2024, 22:41 · Servers
dereckson updated the task description for T1982: Upgrade from Python 3.9 to Python 3.11+.
Jul 23 2024, 22:40 · Servers
dereckson added a task to D3366: Upgrade certbot to Python 3.11: T1982: Upgrade from Python 3.9 to Python 3.11+.
Jul 23 2024, 22:38
dereckson added a revision to T1982: Upgrade from Python 3.9 to Python 3.11+: D3366: Upgrade certbot to Python 3.11.
Jul 23 2024, 22:38 · Servers
dereckson added a task to D3368: Bump default versions to build ports: T1982: Upgrade from Python 3.9 to Python 3.11+.
Jul 23 2024, 22:38