Page MenuHomeDevCentral
Feed All Stories

Jul 25 2024

dereckson triaged T1505: Automate Let's Encrypt TLS certificates management for every server as Normal priority.
Jul 25 2024, 20:42 · Servers
dereckson committed rOPS1e9a54c10365: Deploy Certbot everywhere (authored by dereckson).
Deploy Certbot everywhere
Jul 25 2024, 20:41
dereckson closed D3248: Deploy Certbot everywhere.
Jul 25 2024, 20:41
dereckson updated the diff for D3248: Deploy Certbot everywhere.

Rebased. Use /usr/local/etc/periodic. Clean certbot_dir.

Jul 25 2024, 20:41
DorianWinty updated the diff for D3242: Install postfix.

add hostname

Jul 25 2024, 18:56
dereckson added a comment to T651: Deploy Grafana.

DNS: grafana. CNAME www-dev.nasqueron.org

Jul 25 2024, 18:49 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson added a comment to T651: Deploy Grafana.

Deployment can be using sqlite3 as long as it's still performant
as we want our monitoring tools to be resiliant.

Jul 25 2024, 18:49 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
DorianWinty updated the diff for D3364: Provisioning Dovecot Config.

follow comments

Jul 25 2024, 18:46
dereckson added a comment to T650: Deploy PCP on Docker engines.

Probably a good part of roles/core/monitoring when grains["os_family"] == "RedHat". Eglide has "Debian" for that grain, but not sure if we've enough RAM there.

Jul 25 2024, 18:17 · Monitoring and reporting, Servers
DorianWinty updated the diff for D3364: Provisioning Dovecot Config.

chmod of files and folder + encryption

Jul 25 2024, 18:14
dereckson added a comment to T652: Install PCP on Dwellers.

Just for reference, this was a test deployment. This is not currently installed on Dwellers, and needs to be in Salt as part of T650.

Jul 25 2024, 18:13 · Servers
dereckson renamed T650: Deploy PCP on Docker engines from Give access to Dwellers key statistics to Deploy PCP on Docker engines.
Jul 25 2024, 18:12 · Monitoring and reporting, Servers
dereckson raised the priority of T651: Deploy Grafana from Low to Normal.
Jul 25 2024, 18:12 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson claimed T651: Deploy Grafana.

This task has been created in 2016 to publish metrics from PCP (Performance Co-Pilot) on RHEL-like servers, especially our Docker engines.

Jul 25 2024, 18:12 · Monitoring and reporting, Operations sprints (Ignite Alkane Propulsion), Servers
dereckson added a comment to T1623: Deploy Prometheus to gain observability.

RabbitMQ exporters have been added to NetBox under the tag observability -> https://netbox.nasqueron.org/ipam/services/?tag=observability 🔒

Jul 25 2024, 18:04 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson added parent tasks for T1987: Dovecot Metrics: T1931: Dovecot Provisioning, T1623: Deploy Prometheus to gain observability.
Jul 25 2024, 18:03 · Restricted Project, Mail
dereckson added a subtask for T1623: Deploy Prometheus to gain observability: T1987: Dovecot Metrics.
Jul 25 2024, 18:03 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson added a subtask for T1931: Dovecot Provisioning: T1987: Dovecot Metrics.
Jul 25 2024, 18:03 · Mail, Restricted Project, Servers
dereckson added inline comments to D3364: Provisioning Dovecot Config.
Jul 25 2024, 17:47
dereckson requested changes to D3364: Provisioning Dovecot Config.
Jul 25 2024, 17:45
dereckson added a comment to T1932: ViMbAdmin Provisioning.

Next: memcached

Jul 25 2024, 17:39 · Mail, Restricted Project, Servers
dereckson closed D3373: Enable rabbitmq_prometheus plugin.
Jul 25 2024, 17:37
dereckson committed rDRABBITMQ0b92cff35baa: Enable rabbitmq_prometheus plugin (authored by dereckson).
Enable rabbitmq_prometheus plugin
Jul 25 2024, 17:37
dereckson accepted D3373: Enable rabbitmq_prometheus plugin.
Jul 25 2024, 17:36
dereckson added a comment to T1633: Collect metrics from RabbitMQ.

Pending container redeployment with D3374, we can reach metrics set in D3373 with socat:

Jul 25 2024, 17:36 · Operations sprints (Consolidate them all), Servers
DorianWinty published D3364: Provisioning Dovecot Config for review.
Jul 25 2024, 17:09

Jul 24 2024

DorianWinty added a project to T1987: Dovecot Metrics: Restricted Project.
Jul 24 2024, 21:02 · Restricted Project, Mail
DorianWinty triaged T1987: Dovecot Metrics as Normal priority.
Jul 24 2024, 21:01 · Restricted Project, Mail
dereckson added a parent task for T1986: Upgrade Debian version for docker-nginx-php-fpm image: Unknown Object (Maniphest Task).
Jul 24 2024, 19:55 · Docker images
dereckson triaged T1986: Upgrade Debian version for docker-nginx-php-fpm image as High priority.
Jul 24 2024, 19:54 · Docker images
dereckson added a subtask for T1950: Deploy PHP 8.3: Unknown Object (Maniphest Task).
Jul 24 2024, 19:36 · Servers, PHP 8.x support
DorianWinty closed D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:49
DorianWinty committed rOPSfc0d46d845df: Configure pg_HBA for dovecot user (authored by DorianWinty).
Configure pg_HBA for dovecot user
Jul 24 2024, 17:49
dereckson updated the summary of D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:47
dereckson accepted D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:45
DorianWinty added a revision to T1931: Dovecot Provisioning: D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:27 · Mail, Restricted Project, Servers
DorianWinty updated the summary of D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:27
DorianWinty retitled D3375: Configure pg_HBA for dovecot user from configure pg_HBA for dovecot user to Configure pg_HBA for dovecot user.
Jul 24 2024, 17:26
DorianWinty requested review of D3375: Configure pg_HBA for dovecot user.
Jul 24 2024, 17:26
DorianWinty committed rOPS5903a7ce83ce: Upgrade certbot to Python 3.11 (authored by DorianWinty).
Upgrade certbot to Python 3.11
Jul 24 2024, 15:56
DorianWinty closed D3366: Upgrade certbot to Python 3.11.
Jul 24 2024, 15:56
dereckson updated the task description for T1983: Enable telemetry on Vault.
Jul 24 2024, 00:00 · Vault, Monitoring and reporting

Jul 23 2024

dereckson triaged T1983: Enable telemetry on Vault as Low priority.
Jul 23 2024, 23:57 · Vault, Monitoring and reporting
dereckson requested review of D3374: Expose RabbitMQ metrics on port 15692.
Jul 23 2024, 23:34
dereckson added a revision to T1633: Collect metrics from RabbitMQ: D3374: Expose RabbitMQ metrics on port 15692.
Jul 23 2024, 23:34 · Operations sprints (Consolidate them all), Servers
dereckson updated the task description for T1633: Collect metrics from RabbitMQ.
Jul 23 2024, 23:27 · Operations sprints (Consolidate them all), Servers
dereckson updated the diff for D3373: Enable rabbitmq_prometheus plugin.

+port

Jul 23 2024, 23:25
dereckson updated the summary of D3373: Enable rabbitmq_prometheus plugin.
Jul 23 2024, 23:24
dereckson added a revision to T1633: Collect metrics from RabbitMQ: D3373: Enable rabbitmq_prometheus plugin.
Jul 23 2024, 23:24 · Operations sprints (Consolidate them all), Servers
dereckson requested review of D3373: Enable rabbitmq_prometheus plugin.
Jul 23 2024, 23:15
dereckson added a revision to T1623: Deploy Prometheus to gain observability: D3373: Enable rabbitmq_prometheus plugin.
Jul 23 2024, 23:15 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson committed rOPSd98f98eb5e5d: Collect Docker metrics with Prometheus (authored by dereckson).
Collect Docker metrics with Prometheus
Jul 23 2024, 23:08
dereckson closed D3372: Collect Docker metrics with Prometheus.
Jul 23 2024, 23:08
dereckson added a comment to D3372: Collect Docker metrics with Prometheus.

Deployed on both Docker engines, but docker-002 is still to be restarted, probably good idea to sync that with dnf update

Jul 23 2024, 23:08
dereckson accepted D3372: Collect Docker metrics with Prometheus.
Jul 23 2024, 23:07
dereckson added a revision to T1623: Deploy Prometheus to gain observability: D3372: Collect Docker metrics with Prometheus.
Jul 23 2024, 23:05 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson requested review of D3372: Collect Docker metrics with Prometheus.
Jul 23 2024, 23:05
dereckson added a reverting change for D2609: Install Python 3.9 on CentOS/Rocky 8.5 machines: D3371: Revert "Install Python 3.9 on CentOS/Rocky 8.5 machines".
Jul 23 2024, 22:41
dereckson requested review of D3371: Revert "Install Python 3.9 on CentOS/Rocky 8.5 machines".
Jul 23 2024, 22:41
dereckson added a reverting change for rOPSe509029a4712: Install Python 3.9 on CentOS/Rocky 8.5 machines: D3371: Revert "Install Python 3.9 on CentOS/Rocky 8.5 machines".
Jul 23 2024, 22:41
dereckson added a revision to T1982: Upgrade from Python 3.9 to Python 3.11+: D3371: Revert "Install Python 3.9 on CentOS/Rocky 8.5 machines".
Jul 23 2024, 22:41 · Servers
dereckson updated the task description for T1982: Upgrade from Python 3.9 to Python 3.11+.
Jul 23 2024, 22:40 · Servers
dereckson added a task to D3366: Upgrade certbot to Python 3.11: T1982: Upgrade from Python 3.9 to Python 3.11+.
Jul 23 2024, 22:38
dereckson added a revision to T1982: Upgrade from Python 3.9 to Python 3.11+: D3366: Upgrade certbot to Python 3.11.
Jul 23 2024, 22:38 · Servers
dereckson added a task to D3368: Bump default versions to build ports: T1982: Upgrade from Python 3.9 to Python 3.11+.
Jul 23 2024, 22:38
dereckson added a revision to T1982: Upgrade from Python 3.9 to Python 3.11+: D3368: Bump default versions to build ports.
Jul 23 2024, 22:38 · Servers
dereckson triaged T1982: Upgrade from Python 3.9 to Python 3.11+ as Normal priority.
Jul 23 2024, 22:37 · Servers
dereckson added a revision to T1623: Deploy Prometheus to gain observability: D3370: Deploy Prometheus on WindRiver.
Jul 23 2024, 22:28 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson added a comment to T1931: Dovecot Provisioning.

Also, we need to declare Dovecot ports at https://netbox.nasqueron.org/virtualization/virtual-machines/10/ services table (on the public IP)

Jul 23 2024, 21:36 · Mail, Restricted Project, Servers
dereckson removed a parent task for T1981: Upgrade to FreeBSD 14.1: T1980: ZFS collector doesn't work everywhere.
Jul 23 2024, 21:15 · Servers
dereckson removed a subtask for T1980: ZFS collector doesn't work everywhere: T1981: Upgrade to FreeBSD 14.1.
Jul 23 2024, 21:15 · Monitoring and reporting, Servers
dereckson closed T1980: ZFS collector doesn't work everywhere as Resolved.

I suspect the version 1.6.1 (currently in packages) is compatible with FreeBSD 13 while the version 1.8.2 is compatible with FreeBSD 14.

Jul 23 2024, 21:15 · Monitoring and reporting, Servers
dereckson moved T1978: Document monitoring checks from Backlog to Checks on the Monitoring and reporting board.
Jul 23 2024, 20:58 · Salt, Monitoring and reporting
dereckson moved T1945: Deploy a simple Nagios or Naemon to have a reference implementation from Backlog to Checks on the Monitoring and reporting board.
Jul 23 2024, 20:58 · Monitoring and reporting
dereckson moved T1623: Deploy Prometheus to gain observability from Backlog to Prometheus on the Monitoring and reporting board.
Jul 23 2024, 20:57 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson moved T1980: ZFS collector doesn't work everywhere from Backlog to Prometheus on the Monitoring and reporting board.
Jul 23 2024, 20:57 · Monitoring and reporting, Servers
dereckson moved T1392: Evaluate Prometheus from Backlog to Prometheus on the Monitoring and reporting board.
Jul 23 2024, 20:57 · Monitoring and reporting, Product evaluation
dereckson added a subtask for T1981: Upgrade to FreeBSD 14.1: T1972: Update WindRiver to FreeBSD 14.1.
Jul 23 2024, 20:57 · Servers
dereckson added a parent task for T1972: Update WindRiver to FreeBSD 14.1: T1981: Upgrade to FreeBSD 14.1.
Jul 23 2024, 20:57 · Servers
dereckson triaged T1981: Upgrade to FreeBSD 14.1 as High priority.
Jul 23 2024, 20:57 · Servers
dereckson triaged T1980: ZFS collector doesn't work everywhere as Low priority.
Jul 23 2024, 20:55 · Monitoring and reporting, Servers
dereckson committed rOPSa1b74616cbad: Deploy Prometheus Node Exporter (authored by dereckson).
Deploy Prometheus Node Exporter
Jul 23 2024, 20:10
dereckson closed D3369: Deploy Prometheus Node Exporter.
Jul 23 2024, 20:10
dereckson accepted D3369: Deploy Prometheus Node Exporter.
Jul 23 2024, 20:09
dereckson updated the diff for D3369: Deploy Prometheus Node Exporter.

Align names

Jul 23 2024, 19:58
dereckson updated the diff for D3369: Deploy Prometheus Node Exporter.

hunt-insecable-spaces

Jul 23 2024, 19:57
dereckson updated the diff for D3369: Deploy Prometheus Node Exporter.

Avoid duplicate jinja context for service config

Jul 23 2024, 19:56
dereckson updated the diff for D3369: Deploy Prometheus Node Exporter.

hunt-insecable-spaces

Jul 23 2024, 19:51
dereckson updated the diff for D3369: Deploy Prometheus Node Exporter.

Configure systemd service to restrict to private IP too

Jul 23 2024, 19:50
dereckson updated the diff for D3369: Deploy Prometheus Node Exporter.

no-op on FreeBSD

Jul 23 2024, 19:14
dereckson updated the diff for D3369: Deploy Prometheus Node Exporter.

Service name is different on EPEL package

Jul 23 2024, 19:08
dereckson added a comment to D3369: Deploy Prometheus Node Exporter.

Successfully deployed on FreeBSD servers:

Jul 23 2024, 18:56
dereckson requested review of D3369: Deploy Prometheus Node Exporter.
Jul 23 2024, 18:30
dereckson added a revision to T1623: Deploy Prometheus to gain observability: D3369: Deploy Prometheus Node Exporter.
Jul 23 2024, 18:30 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson added a comment to T1623: Deploy Prometheus to gain observability.

Independently of the 2020 plan for a service mesh, we're going to deploy Prometheus right now to gain observability on currently deployed services.

Jul 23 2024, 18:05 · Monitoring and reporting, Operations sprints (Consolidate them all), Servers
dereckson added a comment to T1392: Evaluate Prometheus.

Prometheus has gained in traction and timeseries become now more and more standard.

Jul 23 2024, 18:01 · Monitoring and reporting, Product evaluation
dereckson renamed T1392: Evaluate Prometheus from Deploy Prometheus to Evaluate Prometheus.
Jul 23 2024, 17:57 · Monitoring and reporting, Product evaluation
dereckson added a project to T1392: Evaluate Prometheus: Monitoring and reporting.
Jul 23 2024, 17:57 · Monitoring and reporting, Product evaluation
dereckson renamed T1392: Evaluate Prometheus from Evaluate Prometheus to Deploy Prometheus.
Jul 23 2024, 17:56 · Monitoring and reporting, Product evaluation
dereckson moved T1392: Evaluate Prometheus from Backlog to Current focus on the Product evaluation board.
Jul 23 2024, 17:56 · Monitoring and reporting, Product evaluation