Page MenuHomeDevCentral

Servers on hyper-001 have network issues
Closed, WontfixPublic

Description

Complector doesn't answer on Vault, SSH or

Worked fine earlier this afternoon.
Still answers to ping.

Disruptive for current work.

Event Timeline

dereckson triaged this task as Unbreak Now! priority.Aug 3 2024, 13:13
dereckson created this task.

Also, at the same time, DevCentral is slow for arc diff or to publish this task. This delay behavior is similar as when DNS resolution timeouts occur.

As of 13:18 UTC, SSH access works.

No charge issue:

load averages:  0.04,  0.07,  0.07
dereckson lowered the priority of this task from Unbreak Now! to High.Aug 3 2024, 13:22
dereckson added a project: security.

Could be at hypervisor level. SSH failed until 13:22 where it worked immediately.

dereckson renamed this task from Server outage: complector to Servers on hyper-001 have network issues.Aug 3 2024, 13:23
dereckson shifted this object from the S1 Nasqueron space to the Restricted Space space.

Stopped currently not needed salt and node-exporter on router-001 to see if that helps.

From router-001 network looks good:

Jul 25 11:40:19 router-001 kernel: Limiting closed port RST response from 250 to 191 packets/sec
Jul 28 15:11:58 router-001 kernel: Limiting closed port RST response from 255 to 197 packets/sec
Jul 31 07:36:56 router-001 kernel: Limiting icmp unreach response from 3230 to 208 packets/sec
Jul 31 07:36:57 router-001 kernel: Limiting icmp unreach response from 3276 to 208 packets/sec
Jul 31 07:36:57 router-001 kernel: Limiting icmp unreach response from 3212 to 187 packets/sec
Jul 31 07:36:58 router-001 kernel: Limiting icmp unreach response from 2674 to 193 packets/sec
Jul 31 07:36:59 router-001 kernel: Limiting icmp unreach response from 3088 to 214 packets/sec
Jul 31 07:37:01 router-001 kernel: Limiting icmp unreach response from 3663 to 197 packets/sec
Jul 31 07:37:02 router-001 kernel: Limiting icmp unreach response from 2954 to 205 packets/sec
Jul 31 07:37:03 router-001 kernel: Limiting icmp unreach response from 3082 to 208 packets/sec
Jul 31 07:37:03 router-001 kernel: Limiting icmp unreach response from 3538 to 197 packets/sec
Jul 31 07:37:04 router-001 kernel: Limiting icmp unreach response from 4057 to 209 packets/sec
Jul 31 07:37:06 router-001 kernel: Limiting icmp unreach response from 4142 to 187 packets/sec
Jul 31 07:37:07 router-001 kernel: Limiting icmp unreach response from 4397 to 189 packets/sec
Jul 31 07:37:07 router-001 kernel: Limiting icmp unreach response from 3558 to 184 packets/sec
Jul 31 07:37:08 router-001 kernel: Limiting icmp unreach response from 3188 to 191 packets/sec
Jul 31 07:37:10 router-001 kernel: Limiting icmp unreach response from 3136 to 185 packets/sec
Jul 31 07:37:10 router-001 kernel: Limiting icmp unreach response from 2623 to 188 packets/sec
Jul 31 07:37:11 router-001 kernel: Limiting icmp unreach response from 2711 to 185 packets/sec
Jul 31 07:37:13 router-001 kernel: Limiting icmp unreach response from 3719 to 197 packets/sec
Jul 31 07:37:22 router-001 kernel: Limiting icmp unreach response from 3626 to 195 packets/sec
Aug  3 01:14:46 router-001 kernel: Limiting closed port RST response from 255 to 211 packets/sec

Last issues was July 31 morning

dereckson claimed this task.

Can't repro

dereckson shifted this object from the Restricted Space space to the S1 Nasqueron space.Sep 12 2024, 18:16