Details

Reviewers

DorianWinty

Maniphest Tasks

T1762: Deploy NetBox

Summary

This change is a proof of concept we can use NetBox
to generate information for node pillar

Usage: utils/netbox/pillarize.py <node name>

Additional credits:
cidr_to_netmask method by Trenton McKinney
See https://stackoverflow.com/a/43904598/1930997

Ref T1762

Test Plan

Compare pillarize output with current pillar/nodes/nodes.sls content

Diff Detail

Repository

rOPS Nasqueron Operations

Lint

Lint Skipped

Unit

No Test Coverage

Branch

netbox-pillar

Build Status

Buildable 6343
Build 6627: arc lint + arc unit

Event Timeline

dereckson requested review of this revision.Feb 8 2023, 20:08

dereckson created this revision.

Harbormaster completed remote builds in B4419: Diff 7084.Feb 8 2023, 20:08

dereckson retitled this revision from Generate a pillar from NetBox information to IWP: Generate a pillar from NetBox information.Feb 8 2023, 20:08

dereckson retitled this revision from IWP: Generate a pillar from NetBox information to WIP: Generate a pillar from NetBox information.

dereckson edited the summary of this revision. (Show Details)

s/RuntimeException/RuntimeError

Harbormaster completed remote builds in B4420: Diff 7085.Feb 8 2023, 20:09

Example run

utils/netbox/pillarize.py router-001

etc_hosts:
- 172.27.27.1 router-001 router-001.nasqueron.drake
- 172.27.27.4 dwellers dwellers.nasqueron.drake
- 172.27.27.5 docker-002 docker-002.nasqueron.drake
- 172.27.27.6 docker-001 docker-001.nasqueron.org
- 172.27.27.7 complector complector.nasqueron.drake
- 172.27.27.8 db-a-001 db-a-001.nasqueron.drake
- 172.27.27.28 cloudhugger cloudhugger.nasqueron.drake
- 172.27.27.33 ysul ysul.nasqueron.drake
- 172.27.27.35 windriver windriver.nasqueron.drake
node:
  forest: nasqueron-infra
  hostname: router-001.nasqueron.drake
  network:
    interfaces:
      intranought:
        device: vmx1
        ipv4:
          address: 172.27.27.1
          netmask: 255.255.255.240
      public:
        device: vmx0
        ipv4:
          address: 51.255.124.8
          gateway: 91.121.86.254
          netmask: 255.255.255.252
  roles:
  - router

dereckson mentioned this in D2826: Update api-datasources service URL.Mar 3 2023, 17:19

We also need pillar/core/network Drake content

Support for IPv6 and for dedicated baremetal instances. Rebased.

Harbormaster completed remote builds in B4789: Diff 7734.Apr 16 2023, 01:23

dereckson mentioned this in T1859: Represent IPv6 gateway in NetBox.May 8 2023, 00:41

Blocked by T1859 change.

Rebased. Use default_gateways and default_gateways_virt fields. Add support for IPv6.

Harbormaster completed remote builds in B4884: Diff 7896.May 13 2023, 16:24

black

Harbormaster completed remote builds in B4885: Diff 7897.May 13 2023, 16:25

What's missing compared to https://devcentral.nasqueron.org/source/operations/browse/main/pillar/nodes/nodes.sls?

IPv6 flags at network level
- ipv6_native`
- ipv6_tunnel
Flags at interface level
- ipv4_ovh_failover: could be derived from a new specific tag "IPFO" applied to failover IPs, and the already existing tag "OVH"
- skip_interface_configuration: used only on Eglide, currently out of scope of NetBox, but if we use this, we'll need to add Eglide too -> custom field with the same name

We then need to carefully compare roles and information, and that's ready to switch from nodes["router-001"] to node directly.

To check if GRE configuration don't try to read IPs from the nodes. If so, we can output a specific pillar entry specific for GRE tunnels, especially as we've all the needed information in the GRE interfaces.

We don't need any flag ipv6_ovh_fix, as we used instead a module method: network_utils.can_directly_be_discovered