Page MenuHomeDevCentral

WIP: Generate a pillar from NetBox information
Needs ReviewPublic

Authored by dereckson on Feb 8 2023, 20:08.
Tags
None
Referenced Files
Unknown Object (File)
Tue, Nov 19, 14:44
Unknown Object (File)
Tue, Nov 19, 13:44
Unknown Object (File)
Tue, Nov 19, 06:44
Unknown Object (File)
Mon, Nov 18, 08:19
Unknown Object (File)
Sun, Nov 17, 16:54
Unknown Object (File)
Fri, Nov 15, 23:18
Unknown Object (File)
Fri, Nov 15, 22:57
Unknown Object (File)
Wed, Nov 13, 23:02
Subscribers
None

Details

Reviewers
DorianWinty
Maniphest Tasks
T1762: Deploy NetBox
Summary

This change is a proof of concept we can use NetBox
to generate information for node pillar

Usage: utils/netbox/pillarize.py <node name>

Additional credits:
cidr_to_netmask method by Trenton McKinney
See https://stackoverflow.com/a/43904598/1930997

Ref T1762

Test Plan

Compare pillarize output with current pillar/nodes/nodes.sls content

Diff Detail

Repository
rOPS Nasqueron Operations
Lint
Lint Errors
SeverityLocationCodeMessage
Errorutils/netbox/pillarize.py:175E302flake8 E302
Errorutils/netbox/pillarize.py:191E501flake8 E501
Errorutils/netbox/pillarize.py:194E501flake8 E501
Errorutils/netbox/pillarize.py:215E501flake8 E501
Errorutils/netbox/pillarize.py:241E501flake8 E501
Errorutils/netbox/pillarize.py:262E302flake8 E302
Errorutils/netbox/pillarize.py:265E302flake8 E302
Errorutils/netbox/pillarize.py:268E302flake8 E302
Unit
No Test Coverage
Branch
netbox-pillar
Build Status
Buildable 4884
Build 5165: arc lint + arc unit

Event Timeline

dereckson created this revision.
dereckson retitled this revision from Generate a pillar from NetBox information to IWP: Generate a pillar from NetBox information.Feb 8 2023, 20:08
dereckson retitled this revision from IWP: Generate a pillar from NetBox information to WIP: Generate a pillar from NetBox information.
dereckson edited the summary of this revision. (Show Details)

s/RuntimeException/RuntimeError

Example run

utils/netbox/pillarize.py router-001

etc_hosts:
- 172.27.27.1 router-001 router-001.nasqueron.drake
- 172.27.27.4 dwellers dwellers.nasqueron.drake
- 172.27.27.5 docker-002 docker-002.nasqueron.drake
- 172.27.27.6 docker-001 docker-001.nasqueron.org
- 172.27.27.7 complector complector.nasqueron.drake
- 172.27.27.8 db-a-001 db-a-001.nasqueron.drake
- 172.27.27.28 cloudhugger cloudhugger.nasqueron.drake
- 172.27.27.33 ysul ysul.nasqueron.drake
- 172.27.27.35 windriver windriver.nasqueron.drake
node:
  forest: nasqueron-infra
  hostname: router-001.nasqueron.drake
  network:
    interfaces:
      intranought:
        device: vmx1
        ipv4:
          address: 172.27.27.1
          netmask: 255.255.255.240
      public:
        device: vmx0
        ipv4:
          address: 51.255.124.8
          gateway: 91.121.86.254
          netmask: 255.255.255.252
  roles:
  - router

We also need pillar/core/network Drake content

Support for IPv6 and for dedicated baremetal instances. Rebased.

Blocked by T1859 change.

Rebased. Use default_gateways and default_gateways_virt fields. Add support for IPv6.

What's missing compared to https://devcentral.nasqueron.org/source/operations/browse/main/pillar/nodes/nodes.sls?

  • IPv6 flags at network level
    • ipv6_native`
    • ipv6_tunnel
  • Flags at interface level
    • ipv4_ovh_failover: could be derived from a new specific tag "IPFO" applied to failover IPs, and the already existing tag "OVH"
    • skip_interface_configuration: used only on Eglide, currently out of scope of NetBox, but if we use this, we'll need to add Eglide too -> custom field with the same name

We then need to carefully compare roles and information, and that's ready to switch from nodes["router-001"] to node directly.

To check if GRE configuration don't try to read IPs from the nodes. If so, we can output a specific pillar entry specific for GRE tunnels, especially as we've all the needed information in the GRE interfaces.

We don't need any flag ipv6_ovh_fix, as we used instead a module method: network_utils.can_directly_be_discovered

Next: ipv6_dhcp_duid flag at interface level

Also gateways are now a multi-value field to allow IPv4 + IPv6

OK, but for hervil, we need to figure how to output 255.255.255.255 as netmask

OK, but for hervil, we need to figure how to output 255.255.255.255 as netmask

Read IPFO / OVH tags to get netmask right

Only require IPFO tag, READ OVH from Tenant

Also missing: "ipv6_dhcp_duid" flag for windriver