Page MenuHomeDevCentral
Differential D2790

WIP: Generate a pillar from NetBox information
Needs ReviewPublic
Actions

Authored by dereckson on Feb 8 2023, 20:08.
Tags
None
Referenced Files
Unknown Object (File)
Thu, Apr 3, 12:22
Unknown Object (File)
Wed, Apr 2, 18:08
Unknown Object (File)
Wed, Apr 2, 10:21
Unknown Object (File)
Wed, Apr 2, 10:21
Unknown Object (File)
Mon, Mar 31, 08:56
Unknown Object (File)
Sat, Mar 29, 11:28
Unknown Object (File)
Fri, Mar 28, 16:32
Unknown Object (File)
Thu, Mar 27, 21:17
View All Files
Subscribers
None

Details

Reviewers
DorianWinty
Maniphest Tasks
T1762: Deploy NetBox
Summary

This change is a proof of concept we can use NetBox
to generate information for node pillar

Usage: utils/netbox/pillarize.py <node name>

Additional credits:
cidr_to_netmask method by Trenton McKinney
See https://stackoverflow.com/a/43904598/1930997

Ref T1762

Test Plan

Compare pillarize output with current pillar/nodes/nodes.sls content

Diff Detail

Repository
rOPS Nasqueron Operations
Lint
Lint Errors
SeverityLocationCodeMessage
Errorutils/netbox/pillarize.py:101E114flake8 E114
Unit
No Test Coverage
Branch
netbox-pillar
Build Status
Buildable 5059
Build 5340: arc lint + arc unit

Event Timeline

dereckson requested review of this revision.Feb 8 2023, 20:08
dereckson created this revision.
Harbormaster completed remote builds in B4419: Diff 7084.Feb 8 2023, 20:08
dereckson retitled this revision from Generate a pillar from NetBox information to IWP: Generate a pillar from NetBox information.Feb 8 2023, 20:08
dereckson retitled this revision from IWP: Generate a pillar from NetBox information to WIP: Generate a pillar from NetBox information.
dereckson edited the summary of this revision. (Show Details)
dereckson updated this revision to Diff 7085.Feb 8 2023, 20:09

s/RuntimeException/RuntimeError

Harbormaster completed remote builds in B4420: Diff 7085.Feb 8 2023, 20:09
dereckson added a comment.Feb 8 2023, 20:12

Example run

utils/netbox/pillarize.py router-001

etc_hosts:
- 172.27.27.1 router-001 router-001.nasqueron.drake
- 172.27.27.4 dwellers dwellers.nasqueron.drake
- 172.27.27.5 docker-002 docker-002.nasqueron.drake
- 172.27.27.6 docker-001 docker-001.nasqueron.org
- 172.27.27.7 complector complector.nasqueron.drake
- 172.27.27.8 db-a-001 db-a-001.nasqueron.drake
- 172.27.27.28 cloudhugger cloudhugger.nasqueron.drake
- 172.27.27.33 ysul ysul.nasqueron.drake
- 172.27.27.35 windriver windriver.nasqueron.drake
node:
  forest: nasqueron-infra
  hostname: router-001.nasqueron.drake
  network:
    interfaces:
      intranought:
        device: vmx1
        ipv4:
          address: 172.27.27.1
          netmask: 255.255.255.240
      public:
        device: vmx0
        ipv4:
          address: 51.255.124.8
          gateway: 91.121.86.254
          netmask: 255.255.255.252
  roles:
  - router
dereckson mentioned this in D2826: Update api-datasources service URL.Mar 3 2023, 17:19
dereckson added a comment.EditedMar 8 2023, 01:37

We also need pillar/core/network Drake content

dereckson updated this revision to Diff 7734.Apr 16 2023, 01:23

Support for IPv6 and for dedicated baremetal instances. Rebased.

Harbormaster completed remote builds in B4789: Diff 7734.Apr 16 2023, 01:23
dereckson mentioned this in T1859: Represent IPv6 gateway in NetBox.May 8 2023, 00:41
dereckson planned changes to this revision.May 8 2023, 00:47

Blocked by T1859 change.

dereckson updated this revision to Diff 7896.May 13 2023, 16:24

Rebased. Use default_gateways and default_gateways_virt fields. Add support for IPv6.

Harbormaster completed remote builds in B4884: Diff 7896.May 13 2023, 16:24
dereckson updated this revision to Diff 7897.May 13 2023, 16:25

black

Harbormaster completed remote builds in B4885: Diff 7897.May 13 2023, 16:25
dereckson planned changes to this revision.May 13 2023, 16:33

What's missing compared to https://devcentral.nasqueron.org/source/operations/browse/main/pillar/nodes/nodes.sls?

  • IPv6 flags at network level
    • ipv6_native`
    • ipv6_tunnel
  • Flags at interface level
    • ipv4_ovh_failover: could be derived from a new specific tag "IPFO" applied to failover IPs, and the already existing tag "OVH"
    • skip_interface_configuration: used only on Eglide, currently out of scope of NetBox, but if we use this, we'll need to add Eglide too -> custom field with the same name

We then need to carefully compare roles and information, and that's ready to switch from nodes["router-001"] to node directly.

To check if GRE configuration don't try to read IPs from the nodes. If so, we can output a specific pillar entry specific for GRE tunnels, especially as we've all the needed information in the GRE interfaces.

We don't need any flag ipv6_ovh_fix, as we used instead a module method: network_utils.can_directly_be_discovered

dereckson added a comment.EditedJun 3 2023, 22:51

Next: ipv6_dhcp_duid flag at interface level

Also gateways are now a multi-value field to allow IPv4 + IPv6

dereckson updated this revision to Diff 8221.Jun 29 2023, 22:25

OK, but for hervil, we need to figure how to output 255.255.255.255 as netmask

Harbormaster completed remote builds in B5059: Diff 8221.Jun 29 2023, 22:25
dereckson updated this revision to Diff 8222.Jun 29 2023, 22:26

OK, but for hervil, we need to figure how to output 255.255.255.255 as netmask

Harbormaster completed remote builds in B5060: Diff 8222.Jun 29 2023, 22:26
dereckson updated this revision to Diff 8224.Jun 29 2023, 23:33

Read IPFO / OVH tags to get netmask right

Harbormaster completed remote builds in B5061: Diff 8224.Jun 29 2023, 23:33
dereckson updated this revision to Diff 8225.Jun 29 2023, 23:35

Only require IPFO tag, READ OVH from Tenant

Harbormaster completed remote builds in B5062: Diff 8225.Jun 29 2023, 23:35
dereckson added a comment.Jul 27 2024, 23:12

Also missing: "ipv6_dhcp_duid" flag for windriver

Revision Contents
Changeset List

PathSize
roles/
salt-primary/
software/
1 line
utils/
netbox/
355 lines

Diff 8221

View Options

roles/salt-primary/software/init.sls

Loading...
View Options

utils/netbox/pillarize.py

Loading...
Nasqueron DevCentral · If it had been much bigger the moon would have had a core of ice. · Powered by Phabricator