Page MenuHomeDevCentral

Provision RabbitMQ configuration
ClosedPublic

Authored by dereckson on Feb 10 2023, 19:38.
Tags
None
Referenced Files
Unknown Object (File)
Wed, Mar 27, 23:01
Unknown Object (File)
Wed, Mar 27, 22:06
Unknown Object (File)
Wed, Mar 27, 21:37
Unknown Object (File)
Wed, Mar 27, 21:33
Unknown Object (File)
Tue, Mar 26, 19:48
Unknown Object (File)
Tue, Mar 26, 02:55
Unknown Object (File)
Sat, Mar 23, 21:43
Unknown Object (File)
Sat, Mar 23, 05:33
Subscribers
None

Details

Summary

RabbitMQ clusters can be configured in pillar/saas/rabbitmq.sls

HTTP API requests through the management plugin are fired
to ensure each cluster is configured as expected.

Vault configuration
The salt-primary node policy has read access to the credentials defined
in rabbitmq_clusters pillar, so it can run the saas-rabbitmq role.

Policies are now templates, and the node policy reads the rendered template
version from /srv/policies instead of Salt roles/ folder.

Ref T752.

Test Plan

Provision white-rabbit cluster

Diff Detail

Repository
rOPS Nasqueron Operations
Lint
Lint Not Applicable
Unit
Tests Not Applicable

Event Timeline

dereckson created this revision.

Here the output of salt-call --local state.show_sls roles/saas-rabbitmq:

1local:
2 rabbitmq_cluster_white-rabbit_user_notifications:
3 __env__: base
4 __sls__: roles/saas-rabbitmq.server.content
5 rabbitmq:
6 - name: notifications
7 - cluster: white-rabbit
8 - credential: ops/secrets/nasqueron.notifications.broker
9 - user_present
10 - order: 10000
11 rabbitmq_cluster_white-rabbit_user_wearg:
12 __env__: base
13 __sls__: roles/saas-rabbitmq.server.content
14 rabbitmq:
15 - name: wearg
16 - cluster: white-rabbit
17 - credential: apps/viperserv/broker
18 - user_present
19 - order: 10001
20 rabbitmq_cluster_white-rabbit_user_notifications-ysul:
21 __env__: base
22 __sls__: roles/saas-rabbitmq.server.content
23 rabbitmq:
24 - name: notifications-ysul
25 - cluster: white-rabbit
26 - credential: ops/secrets/nasqueron/notifications/notifications-cli/ysul
27 - user_present
28 - order: 10002
29 rabbitmq_cluster_white-rabbit_user_notifications-windriver:
30 __env__: base
31 __sls__: roles/saas-rabbitmq.server.content
32 rabbitmq:
33 - name: notifications-windriver
34 - cluster: white-rabbit
35 - credential: ops/secrets/nasqueron/notifications/notifications-cli/windriver
36 - user_present
37 - order: 10003
38 rabbitmq_cluster_white-rabbit_vhost_dev:
39 __env__: base
40 __sls__: roles/saas-rabbitmq.server.content
41 rabbitmq:
42 - name: dev
43 - cluster: white-rabbit
44 - description: Nasqueron dev services
45 - vhost_present
46 - order: 10004
47 rabbitmq_cluster_white-rabbit_vhost_dev_exchange_notifications:
48 __env__: base
49 __sls__: roles/saas-rabbitmq.server.content
50 rabbitmq:
51 - name: notifications
52 - cluster: white-rabbit
53 - vhost: dev
54 - type: topic
55 - exchange_present
56 - order: 10005
57 rabbitmq_cluster_white-rabbit_vhost_dev_queue_wearg-notifications:
58 __env__: base
59 __sls__: roles/saas-rabbitmq.server.content
60 rabbitmq:
61 - name: wearg-notifications
62 - cluster: white-rabbit
63 - vhost: dev
64 - queue_present
65 - order: 10006
66 rabbitmq_cluster_white-rabbit_vhost_dev_binding_1:
67 __env__: base
68 __sls__: roles/saas-rabbitmq.server.content
69 rabbitmq:
70 - queue: wearg-notifications
71 - cluster: white-rabbit
72 - vhost: dev
73 - exchange: notifications
74 - routing_key: '#'
75 - queue_binding
76 - order: 10007
77 rabbitmq_cluster_white-rabbit_vhost_dev_permissions_user_notifications:
78 __env__: base
79 __sls__: roles/saas-rabbitmq.server.content
80 rabbitmq:
81 - cluster: white-rabbit
82 - vhost: dev
83 - user: notifications
84 - permissions:
85 configure: ''
86 read: ''
87 write: ^notifications$
88 - user_permissions
89 - order: 10008
90 rabbitmq_cluster_white-rabbit_vhost_dev_permissions_user_wearg:
91 __env__: base
92 __sls__: roles/saas-rabbitmq.server.content
93 rabbitmq:
94 - cluster: white-rabbit
95 - vhost: dev
96 - user: wearg
97 - permissions:
98 configure: ''
99 read: ^wearg\-notifications$
100 write: ''
101 - user_permissions
102 - order: 10009
103 rabbitmq_cluster_white-rabbit_vhost_dev_permissions_user_notifications-ysul:
104 __env__: base
105 __sls__: roles/saas-rabbitmq.server.content
106 rabbitmq:
107 - cluster: white-rabbit
108 - vhost: dev
109 - user: notifications-ysul
110 - permissions:
111 configure: ^amq\.gen.*$
112 read: ^(amq\.gen.*|notifications)$
113 write: ^amq\.gen.*$
114 - user_permissions
115 - order: 10010
116 rabbitmq_cluster_white-rabbit_vhost_dev_permissions_user_notifications-windriver:
117 __env__: base
118 __sls__: roles/saas-rabbitmq.server.content
119 rabbitmq:
120 - cluster: white-rabbit
121 - vhost: dev
122 - user: notifications-windriver
123 - permissions:
124 configure: ^amq\.gen.*$
125 read: ^(amq\.gen.*|notifications)$
126 write: ^amq\.gen.*$
127 - user_permissions
128 - order: 10011

Switch to Python syntax for roles/saas-rabbitmq/server/content.sls as we've too many for loops,
and as hunting Jinja issues like '#' routing key is parsed as None isn't fun.

It's more easy to read the loops in configure_cluster function than in the previous Jinja/YAML format.

Provide a module and states to query Management API. The module could be generic enough to commit
upstream to Salt. The states could be, but as they're already the rabbitmqctl ones, could be confusing.
In both cases, we've to handle our credential for rabbitmq.user_present from Vault separetely than the upstream verion.

Tested: vhost, exchange, queue

Still to do: user, binding, permission

dereckson added inline comments.
roles/saas-rabbitmq/server/content.sls
146

We can directly use this dictionary.

Use directly privilege in configure_user_permission

User tested. Still to do: binding, permission

Added the mission execution module methods (untested code): list_bindings queue_bind exchange_bind update_permissions. TODO: test, write state module methods for those.

Final version, idempotence checked, see P312.

dereckson retitled this revision from WIP: Provision RabbitMQ configuration to Provision RabbitMQ configuration.Feb 24 2023, 19:12
This revision is now accepted and ready to land.Feb 27 2023, 21:22

Change on operations/_tests/modules/test_rabbitmq.py

Declare notifications exchange as durable

This revision is now accepted and ready to land.Mar 2 2023, 16:25

Wearg queue disappeared on reboot

pillar/saas/rabbitmq.sls
74

Should be durable too.

declare wearg-notifications as durable

This revision is now accepted and ready to land.Mar 31 2023, 21:06

Last tweaks, e.g. read durable parameters and adjust permissions like currently deployed

This revision was landed with ongoing or failed builds.Mar 31 2023, 21:23
This revision was automatically updated to reflect the committed changes.