These conventions apply to Salt state and configuration-management code in this repository.
### State dependencies
-Salt executes states in file order, so execution order within an SLS file is deterministic.
+Salt executes states in file order, so the execution order within an SLS file is deterministic.
Do **not** add `require`, `watch`, `onchanges`, or similar requisites merely to express the dependency between successive states in the same file.
Prefer the simplest code that relies on the file's execution order.
Use requisites when they provide actual behavior that cannot be expressed by file order, especially when a service must be reloaded or restarted when its configuration changes.
For example, if a configuration file should trigger a service reload when changed, use an appropriate requisite rather than relying only on the order of the states.
### Managed file contents
For `file.managed`, store repository-managed file contents in the corresponding `files/` directory.
Do **not** put file contents directly in the `contents` parameter when the content belongs in the repository.
Add a header to repository-managed files that clearly identifies them as being managed by this repository.
For example:
```yaml
/etc/example.conf:
file.managed:
- source: salt://path/to/files/example.conf
```
with the repository file containing an appropriate repository-management header.
### Template file extensions
When a file is a template, normally preserve the original file extension.
For example:
```text
files/foo.conf
files/script.sh
files/config.yaml
```
-If adding Jinja syntax would cause the file to be incorrectly handled by a shell, Python, YAML, or other language linter, append `.jinja` to the original extension:
+If adding Jinja syntax causes the file to be incorrectly handled by a shell, Python, YAML, or other language linter, append `.jinja` to the original extension:
```text
files/foo.sh.jinja
files/config.yaml.jinja
```
Use `.jinja`, **not `.j2`**.
The extension in the repository should reflect the language of the deployed file. When a command or script is deployed without its source-language extension, keep the language extension in the repository but omit it from the deployed filename.
For example:
```text
files/foo.sh
```
may be deployed as:
```text
/usr/local/bin/foo
```
### State block names
When a state deploys a file to a specific path, use that destination path as the state ID.
For example:
```yaml
/etc/example/example.conf:
file.managed:
- source: salt://example/files/example.conf
```
The state ID is therefore the actual path being managed.
Do not introduce an arbitrary descriptive state ID such as:
```yaml
deploy_example_config:
file.managed:
- name: /etc/example/example.conf
```
when the state manages a file at a precise path.
The state ID is implicitly the `name` parameter when `name` is not specified, so using the path directly keeps the state concise and makes the managed resource immediately visible.
Use a descriptive state ID when there are duplicates.
### Template variables
Make template variables visible from the .sls file that uses the template.
Do **not** query Pillar data, grains, or Salt functions directly from inside a Jinja template.
Instead, resolve the values in the `context` of the `file.managed` state and pass them explicitly to the template.
Prefer:
```yaml
/etc/example/example.conf:
file.managed:
- source: salt://example/files/example.conf.jinja
- context:
hostname: {{ grains['host'] }}
setting: {{ pillar['example']['setting'] }}
```
with the template using:
```jinja
hostname = {{ hostname }}
setting = {{ setting }}
```
rather than having the template directly access:
```jinja
{{ grains['host'] }}
{{ pillar['example']['setting'] }}
```
or invoke Salt functions itself.
This makes the inputs to a template visible where the state is declared, simplifies the template, and makes dependencies on external Salt data explicit.
diff --git a/README.md b/README.md
index 3992d22..932dc45 100644
--- a/README.md
+++ b/README.md
@@ -1,181 +1,181 @@
Nasqueron operations
====================
Welcome to **[rOPS](https://devcentral.nasqueron.org/diffusion/OPS/)**,
the Nasqueron operations repository.
------------
Introduction
------------
This repository contains the configuration of our infrastructure servers,
mostly relying on [Salt](https://docs.saltproject.io/en/latest/contents.html)
for deployment and automation.
Additional utilities for humans and Terraform provisioning are also provided.
Nasqueron infrastructure servers support our community
of creative people, writers, developers, and thinkers.
Nasqueron follows the principle of *Infrastructure as Code*
to offer documentation, reproducibility, transparency and
to allow external contributions.
-----------------
Content and scope
-----------------
The repository contains:
- server configuration
- deployment information for our applications and services
Combining this repository with NetBox, the goal is to constitute
an exhaustive inventory and source of truth for our infrastructure.
Both Nasqueron servers and side projects we manage are in scope.
For example, the [Eglide](http://www.eglide.org/) service is configured through
`roles/core` (common to every server) and `roles/shellserver` (specific).
---------
Structure
---------
This repository is organized as follows:
### A. Services
Services are organized in roles and units.
* **Roles:** a role is a full high-level service (e.g., mailserver, paas-docker)
* **Units:** a unit is a component to achieve the service's goals
- (e.g., a userland software collection, a nginx server)
+ (e.g., a userland software collection, an nginx server)
Directories follow `roles/<role>/<unit>`.
Those files are known as **states** or **formulas**.
They are generally written in YAML, with Jinja2 templating.
If configuration files for a unit should be stored,
a subfolder `files/` is created at unit level.
The `top.sls` file defines which states are loaded by server,
while the `services.sls` describes service configurations
that aren't server-specific.
The `map.jinja` file is used to provide “Omni-OS” configuration:
as we deploy onto several Linux distributions (Debian, Rocky) and FreeBSD,
we need to provide a way to map packages and directories.
*Note:* If anything escapes to the role and unit logic organization,
like for CVE hotfixes, `hotfixes/` directory is used.
### B. Structured data
Structured data is stored in the `pillar/` directory.
Those files are known as **pillar**.
This data is structured as we found suitable to express it
cleanly, and queried from states, directly or through functions.
Each pillar .sls file is directly written in YAML.
The `top.sls` file defines which pillar files are loaded by server,
while the `tower.sls` file acts similarly and allows more flexibility.
### C. SaltStack modules
While states offer powerful templating capabilities, they can easily become
too complex to maintain when evolving from declarations to full programs.
For this reason, we use Salt modules to express more complex logic.
Modules are written in Python. The Python functions can directly be called
from states.
The modules are stored in the following directories:
* Execution modules are stored in `_modules/`
* States modules are stored in `_states/`
As any program, modules should be tested. See section E.
### D. UNIX resource assignments
This repository is the source of truth for service accounts, groups, and ports:
* `pillar/core/ids.sls` maps unique usernames and groups with their UIDs/GIDs
* `PORTS` contains the list of reserved application ports
When a service needs one of these resources, assign it in the corresponding file.
### E. Tests
Unit and integration tests are stored in `_tests/`
You can test the correctness of several repository aspects:
- Salt configuration files in `_tests/config/`
- Salt execution modules in `_tests/modules/`
- Salt pillar in `_tests/pillar/`
- Scripts deployed within this repository in `_tests/scripts/`
If you add a new directory, you should add a corresponding entry in `Makefile`.
### F. Resources
Resources used by automated tasks to generate repository files are stored
in `_resources/`, for example, templates for new services.
It also contains JSON schemas to validate key pillar data.
### G. Utilities
Scripts and other utilities to be run by Nasqueron Operations SIG members
are stored in `utils/`.
----------
Contribute
----------
Contributions are welcome to this repository, especially if you wish to:
1. improve our infrastructure
2. install or configure something on a Nasqueron server
3. install or configure something on a project we manage (like Eglide)
4. automate deployment of one of your Nasqueron projects
You can follow this [contributor guide](https://agora.nasqueron.org/How%20to%20contribute%20code)
to send a commit for review. This procedure is open to everyone.
Issues can be reported on the [#Servers component](https://devcentral.nasqueron.org/tag/servers/)
on DevCentral, the Nasqueron Phabricator instance.
Documentation about the repository and tips for IDEs are available on Agora: