Page Menu
Home
DevCentral
Search
Configure Global Search
Log In
Files
F3767514
No One
Temporary
Actions
View File
Edit File
Delete File
View Transforms
Subscribe
Mute Notifications
Award Token
Flag For Later
Size
11 KB
Referenced Files
None
Subscribers
None
View Options
diff --git a/Makefile b/Makefile
index 239fb9b..42c1103 100644
--- a/Makefile
+++ b/Makefile
@@ -1,66 +1,67 @@
# -------------------------------------------------------------
# Salt - Operations repository
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Project: Nasqueron
# Description: Allow to generate repository or API content
# License: Trivial work, not eligible to copyright
# -------------------------------------------------------------
HOST_NAME != hostname -s
HOST_DOMAIN != hostname -d
API_DIR=/var/wwwroot/$(HOST_DOMAIN)/$(HOST_NAME)/datasources/infra
RM=rm -f
MKDIR=mkdir -p
MV=mv
# -------------------------------------------------------------
# Main targets
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
_default: repo
all: repo api
clean: clean-repo clean-api
test:
(cd _tests && make)
# -------------------------------------------------------------
# Build targets - repository
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
repo: roles/webserver-content/init.sls \
roles/webserver-core/nginx/files/ocsp-ca-certs.pem \
.git/hooks/pre-commit
-roles/webserver-content/init.sls: webserver-content-index
+webserver-content-index: clean-webserver-content-index roles/webserver-content/init.sls
-webserver-content-index:
- tmpfile=`mktemp /tmp/make-rOPS-generate-webcontent-index.XXXXXX` ; \
- utils/generate-webcontent-index.py > "$$tmpfile" ;\
- ${MV} "$$tmpfile" roles/webserver-content/init.sls
+roles/webserver-content/init.sls:
+ utils/generate-webcontent-index.py > roles/webserver-content/init.sls
roles/webserver-core/nginx/files/ocsp-ca-certs.pem:
utils/generate-ocsp-bundle.sh > roles/webserver-core/nginx/files/ocsp-ca-certs.pem
.git/hooks/pre-commit:
pre-commit install
-clean-repo:
- ${RM} roles/webserver-content/init.sls .git/hooks/pre-commit
+clean-webserver-content-index:
+ ${RM} roles/webserver-content/init.sls
+
+clean-repo: clean-webserver-content-index
+ ${RM} .git/hooks/pre-commit
${RM} roles/webserver-core/nginx/files/ocsp-ca-certs.pem
# -------------------------------------------------------------
# Build targets - API
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
api: $(API_DIR)/all-states.json
$(API_DIR)/all-states.json:
${MKDIR} ${API_DIR}
utils/show-local-states.py > ${API_DIR}/all-states.json
clean-api:
${RM} ${API_DIR}/all-states.json
diff --git a/README.md b/README.md
index 291d788..52dd1fa 100644
--- a/README.md
+++ b/README.md
@@ -1,110 +1,112 @@
Nasqueron operations
====================
Welcome to **[rOPS](https://devcentral.nasqueron.org/diffusion/OPS/)**,
the Nasqueron operations repository.
----------
Introduction
------------
Nasqueron infrastructure servers support our budding community
of creative people, writers, developers and thinkers.
Nasqueron follows the principle of "Infrastructure as Code"
to offer documentation, reproducibility, transparency and
to allow external contributions.
It contains:
- server configuration
- deployment information for our applications and services
We mainly rely on [Salt](https://docs.saltproject.io/en/latest/contents.html)
for deployment and automation.
Scope
-----
Both Nasqueron servers and side projects we manage are in scope.
For example, the [Eglide](http://www.eglide.org/) service is configured
through roles/core (common to every server) and roles/shellserver (specific).
Structure
---------
A. Services are organized in roles and units.
* Roles: a role is a goal a service accomplishes (e.g. mailserver, paas-docker)
* Units: a unit is a component needed to achieve this goal
(e.g. an userland software collection, a nginx server)
Directories follow `roles/<role>/<unit>`.
Those files are known a **states**.
If configuration files for a unit should be stored,
a subfolder `files` is created at unit level.
If anything escape to the role and unit logic organization,
like for CVE hotfixes, `hotfixes/` directory is used.
B. Structured data is stored in the `pillar/` directory.
Those files are known as **pillar**.
This data is structured as we found suitable to express it
cleanly, and queried from states, directly or through functions.
C. States should mostly be easy-to-read templates, without any more
programmation than simple if and for templating logic.
If more is needed, functions are created in Salt custom modules:
* Execution modules are stored in `_modules/`
* States modules are stored in `_states/`
D. This repository is the source of truth for users, groups and ports:
* UIDs document unique usernames and the UIDs for system accounts
* GIDs document the same information for the groups
* PORTS contain the list of reserved application ports
E. Units and integration tests are stored in `_tests/`
+F. Resources used by automated tasks are stored in `_resources/`
+
Contribute
----------
Contributions are welcome to this repository, especially if you wish to:
1. improve our infrastructure
2. install or configure something on a Nasqueron server
3. install or configure something on a project we manage (like Eglide)
4. help to migrate services to Salt
You can follow this [contributor guide](https://agora.nasqueron.org/How%20to%20contribute%20code)
to send a commit for review. This procedure is open to everyone.
Issues can be reported on the [#Servers component](https://devcentral.nasqueron.org/tag/servers/)
on DevCentral, the Nasqueron Phabricator instance.
Support for contributors is provided on Libera #nasqueron-ops.
Inclusive terminology
---------------------
The repository uses the following terminology:
- **Salt primary server**: server that issues commands to other servers, including itself
- **Node**: a server, baremetal or VM configured by Salt
License
-------
A lot of configuration as code is trivial, and so ineligible for copyright per
[threshold of originality](https://en.wikipedia.org/wiki/Threshold_of_originality)
When this is not the case, the code is licensed under
[BSD-2-Clause](https://opensource.org/licenses/BSD-2-Clause)
if not otherwise specified.
diff --git a/_resources/headers/webserver-content-init b/_resources/headers/webserver-content-init
new file mode 100644
index 0000000..b332609
--- /dev/null
+++ b/_resources/headers/webserver-content-init
@@ -0,0 +1,11 @@
+# -------------------------------------------------------------
+# Salt — Webserver content
+# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
+# Project: Nasqueron
+# License: Trivial work, not eligible to copyright
+# -------------------------------------------------------------
+#
+# <auto-generated>
+# This file is auto-generated by running
+# `make webserver-content-index`.
+# </auto-generated>
diff --git a/_tests/Makefile b/_tests/Makefile
index 9c811d2..59c1c62 100644
--- a/_tests/Makefile
+++ b/_tests/Makefile
@@ -1,16 +1,17 @@
test: test-python test-bats
test-python:
python -m unittest discover modules
python -m unittest discover pillar
python -m unittest discover scripts/python
test-bats:
bats scripts/bats/test_edit_acme_dns_accounts.sh
+ bats roles/bats/test_webserver_content.sh
# -------------------------------------------------------------
# Configuration test specific to the primary server
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
test-config:
cd config/salt-primary && make test
diff --git a/_tests/roles/bats/test_webserver_content.sh b/_tests/roles/bats/test_webserver_content.sh
new file mode 100755
index 0000000..25ee716
--- /dev/null
+++ b/_tests/roles/bats/test_webserver_content.sh
@@ -0,0 +1,10 @@
+#!/usr/bin/env bats
+
+SCRIPT="utils/generate-webcontent-index.py"
+INDEX="roles/webserver-content/init.sls"
+
+@test "Ensure webserver-content index is up-to-date" {
+ cd ..
+ run diff -u <($SCRIPT) <(cat $INDEX)
+ [ "$status" -eq 0 ]
+}
diff --git a/roles/webserver-content/init.sls b/roles/webserver-content/init.sls
index 694fcd6..03be2f7 100644
--- a/roles/webserver-content/init.sls
+++ b/roles/webserver-content/init.sls
@@ -1,24 +1,28 @@
# -------------------------------------------------------------
# Salt — Webserver content
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-# Project: Eglide
-# Created: 2017-11-23
+# Project: Nasqueron
# License: Trivial work, not eligible to copyright
# -------------------------------------------------------------
+#
+# <auto-generated>
+# This file is auto-generated by running
+# `make webserver-content-index`.
+# </auto-generated>
include:
- .be/dereckson
- .org/eglide
- .org/nasqueron/api
- .org/nasqueron/autoconfig
- .org/nasqueron/daeghrefn
- .org/nasqueron/docs
- .org/nasqueron/infra
- .org/nasqueron/labs
- .org/nasqueron/rain
- .org/nasqueron/social
- .org/wolfplex/api
- .org/wolfplex/www
- .space/hypership
- ._generic
diff --git a/utils/generate-webcontent-index.py b/utils/generate-webcontent-index.py
index 5205b6c..887d88e 100755
--- a/utils/generate-webcontent-index.py
+++ b/utils/generate-webcontent-index.py
@@ -1,79 +1,80 @@
#!/usr/bin/env python3
# -------------------------------------------------------------
# rOPS — regenerate roles/webserver-content/init.sls
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
# Project: Nasqueron
# Created: 2017-11-24
# Description: Read the web_content_sls pillar entry
# and regenerate the webserver-content include.
# License: BSD-2-Clause
# -------------------------------------------------------------
import yaml
# -------------------------------------------------------------
# Table of contents
# -------------------------------------------------------------
#
# :: Configuration
# :: Update code
# :: Run task
#
# -------------------------------------------------------------
# -------------------------------------------------------------
# Configuration
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
config = {
+ "header": "_resources/headers/webserver-content-init",
"pillar": "pillar/webserver/sites.sls",
"states": "roles/webserver-content/init.sls",
}
# -------------------------------------------------------------
# Update code
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-def do_update(pillar_file, file_to_update):
- print_header(file_to_update)
+def do_update(header_file, pillar_file, file_to_update):
+ print_file(header_file)
+
print("\ninclude:")
for site in get_sites(pillar_file):
print(" - {}".format(site))
print("")
print(" - ._generic")
def get_pillar_entry(pillar_file, key):
with open(pillar_file) as fd:
pillar = yaml.safe_load(fd.read())
return pillar[key]
def get_sites(pillar_file):
sites = get_pillar_entry(pillar_file, "web_content_sls")
return sorted(set(
[site for sublist in [sites[role] for role in sites] for site in sublist]
))
-def print_header(file_to_update):
- with open(file_to_update) as fd:
+def print_file(file_path):
+ with open(file_path) as fd:
for line in fd:
if not line.startswith("#"):
break
print(line, end="")
-
# -------------------------------------------------------------
# Run task
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
if __name__ == "__main__":
- do_update(config["pillar"], config["states"])
+ do_update(config["header"], config["pillar"], config["states"])
File Metadata
Details
Attached
Mime Type
text/x-diff
Expires
Mon, Nov 25, 00:56 (17 h, 9 m)
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
2259298
Default Alt Text
(11 KB)
Attached To
Mode
rOPS Nasqueron Operations
Attached
Detach File
Event Timeline
Log In to Comment