From c0d961514cf5d01dcb4599244e81b28486068a77 Mon Sep 17 00:00:00 2001
From: alex <alex@pdp7.net>
Date: Tue, 15 Sep 2026 20:41:20 +0200
Subject: Draft /notes/tech/a-monitoring-system

---
 blog/content/notes/index.gmi                    |  1 +
 blog/content/notes/tech/a-monitoring-system.gmi | 53 +++++++++++++++++++++++++
 2 files changed, 54 insertions(+)
 create mode 100644 blog/content/notes/tech/a-monitoring-system.gmi

(limited to 'blog/content/notes')

diff --git a/blog/content/notes/index.gmi b/blog/content/notes/index.gmi
index cafe6192..dea7ed8e 100644
--- a/blog/content/notes/index.gmi
+++ b/blog/content/notes/index.gmi
@@ -45,6 +45,7 @@ Notes about some books and long articles I like:
 => tech/internet-basics Internet basics
 => tech/splicing-mkvs Splicing MKVs
 => tech/what-i-would-like-to-see-in-a-forge What I would like to see in a forge
+=> tech/a-monitoring-system A monitoring system
 => tech/resumes Resumes
 
 ### Gadgets
diff --git a/blog/content/notes/tech/a-monitoring-system.gmi b/blog/content/notes/tech/a-monitoring-system.gmi
new file mode 100644
index 00000000..1a4b29b0
--- /dev/null
+++ b/blog/content/notes/tech/a-monitoring-system.gmi
@@ -0,0 +1,53 @@
+# A monitoring system
+
+Nagios is much-maligned, but pretty much has the most solid design of a monitoring system I have seen:
+
+* Nagios is stable because it is old and focused.
+* Implementing most functionality by launching customizable processes with an argument API makes Nagios very flexible.
+* Configuration as a text file is the best approach.
+
+However, Nagios has some problems:
+
+* The user interface is bad, although there are nicer alternative user interfaces.
+* Metrics collection is an afterthought, and the default implementation is not good.
+* Although Nagios configuration has macros and other features to make the configuration somewhat declarative, it is insufficient and in most complex configurations, using a program to generate the configuration is necessary. Command definition is clunky.
+* The host/service distinction is likely unnecessary, only dependencies are needed.
+* The API for commands is a bit clunky and error-prone. (Generating perfdata and status lines is not straightforward. Return code as command status is error-prone to implement in most programming languages.)
+
+I think this can be improved by:
+
+* Instead of providing a user interface, provide an API as a first-class citizen.
+* Do not implement metrics. Metrics collection is better implemented separately.
+* Make the configuration a JSON-like format that can be generated easily.
+* Only implement checks, dependencies and alerting rules.
+* Use JSON output as the main API for commands.
+
+## Check API
+
+Check plugins should output a JSON blob through stdout.
+
+The output looks like:
+
+```
+{
+  "status": "OK"|"WARNING"|"CRITICAL"|"UNKNOWN",
+  "description": "free text here",
+  "extra": {any JSON value here}
+}
+```
+
+If the process return code is not 0, then the status is equivalent to:
+
+```
+{
+  "status": "UNKNOWN",
+  "description": "command {command} exited with return code {return code}\n\nstdout:\n\n{stdout truncated to a reasonable size}\n\nstderr:\n\n{stderr truncated to a reasonable size}",
+  "extra": {
+    "return-code": "{return-code}",
+    "stdout": "{stdout truncated to a reasonable size}",
+    "stderr": "{stderr truncated to a reasonable size}"
+  }
+}
+```
+
+An adapter executable can be provided to adapt Nagios plugins to the new API.
-- 
cgit v1.2.3