root@debian-server:~$ Dominique Blake-Hofer

Linux, Shell & Tech Notes

Termnoc

[date: 2026-10-20]

TermNOC: Lightweight TUI Fleet Orchestration & Real-Time Infrastructure Monitoring

A terminal-native Operations Center designed for high-density, real-time infrastructure observability and fleet management. Built on Debian GNU/Linux, Python, and the Rich framework, TermNOC delivers actionable telemetry across local KVM hypervisors, bare-metal nodes, remote production environments, and managed customer fleets without the overhead of heavy web GUIs.

---

1. Overview & Philosophy

Modern infrastructure monitoring often suffers from bloat: high-overhead web stacks, brittle JavaScript frontends, and resource-heavy agents that consume memory simply to report state.

TermNOC takes an engineering-first, terminal-driven approach:

  • Zero-Agent Architecture: Uses standard system protocols (ICMP, TCP, SSH, libvirt, wp-cli) instead of proprietary daemons on target nodes.
  • TUI-Centric Visualization: Optimized for vertical (portrait/pivot) displays to maximize readable terminal rows and track hundreds of services at a glance.
  • Low Impact, Decoupled Polling: Protects services from connection fatigue by separating low-latency health checks from heavier inventory audits.
  • Fast Virtual Machine Provisioning: Automates VM creation using netboot and local Debian package caching (apt-cacher-ng).
  • Instant Incident Dispatch: Direct, native integration with Discord webhooks with granular alert policies and automatic node recovery detection.
+-------------------------------------------------------------------------+
|                        TermNOC Operations Center                        |
|                                                                         |
|   +---------------------------------+ +-----------------------------+   |
|   | termnoc-display (Rich TUI 24/7) | | termnoc-menu (CLI Ops Tool) |   |
|   +---------------------------------+ +-----------------------------+   |
+-------------------------------------------------------------------------+
                                     │
           ┌─────────────────────────┼─────────────────────────┐
           ▼                         ▼                         ▼
  +-----------------+       +-----------------+       +-----------------+
  | libvirtd RPC    |       | Network Telemetry|       | Remote Engine   |
  | (qemu:///system)|       | (ICMP, TCP Sync)|       | (SSH / WP-CLI)  |
  +-----------------+       +-----------------+       +-----------------+
           │                         │                         │
  ┌────────┴────────┐       ┌────────┴────────┐       ┌────────┴────────┐
  ▼                 ▼       ▼                 ▼       ▼                 ▼
Local VMs       Hypervisor  LAN Nodes     Gateways    Customer Clusters Web Fleet

---

2. Core Architecture

TermNOC is split into two primary components that operate seamlessly together:

2.1 termnoc-display (Visual Real-Time TUI)

A continuous, event-driven terminal dashboard rendered via Python's rich library. Designed to run as an unmanaged foreground process or as a supervised systemd service on /dev/tty1 over rotated hardware monitors.

Asynchronous Polling Tiers:

  • Host Connectivity (3s Interval): High-frequency ICMP pings determine node availability without saturating connections.
  • KVM Lifecycle (3s Interval): Direct RPC inspection of domains via libvirt to report dynamic state (running, paused, shut off).
  • Service Ports & Host Telemetry (60s Interval): Evaluates service listeners (HTTP, HTTPS, SSH, Webmin/Virtualmin, MySQL) and reads CPU load, RAM allocation, storage capacity, and hardware thermals. Includes automatic connection retries to eliminate false-positive alerts on transient network delays.
  • Application Fleet & WordPress (60m Interval): Low-frequency remote execution checks core status, plugin updates, and theme security patches via wp-cli over secure SSH tunnels.

2.2 termnoc-menu (Fleet Orchestrator)

The interactive administrative console for day-to-day operations:

  • One-click VM lifecycle management (start, shutdown, reboot, destroy).
  • Direct out-of-band console access to guest virtual machines via virtual serial cables (virsh console).
  • Fully automated guest creation via virt-install, kickseeded with Debian preseed files and routed through a local APT mirror.
  • Mass-execution of system upgrades and package maintenance across defined server tiers.

---

3. Configuration & Inventory Schema

TermNOC configures its entire fleet via human-readable YAML manifests located in /etc/termnoc/.

Directory Layout

/etc/termnoc/
├── alerts.yaml        # Discord Webhook & notification rules
├── types.yaml         # Port profiles & server templates
├── local.yaml         # Hypervisor & local KVM guest mappings
├── lan.yaml           # Internal network devices & bare-metal boxes
├── remote.yaml        # Offsite nodes, public servers & VPN tunnels
├── customer.yaml      # Client clusters & business-critical systems
└── wordpress.yaml     # Managed WordPress instances & WP-CLI paths

Manifest Specifications

alerts.yaml

Controls incident alerting logic and webhook destinations.

enabled: true
webhook_url: "https://discord.com/api/webhooks/your/webhook/token"
bot_name: "TermNOC Monitor"
notify_on_node_down: true
notify_on_recovery: true
notify_on_wp_core_update: true
notify_on_wp_plugins_update: false
kvm_alerts_default: false
kvm_alerts_whitelist:
  - "dns-primary"
  - "db-cluster-01"

types.yaml

Defines service port signatures to monitor for various server roles.

types:
  default:
    ports:
      SSH: 22
  web_server:
    ports:
      SSH: 22
      HTTP: 80
      HTTPS: 443
  virtualmin:
    ports:
      SSH: 22
      HTTPS: 443
      VMIN_UI: 10000
      SMTP: 25
  database:
    ports:
      SSH: 22
      MYSQL: 3306

local.yaml

Maps the physical hypervisor and tracks KVM virtual machines.

servers:
  - name: "OBERWIL_CH"
    host: "127.0.0.1"
    type: "hypervisor"
    auth: "none"
    alerts: true

  - name: "dns-primary"
    host: "local-vm"
    type: "kvm_guest"
    alerts: true

wordpress.yaml

Monitors remote CMS environments for uptime, SSL connectivity, and core/plugin/theme patch statuses.

sites:
  - name: "Production-Shop"
    host: "192.168.86.70"
    domain: "shop.example.com"
    auth: "password"
    ssh_user: "adm_web"
    ssh_port: 22
    ssh_pass: "VaultStoredPassword"
    wp_path: "/var/www/shop.example.com/htdocs"
    wp_cli_bin: "/usr/local/bin/wp"
    wp_user: "www-data"
    http_port: 443
    alerts: true

---

4. Operator Manual

Running TermNOC Interactively

To start the dashboard within any terminal session:

termnoc-display

Note: Ensure your terminal emulator supports UTF-8 and 256 colors/Truecolor.

To launch the management interface:

termnoc-menu

Running as a Dedicated Display Appliance (24/7 Vertical Monitor)

TermNOC can turn an inexpensive secondary monitor or a wall-mounted pivot display into an autonomous Operations Center running on Debian's text console (/dev/tty1).

Step 1: Optimize Console Fonts

Install high-legibility fixed-width fonts:

sudo apt update && sudo apt install -y fonts-terminus console-setup

Configure /etc/default/console-setup:

ACTIVE_CONSOLES="/dev/tty[1-6]"
CHARMAP="UTF-8"
CODESET="guess"
FONTFACE="Terminus"
FONTSIZE="8x16"

Apply the font:

sudo setupcon

Step 2: Configure Systemd Service

Mask standard login prompts on tty1 and create the TermNOC service unit:

sudo systemctl stop getty@tty1.service
sudo systemctl mask getty@tty1.service

Write /etc/systemd/system/termnoc-display.service:

[Unit]
Description=TermNOC TUI Operations Center Dashboard (tty1 Portrait)
After=network-online.target libvirtd.service
Wants=network-online.target libvirtd.service
Conflicts=getty@tty1.service

[Service]
Type=simple
User=root
WorkingDirectory=/etc/termnoc

# Rotate display (3 = 270 degrees / Portrait)
ExecStartPre=-/bin/sh -c 'if [ -f /sys/class/graphics/fbcon/rotate ]; then echo 3 > /sys/class/graphics/fbcon/rotate; fi'

# Load console font on tty1
ExecStartPre=-/bin/sh -c 'setupcon --force /dev/tty1 2>/dev/null || true'

# Disable screen blanking & DPMS power-down
ExecStartPre=-/bin/sh -c 'setterm --blank 0 --powerdown 0 </dev/tty1 >/dev/tty1 2>/dev/null || printf "\033[9;0]\033[14;0]" > /dev/tty1'

# Launch TermNOC
ExecStart=/usr/local/bin/termnoc-display

# Bind directly to tty1
StandardInput=tty
StandardOutput=tty
StandardError=journal
TTYPath=/dev/tty1
TTYReset=yes
TTYVHangup=yes
TTYVTDisallocate=yes

Environment=TERM=linux
Environment=LANG=de_CH.UTF-8
Environment=LC_ALL=de_CH.UTF-8
Environment=PYTHONUNBUFFERED=1

Restart=always
RestartSec=3

[Install]
WantedBy=multi-user.target

Enable and launch:

sudo systemctl daemon-reload
sudo systemctl enable --now termnoc-display.service

---

5. Automated Display Sleep Schedules (DPMS & DRM)

To prolong monitor panel life, TermNOC includes scripts for scheduled power management without halting background poller tasks or alert dispatchers.

/usr/local/bin/termnoc-screen-off

#!/bin/sh
setterm --powerdown 1 </dev/tty1 >/dev/tty1 2>/dev/null
setterm --blank force </dev/tty1 >/dev/tty1 2>/dev/null

for conn in /sys/class/drm/card*-*/status; do
    if [ -f "$conn" ] && grep -q "^connected" "$conn" 2>/dev/null; then
        echo "off" > "$conn" 2>/dev/null
    fi
done

/usr/local/bin/termnoc-screen-on

#!/bin/sh
for conn in /sys/class/drm/card*-*/status; do
    if [ -f "$conn" ]; then
        echo "detect" > "$conn" 2>/dev/null
    fi
done

setterm --blank poke </dev/tty1 >/dev/tty1 2>/dev/null
setterm --powerdown 0 </dev/tty1 >/dev/tty1 2>/dev/null
setterm --blank 0 </dev/tty1 >/dev/tty1 2>/dev/null

Scheduling with Cron

# Sleep display at 23:00, wake display at 06:30
00 23 * * * /usr/local/bin/termnoc-screen-off
30 06 * * * /usr/local/bin/termnoc-screen-on

---

6. Summary of Capabilities

| Dimension | Implementation Details |

|---|---|

| Interface | Low-latency TUI using terminal-native cell buffers via rich |

| Orientation | Optimized for 9:16 vertical / portrait displays (1080×1920) |

| Hypervisor Integration | Native KVM / QEMU telemetry via libvirtd system RPC (qemu:///system) |

| Application Layer | Native WordPress update auditing (core, plugins, themes) using wp-cli |

| Alerting | Event-driven Discord Webhook engine with auto-recovery and incident states |

| Provisioning Integration | Automated netboot VM builds backed by local caching (apt-cacher-ng) |

| Footprint | Negligible CPU footprint; zero client-side monitoring daemons required |

TermNOC provides system administrators, operations leads, and self-hosters with a reliable, distraction-free cockpit that brings bare-metal precision back to infrastructure observability.