Development

README.md

@@ -1,6 +1,6 @@
-# Linux Home Router Suite
+# Routlin
 
-A collection of Python scripts that transform an existing Linux server (with at least two Ethernet NICs) into a fully featured home router - eliminating the need for a separate router appliance.
+A collection of Python scripts that transform an existing Linux server (with at least two Ethernet NICs) into a fully featured home router, eliminating the need for a separate router appliance.
 
 ## Why Replace Your Router?
 
@@ -14,7 +14,7 @@ Consumer and prosumer router appliances are constrained by OEM firmware. Securit
 
 ## Summary
 
-These scripts do not run continuously in the background.  They simply install and facilitate the configuration of battle-hardened software (`dnsmasq`, for DHCP and DNS, `nftables` for firewall and NAT, `chrony` for NTP, `freeradius` for RADIUS, `avahi` for mDNS discovery, and `wireguard` for VPN) by using intuitive JSON files that you can edit.  The scripts also install systemd timers to run periodic activities: updating the DNS blocklist(s) of your choice (default once per day), optionally checking if your external IP address changed (default every 5 mins) and if so, updating a DDNS provider.
+These scripts do not run continuously in the background. They install and facilitate the configuration of battle-hardened software (`dnsmasq` for DHCP and DNS, `nftables` for firewall and NAT, `chrony` for NTP, `freeradius` for RADIUS, `avahi` for mDNS discovery, and `wireguard` for VPN) using JSON files that you edit. The scripts also install systemd timers to run periodic activities: updating the DNS blocklist(s) of your choice (default once per day), and optionally checking if your external IP address changed (default every 5 mins) and if so, updating a DDNS provider.
 
 ---
 
@@ -41,8 +41,8 @@ The suite is organized into three independent but complementary scripts, each ma
 - Enforces inter-VLAN isolation by default (forward chain policy drop); specific cross-VLAN traffic is permitted via `inter_vlan_exceptions`
 - Masquerades outbound traffic for all non-WireGuard VLANs automatically
 - Auto-detects active container bridge interfaces (Docker, Podman, libvirt, etc.) and adds forward rules so VLAN clients can reach containerized services
-- Auto-detects active container bridge interfaces and adds DNS listening on each bridge IP, so containers can reach the local DNS resolver during builds and at runtime (container services, e.g. Docker, Podman, must be running at the time of `--apply`)
+- Auto-detects active container bridge interfaces and adds DNS listening on each bridge IP, so containers can reach the local DNS resolver during builds and at runtime
-- Installs a `systemd` boot service (`core-nat.service`) to re-apply firewall rules on every boot
+- Installs a `systemd` boot service (`routlin-nat.service`) to re-apply firewall rules on every boot
 - Co-exists with Docker (does not touch Docker-managed `nat`/`filter` tables)
 - Generates FreeRADIUS `clients.conf` and `users` files from `core.json` reservations, enabling dynamic VLAN assignment via MAC Authentication Bypass (MAB) for both wired (802.1X) and wireless clients
 - Manages a `.radius-secret` shared secret file (generated automatically on first `--apply` if RADIUS is enabled)
@@ -50,9 +50,9 @@ The suite is organized into three independent but complementary scripts, each ma
 
 ### Optional: WireGuard VPN (managed by `core.py` and the dashboard)
 
-- Supports any number of WireGuard interfaces defined in `core.json` (any VLAN with an interface name starting with `wg`)
+- Supports any number of WireGuard interfaces defined in `core.json` (any VLAN with `is_vpn: true`)
 - `core.py --apply` generates the server keypair on first run, writes the server conf to `/etc/wireguard/`, and brings the interface up with `wg-quick`. Subsequent applies sync peer changes live without restarting the interface
-- Peer management is done through the router dashboard: add a peer, set its IP and tunnel mode, and the dashboard generates and downloads the ready-to-import client `.conf` file immediately — the private key is never stored
+- Peer management is done through the Routlin Dashboard: add a peer, set its IP and tunnel mode, and the dashboard generates and downloads the ready-to-import client `.conf` file immediately - the private key is never stored
 - Peer data (name, IP, public key, enabled state) is stored directly in `core.json` alongside the rest of the network config
 - Supports per-peer choice of split-tunnel (VPN subnet only) or full-tunnel (all traffic) routing
 - Reports active peer connections, handshake times, and RX/TX byte counts on the dashboard VPN view
@@ -65,11 +65,18 @@ The suite is organized into three independent but complementary scripts, each ma
 - Installs a `systemd` timer that runs every 5 minutes by default
 - Logs all updates and errors to `ddns.log`
 
+### Optional: Routlin Dashboard
+
+- Web UI for managing all aspects of the router (VLANs, reservations, blocklists, VPN, DDNS, firewall, and more) without editing JSON by hand
+- Runs as a Docker container alongside the existing scripts
+- Changes made in the dashboard are queued and applied to the live system automatically via a 1-minute systemd timer
+- Set up via `install.py`, which configures SMTP, generates a secret key, and starts the container
+
 ---
 
 ## Software Dependencies
 
-These packages are required. `core.py --install` checks that they are installed and will prompt you to install any that are missing.
+These packages are required. `install.py` checks that they are installed and will prompt to install any that are missing.
 
 | Dependency | Purpose | Required By |
 |---|---|---|
@@ -81,6 +88,7 @@ These packages are required. `core.py --install` checks that they are installed
 | `freeradius` | RADIUS server for dynamic VLAN assignment via MAC auth | `core.py` |
 | `avahi-daemon` | mDNS reflector for cross-VLAN service discovery | `core.py` |
 | `wireguard-tools` | WireGuard VPN (`wg`, `wg-quick`) | `core.py` (when WireGuard VLANs are configured) |
+| `docker` | Runs the Routlin Dashboard container | `install.py` (dashboard only) |
 
 ---
 
@@ -117,6 +125,10 @@ All configuration lives in two JSON files. Edit these to match your network befo
 |---|---|
 | `.radius-secret` | Shared secret between FreeRADIUS and RADIUS clients (APs, switches). Generated automatically on first `--apply` when RADIUS is enabled. Root-owned intentionally. |
 | `.wg-<iface>.pub` | WireGuard server public key per interface (e.g. `.wg-wg0.pub`). Written by `core.py --apply`; read by the dashboard to embed in client config downloads. |
+| `.dashboard-queue` | Pending apply commands written by the dashboard; consumed by the 1-minute timer. |
+| `.dashboard-done` | UUIDs of already-processed queue entries; prevents duplicate execution. |
+| `.dashboard-last-run` | Epoch timestamp of the last timer execution. |
+| `.dashboard-lock` | PID lock file preventing concurrent timer runs. |
 | `.ddns-last-ip-*` | Cached public IP per DDNS provider. Managed by `ddns.py`. |
 | `.ddns-last-service` | Tracks IP-check service rotation. Managed by `ddns.py`. |
 
@@ -140,25 +152,23 @@ Edit the top-level blocks:
 
 Edit the `vlans` array to match your network topology. For each VLAN:
 
-- Set `vlan_id` to a unique integer (`1` = untagged physical interface, all others are 802.1Q tagged)
+- Set `subnet` and `subnet_mask`. The VLAN ID is derived automatically from the subnet: for a `/24` it is the third octet (e.g. `192.168.10.0/24` -> VLAN ID `10`); for a `/16` it is the second octet. Ensure this matches the 802.1Q tag configured on your switch. VLAN ID `1` (e.g. `192.168.1.0/24`) is treated as the untagged physical interface.
-- Set `interface` to the NIC name for VLAN 1 (e.g. `enp6s0`); sub-interfaces are named automatically (e.g. `enp6s0.10`). For WireGuard VLANs, use `wg0`, `wg1`, etc.
+- For VLAN 1 (the untagged interface), the physical NIC name is taken from your `general.wan_interface` sibling - set `interface` in `general` to the LAN-facing NIC (e.g. `enp6s0`). Sub-interfaces for all other VLANs are named automatically (e.g. `enp6s0.10`).
 - Set `radius_default` to `true` on exactly one VLAN - unknown MACs will be placed here (typically guest). All other VLANs set this to `false`.
 - Set `use_blocklists` to a list of blocklist names for this VLAN - leave empty for unfiltered DNS
 - Set `server_identities` to the IPs the router itself will hold on this VLAN. The lowest last-octet IP is auto-used as gateway, DNS, and NTP server unless overridden in `dhcp_information.explicit_overrides`.
-- Set `subnet` and `subnet_mask` at the top level of the VLAN object
 - Set `dhcp_information` fields: pool start/end, `lease_time`, and optionally `explicit_overrides` for gateway, dns_server, or ntp_server
 - Add `reservations` for devices that need a known VLAN assignment by MAC address. The `ip` field is optional:
   - Omit `ip`, set it to `""`, or set it to `"dynamic"` to let DHCP assign from the pool (hostname is still set)
   - Set `ip` to a specific address outside the dynamic pool to pin the device to that IP
   - Set `radius_client: true` on any device (AP, switch) that will authenticate other devices via RADIUS
 - Add per-VLAN `port_wrangling` entries to redirect DNS or NTP requests to the local resolver
-- For WireGuard VLANs, include a `vpn_information` block instead of `dhcp_information` and `server_identities`, and a `peers` array instead of `reservations`. Peer management (add, edit, regenerate conf, delete) is done through the dashboard:
+- For WireGuard VLANs, set `is_vpn: true` and include a `vpn_information` block instead of `dhcp_information` and `server_identities`, and a `peers` array instead of `reservations`. WireGuard interface names (`wg0`, `wg1`, ...) are assigned automatically in ascending order of VLAN ID. Peer management (add, edit, regenerate conf, delete) is done through the dashboard:
 
 ```json
 {
-  "vlan_id": 40,
+  "is_vpn": true,
   "name": "vpn",
-  "interface": "wg0",
   "subnet": "192.168.40.0",
   "subnet_mask": "255.255.255.0",
   "radius_default": false,
@@ -173,7 +183,7 @@ Edit the `vlans` array to match your network topology. For each VLAN:
     "explicit_overrides": { "gateway": "", "dns_server": "", "mtu": "" }
   },
   "peers": [],
-  "port_wrangling": [...]
+  "port_wrangling": []
 }
 ```
 
@@ -258,9 +268,10 @@ Configure mDNS reflection with the top-level `mdns_reflection` block in `core.js
 
 ### 2. Edit DDNS Configuration (`ddns.json`)
 
-- Set `provider` to `noip` or `duckdns`
+- Set `provider` to `noip`, `duckdns`, or `cloudflare`
 - For No-IP: set `username`, `password`, and the `hostnames` array
 - For DuckDNS: set `token` and the `subdomains` array
+- For Cloudflare: set `api_token` and the relevant zone/record details
 - Set `timer_interval` to how often the IP should be checked (default: `5m`)
 - The `ip_check_services` list is used in rotation to detect your current public IP - the defaults can be left as-is
 
@@ -269,37 +280,33 @@ Configure mDNS reflection with the top-level `mdns_reflection` block in `core.js
 ## Initial Deployment
 
 ```bash
-sudo python3 core.py --install            # Check and install required packages
+sudo python3 install.py               # Install required packages; optionally set up dashboard and HTTPS
-sudo python3 core.py --apply              # Apply VLANs, DHCP, DNS, firewall, RADIUS, mDNS, timers
+sudo python3 core.py --apply          # Apply VLANs, DHCP, DNS, firewall, RADIUS, mDNS, timers
 sudo python3 core.py --update-blocklists  # Download and apply blocklists
 ```
 
 Optional (if DDNS is desired):
 
 ```bash
-sudo python3 ddns.py --start              # Run an immediate IP update and install the update timer
+sudo python3 ddns.py --start          # Run an immediate IP update and install the update timer
 ```
 
 Optional (if WireGuard VPN is desired):
 
-1. Add a WireGuard VLAN to `core.json` with `interface: "wg0"` (see configuration example above)
+1. Add a WireGuard VLAN to `core.json` with `is_vpn: true` (see configuration example above)
-2. Run `sudo python3 core.py --apply` — this generates the server keypair, writes `/etc/wireguard/wg0.conf`, and brings the interface up
+2. Run `sudo python3 core.py --apply` - this generates the server keypair, writes `/etc/wireguard/wg0.conf`, and brings the interface up
-3. Add peers using one of the two methods below, then run `sudo python3 core.py --apply` again to sync them to the live interface
+3. Add peers through the dashboard (each peer triggers an immediate `.conf` file download), then `--apply` again to sync them to the live interface
-
-**With the router dashboard:**
-Open the VPN view, fill in the Server Endpoint (your public hostname or IP), and add peers — each peer triggers an immediate `.conf` file download ready to import into any WireGuard client.
 
 **Without the dashboard (`create_vpn_peer.py`):**
 
 ```bash
 python3 create_vpn_peer.py --name laptop --ip 192.168.40.2
 python3 create_vpn_peer.py --name laptop --ip 192.168.40.2 --iface wg0
-python3 create_vpn_peer.py --name laptop --ip 192.168.40.2 --vlan-id 40
 python3 create_vpn_peer.py --name phone  --ip 192.168.40.3 --split-tunnel
 python3 create_vpn_peer.py --name tablet --ip 192.168.40.4 --output ~/tablet.conf
 ```
 
-The script reads the specified WireGuard VLAN from `core.json`, validates the IP against the VLAN subnet, generates a keypair, appends the peer to `core.json`, and writes the client `.conf` file. If the config has exactly one WireGuard VLAN, `--iface` and `--vlan-id` are optional. Transfer the `.conf` to the peer device by secure means, then delete it from the server.
+The script reads the specified WireGuard VLAN from `core.json`, validates the IP against the VLAN subnet, generates a keypair, appends the peer to `core.json`, and writes the client `.conf` file. If the config has exactly one WireGuard VLAN, `--iface` is optional. Transfer the `.conf` to the peer device by secure means, then delete it from the server.
 
 ---
 
@@ -307,12 +314,19 @@ The script reads the specified WireGuard VLAN from `core.json`, validates the IP
 
 All scripts are designed to be run multiple times - re-running `--apply` replaces the previous configuration safely.
 
+### install.py
+
+```
+sudo python3 install.py
+```
+
+Interactive setup wizard. Detects the Linux package manager, installs required system packages, and optionally sets up the Routlin Dashboard (Docker container with SMTP configuration) and external HTTPS access via Caddy. Safe to re-run: skips already-installed packages and prompts before reconfiguring an existing dashboard.
+
 ### core.py
 
 Commands that modify system state require `sudo`. Read-only commands do not.
 
 ```
-sudo python3 core.py --install              # Check and interactively install required packages
 sudo python3 core.py --apply                # Apply full config: networkd, dnsmasq, nftables, RADIUS, mDNS, timer, boot service
 sudo python3 core.py --apply --dry-run      # Preview --apply actions without making changes
 sudo python3 core.py --update-blocklists    # Download and merge blocklists, then --apply
@@ -333,18 +347,15 @@ python3 core.py --view-metrics              # Lifetime DNS metrics across all VL
 Does not require `sudo`. Requires `wireguard-tools` (`wg` must be on PATH) and a prior `core.py --apply` to generate the server keypair.
 
 ```
-python3 create_vpn_peer.py --name NAME --ip IP [--iface IFACE | --vlan-id ID] [--split-tunnel] [--output FILE]
+python3 create_vpn_peer.py --name NAME --ip IP [--iface IFACE] [--split-tunnel] [--output FILE]
 
   --name NAME        Peer name (e.g. laptop)
   --ip IP            Peer IP within the VPN subnet (e.g. 192.168.40.2)
-  --iface IFACE      WireGuard interface to add the peer to (e.g. wg0)
+  --iface IFACE      WireGuard interface to add the peer to (e.g. wg0); optional if only one WireGuard VLAN exists
-  --vlan-id ID       VLAN ID of the WireGuard VLAN (e.g. 40); alternative to --iface
   --split-tunnel     Route only VPN subnet traffic through the tunnel (default: full tunnel)
   --output FILE      Output path for the client .conf file (default: vpn-client-<name>.conf)
 ```
 
-`--iface` and `--vlan-id` are mutually exclusive. Both are optional when the config contains exactly one WireGuard VLAN.
-
 ### ddns.py
 
 Only `--start` and `--disable` require `sudo` as they install/remove systemd timer files. All other commands run as a normal user.