Development

2026-06-07 23:32:06 -04:00 · 2026-06-07 23:32:06 -04:00 · 64f83d683e
commit 64f83d683e
parent db837af548
8 changed files with 237 additions and 26 deletions
--- a/docker/routlin-dash/STYLE.md
+++ b/docker/routlin-dash/STYLE.md
@ -0,0 +1,415 @@
+# Developer Guide
+
+This guide covers how to add features, write action handlers, and follow the conventions used throughout the codebase.
+
+---
+
+## Architecture Overview
+
+This app is the web UI for Routlin, a Linux-based router software package. It runs as a Docker container and manages the same configuration files that Routlin's core scripts read and apply to the system. The app does not manage the system directly; it edits `config.json` and queues commands that Routlin's `core.py` executes.
+
+The app is a Flask application in a Docker image.
+
+```
+app/                  Python source (baked into image)
+  factory.py          Converts content.json item trees into HTML strings
+  view_page.py        Routes, data loaders, token assembly, layout rendering
+  navbar.json         Navigation structure
+  pages/              One subdirectory per page
+    <pagename>/
+      content.json    Declarative page layout
+      action.py       Flask Blueprint for POST actions on this page
+      page.js?        Optional page-specific client-side JS (auto-included if present)
+  sanitize.py         Input sanitization (strips dangerous characters)
+  config_utils.py     Config I/O, change history (SQLite), command queue
+  authorized_accounts.json  Web UI user accounts (separate from Routlin users)
+
+www/                  Live-mounted at runtime (./www:/www)
+  styles.css          Application stylesheet (inlined in dev, served as /www/styles.css in production)
+  common.js           Shared client-side JS (inlined in dev, served as /www/common.js in production)
+  icons/              SVG icon files (always inlined by factory.py)
+
+                      # host directory mounted into container: $HOME/routlin -> /routlin_location
+routlin_location/    Routlin install dir
+  config.json         Main Routlin configuration (read on every request, written on save)
+  validation.py       Shared validation module (imported via PYTHONPATH)
+  core.py             Apply script invoked to push config changes to the system
+  ddns.log            DDNS update log
+  blocklists/         Downloaded blocklist files (*.con)
+  .dashboard-snapshots  SQLite database storing change history and revert records
+  .health             Health status JSON written by Routlin
+  .dashboard-queue    Pending commands waiting to be applied
+  .dashboard-done     Record of completed commands
+  .dashboard-pending  Commands queued but not yet run
+  .dashboard-last-run Timestamp of the last apply run
+  .dashboard-lock     Lock file indicating an apply is in progress
+  .ddns-last-ip-*     Cached public IP files, one per DDNS provider
+  .ddns-last-service  Timestamp of the last DDNS service check
+  .<iface>.pub        WireGuard public key files (e.g. .wg0.pub)
+```
+
+---
+
+## Shared Resources with Routlin
+
+The Routlin install directory (`$HOME/routlin` on the host) is mounted into the container at `/routlin_location` and added to `PYTHONPATH`. This is the primary integration point between the dashboard and the router software.
+
+Files and directories under `/routlin_location` that the app reads or writes:
+
+| Path | Access | Description |
+|------|--------|-------------|
+| `config.json` | read/write | Main Routlin configuration. The app reads this on every request and writes it on every save action. |
+| `validation.py` | import | Shared validation module imported as `import validation as validate`. Contains field validators and `validate_config()` used by all action handlers. |
+| `core.py` | exec | Routlin's apply script. The app invokes it as `python3 core.py --apply` or `--update-blocklists` to push config changes to the system. |
+| `ddns.log` | read | DDNS update log shown in the DDNS page. |
+| `blocklists/` | read | Directory of downloaded blocklist files (`*.con`). The app reads them to count entries and report last-updated timestamps. |
+| `.dashboard-snapshots` | read/write | SQLite database storing change history groups and field-level diffs, used for the change history and revert feature. |
+| `.health` | read | JSON file written by Routlin describing service and configuration health status. The dashboard reads it to display the health panel and auto-queue fixes. |
+| `.dashboard-queue` | read/write | Pending commands waiting to be applied. |
+| `.dashboard-done` | read/write | Record of completed commands. |
+| `.dashboard-pending` | read/write | Commands that have been queued but not yet run. |
+| `.dashboard-last-run` | read | Timestamp of the last dashboard apply run. |
+| `.dashboard-lock` | read | Lock file checked to determine if an apply is currently in progress. |
+| `.ddns-last-ip-*` | read | One file per DDNS provider, written by Routlin with the last-known public IP. The app reads all matching files to display the current IP. |
+| `.ddns-last-service` | read | Timestamp written by Routlin each time the DDNS service runs. Displayed as "Last checked" on the DDNS page. |
+| `.<iface>.pub` | read | WireGuard public key files (e.g. `.wg0.pub`). Read by the VPN page to display the server public key. |
+
+The container also mounts `/sys/class/net` and `/sys/devices` read-only to read live network interface state (link status, speed, MAC address, MTU), and `/etc/localtime` read-only for correct timezone display.
+
+---
+
+## Code Style
+
+- **ASCII only in all files.** No em dashes, en dashes, curly quotes, or any non-ASCII character. If a sentence needs a pause that would normally use a dash, restructure it.
+- **No comments unless the WHY is non-obvious.** Well-named functions and variables explain the what. Comments belong on hidden constraints, subtle invariants, or workarounds for specific external bugs.
+- **No docstrings on obvious functions.** Reserve them for functions with non-obvious behavior or important invariants to document.
+- **`import validation as validate`** - the module is aliased because `validate` is a common local variable name and shadowing it causes confusion.
+- **Imports at top, no inline imports** except in rare cases where a circular import cannot otherwise be avoided (e.g. the `from flask import abort` inside `serve_view`).
+
+---
+
+## Naming Conventions
+
+### Underscore prefix
+
+Use `_` prefix only for tiny helpers used in exactly one place. Do not use it for:
+- Module-level constants (`NAVBAR_FILE`, not `_NAVBAR_FILE`)
+- Functions reused by more than one caller
+- Imports (`import factory`, not `import factory as _factory`)
+
+Good uses of `_`:
+- A two-line inner function defined inside a larger function and called once
+- A short closure that is never referenced outside its enclosing scope
+
+### File path constants
+
+Define all file paths as named constants at the top of the module, using `os.path.join`:
+
+```python
+PAGES_DIR    = os.path.join(APP_DIR,     'pages')
+NAVBAR_FILE  = os.path.join(APP_DIR,     'navbar.json')
+CSS_FILE     = os.path.join(DATA_DIR,    'styles.css')
+```
+
+Never hardcode paths inline.
+
+### Section headers in Python files
+
+Use the `# Label ===...` style for major section breaks:
+
+```python
+# File loaders ======================================================
+
+# Config data loaders ===============================================
+
+# Routes ============================================================
+```
+
+---
+
+## HTML Building
+
+All HTML in `factory.py` and `view_page.py` is built by string concatenation using f-strings, not a template engine. Every value that comes from user data or config must be passed through `e()` before interpolation:
+
+```python
+from factory import e
+
+html = f'<td class="table-cell">{e(row["description"])}</td>'
+```
+
+`e()` wraps `html.escape()`. Never skip it for user-controlled or config-sourced values.
+
+When building multi-line HTML inside view_page.py for a token, follow the same `e()` discipline and assign the result to `tokens['MY_TOKEN']`. Mark values as `Markup()` only when they are already-safe assembled HTML that must not be double-escaped at the outer render stage.
+
+Key principle: `factory.py` is a pure HTML builder. It has no routing, no data loading, and no Flask context other than reading the session for access level. All data loading and token assembly happens in `view_page.py`. The two modules are kept separate to avoid circular imports; `view_page.py` injects `factory.load_datasource = load_datasource` at startup.
+
+---
+
+## Adding a New Page
+
+### 1. Create the page directory
+
+```
+app/pages/<pagename>/
+  __init__.py         Empty file (makes it a Python package)
+  content.json        Page layout
+  action.py           POST action handlers
+```
+
+`<pagename>` becomes the URL: `/pagename`.
+
+### 2. Write content.json
+
+Every `content.json` starts with a `client_requirement` and an `items` array:
+
+```json
+{
+  "client_requirement": "client_is_viewer+",
+  "items": [
+    {
+      "type": "header_page_title",
+      "items": [
+        { "type": "h1", "text": "My Page" },
+        { "type": "p",  "text": "Description here." }
+      ]
+    }
+  ]
+}
+```
+
+See `TYPES.md` for the full set of item types and their fields. See `TYPES.md#access-control` for the full set of `client_requirement` values.
+
+### 3. Write action.py
+
+```python
+from pathlib import Path
+from flask import Blueprint, request, redirect, flash
+from auth import require_level
+from config_utils import load_config, record_group, diff_fields, verify_config_hash
+import sanitize
+import validation as validate
+
+_PAGE = Path(__file__).parent.name
+bp = Blueprint(_PAGE, __name__)
+```
+
+`_PAGE` is the canonical page name and redirect target. Using `Path(__file__).parent.name` means it stays correct even if the directory is renamed.
+
+### 4. Register the blueprint
+
+In `app/main.py` (or wherever blueprints are registered), add the new action blueprint alongside the existing ones.
+
+### 5. Add page.js (optional)
+
+If the page needs client-side behavior not shared with other pages, create `app/pages/<pagename>/page.js`. It is automatically included - no registration needed. See the "Page-Specific JavaScript" section for guidance on what belongs here vs `common.js`.
+
+---
+
+## Token System
+
+Tokens are `%LIKE_THIS%` placeholders in `content.json` string fields. They are substituted before the page is rendered. Tokens carry dynamic data (config values, computed HTML, JSON arrays) from Python into the declarative layout without embedding Python logic in the JSON.
+
+Tokens are assembled in `view_page.py`'s `collect_tokens()` function:
+
+```python
+tokens['VLAN_SUBNET'] = vlan.get('subnet', '')
+tokens['EXISTING_VLAN_IDS_JSON'] = json.dumps([v['vlan_id'] for v in vlans])
+```
+
+String tokens that resolve to a JSON array or object are automatically parsed back into Python structures by `factory.expand_fields()`, so they can be serialized correctly into `data-fields` attributes.
+
+Token values are not HTML-escaped by default. When a token is used in an HTML attribute or text node, `factory.py` calls `e()` on it. When a token resolves to raw HTML meant for injection (e.g. `PENDING_ACTIONS_HTML`), it is used as-is and must be safe before it enters the token map.
+
+---
+
+## Action Handler Pattern
+
+Every POST action follows the same sequence: parse, validate input, check config hash, mutate config, validate config, save.
+
+```python
+@bp.route('/action/bannedips/addip_add', methods=['POST'])
+@require_level('administrator')
+def addip_add():
+    # 1. Parse and validate individual fields first, before touching config.
+    raw = request.form.get('ip', '').strip()
+    ip = validate.banned_ip(raw)
+    if not ip:
+        flash('Invalid IP address.', 'error')
+        return redirect(f'/{_PAGE}')
+
+    description = sanitize.text(request.form.get('description', ''))
+
+    # 2. Check config hash (stale-config detection / CSRF guard).
+    if not verify_config_hash(request.form.get('config_hash', '')):
+        flash('Configuration was modified by another session. Please refresh and try again.', 'error')
+        return redirect(f'/{_PAGE}')
+
+    # 3. Load, mutate, validate the full config.
+    cfg = load_config()
+    entry = {'ip': ip, 'description': description, 'enabled': True}
+    cfg.setdefault('banned_ips', []).append(entry)
+
+    errors = validate.validate_config(cfg)
+    if errors:
+        for msg in errors:
+            flash(msg, 'error')
+        return redirect(f'/{_PAGE}')
+
+    # 4. Record, save, and queue.
+    changes = diff_fields({}, entry)
+    flash(record_group(cfg, 'banned_ips', 'ip', ip, changes, 'core apply'), 'success')
+    return redirect(f'/{_PAGE}')
+```
+
+Rules:
+- Always redirect back to `/{_PAGE}` after a POST, never render HTML directly.
+- `flash()` is the only feedback mechanism. Use `'error'` or `'success'` as the category.
+- Parse all user input before checking the config hash. If input is invalid you want to bail out cheaply before acquiring anything.
+- Always call `validate.validate_config(cfg)` on the mutated config before saving, even for simple operations.
+- `record_group(cfg, parent_path, item_key, item_value, changes, cmd)` saves the config, records the change in the SQLite history, queues the command, and returns a human-readable flash message string. Use `diff_fields(before, after)` to produce the `changes` argument.
+
+### Row index actions
+
+For table row operations, extract the row index and bounds-check it:
+
+```python
+def _row_index():
+    try:
+        return int(request.form.get('row_index', ''))
+    except (ValueError, TypeError):
+        return None
+
+idx = _row_index()
+if idx is None:
+    flash('Invalid request.', 'error')
+    return redirect(f'/{_PAGE}')
+
+items = cfg.get('my_list', [])
+if idx < 0 or idx >= len(items):
+    flash('Entry not found.', 'error')
+    return redirect(f'/{_PAGE}')
+```
+
+---
+
+## Input Handling
+
+### Sanitize vs. validate
+
+- `sanitize.*` functions strip or normalize raw input so it is safe to store. They do not return errors; they return a cleaned value.
+- `validate.*` functions check whether a value is semantically correct for a specific field. They return a cleaned/normalized value on success or a falsy value (empty string, `None`) on failure.
+
+```python
+description = sanitize.text(request.form.get('description', ''))  # always safe
+ip = validate.banned_ip(raw)                                       # returns None on failure
+```
+
+### Checkbox fields
+
+Checkboxes are absent from the form body when unchecked. Both forms are equivalent and acceptable:
+
+```python
+enabled  = request.form.get('enabled') == 'on'   # explicit value comparison
+logging  = 'logging' in request.form              # presence check
+```
+
+### JSON fields from record_editor
+
+`record_editor` submits its data as a JSON string in a hidden input. Parse it defensively:
+
+```python
+import json
+raw = request.form.get('server_identities', '[]')
+try:
+    identities = json.loads(raw)
+    if not isinstance(identities, list):
+        raise ValueError
+except (ValueError, TypeError):
+    flash('Invalid identity data.', 'error')
+    return redirect(f'/{_PAGE}')
+```
+
+---
+
+## Access Control
+
+### In content.json
+
+Any item can carry `"client_requirement": "client_is_administrator+"`. Items that fail the check are omitted from the rendered output entirely. See `TYPES.md#access-control` for the full table.
+
+The page-level `client_requirement` is inherited by all child items that do not declare their own. This means you can set `client_is_viewer+` at the page level and only gate specific cards or buttons at a higher level.
+
+### In action.py
+
+```python
+@require_level('administrator')
+def my_action():
+    ...
+```
+
+Valid levels: `nothing`, `viewer`, `administrator`, `manager`. The decorator enforces the minimum rank and handles the redirect automatically.
+
+---
+
+## Config Persistence
+
+`load_config()` reads `config.json` fresh on every call. Do not cache it across a request.
+
+`record_group(cfg, parent_path, item_key, item_value, changes, cmd)` saves the config, records a change group in the SQLite history database, and queues the apply command. `diff_fields(before, after)` computes the field-level diff to pass as `changes`. Always take a `copy.deepcopy` of the relevant sub-object before mutating so `before` is accurate.
+
+`verify_config_hash(hash)` confirms the config has not changed since the form was rendered. Always check it before mutating the config.
+
+---
+
+## Client-Side Validation
+
+Attach a `validate` field to any `field` item in `content.json` to enable live validation. The value is one or more `VALIDATION_*` flag names joined by `|`:
+
+```json
+{ "type": "field", "name": "nat_ip", "validate": "VALIDATION_IPV4_FORMAT|VALIDATION_UNRESTRICTED" }
+```
+
+Factory converts the flag names to a bitmask integer stored in `data-validate`. The client-side `bigValidate()` function reads this mask and runs only the enabled checks on every keystroke, marking the field valid (green), incomplete (yellow), or invalid (red). The form submit button stays disabled until all validated fields pass.
+
+See `TYPES.md#validation` for the full flag table.
+
+`VALIDATION_UNRESTRICTED` is special: when present, factory automatically reads the current restricted VLAN subnets from config and injects them into `data-existing-ids` on the input. No extra token is needed.
+
+For `record_editor` fields, use `validate` (or the legacy alias `valtype`) on each field definition. The same `VALIDATION_*` flags apply.
+
+For data that must be passed to a JS validator (such as existing VLAN IDs for uniqueness checking), use a `data-*` HTML attribute on the input rather than a global JS variable. The `existing_ids` field on a `field` item emits `data-existing-ids` on the rendered input.
+
+---
+
+## Page-Specific JavaScript
+
+If a page needs client-side behavior that is not shared with any other page, put it in `app/pages/<pagename>/page.js`. `view_page.py` automatically appends this file to the inline script bundle when it exists.
+
+```
+app/pages/ddns/page.js               -- credential provider toggle
+app/pages/physicalinterfaces/page.js -- interface config card wiring
+app/pages/actions/page.js            -- history revert button state
+app/pages/networklayout/page.js      -- is_vpn -> mdns_reflection sync
+```
+
+`page.js` runs after `common.js`, so all shared utilities (`htmlEsc`, `showCard`, `tablePickerCloseAll`, etc.) are available.
+
+In production mode (`PRODUCTION_MODE=1` env var), `common.js` and `styles.css` are served as separate files from `/www/` with `Cache-Control: public, max-age=86400` and an mtime-based `?v=` query string for cache busting. `page.js` and the generated `bigValidate()` function are always inlined regardless of mode.
+
+Rules for what belongs in `page.js` vs `common.js`:
+
+- **`page.js`**: behavior that only activates on one page (specific field names, CSS classes, or form actions that appear nowhere else).
+- **`common.js`**: shared infrastructure used by multiple pages, or global event handlers (click-to-close, UUID hover highlight, stat card edit mode).
+- **Inline `<script>` from factory**: per-element wiring emitted by factory at render time (form validation, table-picker row wiring). Factory uses `document.currentScript.previousElementSibling` to scope the script to the element it follows.
+
+---
+
+## JS Table Workers
+
+Non-standard `input_type` values in `inline_edit` row actions require a table worker. `factory.py` detects any `input_type` not in `STANDARD_INPUT_TYPES = {'text', 'password', 'number', 'checkbox', 'select', 'textarea'}` and emits a `<script>` block via `build_table_worker_script()` that registers the worker using `registerTableWorker(id, impl)`.
+
+Workers implement:
+- `renderCell(fDef, td, val, row)` - returns `true` if handled, `false` to fall through to default text input
+- `afterRowOpen(tr, row)` - optional; called after all cells are rendered; use it to wire cross-field listeners
+
+The worker ID is derived from the table's `datasource` field by stripping the `config:` or `live:` prefix. The only current non-standard type is `credentials` on the DDNS accounts table.