Development

This commit is contained in:
Matthew Grotke 2026-06-06 14:55:29 -04:00
parent 33ec9e7f1c
commit 0cec7d69c9
12 changed files with 124 additions and 92 deletions

View file

@ -21,13 +21,13 @@ app/ Python source (baked into image)
action.py Flask Blueprint for POST actions on this page
page.js? Optional page-specific client-side JS (auto-included if present)
sanitize.py Input sanitization (strips dangerous characters)
config_utils.py Config I/O, snapshot system, command queue
config_utils.py Config I/O, change history (SQLite), command queue
authorized_accounts.json Web UI user accounts (separate from Routlin users)
data/ Live-mounted at runtime (./data:/data)
styles.css Application stylesheet
common.js Shared client-side interaction logic (all pages)
validation.js Client-side field validation
www/ Live-mounted at runtime (./www:/www)
styles.css Application stylesheet (inlined in dev, served as /www/styles.css in production)
common.js Shared client-side JS (inlined in dev, served as /www/common.js in production)
icons/ SVG icon files (always inlined by factory.py)
# host directory mounted into container: $HOME/routlin -> /routlin_location
routlin_location/ Routlin install dir
@ -36,7 +36,7 @@ routlin_location/ Routlin install dir
core.py Apply script invoked to push config changes to the system
ddns.log DDNS update log
blocklists/ Downloaded blocklist files (*.con)
.snapshots/ Config snapshot JSON files (one per saved change)
.dashboard-snapshots SQLite database storing change history and revert records
.health Health status JSON written by Routlin
.dashboard-queue Pending commands waiting to be applied
.dashboard-done Record of completed commands
@ -63,7 +63,7 @@ Files and directories under `/routlin_location` that the app reads or writes:
| `core.py` | exec | Routlin's apply script. The app invokes it as `python3 core.py --apply` or `--update-blocklists` to push config changes to the system. |
| `ddns.log` | read | DDNS update log shown in the DDNS page. |
| `blocklists/` | read | Directory of downloaded blocklist files (`*.con`). The app reads them to count entries and report last-updated timestamps. |
| `.snapshots/` | read/write | Directory of config snapshot JSON files. One file per saved change, used for the change history and revert feature. |
| `.dashboard-snapshots` | read/write | SQLite database storing change history groups and field-level diffs, used for the change history and revert feature. |
| `.health` | read | JSON file written by Routlin describing service and configuration health status. The dashboard reads it to display the health panel and auto-queue fixes. |
| `.dashboard-queue` | read/write | Pending commands waiting to be applied. |
| `.dashboard-done` | read/write | Record of completed commands. |
@ -185,7 +185,7 @@ See `TYPES.md` for the full set of item types and their fields. See `TYPES.md#ac
from pathlib import Path
from flask import Blueprint, request, redirect, flash
from auth import require_level
from config_utils import load_config, save_config_with_snapshot, verify_config_hash
from config_utils import load_config, record_group, diff_fields, verify_config_hash
import sanitize
import validation as validate
@ -255,13 +255,9 @@ def addip_add():
flash(msg, 'error')
return redirect(f'/{_PAGE}')
# 4. Save with a snapshot entry.
flash(save_config_with_snapshot(
cfg,
path='banned_ips', key=ip, operation='add',
before=None, after=entry,
description=f'Added banned IP: {ip}',
), 'success')
# 4. Record, save, and queue.
changes = diff_fields({}, entry)
flash(record_group(cfg, 'banned_ips', 'ip', ip, changes, 'core apply'), 'success')
return redirect(f'/{_PAGE}')
```
@ -270,7 +266,7 @@ Rules:
- `flash()` is the only feedback mechanism. Use `'error'` or `'success'` as the category.
- Parse all user input before checking the config hash. If input is invalid you want to bail out cheaply before acquiring anything.
- Always call `validate.validate_config(cfg)` on the mutated config before saving, even for simple operations.
- `save_config_with_snapshot` returns a human-readable success message string suitable for passing directly to `flash()`.
- `record_group(cfg, parent_path, item_key, item_value, changes, cmd)` saves the config, records the change in the SQLite history, queues the command, and returns a human-readable flash message string. Use `diff_fields(before, after)` to produce the `changes` argument.
### Row index actions
@ -310,10 +306,11 @@ ip = validate.banned_ip(raw) # returns Non
### Checkbox fields
Checkboxes are absent from the form body when unchecked. Always compare to `'on'`:
Checkboxes are absent from the form body when unchecked. Both forms are equivalent and acceptable:
```python
enabled = request.form.get('enabled') == 'on'
enabled = request.form.get('enabled') == 'on' # explicit value comparison
logging = 'logging' in request.form # presence check
```
### JSON fields from record_editor
@ -358,7 +355,7 @@ Valid levels: `nothing`, `viewer`, `administrator`, `manager`. The decorator enf
`load_config()` reads `config.json` fresh on every call. Do not cache it across a request.
`save_config_with_snapshot(cfg, path, key, operation, before, after, description)` writes the config and records a snapshot entry for the change history. Always pass `before` and `after` as the specific sub-object that changed, not the whole config.
`record_group(cfg, parent_path, item_key, item_value, changes, cmd)` saves the config, records a change group in the SQLite history database, and queues the apply command. `diff_fields(before, after)` computes the field-level diff to pass as `changes`. Always take a `copy.deepcopy` of the relevant sub-object before mutating so `before` is accurate.
`verify_config_hash(hash)` confirms the config has not changed since the form was rendered. Always check it before mutating the config.
@ -366,15 +363,19 @@ Valid levels: `nothing`, `viewer`, `administrator`, `manager`. The decorator enf
## Client-Side Validation
Attach a `validate` field to any `field` item in `content.json` to enable live validation:
Attach a `validate` field to any `field` item in `content.json` to enable live validation. The value is one or more `VALIDATION_*` flag names joined by `|`:
```json
{ "type": "field", "name": "subnet", "validate": "ipv4" }
{ "type": "field", "name": "nat_ip", "validate": "VALIDATION_IPV4_FORMAT|VALIDATION_UNRESTRICTED" }
```
See `TYPES.md#validation` for the full table of validate values and what each accepts.
Factory converts the flag names to a bitmask integer stored in `data-validate`. The client-side `bigValidate()` function reads this mask and runs only the enabled checks on every keystroke, marking the field valid (green), incomplete (yellow), or invalid (red). The form submit button stays disabled until all validated fields pass.
For `record_editor` fields that need subnet-aware validation, use `valtype` instead of `validate` and wire `data-dep-subnet` / `data-dep-mask` attributes via the `attrs` field.
See `TYPES.md#validation` for the full flag table.
`VALIDATION_UNRESTRICTED` is special: when present, factory automatically reads the current restricted VLAN subnets from config and injects them into `data-existing-ids` on the input. No extra token is needed.
For `record_editor` fields, use `validate` (or the legacy alias `valtype`) on each field definition. The same `VALIDATION_*` flags apply.
For data that must be passed to a JS validator (such as existing VLAN IDs for uniqueness checking), use a `data-*` HTML attribute on the input rather than a global JS variable. The `existing_ids` field on a `field` item emits `data-existing-ids` on the rendered input.
@ -391,7 +392,9 @@ app/pages/actions/page.js -- history revert button state
app/pages/networklayout/page.js -- is_vpn -> mdns_reflection sync
```
`page.js` runs after `common.js` and `validation.js`, so all shared utilities (`htmlEsc`, `showCard`, `tablePickerCloseAll`, etc.) are available.
`page.js` runs after `common.js`, so all shared utilities (`htmlEsc`, `showCard`, `tablePickerCloseAll`, etc.) are available.
In production mode (`PRODUCTION_MODE=1` env var), `common.js` and `styles.css` are served as separate files from `/www/` with `Cache-Control: public, max-age=86400` and an mtime-based `?v=` query string for cache busting. `page.js` and the generated `bigValidate()` function are always inlined regardless of mode.
Rules for what belongs in `page.js` vs `common.js`:

View file

@ -318,7 +318,7 @@ Form-group layout with a badge showing interface state (`UP`/`DOWN`/`INVALID`/ot
"scroll_to_bottom?": true
}
```
`<pre>` block for log output. Setting `scroll_to_bottom: true` adds a `data-scroll-bottom` attribute that JS uses to auto-scroll.
`<pre>` block for log output. Lines do not wrap; content scrolls horizontally when lines exceed the container width. Setting `scroll_to_bottom: true` adds a `data-scroll-bottom` attribute that JS uses to auto-scroll to the bottom on load and on update.
---
@ -507,47 +507,40 @@ Group of hidden credential sub-groups (API token vs. No-IP username/password). J
## Validation
The `validate` field attaches a named validator to an input. Validators run live as the user types, show inline hint messages, and keep the form's submit button disabled until all validated fields pass.
The `validate` field attaches one or more `VALIDATION_*` flags to an input. Validators run live as the user types, show inline hint messages, and keep the form's submit button disabled until all validated fields pass. Flags are combined with `|`:
Three mechanisms exist depending on context:
```json
{ "type": "field", "name": "nat_ip", "validate": "VALIDATION_IPV4_FORMAT|VALIDATION_UNRESTRICTED" }
```
- `field`, `editable_list`: sets `data-validate` on the input. The named classifier runs on every keystroke and marks the field valid, incomplete (warning), or invalid (error).
- `overridable_textarea`: sets `data-validate-lines` on the textarea. When the override checkbox is checked, each non-empty line is validated as an IP address. If subnet context is available (from sibling subnet/mask inputs), each line is also checked against that subnet.
- `record_editor` fields: use `valtype` instead of `validate`. The field is wired to sibling subnet/mask inputs via `data-dep-subnet` and `data-dep-mask` attributes.
All three contexts use the same flag system:
### `validate` values (for `field` and `editable_list`)
- `field`, `editable_list`: sets `data-validate` (bitmask integer) on the input. Validation fires on every keystroke.
- `overridable_textarea`: sets `data-validate-lines`. When the override checkbox is checked, each non-empty line is validated individually.
- `record_editor` fields: `validate` (or its legacy alias `valtype`) sets `data-validate` on the per-field input.
| Value | Accepts |
|-------|---------|
| `ipv4` | IPv4 address. Four dot-separated octets, each 0-255. |
| `mac` | MAC address. Six colon-separated two-digit hex groups. |
| `url` | HTTP or HTTPS URL. Must begin with `http://` or `https://`, have a valid hostname, and an optional port in range 1-65535. |
| `port` | Port number. Integer 1-65535, digits only. |
| `ipv4cidr` | IPv4 address or CIDR notation. Accepts a bare IP or `IP/prefix` where prefix is 0-32. |
| `endpoint` | Network endpoint. Accepts an IPv4 address, IPv6 address, or hostname. Hostnames may include a `:port` suffix. |
| `dashname` | Dash-delimited identifier. Lowercase letters, digits, and hyphens. No leading, trailing, or consecutive hyphens. |
| `domainname` | Domain name. Letters, digits, hyphens, and dots. Each label must not start or end with a hyphen. |
| `networkname` | Network identifier. Letters, digits, hyphens, and underscores. No leading, trailing, or consecutive special characters. |
| `time_24h` | 24-hour time. Format `HH:MM`. Hours 00-23, minutes 00-59. |
| `positive_int` | Positive integer. Digits only. Respects the field's `min` and `max` bounds when present. |
| `vlan_id` | VLAN ID. Integer 1-4094. Also checks uniqueness against existing IDs read from the input's `data-existing-ids` attribute. |
`number` input_type defaults to `VALIDATION_RANGE_INT` when no `validate` is specified.
### `validate` value (for `overridable_textarea`)
### `VALIDATION_*` flags
| Value | Behavior |
|-------|---------|
| `ip_in_subnet` | Each non-empty line must be a valid IPv4 address. When the VLAN subnet and mask are both filled in, also verifies each address falls within that subnet. |
### `valtype` values (for `record_editor` fields)
`valtype` wires subnet-aware real-time validation using sibling inputs pointed to by `data-dep-subnet` and `data-dep-mask` attributes on the field.
| Value | Validates |
|-------|---------|
| `address` | IPv4 host address. Must be a valid IP within the known subnet, and must not be the network or broadcast address. |
| `subnet` | IPv4 subnet address. All host bits must be zero for the given mask. |
| `mask` | Prefix length. Integer 1-30. |
| `format` | IPv4 address format only. No subnet membership check. |
| Flag | Accepts |
|------|---------|
| `VALIDATION_IPV4_FORMAT` | IPv4 address. Four dot-separated octets, each 0-255. |
| `VALIDATION_IPV6_FORMAT` | IPv6 address. |
| `VALIDATION_SUBNET` | IPv4 subnet address. All host bits must be zero for the prefix length implied by sibling inputs. |
| `VALIDATION_ADDRESS` | IPv4 host address. Must fall within the subnet implied by sibling inputs and must not be the network or broadcast address. |
| `VALIDATION_MAC` | MAC address. Six colon-separated two-digit hex groups. |
| `VALIDATION_URL` | HTTP or HTTPS URL. Must begin with `http://` or `https://` and have a valid hostname. |
| `VALIDATION_PORT` | Port number. Integer 1-65535. |
| `VALIDATION_DASH_NAME` | Lowercase letters, digits, and hyphens. No leading, trailing, or consecutive hyphens. |
| `VALIDATION_NETWORK_NAME` | Letters, digits, hyphens, and underscores. No leading, trailing, or consecutive special characters. |
| `VALIDATION_DOMAIN_NAME` | Domain name. Labels of letters, digits, and hyphens separated by dots. No label may start or end with a hyphen. |
| `VALIDATION_TIME24H` | 24-hour time. Format `HH:MM`. Hours 00-23, minutes 00-59. |
| `VALIDATION_RANGE_INT` | Integer. Respects the field's `min` and `max` bounds when present. Default for `number` input_type. |
| `VALIDATION_IPV4_CIDR` | Strict CIDR. A prefix is always required (`192.168.1.0/24`). Host bits must be zero. |
| `VALIDATION_IPV4_CIDRFLEX` | Flexible CIDR. Accepts a bare IPv4 address (yellow/incomplete when last octet is 0) or `IP/prefix`. |
| `VALIDATION_UNRESTRICTED` | IPv4 address must not fall within any restricted VLAN subnet. Factory automatically injects the current restricted subnets into `data-existing-ids` on the input. |
| `VALIDATION_IP_OR_DOMAIN_NAME` | IPv4 address or hostname/domain name. |
---

View file

@ -839,10 +839,10 @@ def load_datasource(spec):
def collect_layout_tokens(cfg):
import license
import settings as settings
net = cfg.get('network_interfaces', {})
return {
'GENERAL_LAN_INTERFACE': str(net.get('lan_interface', '-')),
'VPN_VLAN_COUNT': str(sum(1 for v in cfg.get('vlans', []) if v.get('is_vpn'))),
'PRO_LICENSE_JS': 'true' if license.is_pro() else 'false',
'PRO_LICENSE_JS': 'true' if settings.is_pro() else 'false',
}

View file

@ -15,10 +15,19 @@ from config_utils import (
_is_locked, _lock_mtime, _entry_ts_from_queue,
)
PAGES_DIR = os.path.join(APP_DIR, 'pages')
NAVBAR_FILE = os.path.join(APP_DIR, 'navbar.json')
CSS_FILE = os.path.join(DATA_DIR, 'styles.css')
COMMON_JS_FILE = os.path.join(DATA_DIR, 'common.js')
import settings as settings
PAGES_DIR = os.path.join(APP_DIR, 'pages')
NAVBAR_FILE = os.path.join(APP_DIR, 'navbar.json')
CSS_FILE = os.path.join(WWW_DIR, 'styles.css')
COMMON_JS_FILE = os.path.join(WWW_DIR, 'common.js')
def _file_version(path):
try:
return int(os.path.getmtime(path))
except OSError:
return 0
# Constants ===========================================================
@ -91,21 +100,20 @@ def load_icon(name):
return ''
def inline_js(page_name=None):
big_validate_js = build_big_validate()
try:
with open(COMMON_JS_FILE) as f:
app_js = f.read()
except Exception:
app_js = ''
page_js = ''
if page_name:
page_js_path = os.path.join(PAGES_DIR, page_name, 'page.js')
parts = [build_big_validate()]
if not settings.is_production():
try:
with open(page_js_path) as f:
page_js = f.read()
with open(COMMON_JS_FILE) as f:
parts.append(f.read())
except Exception:
pass
return big_validate_js + '\n' + app_js + ('\n' + page_js if page_js else '')
if page_name:
try:
with open(os.path.join(PAGES_DIR, page_name, 'page.js')) as f:
parts.append(f.read())
except Exception:
pass
return '\n'.join(parts)
# Utilities ===========================================================
@ -1512,7 +1520,6 @@ def build_item(item, tokens, inherited_req=None):
# Layout renderer =====================================================
def render_layout(view_id, content_html, tokens, page_name=None):
css = load_css()
level = client_level()
has_pending_alert = not _apply_changes_immediately() and bool(get_dashboard_pending())
titlebar_html = f'<div class="titlebar"><span class="titlebar-brand">{WEB_APP_DISPLAY_NAME}</span></div>'
@ -1636,17 +1643,27 @@ def render_layout(view_id, content_html, tokens, page_name=None):
'</div>\n'
)
if settings.is_production():
css_ver = _file_version(CSS_FILE)
js_ver = _file_version(COMMON_JS_FILE)
css_tag = f' <link rel="stylesheet" href="/www/styles.css?v={css_ver}">\n'
common_js = f'<script src="/www/common.js?v={js_ver}"></script>\n'
else:
css_tag = f' <style>{load_css()}</style>\n'
common_js = ''
return (
'<!DOCTYPE html>\n<html lang="en">\n<head>\n'
' <meta charset="UTF-8"/>\n'
' <meta name="viewport" content="width=device-width, initial-scale=1.0"/>\n'
f' <title>{WEB_APP_DISPLAY_NAME}</title>\n'
f' <style>{css}</style>\n'
f'{css_tag}'
'</head>\n<body>\n'
f'{titlebar_html}\n'
f'{navbar_html}\n'
f'<main class="main-content">\n{pending_bar}{problem_bars}{other_bars}{content_html}\n</main>\n'
f'{footer_html}\n'
f'{common_js}'
f'<script>var CONFIG_HASH="{page_hash}";var LAN_IFACE="{lan_iface}";var VPN_VLAN_COUNT={vpn_count};var APPLY_UUID={json.dumps(my_uuid)};</script>\n'
f'<script>{inline_js(page_name)}</script>\n'
'</body>\n</html>'

View file

@ -1,2 +0,0 @@
def is_pro():
return True

View file

@ -1,14 +1,15 @@
import os, json, sys, importlib.util as _importlib_util
from flask import Flask, Blueprint, session, redirect, get_flashed_messages
from flask import Flask, Blueprint, session, redirect, get_flashed_messages, send_from_directory
from markupsafe import Markup
from config_utils import (
ACCOUNTS_FILE, APP_DIR, CONFIGS_DIR, HEALTH_FILE,
ACCOUNTS_FILE, APP_DIR, CONFIGS_DIR, HEALTH_FILE, WWW_DIR,
load_config, queue_command, _find_cmd_in_queues,
)
from factory import (
LEVEL_RANK, PAGES_DIR, e, client_level, passes, build_items,
load_json, render_layout,
)
import settings as settings
from pages.actions.action import bp as actions_bp
from pages.bannedips.action import bp as bannedips_bp
from pages.ddns.action import bp as ddns_bp
@ -35,6 +36,16 @@ from api_apply_health import bp as api_apply_health_bp
app = Flask(__name__)
app.secret_key = os.environ.get('SECRET_KEY', os.urandom(24))
# Static www/ serving =================================================
@app.route('/www/<path:filename>')
def serve_www(filename):
response = send_from_directory(WWW_DIR, filename)
if settings.is_production():
response.cache_control.max_age = 86400
response.cache_control.public = True
return response
# View blueprint ======================================================
bp = Blueprint('view_page', __name__)

View file

@ -8,11 +8,11 @@ from auth import require_level
from config_utils import load_config, record_group, diff_fields, verify_config_hash
import sanitize
import mod_validation as validate
import license
import settings as settings
_PAGE = Path(__file__).parent.name
PRO_LICENSE = license.is_pro()
PRO_LICENSE = settings.is_pro()
bp = Blueprint(_PAGE, __name__)

View file

@ -1,9 +1,9 @@
import json
from config_utils import collect_layout_tokens, load_datasource
from factory import load_json, build_table, table_token_key, iter_table_items, PAGES_DIR
import license
import settings as settings
PRO_LICENSE = license.is_pro()
PRO_LICENSE = settings.is_pro()
def collect_tokens(cfg):

View file

@ -8,11 +8,11 @@ from flask import Blueprint, request, redirect, flash, send_file, abort, jsonify
from auth import require_level
from config_utils import CONFIGS_DIR, load_config, record_group, diff_fields
import mod_validation as validate
import license
import settings as settings
_PAGE = Path(__file__).parent.name
PRO_LICENSE = license.is_pro()
PRO_LICENSE = settings.is_pro()
bp = Blueprint(_PAGE, __name__)

View file

@ -1,9 +1,9 @@
import json
import os
from config_utils import collect_layout_tokens, CONFIGS_DIR
import license
import settings as settings
PRO_LICENSE = license.is_pro()
PRO_LICENSE = settings.is_pro()
RADIUS_LOG_MAX = 50
RADIUS_LOG_FILE = '/var/log/freeradius/radius.log'

View file

@ -0,0 +1,9 @@
import os
def is_production():
return os.environ.get('PRODUCTION_MODE', '').lower() in ('1', 'true', 'yes')
def is_pro():
return bool(os.environ.get('LICENSE', '').strip())

View file

@ -25,4 +25,5 @@ services:
- SMTP_USER=grotek.industries@gmail.com
- SMTP_PASSWORD=lfhrygyuwvlaczaw
- SMTP_FROM=grotek.industries@gmail.com
- LICENSE=
restart: unless-stopped