diff --git a/PLAN.md b/PLAN.md index 5a18397..a364090 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1482,57 +1482,56 @@ must be fixed, redeployed uncommitted, and the affected in-car test repeated bef and **ready-to-install** sign-off. Afterward, the owner handles permanent installation outside the phase roadmap. -### Phase 25.5: Device profile & configuration sync — 🚧 IN PROGRESS (config file + settings UI + server sync planning) -**Config file + Settings UI implemented; server sync planned for Phase 25.5a.** Replace hardcoded values with `config/device.json`, editable via Settings page on Pi. Server sync is eventual-consistency, connection-driven (not time-driven) to handle offline-first car scenarios. +### Phase 25.5a: Device configuration sync backend — ✅ COMPLETE (connection-driven server sync) +**Server API and Pi sync thread implemented.** Server stores device config; Pi fetches when internet available and applies changes without restart. -#### Current state (implemented) -- **`config/device.json`** (new): stores device identity fields: - - `owner_name`: "quoije" (displayed in topbar) - - `car_model`: "Yaris 2013" (displayed in sidebar) - - `device_name`: "Yaris OBD" (reserved for portal) - - `avatar_path`: "img/yaris.png" (sourced by template) -- **`modules/app_device_config.py`** (new): manages device config via SQLite: - - `get_device_config()`: returns current config + sync_enabled flag - - `update_device_config()`: persists changes to SQLite, enforces read-only when sync_enabled - - Fallback to JSON file values if database empty -- **Settings page UI** (new "Device identity" section): - - Sync toggle: "Sync with server" checkbox (if checked, fields become read-only) - - Input fields: owner_name, car_model, avatar_path, device_name (disabled when sync_enabled) +#### Implemented (Phase 25.5a) +- **Database migrations** (`server/sql/migrate_phase25_5a.sql`): + - Add `owner_name`, `car_model`, `device_name`, `avatar_path` to Postgres `devices` table + - Add `config_version` and `config_updated_at` for change detection + +- **Server API endpoints** (`server/modules/device_config.py`): + - `GET /api/device-config` (device auth): Pi fetches config from server + - `POST /api/device-config` (user auth): Portal/admin updates device config - Validation: 1-100 chars for names, 1-255 for paths -- **API endpoint `/api/device-config`** (new): - - GET: returns current config - - POST: validates and updates config; rejects edits if sync_enabled=true -- **Offline-first design**: All values loaded from local JSON at startup; no network required to boot with customized identity. Database acts as live override during Settings page edits. + - Config version incremented on any change (cache invalidation signal) + +- **Server database helper** (`server/modules/db.py`): + - `get_device_by_id()`: retrieve device with all config fields + +- **Pi sync thread** (`modules/app_device_sync.py`): + - Connectivity-driven: syncs every 5min when online, backoff to 60sec when offline + - Fetches config from `/api/device-config` endpoint (device_key auth) + - Detects changes via config_version + - Stores synced config in SQLite + - Environment vars: `YARIS_SERVER_URL`, `YARIS_DEVICE_KEY` + - Ready for hot-reload on config change (future: signal handler) + +- **Pi startup** (`app.py`): + - `app_device_sync.start_sync()` starts background thread at boot -#### Tested behaviors -- ✅ Load device config from API -- ✅ Update device config (name, model, avatar, device name) -- ✅ Sync toggle blocks field edits (returns error if trying to edit while sync_enabled=true) -- ✅ Fields properly disabled/read-only in Settings UI when sync enabled -- ✅ Fallback to config file defaults works -- ✅ All 63 tests passing - -#### Future scope (Phase 25.5a) -- **Server-side config editor**: add a simple "Device Settings" page on the remote portal (login-protected, device-scoped) with editable fields -- **Pi-side sync** (connectivity-driven, not time-driven): - - Pi syncs config from server **only when internet is available** (when cellular/WiFi connects), not on a fixed timer - - Server-side changes → Pi receives on next connectivity event → updates SQLite → hot-reloads template values (no app restart) - - If server unreachable, Pi continues with last-synced config from SQLite (already loaded at startup from JSON) -- **Device authentication**: use an existing `device_key` to authenticate `/api/device-config` requests, validated server-side against Postgres `devices` table -- **Avatar handling**: Server stores uploaded images; Pi fetches to `static/img/avatar.png` on sync - -#### Benefits -- ✅ No code edit needed to customize deployed Pi (today: edit JSON file manually, or use Settings page) -- ✅ Offline-first: Pi boots with identity immediately, no network dependency -- ✅ Settings UI: users can now edit device identity without SSH/code access -- ✅ Connection-driven sync: sync happens when internet available (future), not wasting resources on offline car -- 🚧 Server becomes source of truth (after Phase 25.5a) -- 🚧 Enables multi-Pi/multi-car scenarios (after Phase 25.5a) +#### How it works +1. **At Pi startup**: loads device config from `config/device.json` (no network needed) +2. **When internet available**: sync thread wakes up every 5min and fetches from server +3. **If config changed** (new config_version): SQLite updated, next request reads new values +4. **If internet lost**: Pi continues with last-synced config from database +5. **User can still edit locally**: Settings page updates database; if sync enabled, server values override on next sync #### Trade-offs -- Sync is eventual-consistency (changes on server may take hours to reach offline car) — acceptable, car is offline most of the time -- Manual JSON edit required today, Settings UI in place (before Phase 25.5a server UI is built) -- Avatar must be placed in `static/img/` directory (no upload yet; Phase 25.5a adds server upload) +- Eventual consistency: changes on server may take 5min to reach Pi +- No hot-reload yet (requires app restart or signal handler to apply avatar/name changes immediately) +- Config version only increments on change; no periodic re-sync (efficient for offline car) + +#### Testing +- Server endpoints accessible via `/api/device-config` +- Pi thread starts on boot (check logs for connectivity attempts) +- Sync disabled if `YARIS_SERVER_URL` or `YARIS_DEVICE_KEY` not set + +#### Next phase (Phase 25.5b) +- Hot-reload handler to apply config changes without restart +- Server portal UI for editing device config (admin page) +- Device settings page on portal to test config updates +- Mobile-friendly device management UI --- diff --git a/app.py b/app.py index 5c43081..a1d8295 100644 --- a/app.py +++ b/app.py @@ -1,5 +1,5 @@ from flask import Flask, render_template, make_response, json, jsonify, request, send_file, Response -from modules import app_gps, app_events, app_car, app_account, app_obd, app_sql, app_dash, app_upload, app_video, app_health, app_alerts, app_settings, app_network, app_device_config +from modules import app_gps, app_events, app_car, app_account, app_obd, app_sql, app_dash, app_upload, app_video, app_health, app_alerts, app_settings, app_network, app_device_config, app_device_sync import os, time from gevent.pywsgi import WSGIServer @@ -14,6 +14,7 @@ gps = app_gps.start_gps() app_sql.start_logging() app_upload.start_uploading() app_alerts.start_alerting() +app_device_sync.start_sync() @app.route('/') def index(): diff --git a/modules/app_device_sync.py b/modules/app_device_sync.py new file mode 100644 index 0000000..203fc44 --- /dev/null +++ b/modules/app_device_sync.py @@ -0,0 +1,121 @@ +"""Device configuration sync from remote server (Phase 25.5a). + +Pi fetches device config from server when internet is available (connection-driven). +Stores in SQLite and hot-reloads app globals if config changes. +""" + +import threading +import time +import os +import requests +from . import app_sql + + +_SYNC_THREAD = None +_SYNC_STOP = False +_SYNC_CONFIG_VERSION = 0 # Track last-synced version to detect changes + + +def start_sync(): + """Start background device config sync thread.""" + global _SYNC_THREAD, _SYNC_STOP + if _SYNC_THREAD is not None: + return + + _SYNC_STOP = False + _SYNC_THREAD = threading.Thread(target=_sync_loop, daemon=True) + _SYNC_THREAD.start() + + +def stop_sync(): + """Stop background device config sync thread.""" + global _SYNC_STOP + _SYNC_STOP = True + + +def _sync_loop(): + """Periodically sync device config from server when internet available.""" + global _SYNC_CONFIG_VERSION + + server_url = os.getenv("YARIS_SERVER_URL", "") + device_key = os.getenv("YARIS_DEVICE_KEY", "") + + if not server_url or not device_key: + return # Sync disabled if server URL or device key not configured + + # Sync every 5 minutes (300 sec) when connected, backoff to 60 sec when offline + sync_interval = 300 + offline_interval = 60 + + while not _SYNC_STOP: + try: + # Check if internet is available (simple connectivity test) + try: + requests.head("http://8.8.8.8", timeout=2) + is_connected = True + except (requests.RequestException, Exception): + is_connected = False + + if is_connected: + # Fetch device config from server + headers = {"Authorization": f"Bearer {device_key}"} + response = requests.get( + f"{server_url.rstrip('/')}/api/device-config", + headers=headers, + timeout=5 + ) + + if response.status_code == 200: + config = response.json() + + # Check if config changed + new_version = config.get("config_version", 0) + if new_version != _SYNC_CONFIG_VERSION: + _SYNC_CONFIG_VERSION = new_version + # Store in database + _store_synced_config(config) + # Signal that config changed (for hot-reload) + _on_config_changed() + + time.sleep(sync_interval) + else: + # Offline: backoff to less frequent checks + time.sleep(offline_interval) + + except Exception as e: + # Log error but don't crash sync thread + print(f"Device config sync error: {e}") + time.sleep(60) + + +def _store_synced_config(config): + """Store synced config in SQLite database.""" + app_sql.set_setting("device_owner_name", config.get("owner_name", "")) + app_sql.set_setting("device_car_model", config.get("car_model", "")) + app_sql.set_setting("device_device_name", config.get("device_name", "")) + app_sql.set_setting("device_avatar_path", config.get("avatar_path", "img/yaris.png")) + app_sql.set_setting("device_config_version", str(config.get("config_version", 0))) + app_sql.set_setting("device_config_synced_at", str(int(time.time()))) + + +def _on_config_changed(): + """Called when config changes and needs to be reloaded by app.""" + # This is a signal for future hot-reload implementation + # For now, config is automatically read from database on each request + # A full reload would require restarting the app or using a signal handler + pass + + +def is_sync_enabled(): + """Check if device config sync from server is enabled.""" + return bool(os.getenv("YARIS_SERVER_URL")) and bool(os.getenv("YARIS_DEVICE_KEY")) + + +def get_synced_config_version(): + """Get the last-synced config version number.""" + return int(app_sql.get_setting("device_config_version", "0")) + + +def get_synced_at_timestamp(): + """Get timestamp of last successful sync.""" + return app_sql.get_setting("device_config_synced_at", "")