diff --git a/PLAN.md b/PLAN.md index a9b510e..96c47f5 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1398,89 +1398,9 @@ including two UX polish passes and multiple bug-fix rounds. - **Dispatcher hook fix (`deploy/90-yaris-uplink`)** β€” previously ignored wlan1 events; now listens for both `wlan0` and `wlan1` so uplink transition recording remains accurate on current hardware. - **README hardware photo** β€” added a current hardware build image (`pics/yaris-obd-hardware.jpg`) at the bottom of `readme.md`; saved without EXIF metadata and resized for lighter page load. -### Phase 25: Pre-install Yaris 2013 compatibility test β€” 🚧 PLANNED (core-system gate) -**Added 2026-08-09; intentionally immediately before the Android app.** Bench tests, synthetic -routes, attached-hardware tests, and stationary Pi runs cannot prove the complete system works -when temporarily connected to and powered by the actual 2013 Toyota Yaris. This phase is the -pre-installation acceptance gate for the complete core Pi/web/server system after Phase 24. It is -verification and targeted correctionβ€”not permanent installation or another redesignβ€”and the -system is not considered ready to install until the results are recorded here. Permanent mounting, -cable routing, trim work, and final installation are explicitly outside the roadmap and will be -handled by the owner after this phase passes. - -Prepare a temporary, removable test setup before driving: -- Connect the Pi, SIM868 HAT/antennas, USB OBD adapter, dashcam, and representative intended power - source without permanently mounting or routing anything. Secure every temporary component and - cable so it cannot interfere with pedals, steering, airbags, visibility, occupants, or loose-item - safety during the controlled drive; remove the setup after testing. -- Resolve the Phase 19/20 undervoltage/throttling warning before the test; confirm - `vcgencmd get_throttled` shows no current power fault at idle and under load with the temporary - representative power setup. This validates electrical compatibility, not final cable routing. -- Back up the Pi SQLite database and record baseline row/trip/video/upload counts on the Pi and - remote server. Use a uniquely timestamped test window so generated records can be reconciled - without deleting unrelated history. -- Verify the service, watchdog, storage mounts, hotspot, `http://yaris.local/`, alert engine, GPS - antenna view, dashcam recording mode, free space, and remote server health before moving. -- Use a second person for UI/log observations while the vehicle is moving, or record diagnostics - for review afterward. The driver must not operate or troubleshoot the UI while driving. - -Run a controlled route containing ignition-off/on, stationary idle, low-speed streets, normal road -speed, several complete stops, turns, a short GPS-obstructed segment if safely available, and a -clean trip end: -- **OBD/ECU:** confirm the ELM327 reaches `CAR_CONNECTED`, record negotiated Toyota protocol and - adapter firmware, and verify real speed/RPM/coolant/fuel/throttle/MIL/DTC values. Compare wheel - speed against the instrument cluster at several safe speeds. Record unsupported PIDs (especially - odometer PID A6) as capabilities, not failures, and verify reconnect after ignition cycling and a - deliberate USB disconnect/reconnect. -- **GPS and motion logging:** confirm fixes while stationary and moving, route coordinates on the - offline map, OBD-speed-first motion detection, GPS displacement fallback when OBD is deliberately - unavailable, one-minute idle heartbeat behavior, and no false trip from stationary GPS drift. -- **Trip boundary/replay:** after the inactivity timeout, verify exactly one trip (or the expected - number for deliberate ignition gaps), correct start/end time and coordinates, plausible - distance/average/max speed, km/mi and km/h/mph values, route polyline, start/end markers, timeline - scrubber, play/pause marker position, and stopped portions of the route. -- **Local web UI:** test the final Phase 23 interface on the actual phone over the Pi/phone network at - `http://yaris.local/`; verify the explicit **In-car / Local Pi** identity, dashboard telemetry, - DTC page, trip history, dashcam library, settings, alerts, responsive layout, reconnect behavior, - and no new browser console/API errors. -- **Dashcam workflow:** create a known short real clip, switch the camera safely into USB storage - mode, detect/list it without confusing the OBD adapter, import only that selected file, verify - thumbnail/playback/Range seeking, preserve the source until checks finish, and confirm the camera - returns to recording afterward. Test deletion only with the disposable clip. -- **Alerts:** safely exercise conditions that can be simulated without damaging hardware (upload - backlog, temporary GPS loss while OBD reports motion, and disposable threshold overrides). - Confirm local bell transition/recovery/cooldown behavior and offline alert-state reconciliation - with the Phase 24 server after connectivity returns. Do not intentionally overheat, undervolt, - fill storage, or trigger a real MIL. With a real SIM, explicitly determine whether the SIM868 - transport can reliably deliver ntfy notifications while WiFi/Ethernet is unavailable, including - authentication, timeout, retry, and recovery. If that cannot be made reliable on the actual - modem/network, remove the ntfy feature and its settings rather than retain a transport that fails - in the only situation where it is useful. -- **Power/reliability:** perform multiple normal ignition/power cycles, confirm automatic boot, - service/watchdog startup, SIM868 power-on, OBD/GPS recovery, no SQLite corruption, and no current - undervoltage/throttling. The deferred Phase 14 clean-shutdown limitation must be explicitly - accepted or resolved with representative delayed-power/ignition-sense hardware before declaring - the design ready for permanent installation. - -Verify the complete remote path after connectivity returns: -- Confirm every test-window tracking row uploads exactly once and the selected dashcam clip resumes - or completes without corruption; compare counts, timestamps, hashes, and byte sizes locally and - remotely. -- On the Phase 24 portal, verify the explicit **Remote server** identity, authenticated - vehicle-scoped trip summary/replay, route and timeline agreement with the local Pi, archived clip - thumbnail/playback/Range seeking/download, synchronized alert history/current state without - duplicate transitions after an offline interval, and absence of local-only OBD/cellular/Pi - controls. -- The Android companion is intentionally outside this core acceptance gate and remains the final, - low-priority Phase 26. Its own hotspot/cellular binding, trip display, and disposable clip relay - require separate real-phone/in-car acceptance when that phase is implemented. - -Acceptance requires a written result for every item, captured negotiated protocol/PID support, -measured trip/count/hash comparisons, relevant logs/screenshots, no unexplained tracebacks or data -loss, and cleanup of every disposable record/file while preserving the real drive. Any defect found -must be fixed, redeployed uncommitted, and the affected in-car test repeated before the final commit -and **ready-to-install** sign-off. Afterward, the owner handles permanent installation outside the -phase roadmap. +**Roadmap ordering note (2026-08-10):** the late phases are intentionally sequenced as +Phase 25.5a–25.5d (configuration/sync/status work) -> Phase 25 (real-car acceptance gate) -> +Phase 26 (Android, always last). ### Phase 25.5a: Device configuration sync backend β€” βœ… COMPLETE (connection-driven server sync) **Server API and Pi sync thread implemented.** Server stores device config; Pi fetches when internet available and applies changes without restart. @@ -1570,310 +1490,395 @@ updates its display-facing values without a manual restart once sync lands. --- - ### Phase 25.5c: Multi-device provisioning + force-sync UX β€” βœ… COMPLETE - Extend the current single-device portal page into a real multi-device management - surface: register devices from the server, mint/rotate API keys, help the Pi - consume those credentials without SSH-heavy manual steps, and expose an honest - **force sync** action that works with the system's pull-based/offline-first - architecture. +### Phase 25.5c: Multi-device provisioning + force-sync UX β€” βœ… COMPLETE +Extend the current single-device portal page into a real multi-device management +surface: register devices from the server, mint/rotate API keys, help the Pi +consume those credentials without SSH-heavy manual steps, and expose an honest +**force sync** action that works with the system's pull-based/offline-first +architecture. - #### Goals - - **Multi-device server view**: stop assuming one `DEVICE_NAME` in the portal; - show all registered devices and let the user select which one is being edited - - **Device registration from the portal**: create a new device row and return a - one-time raw API key directly in the browser UI (same behavior as - `server/scripts/create_device.py`, but web-managed) - - **Credential handoff to the Pi**: make it possible to put `YARIS_SERVER_URL`, - `YARIS_DEVICE_NAME`, and `YARIS_API_KEY` onto the Pi from the app UI rather - than editing `db/upload_creds.env` manually over SSH - - **Force sync button**: let the user request an immediate config pull without - pretending the cloud server can magically push into an offline car +#### Goals +- **Multi-device server view**: stop assuming one `DEVICE_NAME` in the portal; + show all registered devices and let the user select which one is being edited +- **Device registration from the portal**: create a new device row and return a + one-time raw API key directly in the browser UI (same behavior as + `server/scripts/create_device.py`, but web-managed) +- **Credential handoff to the Pi**: make it possible to put `YARIS_SERVER_URL`, + `YARIS_DEVICE_NAME`, and `YARIS_API_KEY` onto the Pi from the app UI rather + than editing `db/upload_creds.env` manually over SSH +- **Force sync button**: let the user request an immediate config pull without + pretending the cloud server can magically push into an offline car - #### Proposed implementation - 1. **Server: device management model + portal** - - Add a device list page/card in the remote portal, with: - - registered devices table - - "Add device" action - - "Rotate API key" action - - per-device "Edit settings" action - - Replace the single-device lookup helper with: - - list devices - - fetch one device by id - - update one device's identity/config - - Keep raw API keys **write-only/one-time reveal**: - - show only immediately after create/rotate - - never store raw keys server-side - - only store hash as today +#### Proposed implementation +1. **Server: device management model + portal** + - Add a device list page/card in the remote portal, with: + - registered devices table + - "Add device" action + - "Rotate API key" action + - per-device "Edit settings" action + - Replace the single-device lookup helper with: + - list devices + - fetch one device by id + - update one device's identity/config + - Keep raw API keys **write-only/one-time reveal**: + - show only immediately after create/rotate + - never store raw keys server-side + - only store hash as today - 2. **Server: browser APIs** - - Add login-protected endpoints for: - - list devices - - create device + return one-time API key - - rotate device API key + return one-time API key - - get/update selected device config - - request force sync for a selected device - - Device settings UI should populate from the selected device row, not from a - single hardcoded portal `DEVICE_NAME` +2. **Server: browser APIs** + - Add login-protected endpoints for: + - list devices + - create device + return one-time API key + - rotate device API key + return one-time API key + - get/update selected device config + - request force sync for a selected device + - Device settings UI should populate from the selected device row, not from a + single hardcoded portal `DEVICE_NAME` - 3. **Pi: credential management UI** - - Add a local Settings section for remote server credentials: - - server URL - - device name - - API key - - connection test button - - Persist those values in the existing gitignored - `db/upload_creds.env` (or an equivalent sqlite-backed secret store if we - later choose to migrate away from env files) - - After saving: - - reload upload client / sync client config - - allow enabling **Sync with server** only when creds test cleanly +3. **Pi: credential management UI** + - Add a local Settings section for remote server credentials: + - server URL + - device name + - API key + - connection test button + - Persist those values in the existing gitignored + `db/upload_creds.env` (or an equivalent sqlite-backed secret store if we + later choose to migrate away from env files) + - After saving: + - reload upload client / sync client config + - allow enabling **Sync with server** only when creds test cleanly - 4. **Force sync semantics** - - **Constraint**: the server cannot directly reach a Pi behind hotspot/NAT, - and the car is often offline. So a server-side force sync button cannot be - a true push. - - Proposed behavior: - - server sets a per-device `force_sync_requested_at` flag/version bump - - Pi checks that flag during normal online polling - - local Pi UI also gets a **Sync now** button that triggers an immediate - fetch against the server using the saved credentials - - Result: - - from server: "sync on next online contact" - - from Pi: "sync immediately if internet is currently available" +4. **Force sync semantics** + - **Constraint**: the server cannot directly reach a Pi behind hotspot/NAT, + and the car is often offline. So a server-side force sync button cannot be + a true push. + - Proposed behavior: + - server sets a per-device `force_sync_requested_at` flag/version bump + - Pi checks that flag during normal online polling + - local Pi UI also gets a **Sync now** button that triggers an immediate + fetch against the server using the saved credentials + - Result: + - from server: "sync on next online contact" + - from Pi: "sync immediately if internet is currently available" - #### Data model changes - - Add server columns such as: - - `last_config_sync_at` - - `last_config_sync_version` - - `force_sync_requested_at` - - `api_key_rotated_at` - - Optionally add a lightweight pairing/claim model later if copying raw API keys - manually into the Pi UI feels too clunky +#### Data model changes +- Add server columns such as: + - `last_config_sync_at` + - `last_config_sync_version` + - `force_sync_requested_at` + - `api_key_rotated_at` +- Optionally add a lightweight pairing/claim model later if copying raw API keys + manually into the Pi UI feels too clunky - #### UI/UX shape - - **Remote portal** - - left: device list / selector - - right: settings form for selected device - - actions: Add device, Rotate key, Copy creds, Request sync - - status badges: last seen, last config version, last sync requested, maybe - last successful contact - - **Pi local app** - - server credentials card in Settings - - Test connection button - - Sync now button - - clear warning when local edits are locked because server sync is enabled +#### UI/UX shape +- **Remote portal** + - left: device list / selector + - right: settings form for selected device + - actions: Add device, Rotate key, Copy creds, Request sync + - status badges: last seen, last config version, last sync requested, maybe + last successful contact +- **Pi local app** + - server credentials card in Settings + - Test connection button + - Sync now button + - clear warning when local edits are locked because server sync is enabled - #### Security / operational notes - - Never display a stored API key after the creation/rotation response is gone - - Keep key rotation explicit; rotating a key should warn that the old Pi config - will stop authenticating until updated - - Avoid logging raw keys in Flask logs, JS errors, or shell output captured in - committed docs - - Because the Pi is usually offline, all flows must degrade gracefully when the - remote portal requests work the Pi cannot act on yet +#### Security / operational notes +- Never display a stored API key after the creation/rotation response is gone +- Keep key rotation explicit; rotating a key should warn that the old Pi config + will stop authenticating until updated +- Avoid logging raw keys in Flask logs, JS errors, or shell output captured in + committed docs +- Because the Pi is usually offline, all flows must degrade gracefully when the + remote portal requests work the Pi cannot act on yet - #### Suggested execution order - 1. Server multi-device DB/API refactor - 2. Remote portal device list + register/rotate key UI - 3. Pi local server-credentials UI + connection test - 4. Pi/local immediate **Sync now** button - 5. Server "request sync" flag + per-device status indicators - 6. End-to-end validation: - - create device on server - - copy/paste creds into Pi UI - - test connection - - enable sync - - change device identity on server - - verify Pi receives it - - rotate key and verify old key stops working until Pi updated +#### Suggested execution order +1. Server multi-device DB/API refactor +2. Remote portal device list + register/rotate key UI +3. Pi local server-credentials UI + connection test +4. Pi/local immediate **Sync now** button +5. Server "request sync" flag + per-device status indicators +6. End-to-end validation: + - create device on server + - copy/paste creds into Pi UI + - test connection + - enable sync + - change device identity on server + - verify Pi receives it + - rotate key and verify old key stops working until Pi updated - #### Open constraint to keep in mind - - If you want **zero manual copy/paste** for API key provisioning, that becomes - a pairing problem (claim code / QR / short-lived token). That is doable, but - it is a bigger scope than simply exposing key create/copy fields in the two - existing UIs. My recommendation is to ship copy/paste provisioning first, then - add a pairing flow only if it still feels too annoying. +#### Open constraint to keep in mind +- If you want **zero manual copy/paste** for API key provisioning, that becomes + a pairing problem (claim code / QR / short-lived token). That is doable, but + it is a bigger scope than simply exposing key create/copy fields in the two + existing UIs. My recommendation is to ship copy/paste provisioning first, then + add a pairing flow only if it still feels too annoying. - #### Implementation progress (2026-08-10) - - **Server-side multi-device API/UI** - - Added device listing, create-device, rotate-key, per-device config, and - request-sync endpoints in the remote portal - - Reworked `/settings/device` into a multi-device management page with a - registered-device list, selected-device editor, one-time API-key reveal, and - copy-credentials helper - - **Server schema** - - Added Phase 25.5c migration for `last_config_sync_at`, - `last_config_sync_version`, `force_sync_requested_at`, and - `api_key_rotated_at` - - Added device sync acknowledgement endpoint so the Pi can report when a - config version was actually applied - - **Pi-side provisioning** - - Added local Settings UI/API for `YARIS_SERVER_URL`, `YARIS_DEVICE_NAME`, and - `YARIS_API_KEY` - - Added **Test connection** and **Sync now** actions in the Pi app - - Reused the existing gitignored `db/upload_creds.env` store instead of - inventing a second credential location - - **Runtime behavior fixes** - - Upload client now reads credentials dynamically instead of only at import - - Device sync thread can restart after credentials are added later - - Enabling **Sync with server** now validates that remote credentials exist +#### Implementation progress (2026-08-10) +- **Server-side multi-device API/UI** + - Added device listing, create-device, rotate-key, per-device config, and + request-sync endpoints in the remote portal + - Reworked `/settings/device` into a multi-device management page with a + registered-device list, selected-device editor, one-time API-key reveal, and + copy-credentials helper +- **Server schema** + - Added Phase 25.5c migration for `last_config_sync_at`, + `last_config_sync_version`, `force_sync_requested_at`, and + `api_key_rotated_at` + - Added device sync acknowledgement endpoint so the Pi can report when a + config version was actually applied +- **Pi-side provisioning** + - Added local Settings UI/API for `YARIS_SERVER_URL`, `YARIS_DEVICE_NAME`, and + `YARIS_API_KEY` + - Added **Test connection** and **Sync now** actions in the Pi app + - Reused the existing gitignored `db/upload_creds.env` store instead of + inventing a second credential location +- **Runtime behavior fixes** + - Upload client now reads credentials dynamically instead of only at import + - Device sync thread can restart after credentials are added later + - Enabling **Sync with server** now validates that remote credentials exist - #### Validation - - Local tests expanded to `72 passed` - - Deployed Phase 25.5c to unraid and applied `migrate_phase25_5c.sql` - - Deployed Phase 25.5c to the Pi at `192.168.50.213`; `yaris-obd.service` - restarted cleanly - - Pi local API checks: - - `GET /api/remote-config` shows the saved remote server credentials state - - `POST /api/remote-config/test` succeeds with authenticated device creds - - `POST /api/remote-config/sync-now` succeeds (`no_change` when already current) - - Server sync-ack metadata verified live: `last_config_sync_version` matched the - current `config_version` for `yaris-2013`, and `last_config_sync_at` populated - - Temporary create/delete validation of the new device-provisioning helper - succeeded on the live server (`phase25c-validate-temp`) +#### Validation +- Local tests expanded to `72 passed` +- Deployed Phase 25.5c to unraid and applied `migrate_phase25_5c.sql` +- Deployed Phase 25.5c to the Pi at `192.168.50.213`; `yaris-obd.service` + restarted cleanly +- Pi local API checks: + - `GET /api/remote-config` shows the saved remote server credentials state + - `POST /api/remote-config/test` succeeds with authenticated device creds + - `POST /api/remote-config/sync-now` succeeds (`no_change` when already current) +- Server sync-ack metadata verified live: `last_config_sync_version` matched the + current `config_version` for `yaris-2013`, and `last_config_sync_at` populated +- Temporary create/delete validation of the new device-provisioning helper + succeeded on the live server (`phase25c-validate-temp`) - #### Remaining polish ideas (not blockers) - - Exercise the authenticated browser flow for create/rotate/request-sync in the - live portal UI itself, not just the backing APIs/helpers - - If copy/paste provisioning feels too clunky in practice, add a later QR / - claim-code pairing flow - - Main portal overview/history/dashcam pages are **not** multi-device yet: - the data model already separates rows by `device_id`, but those pages still - read from the single configured `DEVICE_NAME`. Today that means the new - multi-device management page can manage several devices, but the normal trip - history view still shows one configured car/device at a time until a later - portal-wide device selector/filter is implemented. +#### Remaining polish ideas (not blockers) +- Exercise the authenticated browser flow for create/rotate/request-sync in the + live portal UI itself, not just the backing APIs/helpers +- If copy/paste provisioning feels too clunky in practice, add a later QR / + claim-code pairing flow +- Main portal overview/history/dashcam pages are **not** multi-device yet: + the data model already separates rows by `device_id`, but those pages still + read from the single configured `DEVICE_NAME`. Today that means the new + multi-device management page can manage several devices, but the normal trip + history view still shows one configured car/device at a time until a later + portal-wide device selector/filter is implemented. - ### Phase 25.5d: GPS dashboard visibility / live-status debugging β€” 🚧 IN PROGRESS - - Add an explicit Pi dashboard GPS status indicator (thread alive / live fix / stale fix / no fix) - - Expose a local `/api/gps/status` endpoint so GPS health can be checked without relying on - socket events alone - - Verify on the real Pi whether GPS is actually still producing fixes or whether the issue is - now hardware/module/fix availability rather than a front-end live-update regression - - **Live Pi check 2026-08-10**: after deploying the new endpoint, the real Pi reported - `thread_alive=true` but `has_fix=false`, `latitude=null`, `longitude=null`, and - `fix_age_seconds=null`. That means the local app/socket path is still running, but there is - currently no GNSS fix to stream into the dashboard. - - **Follow-up recheck 2026-08-10**: the Pi later recovered a live fix without further code - changes. `/api/gps/status` returned `thread_alive=true`, `has_fix=true`, - `latitude=45.544198`, `longitude=-73.536626`, and a fresh `fix_age_seconds` (~2.7s), which - confirms the live-update path still works and the earlier issue was temporary lack of GNSS fix, - not a dashboard/socket regression. - - **Dashboard fit polish 2026-08-10**: tightened the Pi dashboard layout so the GPS and Overview - cards no longer stretch to the speed graph's full height, shortened the speed graph on - Pi-sized screens, and reduced excess map/footer whitespace that previously forced unnecessary - scrolling. - - **Next local UX follow-up 2026-08-10**: split the bulky Pi-health block out of the dashboard - into a dedicated local **Status** page and expand it into a broader runtime diagnostics view: - local `systemd` service states, server reachability/auth status, Wi-Fi uplink state, GPS fix - state, SIM868/cellular-HAT responsiveness, upload backlog, and OBD adapter presence. - - **Implemented + verified 2026-08-10**: added a new local `/status` page plus `/api/status` - aggregator. The dashboard no longer carries the bulky Pi-health block; instead the new page - shows Pi health, remote-server reachability/auth, upload backlog, Wi-Fi/uplink mode, GPS fix - status, SIM868 UART/modem responsiveness, OBD presence, and key `systemd` units - (`yaris-obd.service`, `NetworkManager.service`, `yaris-netwatch.timer`, - `yaris-uplink-wait.service`). Deployed to the real Pi (`192.168.50.213`) and confirmed the - page/API served live values: server reachable+authenticated, uplink on Wi-Fi with internet, - GPS thread alive with live fix, SIM868 responding on UART, OBD adapter present, and the local - services reporting expected active/inactive states. - - **Next status-page polish 2026-08-10**: add quick action buttons directly on `/status` for - server test, sync-now, uplink reconnect, and app-service restart so the Pi can be diagnosed and - nudged from one screen instead of bouncing back to Settings/SSH. - - **Implemented + verified 2026-08-10**: `/status` now includes quick actions for server test, - sync-now, uplink reconnect, and app restart. The first three reuse the existing local APIs; app - restart adds a minimal sudo-backed `systemctl restart --no-block yaris-obd.service` path, - installed by `deploy/install_service.sh` via `/etc/sudoers.d/yaris-status-actions`. Verified on - the real Pi by deploying the sudoers update, loading the new buttons, exercising - `/api/remote-config/test` (200/ok), `/api/remote-config/sync-now` (200/no_change), and - `/api/status/restart-app` (202/restart_scheduled), then confirming `yaris-obd.service` - returned to `active` and `/api/status` recovered with live values. The uplink-reconnect button - was left unclicked during verification because this Pi currently has a saved phone hotspot and - the action intentionally may switch networks/drop the active SSH session. - - **Dashboard card order tweak 2026-08-10**: swapped the local dashboard layout so the Overview - card now appears before the Live GPS map. Deployed to the real Pi and confirmed the rendered - `/` HTML places `Overview` before `Live GPS`. +### Phase 25.5d: GPS dashboard visibility / live-status debugging β€” βœ… COMPLETE +- Add an explicit Pi dashboard GPS status indicator (thread alive / live fix / stale fix / no fix) +- Expose a local `/api/gps/status` endpoint so GPS health can be checked without relying on + socket events alone +- Verify on the real Pi whether GPS is actually still producing fixes or whether the issue is + now hardware/module/fix availability rather than a front-end live-update regression +- **Live Pi check 2026-08-10**: after deploying the new endpoint, the real Pi reported + `thread_alive=true` but `has_fix=false`, `latitude=null`, `longitude=null`, and + `fix_age_seconds=null`. That means the local app/socket path is still running, but there is + currently no GNSS fix to stream into the dashboard. +- **Follow-up recheck 2026-08-10**: the Pi later recovered a live fix without further code + changes. `/api/gps/status` returned `thread_alive=true`, `has_fix=true`, + `latitude=45.544198`, `longitude=-73.536626`, and a fresh `fix_age_seconds` (~2.7s), which + confirms the live-update path still works and the earlier issue was temporary lack of GNSS fix, + not a dashboard/socket regression. +- **Dashboard fit polish 2026-08-10**: tightened the Pi dashboard layout so the GPS and Overview + cards no longer stretch to the speed graph's full height, shortened the speed graph on + Pi-sized screens, and reduced excess map/footer whitespace that previously forced unnecessary + scrolling. +- **Next local UX follow-up 2026-08-10**: split the bulky Pi-health block out of the dashboard + into a dedicated local **Status** page and expand it into a broader runtime diagnostics view: + local `systemd` service states, server reachability/auth status, Wi-Fi uplink state, GPS fix + state, SIM868/cellular-HAT responsiveness, upload backlog, and OBD adapter presence. +- **Implemented + verified 2026-08-10**: added a new local `/status` page plus `/api/status` + aggregator. The dashboard no longer carries the bulky Pi-health block; instead the new page + shows Pi health, remote-server reachability/auth, upload backlog, Wi-Fi/uplink mode, GPS fix + status, SIM868 UART/modem responsiveness, OBD presence, and key `systemd` units + (`yaris-obd.service`, `NetworkManager.service`, `yaris-netwatch.timer`, + `yaris-uplink-wait.service`). Deployed to the real Pi (`192.168.50.213`) and confirmed the + page/API served live values: server reachable+authenticated, uplink on Wi-Fi with internet, + GPS thread alive with live fix, SIM868 responding on UART, OBD adapter present, and the local + services reporting expected active/inactive states. +- **Next status-page polish 2026-08-10**: add quick action buttons directly on `/status` for + server test, sync-now, uplink reconnect, and app-service restart so the Pi can be diagnosed and + nudged from one screen instead of bouncing back to Settings/SSH. +- **Implemented + verified 2026-08-10**: `/status` now includes quick actions for server test, + sync-now, uplink reconnect, and app restart. The first three reuse the existing local APIs; app + restart adds a minimal sudo-backed `systemctl restart --no-block yaris-obd.service` path, + installed by `deploy/install_service.sh` via `/etc/sudoers.d/yaris-status-actions`. Verified on + the real Pi by deploying the sudoers update, loading the new buttons, exercising + `/api/remote-config/test` (200/ok), `/api/remote-config/sync-now` (200/no_change), and + `/api/status/restart-app` (202/restart_scheduled), then confirming `yaris-obd.service` + returned to `active` and `/api/status` recovered with live values. The uplink-reconnect button + was left unclicked during verification because this Pi currently has a saved phone hotspot and + the action intentionally may switch networks/drop the active SSH session. +- **Dashboard card order tweak 2026-08-10**: swapped the local dashboard layout so the Overview + card now appears before the Live GPS map. Deployed to the real Pi and confirmed the rendered + `/` HTML places `Overview` before `Live GPS`. - --- +--- +### Phase 25: Pre-install Yaris 2013 compatibility test β€” 🚧 PLANNED (core-system gate) +**Added 2026-08-09; intentionally immediately before the Android app.** Bench tests, synthetic +routes, attached-hardware tests, and stationary Pi runs cannot prove the complete system works +when temporarily connected to and powered by the actual 2013 Toyota Yaris. This phase is the +pre-installation acceptance gate for the complete core Pi/web/server system after Phase 24. It is +verification and targeted correctionβ€”not permanent installation or another redesignβ€”and the +system is not considered ready to install until the results are recorded here. Permanent mounting, +cable routing, trim work, and final installation are explicitly outside the roadmap and will be +handled by the owner after this phase passes. - ### Phase 26: Native Android companion app β€” 🚧 PLANNED (low priority, always last) - Revives the phone-relay goal (Phase 11/12) properly this time, using a real native app instead of - a web page - the only way to actually hold a WiFi connection to the Pi's hotspot and a cellular - connection to the internet open simultaneously (`ConnectivityManager.bindProcessToNetwork()`, +Prepare a temporary, removable test setup before driving: +- Connect the Pi, SIM868 HAT/antennas, USB OBD adapter, dashcam, and representative intended power + source without permanently mounting or routing anything. Secure every temporary component and + cable so it cannot interfere with pedals, steering, airbags, visibility, occupants, or loose-item + safety during the controlled drive; remove the setup after testing. +- Resolve the Phase 19/20 undervoltage/throttling warning before the test; confirm + `vcgencmd get_throttled` shows no current power fault at idle and under load with the temporary + representative power setup. This validates electrical compatibility, not final cable routing. +- Back up the Pi SQLite database and record baseline row/trip/video/upload counts on the Pi and + remote server. Use a uniquely timestamped test window so generated records can be reconciled + without deleting unrelated history. +- Verify the service, watchdog, storage mounts, hotspot, `http://yaris.local/`, alert engine, GPS + antenna view, dashcam recording mode, free space, and remote server health before moving. +- Use a second person for UI/log observations while the vehicle is moving, or record diagnostics + for review afterward. The driver must not operate or troubleshoot the UI while driving. + +Run a controlled route containing ignition-off/on, stationary idle, low-speed streets, normal road +speed, several complete stops, turns, a short GPS-obstructed segment if safely available, and a +clean trip end: +- **OBD/ECU:** confirm the ELM327 reaches `CAR_CONNECTED`, record negotiated Toyota protocol and + adapter firmware, and verify real speed/RPM/coolant/fuel/throttle/MIL/DTC values. Compare wheel + speed against the instrument cluster at several safe speeds. Record unsupported PIDs (especially + odometer PID A6) as capabilities, not failures, and verify reconnect after ignition cycling and a + deliberate USB disconnect/reconnect. +- **GPS and motion logging:** confirm fixes while stationary and moving, route coordinates on the + offline map, OBD-speed-first motion detection, GPS displacement fallback when OBD is deliberately + unavailable, one-minute idle heartbeat behavior, and no false trip from stationary GPS drift. +- **Trip boundary/replay:** after the inactivity timeout, verify exactly one trip (or the expected + number for deliberate ignition gaps), correct start/end time and coordinates, plausible + distance/average/max speed, km/mi and km/h/mph values, route polyline, start/end markers, timeline + scrubber, play/pause marker position, and stopped portions of the route. +- **Local web UI:** test the final Phase 23 interface on the actual phone over the Pi/phone network at + `http://yaris.local/`; verify the explicit **In-car / Local Pi** identity, dashboard telemetry, + DTC page, trip history, dashcam library, settings, alerts, responsive layout, reconnect behavior, + and no new browser console/API errors. +- **Dashcam workflow:** create a known short real clip, switch the camera safely into USB storage + mode, detect/list it without confusing the OBD adapter, import only that selected file, verify + thumbnail/playback/Range seeking, preserve the source until checks finish, and confirm the camera + returns to recording afterward. Test deletion only with the disposable clip. +- **Alerts:** safely exercise conditions that can be simulated without damaging hardware (upload + backlog, temporary GPS loss while OBD reports motion, and disposable threshold overrides). + Confirm local bell transition/recovery/cooldown behavior and offline alert-state reconciliation + with the Phase 24 server after connectivity returns. Do not intentionally overheat, undervolt, + fill storage, or trigger a real MIL. With a real SIM, explicitly determine whether the SIM868 + transport can reliably deliver ntfy notifications while WiFi/Ethernet is unavailable, including + authentication, timeout, retry, and recovery. If that cannot be made reliable on the actual + modem/network, remove the ntfy feature and its settings rather than retain a transport that fails + in the only situation where it is useful. +- **Power/reliability:** perform multiple normal ignition/power cycles, confirm automatic boot, + service/watchdog startup, SIM868 power-on, OBD/GPS recovery, no SQLite corruption, and no current + undervoltage/throttling. The deferred Phase 14 clean-shutdown limitation must be explicitly + accepted or resolved with representative delayed-power/ignition-sense hardware before declaring + the design ready for permanent installation. + +Verify the complete remote path after connectivity returns: +- Confirm every test-window tracking row uploads exactly once and the selected dashcam clip resumes + or completes without corruption; compare counts, timestamps, hashes, and byte sizes locally and + remotely. +- On the Phase 24 portal, verify the explicit **Remote server** identity, authenticated + vehicle-scoped trip summary/replay, route and timeline agreement with the local Pi, archived clip + thumbnail/playback/Range seeking/download, synchronized alert history/current state without + duplicate transitions after an offline interval, and absence of local-only OBD/cellular/Pi + controls. +- The Android companion is intentionally outside this core acceptance gate and remains the final, + low-priority Phase 26. Its own hotspot/cellular binding, trip display, and disposable clip relay + require separate real-phone/in-car acceptance when that phase is implemented. + +Acceptance requires a written result for every item, captured negotiated protocol/PID support, +measured trip/count/hash comparisons, relevant logs/screenshots, no unexplained tracebacks or data +loss, and cleanup of every disposable record/file while preserving the real drive. Any defect found +must be fixed, redeployed uncommitted, and the affected in-car test repeated before the final commit +and **ready-to-install** sign-off. Afterward, the owner handles permanent installation outside the +phase roadmap. + +--- + +### Phase 26: Native Android companion app β€” 🚧 PLANNED (low priority, always last) +Revives the phone-relay goal (Phase 11/12) properly this time, using a real native app instead of +a web page - the only way to actually hold a WiFi connection to the Pi's hotspot and a cellular +connection to the internet open simultaneously (`ConnectivityManager.bindProcessToNetwork()`, unavailable to browser JS - see the Phase 11 update). Explicitly **not urgent** and intentionally kept at the end of the roadmap, after all web/server work and the Phase 25 core in-car acceptance test are done. Permanent vehicle installation remains owner-handled and is not an implementation phase. Scope, at a high level (not yet designed in detail): - **Dashcam file transfer**: browse/select/delete dashcam clips already imported to the Pi (same - data as the Phase 17 local video browser, just a native client instead of/alongside the web UI), - download them to the phone, and upload to the remote server over cellular while the phone stays - on the Pi's hotspot for the download leg - the true simultaneous dual-network relay the web - version could never deliver. +data as the Phase 17 local video browser, just a native client instead of/alongside the web UI), +download them to the phone, and upload to the remote server over cellular while the phone stays +on the Pi's hotspot for the download leg - the true simultaneous dual-network relay the web +version could never deliver. - **Trip/location info**: surface live GPS position, speed, and trip history (Phase 18) pulled - from the Pi while connected to its hotspot (reusing the existing `/api/gps`-style data feeds), - so the app is useful as a real "check on the car" companion, not just a file-transfer tool. +from the Pi while connected to its hotspot (reusing the existing `/api/gps`-style data feeds), +so the app is useful as a real "check on the car" companion, not just a file-transfer tool. - Not yet scoped: distribution (sideload vs Play Store), whether it talks to the Pi directly over - the hotspot only or also to the remote server directly for trip history when away from the car, - and how much of the existing REST API surface can be reused as-is vs needs new - mobile-friendly endpoints. To be designed once Phases 12-24 (web-side cleanup + hardening, - redesign, remote portal, and core in-car acceptance) are done. +the hotspot only or also to the remote server directly for trip history when away from the car, +and how much of the existing REST API surface can be reused as-is vs needs new +mobile-friendly endpoints. To be designed once Phases 12-24 (web-side cleanup + hardening, +redesign, remote portal, and core in-car acceptance) are done. ## Working Notes / Environment Gotchas - **Pi credentials/SSH**: `ssh yaris@`. Confirm current IP before connecting (varies by - network β€” check locally, not recorded here) β€” usually reachable via `yaris.local` too (mDNS, - see Phase 6). **Current verified LAN IP (2026-08-10): `192.168.50.213`** after the TP-Link - Archer T3U WiFi change. +network β€” check locally, not recorded here) β€” usually reachable via `yaris.local` too (mDNS, +see Phase 6). **Current verified LAN IP (2026-08-10): `192.168.50.213`** after the TP-Link +Archer T3U WiFi change. - **Local dev repo**: work happens on a separate dev machine, not on the Pi. Push to - `origin/shameless-aislop`, the repo's current default branch (repo is public). +`origin/shameless-aislop`, the repo's current default branch (repo is public). - **unraid server SSH**: `ssh root@192.168.10.10` (alias `unraid`). Real git checkout lives at - `/mnt/user/app/git/yaris-obd` (not `/mnt/user/appdata/...`, which is only the docker-managed data - volume for the `db`/`dashcam` containers). **Re-verified 2026-08-10**: alias works and lands in - `/root`. Deploy with - `cd /mnt/user/app/git/yaris-obd && git pull --ff-only && cd server && docker compose up -d --build`. - Container names: `yaris-obd-db` (Postgres 17), `yaris-obd-api`. For existing databases, also run - the mounted migration scripts manually, e.g. - `docker compose exec db psql -U yarisobd -d yarisobd -f /docker-entrypoint-initdb.d/migrate_phase25_5a.sql`. - **Important 2026-08-10**: this checkout had a long-diverged local `shameless-aislop` - (`ahead 136, behind 146`), so `git pull --ff-only` failed. To avoid destroying local history, the - Phase 25.5a deploy was performed from a fresh temporary clone while reusing the same `.env` and - Docker data. Reconcile or replace the old checkout before relying on it for future ff-only deploys. +`/mnt/user/app/git/yaris-obd` (not `/mnt/user/appdata/...`, which is only the docker-managed data +volume for the `db`/`dashcam` containers). **Re-verified 2026-08-10**: alias works and lands in +`/root`. Deploy with +`cd /mnt/user/app/git/yaris-obd && git pull --ff-only && cd server && docker compose up -d --build`. +Container names: `yaris-obd-db` (Postgres 17), `yaris-obd-api`. For existing databases, also run +the mounted migration scripts manually, e.g. +`docker compose exec db psql -U yarisobd -d yarisobd -f /docker-entrypoint-initdb.d/migrate_phase25_5a.sql`. +**Important 2026-08-10**: this checkout had a long-diverged local `shameless-aislop` +(`ahead 136, behind 146`), so `git pull --ff-only` failed. To avoid destroying local history, the +Phase 25.5a deploy was performed from a fresh temporary clone while reusing the same `.env` and +Docker data. Reconcile or replace the old checkout before relying on it for future ff-only deploys. - **Pi deploy loop**: `ssh yaris@` (alias `yarisobd`) -> - `cd ~/app/yaris-obd && git pull --ff-only` -> `sudo systemctl restart yaris-obd.service` - (passwordless sudo confirmed via `yaris ALL=(ALL) NOPASSWD: ALL`). App stdout/stderr goes to - `~/app/yaris-obd/yaris-obd.log` (not journald - the systemd unit redirects both streams there), - so use `tail`/`grep` on that file, not `journalctl`, to see application-level prints (upload - progress, OBD/GNSS errors, etc.) - `journalctl -u yaris-obd.service` only shows service - start/stop/crash lifecycle events. **Update 2026-08-09**: plain `print()` calls were silently - invisible in that log for months because Python fully-buffers stdout when it's not a TTY (a - file, in this case) β€” `logging`-based lines (e.g. the `obd` library's own logger, which uses - stderr) showed up fine, but bare `print()` diagnostics from `app_gps.py` never did. Fixed by - adding `Environment=PYTHONUNBUFFERED=1` to the systemd unit - (`deploy/yaris-obd.service`) β€” **remember to also copy the updated unit file to - `/etc/systemd/system/yaris-obd.service` and run `sudo systemctl daemon-reload`** on the Pi, a - plain `git pull` + restart does NOT pick up systemd unit file changes (the live - `/etc/systemd/system/` copy is independent of the repo's `deploy/` copy). +`cd ~/app/yaris-obd && git pull --ff-only` -> `sudo systemctl restart yaris-obd.service` +(passwordless sudo confirmed via `yaris ALL=(ALL) NOPASSWD: ALL`). App stdout/stderr goes to +`~/app/yaris-obd/yaris-obd.log` (not journald - the systemd unit redirects both streams there), +so use `tail`/`grep` on that file, not `journalctl`, to see application-level prints (upload +progress, OBD/GNSS errors, etc.) - `journalctl -u yaris-obd.service` only shows service +start/stop/crash lifecycle events. **Update 2026-08-09**: plain `print()` calls were silently +invisible in that log for months because Python fully-buffers stdout when it's not a TTY (a +file, in this case) β€” `logging`-based lines (e.g. the `obd` library's own logger, which uses +stderr) showed up fine, but bare `print()` diagnostics from `app_gps.py` never did. Fixed by +adding `Environment=PYTHONUNBUFFERED=1` to the systemd unit +(`deploy/yaris-obd.service`) β€” **remember to also copy the updated unit file to +`/etc/systemd/system/yaris-obd.service` and run `sudo systemctl daemon-reload`** on the Pi, a +plain `git pull` + restart does NOT pick up systemd unit file changes (the live +`/etc/systemd/system/` copy is independent of the repo's `deploy/` copy). - **Pi deploy loop, requirements.txt changes**: after any `requirements.txt` change (e.g. adding - `RPi.GPIO`), also run `~/app/yaris-obd/venv/bin/pip install -r requirements.txt` on the Pi β€” a - `git pull` alone doesn't install new dependencies into the venv. +`RPi.GPIO`), also run `~/app/yaris-obd/venv/bin/pip install -r requirements.txt` on the Pi β€” a +`git pull` alone doesn't install new dependencies into the venv. - **Pi memory budget**: ~900MB usable RAM, no swap. Any code path that might buffer a whole - dashcam-sized file (hundreds of MB) in memory is unsafe on this hardware - stream, don't buffer - (see Phase 10). +dashcam-sized file (hundreds of MB) in memory is unsafe on this hardware - stream, don't buffer +(see Phase 10). - **Git push auth**: if `git push` fails with a credential/auth error, run `gh auth login` - (device-code browser flow) then `gh auth setup-git` to wire git to use the `gh` CLI's stored - credentials β€” this fixed push auth issues in this environment without needing SSH keys or PATs. +(device-code browser flow) then `gh auth setup-git` to wire git to use the `gh` CLI's stored +credentials β€” this fixed push auth issues in this environment without needing SSH keys or PATs. - **Pi environment gap**: Debian Bookworm blocks system-wide `pip install` (PEP 668, - "externally-managed-environment"). Use a venv for the real app dependency install (Phase 5) - rather than `--break-system-packages` everywhere. (Resolved in Phase 5 β€” venv at - `/home/yaris/app/yaris-obd/venv`.) +"externally-managed-environment"). Use a venv for the real app dependency install (Phase 5) +rather than `--break-system-packages` everywhere. (Resolved in Phase 5 β€” venv at +`/home/yaris/app/yaris-obd/venv`.) - **Pi β†’ GitHub auth for `git pull` (private repo)**: the Pi's pre-existing - `~/.ssh/id_ed25519.pub` is an *inbound* key only (private half is on the dev machine) β€” can't - be reused for the Pi's own outbound git auth. Generated a dedicated keypair on the Pi - (`~/.ssh/yaris_deploy_key`), added it as a GitHub **read-only deploy key** on this repo, and - added an `~/.ssh/config` entry pinning `github.com` to that key. `git@github.com:...` (SSH - remote) then works standalone on the Pi, no PAT/interactive login needed. +`~/.ssh/id_ed25519.pub` is an *inbound* key only (private half is on the dev machine) β€” can't +be reused for the Pi's own outbound git auth. Generated a dedicated keypair on the Pi +(`~/.ssh/yaris_deploy_key`), added it as a GitHub **read-only deploy key** on this repo, and +added an `~/.ssh/config` entry pinning `github.com` to that key. `git@github.com:...` (SSH +remote) then works standalone on the Pi, no PAT/interactive login needed. - **piwheels**: pip on the Pi automatically uses `piwheels.org` as an extra index (Raspberry Pi - OS default `pip.conf`) β€” this is why `gevent` (which normally needs a C compile) installed in - seconds from a prebuilt ARM wheel instead of taking many minutes to build from source. +OS default `pip.conf`) β€” this is why `gevent` (which normally needs a C compile) installed in +seconds from a prebuilt ARM wheel instead of taking many minutes to build from source. - **python-obd note**: `obd.OBD()` (or with `portstr=`) performs the ELM327 handshake at - connection time; if there's no car on the other end, it raises rather than silently returning - a dead handle β€” `app_obd.py`'s `connect_obd()` already catches this and sets `connection = None`. +connection time; if there's no car on the other end, it raises rather than silently returning +a dead handle β€” `app_obd.py`'s `connect_obd()` already catches this and sets `connection = None`. \ No newline at end of file diff --git a/readme.md b/readme.md index c4a9f84..02159c8 100644 --- a/readme.md +++ b/readme.md @@ -59,8 +59,8 @@ Dashboard β€” mobile (390 px) and desktop (1280 px): trip history with OSM/CARTO live map and route replay, dashcam archive browsing with inline video player and Range-request seeking, overview stats dashboard, delete buttons for trips and clips, and a device-settings page for editing the synced owner/car/avatar values that the Pi - pulls from the server. The portal can register multiple devices, reveal a one-time API key at - creation/rotation time, and request a sync for a selected device. When the Pi has + pulls from the server. The portal can register/delete multiple devices, reveal a one-time API + key at creation/rotation time, and request a sync for a selected device. When the Pi has **Sync with server** enabled, those identity changes hot-reload into the running local UI on the next successful sync instead of waiting for a manual restart. Shared `history.js` code runs on both the Pi and the server β€” tile source and zoom limits are configured per-deployment via HTML @@ -71,8 +71,9 @@ Dashboard β€” mobile (390 px) and desktop (1280 px): diagnostics, plus quick actions for server test, sync-now, uplink reconnect, and app restart - πŸ—„οΈ **Motion-aware local-first SQLite logging** β€” five-second motion checks without five-second parked writes, plus retry-safe remote sync and automatic backoff -- 🌑️ **Pi health monitoring** β€” dashboard temperature, RAM, disk space, uptime, and explicit - undervoltage/throttling warnings from the Raspberry Pi firmware +- 🌑️ **Pi health + diagnostics** β€” a dedicated local Status page shows temperature, RAM, disk, + uptime, undervoltage/throttling warnings, service state, uplink/server reachability, GPS state, + OBD presence, and safe quick actions - πŸ”” **Offline-first alerts** β€” a notification bell on every page reports check-engine state, storage pressure, upload backlog, temperature/power faults, and GPS loss while moving. Optional ntfy delivery is available but remains disabled until configured @@ -122,7 +123,14 @@ For the remote server (receives trip logs + dashcam uploads), see [server/README ## Customizing device identity -The Pi displays a username, car model, and avatar image. All customizable values are stored in `config/device.json` (no code edits needed). +The Pi displays an owner name, car model, device name, and avatar. You can manage those values in +two ways: + +- **Locally on the Pi** through the Settings page +- **From the server portal** when **Sync with server** is enabled on the Pi + +The local fallback/default source remains [config/device.json](./config/device.json), so the car +still boots with sensible identity values even when fully offline. ### Edit device configuration @@ -141,10 +149,10 @@ Edit [config/device.json](./config/device.json): |-------|----------|---------| | `owner_name` | Top-right of app topbar | Your name/username | | `car_model` | Left sidebar (next to car icon) | Vehicle model/year | -| `device_name` | (Reserved) | Future server-side use in Phase 25.5 | +| `device_name` | Local settings / remote portal | Device identity used by server sync/provisioning | | `avatar_path` | Top-right avatar circle | Profile picture (relative to `static/`) | -After editing, restart the app: +If you edit the JSON file directly, restart the app: ```bash # If running as a systemd service: @@ -187,15 +195,26 @@ rm db/sql/quoije_Yaris2013.db ## Project status -Phases 1–24.7 are complete. The system is fully built and audited, ready for the Phase 25 real-vehicle test: +The build is now at the endgame stage: the Pi app, remote server, and device-management/sync work +are implemented and verified on the real Pi/unraid setup. The remaining roadmap phases are the +temporary real-vehicle acceptance test and the low-priority Android companion. - **Phases 1–23.5**: core Pi system β€” GPS, OBD, dashcam, trip history, web UI redesign, WiFi upgrade, cellular fallback, alerts, settings, upload - **Phase 24**: remote web portal at `https://yaris.ceaf.dev/` β€” login-protected, trip history with CARTO live map, dashcam archive, delete, overview stats - **Phase 24.5**: full code audit β€” 6 bugs fixed across Pi modules and server, 63 tests passing - **Phase 24.6**: Yaris 2013 OBD compatibility hardening β€” `_decode_odometer` no longer kills the OBD session on unsupported PIDs, sensor globals reset on reconnect, `OBD_SKIP_VOLTAGE_CHECK` escape hatch for clone ELM327 adapters - **Phase 24.7**: diagnostic script (`scripts/diagnose.sh`), map tile switch to CARTO, zoom limit fix +- **Phase 24.8**: deployment/readme cleanup and uplink interface consistency +- **Phases 25.5a–25.5d**: offline-first device identity sync, remote portal device settings, + Pi-side hot-reload, multi-device provisioning, one-time API key rotation/registration, local + Settings UI for remote credentials, Sync now / Test connection, GPS live-status verification, + dedicated local Status page, and status-page quick actions +- **Current known limitation**: the remote portal's device-management page is multi-device, but the + main remote overview/history/dashcam browsing views still follow one configured `DEVICE_NAME` + until a later portal-wide device selector/filter is added -**Next**: Phase 25 β€” temporary real-vehicle test in the 2013 Yaris (see PLAN.md for the full acceptance checklist). Phase 26 (Android companion app) follows after that. +**Next**: Phase 25 β€” temporary real-vehicle test in the 2013 Yaris (see [PLAN.md](./PLAN.md) for +the full acceptance checklist). Phase 26 (Android companion app) follows after that. Phase 14 is partially complete: the Pi's hardware watchdog is deployed and reboot-verified. Read-only-root storage hardening is deferred until persistent storage is moved off the SD root @@ -205,12 +224,12 @@ The browser-based phone relay was removed after real-world testing confirmed tha cannot reliably use the Pi's WiFi and cellular data simultaneously. A native Android companion is planned later instead (Phase 26). -The local Web UI includes a Settings page for safe runtime configuration: dashcam uploads, -uploaded-video retention, cellular APN fallback, optional ntfy delivery, and alert thresholds. -Notification secrets are write-only in the browser API, and network credentials that could lock -the Pi out remain SSH/file-managed. When cellular is configured, small ntfy notifications use the -SIM868 first and WiFi/Ethernet only as fallback; that modem path is implemented and mock-tested but -still needs real-SIM/carrier verification. +The local web UI includes a Settings page for safe runtime configuration: dashcam uploads, +uploaded-video retention, cellular APN fallback, optional ntfy delivery, alert thresholds, local +device identity, and remote server credentials/API key. Notification secrets remain write-only in +the browser API. When cellular is configured, small ntfy notifications use the SIM868 first and +WiFi/Ethernet only as fallback; that modem path is implemented and mock-tested but still needs +real-SIM/carrier verification. The Pi can also join a configured phone hotspot for outbound internet when no SIM is available, including trip synchronization and WiFi dashcam uploads, with automatic recovery to home WiFi or diff --git a/server/README.md b/server/README.md index 550e33f..7fc34b3 100644 --- a/server/README.md +++ b/server/README.md @@ -21,6 +21,13 @@ docker-compose stack on an unraid server (or any Docker host). staged on disk and reassembled + sha256-verified once all have arrived, then recorded the same way as the single-shot endpoint. This is what the Pi actually uses for real dashcam files β€” see "Why chunked uploads" below. +- `GET /api/device-config` β€” device-authenticated config fetch used by the Pi's + identity-sync thread and local **Sync now** action. +- `POST /api/v1/device-config/ack` β€” device-authenticated config-sync + acknowledgement so the server can record the last applied version/time. +- `/settings/device` + `/api/v1/portal/devices*` β€” login-protected multi-device + management UI/APIs for create, delete, rotate key, edit config, and request + sync. Both `/api/v1/tracking/batch` and the dashcam upload endpoints require two headers: `X-Device-Name` and `X-Api-Key` (see @@ -28,13 +35,11 @@ headers: `X-Device-Name` and `X-Api-Key` (see Postgres is **not** published to the LAN β€” only the `api` container talks to it, over the internal compose network. -**CORS is enabled on every response** (`Access-Control-Allow-Origin: *`) so a -device's registered API key can be used directly from browser JS running on a -*different* origin β€” specifically, the Pi's phone-relay feature (see -[PLAN.md](../PLAN.md) Phase 11), where a phone's browser calls this API -straight from a page served by the Pi itself. This is safe here because -auth is header-based (`X-Device-Name`/`X-Api-Key`), not cookie/session-based, -so a permissive origin can't be used to steal a logged-in session. +**CORS is enabled on every response** (`Access-Control-Allow-Origin: *`) for the +device-authenticated API surface. This keeps header-authenticated device calls +usable from browser-based tooling and same-user cross-origin utilities without +introducing cookie/session theft risk, because device auth is explicit +(`X-Device-Name`/`X-Api-Key`) rather than ambient session auth. ## Deploying on unraid @@ -65,12 +70,11 @@ so a permissive origin can't be used to steal a logged-in session. docker compose ps curl http://localhost:8098/health ``` -5. Sign in to the remote portal and open `/settings/device` to edit the synced - owner/car/device/avatar values for the configured `DEVICE_NAME`. If the Pi - has local **Sync with server** enabled, those values hot-reload into the - running Pi UI after the next successful sync. Phase 25.5c also adds a - multi-device list, one-time API-key reveal, key rotation, and a request-sync - action on that page. +5. Sign in to the remote portal and open `/settings/device` to manage devices: + create/delete devices, reveal a one-time API key, rotate keys, edit synced + owner/car/device/avatar values, and request a sync. If the Pi has local + **Sync with server** enabled, those values hot-reload into the running Pi UI + after the next successful sync. ### Upgrading an existing unraid deployment @@ -87,10 +91,12 @@ docker compose up -d --build Phase 25.5a adds the device identity sync columns (`owner_name`, `car_model`, `device_name`, `avatar_path`, `config_version`, `config_updated_at`) used by the -Pi/server config sync flow. The browser portal page at `/settings/device` edits -those values; the Pi applies them on its next successful sync when local -**Sync with server** is enabled, and Phase 25.5b now hot-reloads the in-process -UI fields instead of requiring an app restart. +Pi/server config sync flow. Phase 25.5c adds multi-device provisioning/status +columns (`last_config_sync_at`, `last_config_sync_version`, +`force_sync_requested_at`, `api_key_rotated_at`). The browser portal page at +`/settings/device` edits those values; the Pi applies them on its next +successful sync when local **Sync with server** is enabled, and Phase 25.5b +hot-reloads the in-process UI fields instead of requiring an app restart. ## Registering a device and getting an API key @@ -104,14 +110,23 @@ This prints a one-time API key β€” copy the `YARIS_DEVICE_NAME` / `YARIS_API_KEY` values into the Pi's upload-client config. Only the key's sha256 hash is stored server-side; the raw key is never shown again. +On the Pi, those credentials can now be entered through the local Settings page +instead of editing `db/upload_creds.env` over SSH. The same page also exposes +**Test connection** and **Sync now** actions. + The browser portal at `/settings/device` can now do the same workflow without shell access: - create a new device +- delete a non-default device - reveal a one-time API key - rotate the API key later - request a sync for the selected device +Current limitation: device management is multi-device, but the main remote +overview/history/dashcam browsing pages still scope themselves to one +configured `DEVICE_NAME` until a portal-wide device selector/filter is added. + ## Exposing it outside the LAN Two options, either works with this API unchanged: @@ -149,7 +164,7 @@ byte-for-byte sha256-verified against the source on the Pi. The Pi's web UI has a setting to pause dashcam file uploads while keeping GPS/trip log uploads running β€” no server-side change needed, the Pi simply -stops calling `/api/v1/dashcam/upload` while the toggle is off. +stops calling the dashcam upload status/chunk endpoints while the toggle is off. ## Local development / testing without unraid