# yaris-obd Revival Plan

This document tracks the plan to revive and finish this project: a Raspberry Pi 3-based
OBD-II + GPS car tracker for a 2013 Toyota Yaris, hosting a local web UI over a self-broadcast
WiFi hotspot, with dashcam file retrieval and remote upload of itinerary/dashcam data.

Keep this file up to date as phases complete or plans change, so any future session/agent can
pick up the project without needing prior chat history.

## Hardware

- **Board**: Raspberry Pi 3 Model B Rev 1.2, 906MB RAM. Confirmed too weak to run an AI coding
  agent — all agent-assisted dev work happens on a separate local machine; the Pi is only used
  for hardware bring-up/testing/deployment via SSH (`yaris@<pi-ip>`).
- **WiFi**: Dual-radio setup — **TP-Link Archer T3U (RTL8812BU)** USB dongle as `wlan1` and the
  onboard BCM43438 as `wlan0`. **2026-08-12 role swap**: the fast T3U now hosts the in-car AP
  ("vroom vroom yaris de course" on 5GHz at 10.42.0.1) for fast phone access (Android dev, video
  browsing); the weak onboard radio carries the internet uplink (home WiFi "BuzzBis" +
  phone hotspot, static IP 192.168.50.220, gw 192.168.50.1). Both radios active simultaneously.
  Tradeoff: away-from-home uploads (dashcam/trip sync) now run over the weak onboard radio — a
  background task, accepted by owner. A single radio cannot be AP + client at once, so the
  phone-hotspot uplink cannot also live on wlan1 while it hosts the AP.
- **GPS/GSM**: Waveshare **SIM868 GSM/GPRS/GNSS/Bluetooth HAT**. GNSS provides live GPS fixes
  (confirmed working, ~7 satellites, ~80s cold-start TTFF). GSM/GPRS cellular is **not in use**
  (no SIM card available); a Huawei **E3372h-153 4G LTE modem** has been purchased as a future
  cellular uplink replacement (FEAT-04). SIM868 GPS antenna confirmed connected to correct u.FL
  port (was briefly disconnected — reconnected and verified 2026-08-11).
- **OBD-II adapter**: a cheap unbranded USB adapter using a **PL2303** USB-to-serial chip,
  ELM327-compatible (works with the `python-obd` library). Originally Bluetooth, switched to USB.
- **Power**: Hard-wired 5V/GND directly to GPIO header (red to Pin 2, black to Pin 6), bypassing
  the worn micro-USB connector. `vcgencmd get_throttled` returns `0x0` — no undervoltage or
  throttling. Pi runs at full 1.2 GHz (was ~600 MHz downclocked before the fix).
- **Car**: 2013 Toyota Yaris.

### SIM868 HAT bring-up notes (important gotchas)
- UART routing is a 3-position jumper: **A** = USB↔SIM868 (via onboard CP2102), **B** = Pi
  GPIO↔SIM868, **C** = USB↔Pi console. Must be on **B** for the Pi to talk to the module over
  its own GPIO UART.
- Pi-side UART setup required: `enable_uart=1` and `dtoverlay=disable-bt` in
  `/boot/firmware/config.txt` (frees the full PL011 UART — `/dev/ttyAMA0`/`/dev/serial0` — from
  the onboard Bluetooth controller), `raspi-config nonint do_serial_cons 1` (stops a login shell
  from claiming the serial port), and disable the `hciuart` systemd service.
- **The SIM868 module chip itself needs its PWRKEY button pressed and held (~1-2s) to power on**
  — this is separate from the board's 5V/PWR LED, which can be lit while the module chip is
  fully off. AT commands get zero response if the module chip isn't powered, even with perfect
  UART wiring/config. **Update 2026-08-09: this HAT revision DOES have the "P4" GPIO-power-on
  mod wire installed** (per the Waveshare FAQ/`GSM_PWRKEY.rar` script) — confirmed by pulsing
  physical header pin 7 (BCM GPIO4) LOW for 4s then HIGH while the module was unresponsive, after
  which it immediately answered `AT`. This is now automated in `modules/app_gps.py`
  (`_ensure_module_powered()`), which checks `AT` first and only pulses PWRKEY if the module
  isn't already responding (PWRKEY is a toggle — pulsing an already-on module would turn it back
  off, so this check-first behavior is required). No more manual button-pressing needed after a
  Pi power cycle.
- Confirmed AT command baud rate: **115200**.
- GNSS workflow: `AT+CGNSPWR?` to check GNSS power state, `AT+CGNSPWR=1` to power on,
  `AT+CGNSINF` to poll for a fix. Response format:
  `+CGNSINF: run,fix,utc_time,lat,lon,alt,speed,course,fixmode,reserved1,hdop,pdop,vdop,reserved2,gps_sat_view,gnss_sat_used,gnss_sat_view,cn0,hpa,vpa`
  — field 1 (`fix`) must be `'1'` for a valid fix.
- **The GPS/GNSS antenna is separate from the GSM antenna** — both need to be physically
  connected to their respective u.FL connectors on the HAT. A real fix was obtained only after
  discovering the GPS antenna wasn't plugged in (only GSM was).
- Confirmed real fix obtained near a window: ~80s cold-start TTFF, 5-7 satellites, HDOP ~1.2-1.6.

## Software Architecture Decisions

1. **Offline maps**: Pre-cache/download offline map tiles for the car's usual driving area so
   the live map works fully in the car with no internet (not just numeric GPS readout).
   Tooling: MapTiler Cloud free tier (raster XYZ tiles, `streets-v2` style) — one-time download
   on the dev machine via `scripts/download_map_tiles.py`, output shipped to the Pi as static
   PNGs. Bulk-downloading directly from `tile.openstreetmap.org` was ruled out — it violates the
   OSM tile usage policy. Bounding box is a required CLI arg (the user's own driving area, not
   committed here), zoom 8-15 by default (~14k tiles for a city-plus-suburbs-sized area, well
   under MapTiler's 100k/month free quota).
2. **WiFi hotspot mode**: **Dual-radio** — the fast TP-Link Archer T3U (wlan1) hosts the
   in-car AP ("vroom vroom yaris de course" on 5GHz at 10.42.0.1), while the onboard BCM43438
   (wlan0) carries all internet uplink traffic (home WiFi, phone hotspot) and is never taken
   down by the uplink manager. The hotspot is *only* for local web-UI/Android access while
   driving — it is not part of the upload path. Remote-server uploads happen independently via
   home WiFi (when in range), phone hotspot (Phase 22, now on wlan0), or cellular fallback
   (decision #5). Phone-hotspot auto-connects on wlan0 within ~60s of appearing, falls back to
   home-wifi when it disappears. **2026-08-12 swap** from the previous arrangement (uplink on
   wlan1, AP on wlan0) to put the fast radio on the AP for interactive phone work; the weak
   radio's uplink slowness is accepted because away-from-home upload is a background task.
3. **Local/remote persistence**: Keep SQLite locally (reliable local write path per trip). The
   original plan paired this with MongoDB Atlas for remote sync (`modules/app_db.py`, fixed in
   Phase 7) — **superseded by decision #4/Phase 9**: the actual remote store is the self-hosted
   Postgres server, uploaded via `modules/app_upload.py`. `app_db.py`'s Mongo path is not wired
   into `app.py` and is effectively dead code at this point, kept only for reference.
4. **Remote server**: Self-hosted — a custom server/API (e.g. on a home server or VPS) receives
   itinerary and dashcam uploads, rather than 3rd-party managed cloud services.
5. **Remote upload uplink**: Three paths, tried in order: (1) WiFi/Ethernet via `requests`,
   (2) phone hotspot via wlan0 when configured and in range (Phase 22), (3) SIM868 GSM/GPRS
   cellular fallback (AT-command HTTP client, tracking batches only — no dashcam video).
   Cellular is **untested** (no SIM card). A Huawei E3372h-153 4G LTE modem has been purchased
   to replace the 2G-only SIM868 cellular path once a SIM is available (FEAT-04).

## Phases

### Phase 1: Real GPS integration — ✅ DONE
`modules/app_gps.py` rewritten to read real SIM868 GNSS data via serial AT commands
(`/dev/serial0` @ 115200 baud) instead of hardcoded fake coordinates. Background daemon thread
polls `AT+CGNSINF` every 2s, parses fix data, emits `gps_location` via socketio only on an actual
fix. Verified against real hardware (see bring-up notes above).

### Phase 2: OBD fixes for USB adapter — ✅ DONE
`modules/app_obd.py` fixed (commit "Fix app_obd.py bugs and adapt for USB OBD-II adapter",
pushed to `origin/shameless-aislop`):
- `connection` is never left undefined on failed connect (was a `NameError` crash).
- Added `_find_obd_port()`/`connect_obd()`: auto-detects `/dev/ttyUSB*` (the PL2303 adapter),
  honors an `OBD_PORT` env override, falls back to python-obd's own port auto-scan (still covers
  the old `/dev/rfcomm*` Bluetooth path).
- Fixed stale-response bug: `obd_speed()`/`obd_rpm()` now re-query every loop iteration instead
  of reusing one response forever.
- Fixed `obd_rpm()` using the wrong lock, never updating the global `rpm`, and emitting to the
  wrong socketio event (`obd_speed` instead of `obd_rpm`).
- Fixed `kill_obd()` being a no-op — now uses `threading.Event` flags actually checked in the
  polling loops.
- Fixed inverted thread-start logic (threads were starting on failed connection, not success).
- Fixed `obm_error` → `obd_error` typo (frontend listens for `obd_error`).
- Added `obd` (python-obd) to `requirements.txt`.
- **Still untested**: a real connection to the actual car (adapter has only been tested
  standalone on the Pi so far — correctly reports "no connection to car" with nothing attached).

### Phase 3: Web UI must work fully offline — ✅ DONE
The whole point of the project is a Pi hotspot with **no internet in the car**, but the current
UI silently depended on the internet for core functionality:
- `templates/index.html`/`dash.html`/`dtc.html` loaded Google Fonts, FontAwesome, Leaflet JS/CSS,
  and the Socket.IO client from public CDNs — none of this loads offline, breaking layout, the
  websocket connection itself, and the map entirely.
- `static/js/yaris.js` loaded live map tiles from `tile.openstreetmap.org` (also needs internet).
- A local `static/js/chart.min.js` existed but was unused — `index.html` loaded Chart.js from
  `cdn.jsdelivr.net` instead (turned out the local copy was a stale Chart.js **v2.8.0** while
  `yaris.js`'s config uses v3/v4-style `scales: {x:{...}, y:{...}}` syntax — that mismatch is
  exactly why the CDN was used instead. Replaced the local file with a current v4.5.0 UMD build).

**What was done**:
- Vendored Socket.IO client (4.8.1, matches Flask-SocketIO 5.6.1's Engine.IO v4 protocol) under
  `static/vendor/socketio/`, and Leaflet (1.9.4 JS/CSS + marker/layers images) under
  `static/vendor/leaflet/`.
- FontAwesome 5.12.0 was already vendored locally at `static/fonts/` (self-consistent, just never
  referenced) — repointed all three templates to it instead of the CDN.
- Dropped the Google Fonts (Nunito) `<link>` entirely rather than self-hosting 12 font-weight
  files — Bootstrap's `--bs-font-sans-serif` variable already falls back cleanly to system fonts
  (`-apple-system, Segoe UI, Roboto, ...`) after Nunito, so this is a clean no-cost simplification.
- Replaced `static/js/chart.min.js` with a current Chart.js v4.5.0 UMD build (auto-registering,
  same drop-in behavior as the old CDN `<script src="https://cdn.jsdelivr.net/npm/chart.js">`).
- Offline map tiles: added `scripts/download_map_tiles.py`, a dev-machine-only one-time tile
  downloader using the MapTiler Cloud free tier (raster XYZ, `streets-v2` style, 256px tiles).
  Reads `MAPTILER_KEY` from an env var/CLI arg only — the key itself is never committed or
  deployed, only the resulting static PNG tiles are. Bounding box is a required CLI arg (your own
  driving area's coordinates - not committed/hardcoded) at zoom 8-15 (~14k tiles for a
  city-plus-suburbs-sized area, well under MapTiler's 100k tiles/month free quota). Output goes to
  `static/vendor/tiles/{z}/{x}/{y}.png` (gitignored — regenerate locally, rsync to the Pi as part
  of deployment, don't bloat the git repo with binary tile data).
  - **Tiles populated**: user signed up for a free MapTiler key and the full zoom 8-15 download
    ran to completion — 14,241 downloaded, 24 skipped (out-of-bounds), 0 failed (14,265 tiles
    total, ~285MB). Rsynced to the Pi's `static/vendor/tiles/` (tile counts verified identical
    on both sides). Live Pi instance re-confirmed serving HTTP 200 with the complete offline
    tile set.
- Updated `static/js/yaris.js`'s Leaflet init to point at `/static/vendor/tiles/{z}/{x}/{y}.png`
  (`minZoom`/`maxZoom` 8/15 to match). The initial map center is just a generic placeholder (not
  a real location) - it re-centers automatically to the real live GPS fix once one comes in, and
  the marker is exposed as `carMarker` so Phase 4 can move it on live fixes.
- Added a repo-root `.gitignore` (Python caches, venvs, `.vscode/`, `static/vendor/tiles/`,
  `*.env` except the already-tracked `db/creds.env` placeholder) — there wasn't one before, which
  is why stray `__pycache__`/`.pyc` files kept showing up in source control.
- **Verified**: ran the Flask app in a local venv, confirmed all three pages (`/`, `/dtc`, `/dash`)
  render with **zero external URLs** in the served HTML, every local static asset (JS, CSS, fonts,
  Leaflet images) returns HTTP 200, and all vendored JS files pass a Node.js syntax check.

### Phase 4: Map/GPS display bug — ✅ DONE
`yaris.js`'s Leaflet marker (`carMarker`) was never moved on incoming `gps_location` socket
events — only two text spans (`gpsLocationX`/`Y`) got updated. Fixed: `carMarker.setLatLng(...)`
is now called on every `gps_location` event, and the map re-centers (zoom 15) on the first fix
only (`mapCentered` flag), so the user isn't fighting the map re-centering under them while
panning/zooming during a trip.

**Verified**: also used this as an opportunity to test Phase 3 end-to-end — ran
`scripts/download_map_tiles.py` for real with the user's actual MapTiler API key (smoke-tested
zoom 8-10 first, confirmed idempotent re-run skips existing tiles, confirmed a downloaded tile
is a real map image of the target driving area), then started the full zoom 8-15
download. Took a **headless Chromium screenshot** of the running app's `/` page and confirmed:
sidebar/icons/fonts render correctly (FontAwesome working), the Speed Graph (Chart.js v4) renders,
and **the Leaflet map renders real offline map tiles with the marker in place** — full offline
end-to-end proof, not just "no CDN URLs in the HTML".
- Fixed (was noted here as out of scope, now addressed in Phase 8): `/dash` and `/dtc` used to
  render near-identical body content (both showed "DTC Codes") — `dash.html` now has real
  dashcam-specific content.
- **Full tile download completed** (see Phase 3 update above) and rsynced to the Pi — Phase
  3/4's offline map is now 100% functional end-to-end on the actual deployed device, not just
  the dev machine.

### Phase 5: Pi deployment / autostart — ✅ DONE
Set up a proper venv (`/home/yaris/app/yaris-obd/venv`), installed all requirements (all found
as prebuilt ARM wheels via piwheels — no slow source compiles needed, even for gevent).
Added `deploy/yaris-obd.service` (systemd unit: runs as the `yaris` user, `Restart=on-failure`,
5s `ExecStartPre` sleep to let USB/serial devices enumerate after boot, logs to
`yaris-obd.log`) and `deploy/install_service.sh` (idempotent one-time install helper — also
creates the venv if missing). Installed and enabled on the Pi (`systemctl enable` — survives
reboot).

**Also had to fix Pi → GitHub auth for `git pull`**: the repo is private, and the Pi's existing
`~/.ssh/id_ed25519.pub` turned out to be an *inbound* SSH key (its private half lives on the dev
machine, not the Pi) — useless for the Pi to authenticate outbound. Generated a dedicated
ed25519 keypair **on the Pi itself** (`~/.ssh/yaris_deploy_key`), registered it as a **read-only
deploy key** on the GitHub repo (least-privilege — the Pi only ever needs to pull), and added an
`~/.ssh/config` entry so git transparently uses it for `github.com`. Switched the Pi's `origin`
remote to SSH. `git pull` now works standalone on the Pi with no PAT/interactive auth needed.

**Verified end-to-end**: pulled latest code onto the Pi (was 5 commits stale), confirmed all
Python deps import cleanly, ran the app manually first (HTTP 200, all local static assets —
Leaflet/Socket.IO/Chart.js/FontAwesome — served correctly from the Pi itself), then installed +
started the systemd service and confirmed `active (running)` + `enabled`. **Tested crash
recovery**: sent SIGKILL to the running process, confirmed systemd auto-restarted it (through the
5s `ExecStartPre` delay) and the app was serving HTTP 200 again within ~10s, with zero manual
intervention.

### Phase 6: WiFi hotspot mode — ✅ DONE
Used NetworkManager (already the network manager in use, not hostapd/dnsmasq directly — those
were leftover/unused) since it natively supports AP mode via `nmcli`. Created two connection
profiles on the Pi: `home-wifi` (your real home WiFi network, `autoconnect-priority 10`) and
`Hotspot` (AP mode, `ipv4.method shared`, its own dedicated SSID/password set locally on the Pi -
not committed here, WPA2-PSK, `autoconnect-priority -10`). NM's own autoconnect logic handles
"leaving home WiFi range -> fall back to hotspot" automatically since `home-wifi` has higher
priority and NM always tries it first.

For the reverse direction (home WiFi back in range while sitting in AP mode) added
`deploy/yaris-netwatch.sh` + `.service`/`.timer`: since the Pi's onboard BCM43438 radio can't be
an AP and background-scan for client networks simultaneously, a systemd timer runs every 3
minutes, briefly attempts reconnecting to `home-wifi`, and restores the hotspot if home WiFi still
isn't in range. Installed via `deploy/install_netwatch.sh`.

**Root-caused and fixed a real bug during testing**: the hotspot's SSID broadcast worked
immediately, but its DHCP server initially failed (`dnsmasq: failed to create listening socket
for 10.42.0.1: Address already in use`) — a pre-existing **orphaned system-wide `dnsmasq.service`**
(noted as "likely unused" in earlier investigation) was already bound to the address space
NetworkManager's own internal dnsmasq needs for the hotspot. Confirmed the system dnsmasq's
config was genuinely empty/default (nothing in this project depends on it) and disabled it
(`systemctl disable --now dnsmasq`).

Also added easy access without needing to know the Pi's IP:
- **Renamed the Pi's hostname** from `yarisobd` to `yaris` — `avahi-daemon` (already installed
  and running) now publishes it as `yaris.local` via mDNS.
- **Port 80 -> 5000 redirect** via nftables (`deploy/nftables-yaris.conf` +
  `install_port_redirect.sh`, loaded through an `include` in `/etc/nftables.conf`), so the app is
  reachable at plain `http://yaris.local/` (no port number) instead of `:5000`.

**Verified on real hardware, very carefully** since the SSH session used to do all this testing
itself runs over `wlan0`: used a "dead-man's-switch" pattern for every risky test — a detached
background command (`setsid nohup bash -c 'sleep N && nmcli con up home-wifi' & disown`) armed
*before* each hotspot activation, so the Pi automatically reverts to home WiFi (and SSH access
comes back) even if something goes wrong, no physical access needed. Confirmed: hotspot broadcasts
and a real phone gets a genuine DHCP lease from NetworkManager's own DHCP server and can load the
web UI; `yaris-netwatch` correctly detects AP mode and reconnects to home WiFi when in range (and
is a no-op when already on home WiFi); `http://yaris.local/` and `http://<pi-lan-ip>/` both return
HTTP 200 with no port number from another machine on the network. (Real hotspot SSID/password and
the Pi's LAN IP are intentionally not recorded in this file — see local, non-committed notes.)

**Update — real-hardware bug found + fixed (hotspot->home reconnect was flaky)**: while testing
the phone-relay feature live (see Phase 11 update below), `yaris-netwatch` repeatedly logged "home
WiFi not found" 4 times in a row over ~13 minutes while sitting right next to the router — a real
regression from the original design, not a one-off fluke. Root cause: a plain `nmcli con up
home-wifi` straight out of AP mode relies on NM's scan cache for `wlan0`, which is stale/empty
coming out of AP mode (the radio can't scan while acting as an AP - see above). A single
`--wait 20` connect attempt gave up before a fresh scan could land. Fixed `yaris-netwatch.sh` to
explicitly drop the AP connection first (releases the radio), force `nmcli device wifi rescan`,
wait for it to settle, then retry the connect (longer 30s wait) up to 2 times before conceding and
restoring the hotspot. Bumped `yaris-netwatch.service`'s `TimeoutStartSec` to 180s (retry loop can
take up to ~80s, default systemd 90s start timeout was too tight). Verified fixed on real hardware:
both hotspot->home transitions after the fix recovered in ~7 seconds (previously failing outright).
Deployed: `sudo cp deploy/yaris-netwatch.sh /usr/local/bin/... && cp .service ... && daemon-reload`.

### Phase 7: app_db.py / app_sql.py fixes — ✅ DONE
Neither file was called from `app.py` or anywhere else - orphaned from a prior iteration. Fixed:
- `app_db.py`: applied a prior-session stash fix - resolves `.env` path via `__file__` (was
  cwd-relative), fixes `os.getenv()` calls to pass the actual key names (was called with zero
  args, an immediate `TypeError` if ever invoked), added a clear `RuntimeError` guard if
  credentials are still empty placeholders, returns `client[mDB]`.
- `app_sql.py`: rewritten - resolves `db/sql/` via `__file__` + `os.makedirs` (was cwd-relative),
  fixed `create_conn()` closing the connection immediately and returning `None` (so table
  creation never ran and every caller got a dead connection), added `get_conn()` (lazy singleton
  + table creation for both `tracking` and `dtc` tables), `insert_tracking()`, and a background
  `start_logging()`/`_log_loop()` thread (5s interval) that snapshots GPS (`app_gps.LocationX/Y`)
  + OBD speed (`app_obd.speed`) into the local `tracking` table. Used a deferred (function-local)
  import of `app_obd` inside `_log_loop()` to keep the pre-existing `app_obd`<->`app_sql`
  top-level circular import safe/explicit.
- Also fixed a Python 3.12+/3.14 compat bug found during testing: `sqlite3.version` (the
  sqlite3 *module's* version) was removed/deprecated - switched to `sqlite3.sqlite_version`
  (the actual SQLite library version), which is what was intended.
- Wired `app_sql.start_logging()` into `app.py` alongside the other startup calls
  (`app_gps.start_gps()`, `app_obd.OBD()`).
- `db/sql/` added to `.gitignore` (per-device local trip data, not source).
- Verified locally: `get_conn()` creates `db/sql/<account>_<car>.db` with both tables;
  `insert_tracking()` writes a real row; `start_logging()` thread starts and logs
  "SQL tracking-logger thread started, alive: True"; full `app.py` run confirms the thread
  starts cleanly and (with no real GPS/OBD hardware on the dev machine) correctly logs read
  errors instead of crashing; `app_db.get_database()` raises the new clear `RuntimeError` instead
  of crashing, since `db/creds.env` still holds empty placeholder credentials (real Mongo
  credentials are a user-provided follow-up, out of scope here).

### Phase 8: Dashcam retrieval (app_dash.py) — ✅ DONE, verified on real hardware
Rewrote `app_dash.py` to be brand/device-agnostic: detection via `lsblk -J`, matching partitions
whose parent disk reports `TRAN == "usb"` (this is what safely distinguishes a plugged-in
dashcam/USB stick from the Pi's own onboard storage, which is always `mmc` transport - no more
hardcoded `/dev/sda`). Walks the whole mounted filesystem for video file extensions
(`.mp4/.mov/.avi/.ts/.mkv`) instead of assuming a specific brand's folder layout (DCIM, MOVIE,
NORM/EVENT/PARKING, etc. all just work), preserves each file's relative sub-folder path when
copying (avoids filename collisions across dashcam event/date folders), and skips files already
copied (same relative path + size) so re-running an import is cheap. Mounts read-only to avoid
any risk of corrupting the dashcam's SD card. Dropped the unused/uninstalled `tqdm` CLI-progress
dependency (never actually in `requirements.txt`) since this now runs headless inside the always-
on web server, not interactively.

Wired into `app.py`: `GET /api/dash/devices` (list detected USB storage) and
`POST /api/dash/import` (run the mount->copy->unmount pipeline), both JSON. Fixed the pre-existing
`/dash` vs `/dtc` duplicate-template bug — `dash.html` now has real dashcam content (scan/import
buttons + live status) instead of a copy-pasted "DTC Codes" body. Added a dedicated `dash.js`
(not the shared `yaris.js`, which unconditionally initializes the `/index`-only map/chart and
would throw/crash on `/dash` and `/dtc` since those pages don't have those DOM elements - a
pre-existing bug, worked around by not depending on that shared file for this page's new JS).
Added `dash/videos/` to `.gitignore` (imported footage is per-device data, not source).

**Tested thoroughly without the physical dashcam** (user didn't have it on hand this session):
1. Unit-tested the `lsblk`-JSON detection/filtering logic against a realistic synthetic sample
   (Pi's own `mmcblk0` boot+root partitions plus a simulated USB dashcam partition) - confirmed
   it correctly finds only the USB partition and ignores the Pi's own storage.
2. Integration-tested the real mount -> find-videos -> copy (preserving structure) -> dedupe ->
   unmount pipeline on the actual Pi using a loop-device-backed FAT32 image (`mkfs.vfat` +
   `losetup`) populated with fake nested dashcam video files (`DCIM/100EVENT/...`,
   `DCIM/100NORMAL/...`) - confirmed real mount/copy/unmount all work, confirmed a second import
   run correctly skips already-copied files, confirmed non-video files (e.g. `thumbs.db`) are
   ignored. All simulation artifacts (loop device, test image, temp files) cleaned up afterward.
3. Verified the full `/dash` page + both new API routes locally (Flask dev run + headless
   browser check): auto-scan on page load correctly reports "No USB dashcam/storage device
   detected" (accurate, since none is plugged into the dev machine), Import button correctly
   stays disabled until a device is found, no JS console errors, no regression on `/` or `/dtc`.

**Verified against the real physical dashcam** (Jieli-chipset USB ID `1224:2725`, 29.7GB SD card,
`DCIM/1/VID_###.MOV` layout, 87 files, ~28GB total). This first live hardware test found and fixed
three real bugs the loop-device simulation couldn't have caught:
1. **Superfloppy SD card** - the card has no partition table at all (`lsblk` reports the whole
   `/dev/sda` disk directly as `fstype: vfat`, no child partition) - the original detection assumed
   a partition would always exist. Fixed to also match a `TRAN == "usb"` disk with its own
   `fstype` set directly, not just partitions under it.
2. **Mount permission error** - `mount_device()` called `os.makedirs()` directly, but the service
   runs as unprivileged `yaris` and `/mnt` is root-owned. Fixed to `sudo mkdir -p <mount_point>`
   before the existing `sudo mount`, mirroring that pattern.
3. **OOM crash during upload** (see Phase 10 below) - not a copy-path bug, but discovered
   immediately after the first successful real import.

End-to-end result: full 28GB/87-file import from the SD card completed successfully (resumed
cleanly after being interrupted once by the OOM crash below - `copy_files()`'s skip-if-already-
copied logic meant re-running `/api/dash/import` just picked up the remaining files, no data loss
or re-copy of the first 19 files). SD card unmounted cleanly afterward.

### Phase 9: Remote server for itinerary/dashcam upload — ✅ DONE
Per architecture decisions #4 and #5: self-hosted server/API to receive itinerary (from the
SQLite pipeline) and dashcam file uploads.

Decided/built: Flask API + dedicated Postgres, deployed as a docker-compose stack on the user's
unraid server (separate from any existing containers/DBs there), living in `server/` in this
same repo. Auth: simple `X-Device-Name`/`X-Api-Key` header pair per device, key hash stored
server-side, real key never committed (raw key printed once by `server/scripts/create_device.py`,
saved locally on the Pi only, in gitignored `db/upload_creds.env`). External access: exposed at
`https://yaris.ceaf.dev` via the user's existing Cloudflare Tunnel (public-hostname rule added in
the Cloudflare dashboard, pointing at `192.168.10.10:8098`) — router port-forward + the user's
Unifi firewall is the documented fallback if ever needed instead. Large dashcam uploads can
still prefer a local-network path (e.g. an AdGuard DNS override to bypass the tunnel while on
home WiFi) since tunnels typically cap per-request upload size — noted in `server/README.md`,
not yet needed/built.

Built (`server/` folder):
- `server/sql/init.sql` — Postgres schema: `devices` (name + sha256 api-key-hash), `tracking_points`
  (device_id FK, GPS/speed/odometer + `recorded_at`, unique per device+timestamp so retried
  batches can't duplicate rows), `dashcam_files` (device_id FK, relative_path + sha256, unique per
  device+path+hash so re-uploads are no-ops).
- `server/modules/db.py` — psycopg3 connection pool + query helpers (bulk tracking insert with
  `ON CONFLICT DO NOTHING`, dashcam dedupe-check-before-write).
- `server/modules/auth.py` — `require_device_auth` decorator checking the two headers against
  the stored key hash.
- `server/app.py` — Flask app: `GET /health` (unauthenticated), `POST /api/v1/tracking/batch`
  (bulk JSON insert, capped at 5000 points/request as a sanity guard), `POST /api/v1/dashcam/upload`
  (multipart, streamed to disk with sha256 verification + path-traversal guard on `relative_path`).
- `server/Dockerfile` + `server/docker-compose.yml` — `api` (gunicorn, published on host port
  8098 by default) + `db` (Postgres 17, internal-only, not published to the LAN) services, data
  volume defaults to `server/data/` but pointed at `/mnt/user/appdata/yaris-obd-server` on unraid.
- `server/.env.example` — template for `POSTGRES_PASSWORD`/`API_PORT`/`DATA_DIR`/upload size cap.
- `server/scripts/create_device.py` — run once per device inside the `api` container to register
  it and print its one-time API key. Ran once for device `yaris-2013`.
- `server/README.md` — full unraid deployment + Cloudflare Tunnel exposure + device-registration
  instructions.

Pi-side upload client (`modules/app_upload.py`):
- Background thread, uploads unsent `tracking` rows (batches of 500) and not-yet-uploaded
  dashcam files (sha256-verified) via the API above, with exponential retry/backoff on failure.
- Independent dashcam-upload-only pause toggle (persisted `settings` table + a switch on the
  `/dash` web UI) — trip/GPS logs always keep uploading regardless of the toggle, per explicit
  user request (useful on a slow/metered connection).
- **Backlog-drain tuning**: since tracking rows log every 5s locally, an offline/stale device can
  queue up a large backlog. The loop drains back-to-back (short 1s pause between batches) while a
  batch comes back full, only settling into the slow 30s idle cadence once actually caught up —
  a simulated 1250-row backlog now drains in 3 batches (~0.35s) instead of 13 batches over ~6.5
  minutes with the original fixed 100-row/30s-sleep design.
- `modules/app_sql.py` extended: `UPLOADED` column + migration on `tracking`, `settings` table,
  `dashcam_uploads` dedupe table, and helper functions backing all of the above.

Tested end-to-end against the real deployed server (tracking batch + dedupe, dashcam upload +
dedupe + path-traversal rejection, bad-API-key rejection, toggle-off correctly pausing only
dashcam uploads) and on the real Pi (its actual ~1800-row pre-feature backlog drained and
confirmed matching row counts on the server via direct psql query, live upload status visible on
`/dash`). Committed+pushed (`4ff99f0`, `c4db1ab`, `8dc304e`), deployed on both unraid and the Pi.
Exposed externally at `https://yaris.ceaf.dev` via the user's Cloudflare Tunnel (public hostname
added in the Cloudflare dashboard, pointing at `192.168.10.10:8098`).

### Phase 10: Dashcam upload memory/size fixes — ✅ DONE, verified on real hardware
Two real-hardware-only bugs surfaced once actual ~300MB+ video files (not small test fixtures) hit
the upload path for the first time:

1. **OOM crash from in-memory multipart uploads.** `modules/app_upload.py` originally uploaded
   dashcam files via `requests.post(..., files={"file": f})`. The `requests` library builds the
   *entire* multipart/form-data body in memory before sending when using `files=` - a well-known
   gotcha, not specific to this codebase. On the Pi 3's ~900MB RAM, uploading one ~320MB video this
   way spiked the process past 600MB+ RSS, triggering the kernel OOM killer within minutes
   (confirmed via `dmesg`: repeated `Out of memory: Killed process ... anon-rss:823928kB`, and via
   `pmap -x` showing a single ~362MB anonymous mapping matching the file size almost exactly).
   **Fixed**: added `requests-toolbelt` and rewrote the upload body construction to use
   `MultipartEncoder`, which streams the file in chunks as the request is sent instead of
   materializing it up front. Confirmed via an isolated 300MB test (628MB RSS old way -> 29MB new
   way, byte-identical result) and live on the Pi (RSS stayed flat ~55-59MB during real uploads,
   vs. previously crashing every few minutes).
2. **Cloudflare Tunnel's hard ~100MB request-body cap.** Even after the memory fix, real uploads
   still failed once routed through `https://yaris.ceaf.dev` - a bare `413 Payload Too Large` from
   Cloudflare itself (confirmed: the identical file succeeds direct-LAN, 413s only through the
   tunnel). This is a free-tier platform limit, unrelated to `MAX_DASHCAM_UPLOAD_MB` on the Flask
   side. **Fixed** via a chunked/resumable upload redesign (well under the 100MB cap):
   - Client (`modules/app_upload.py`): `DASHCAM_CHUNK_SIZE = 40MB`; new
     `_upload_video_chunked()` checks `/status` first (enables resume across restarts/failures),
     then POSTs missing chunks one at a time via `MultipartEncoder`.
   - Server (`server/app.py`): new `GET /api/v1/dashcam/upload/status` (reports
     `already_uploaded` or `received_chunk_indices`) and `POST /api/v1/dashcam/upload/chunk`
     (stages chunks under `STORAGE_DIR/.upload_tmp/<device>/<sha256>/`, reassembles + verifies
     SHA256 once all chunks arrive, records via the existing `db.insert_dashcam_file`, cleans up
     the staging dir). A `.total_chunks` marker guards against stale/mismatched partial sets if the
     chunk size ever changes between client versions. The original single-shot
     `/api/v1/dashcam/upload` endpoint is untouched, kept for reference/compatibility.

**Verified end-to-end against the real deployed server through the real Cloudflare tunnel**,
first with a synthetic interrupted-then-resumed 250MB test (status transitions
`not_started` -> partial upload -> `in_progress` w/ correct `received_chunk_indices` -> resume ->
`complete` -> `already_uploaded`, SHA256 match confirmed on disk), then with real dashcam footage:
the Pi copied all 87 real files locally and the background upload thread drained the real backlog
one file at a time, each one verified byte-for-byte (SHA256 match Pi vs. server) and correctly
recorded in Postgres, while the Pi's service ran 50+ minutes straight with RSS staying in a bounded
~55-141MB range (no OOM kills) - a stark contrast to the previous crash-every-few-minutes behavior.
Given home-upload-bandwidth constraints, the full 28GB backlog drains at roughly one ~350MB file
per 4-5 minutes and is expected to take several hours total - this runs unattended in the
background with no user action needed once started.

**Not yet built**: parallel/concurrent chunk uploads (currently strictly sequential, one chunk
then one file at a time) - would speed up the multi-hour full-backlog drain but adds complexity;
not requested by the user yet, flagged here as a possible future optimization if the sequential
pace becomes annoying in practice.

**Known manual step (not yet automated)**: this dashcam model boots into a "select mode" menu
(dashcam / mass-storage / webcam) when it senses USB data lines are live, and needs USB data
communication physically absent to boot straight into recording mode, but present when plugging
into the Pi for import. No code in this repo handles that switch - it's a manual physical step by
the user (e.g. a data-blocking USB cable/adapter, or plugging/unplugging appropriately) each time
before recording vs. before importing. Worth revisiting if it becomes tedious in daily use (e.g. a
Pi-controlled USB data-line switch), but out of scope for now.

### SIM868 cellular fallback for tracking uploads — ⚠️ built, UNTESTED (no SIM card available)
`modules/app_cellular.py` gives the tracking-upload path a fallback over the SIM868's cellular
data connection when WiFi/Ethernet is unreachable. Design notes:
- Uses the SIM868's built-in AT-command HTTP(S) client (`AT+HTTPINIT`/`HTTPPARA`/`HTTPACTION`/
  `HTTPREAD`) rather than a real `ppp0` network interface. Reason: the SIM868's GNSS (polled
  continuously by `modules/app_gps.py`) and its GSM/GPRS radio share the *same* physical UART
  (`/dev/serial0`) — a classic `pppd`/`chat` dial-up would put that port into raw PPP framing for
  as long as the link is up, starving the GNSS polling thread. Staying in AT-command mode lets
  cellular HTTP calls interleave safely through the same serial lock
  (`modules.app_gps._serial_lock`) — each exchange just briefly delays the next GPS poll.
- Only carries small JSON tracking batches, **never** dashcam video — cellular data is far too
  slow/limited for multi-MB video files, and the modem's HTTP AT interface has a small internal
  buffer unsuited to them anyway. Dashcam uploads remain WiFi/Ethernet-only, unchanged.
- `modules/app_upload.py`'s tracking-batch upload always tries WiFi/Ethernet (`requests`) first;
  only on a connection-level failure (no route/DNS/timeout — as opposed to an HTTP error status,
  which means the server WAS reached) does it retry the same batch over cellular.
- Feature-flagged off by default: `YARIS_CELLULAR_ENABLED`/`YARIS_CELLULAR_APN`/
  `YARIS_CELLULAR_APN_USER`/`YARIS_CELLULAR_APN_PASS` added to `db/upload_creds.env`, all blank
  by default so this has zero effect on the already-verified WiFi upload path until the user
  fills in an APN once they get a SIM card. Regression-tested (against the real server) that
  normal WiFi uploads are completely unaffected while cellular is unconfigured.
- `/dash` upload-status card now shows which path (`wifi`/`cellular`) the last successful batch
  used, and whether cellular fallback is configured.
- **Needs validation once a SIM card is available** — exact AT command dialect/timing (SIMCOM
  firmware revisions vary slightly), and whether `AT+HTTPSSL` works for the HTTPS
  `yaris.ceaf.dev` endpoint on this specific module's firmware (if not, a documented fallback is
  a plain-HTTP router port-forward path, avoiding the need for the modem's TLS stack to validate
  a cert chain).

### Phase 11: Phone-relay dashcam upload — ⛔ REMOVED (see update below - hit a hard platform limitation)
Root cause investigation into "why is the transfer from the Pi to the server so slow?" (real
dashcam backlog draining at only ~350MB/4-5min) found the bottleneck is the **Pi's own onboard
BCM43438 WiFi chip** (2.4GHz-only, congested/interfered channel) — confirmed via `iperf3`
raw-TCP tests showing only ~14.7Mbit/s with active retransmits even over a same-subnet direct-LAN
hop that never touches the internet/Cloudflare, ruling out ISP/Cloudflare limits. Since the Pi 3 B
has no 5GHz radio and can't be fixed in software, added a way to bypass its weak WiFi for the
internet-bound leg entirely: a phone opens the same `/dash` page (over the Pi's hotspot/home WiFi)
and manually relays pending dashcam files straight to the remote server using the phone's own
network (WiFi or cellular data), instead of routing everything through the Pi.

**Zero new remote-server upload logic** — the phone's browser JS calls the *exact same*
`/api/v1/dashcam/upload/status` + `/api/v1/dashcam/upload/chunk` endpoints the Pi's own Python
client already uses, in the same 40MB-chunk protocol, just orchestrated by `fetch()`/`FormData`
instead of `requests`/`MultipartEncoder`. The server dedupes purely by SHA256 regardless of which
device sends the bytes, so no server-side changes were needed for the upload path itself — only a
CORS `after_request` hook (`server/app.py`) was added, since the phone's browser calls
`yaris.ceaf.dev` from a page served by a *different* origin (the Pi), which browsers block without
`Access-Control-Allow-*` headers on both the real request and its preflight `OPTIONS`. Wildcard
origin is fine here — auth is header-based (`X-Device-Name`/`X-Api-Key`), not cookie/session-based.

**New Pi-side endpoints** (`app.py`):
- `GET /api/dash/pending` — lists not-yet-uploaded dashcam files (`relative_path`, `sha256`,
  `size`) via a new `app_upload.list_pending_dashcam_files()`.
- `GET /api/dash/file/<path:relpath>` — serves a dashcam file's bytes for the phone to download,
  via `send_file(..., conditional=True)` for **Range-request support** (206 Partial Content) so
  the phone downloads (and immediately re-uploads) one 40MB slice at a time instead of ever
  holding a whole 300+MB file in the browser's memory — the exact same memory-safety lesson
  learned the hard way in Phase 10, applied symmetrically to the download side. Path-traversal
  safe via Werkzeug's `safe_join` against `app_dash.DEFAULT_DESTINATION`.
- `GET /api/dash/relay/config` — hands the phone the same `SERVER_URL`/`DEVICE_NAME`/`API_KEY`
  the Pi's own client uses, so its JS can auth directly with the remote server. Same trust
  boundary as the rest of this project's unauthenticated local UI (anyone on the Pi's own
  hotspot/LAN is already implicitly trusted).
- `POST /api/dash/relay/heartbeat` / `POST /api/dash/relay/stop` — a **lease-based pause**
  (`RELAY_LEASE_SECONDS = 45`) for the Pi's own background dashcam uploader, chosen over a plain
  on/off toggle specifically to self-heal if the phone's browser tab closes/loses connection
  uncleanly (a simple toggle could get stuck "paused" forever). The phone page heartbeats every
  15s while relaying; `_upload_dashcam_files()` skips entirely while a lease is active, re-checked
  between files so it stops promptly if a relay starts mid-drain. Also released via
  `navigator.sendBeacon()` on `beforeunload` as a best-effort immediate release.
- `POST /api/dash/relay/mark_uploaded` — lets the phone tell the Pi "I just uploaded this file"
  right after a successful relay, so the local dedupe table (and the pending list / Pi's own
  background loop) reflect it immediately instead of waiting for the next full backlog scan.

**Performance side-fix**: `modules/app_upload.py`'s dashcam loop previously recomputed SHA256 for
*every* candidate file on *every* loop iteration, including files long since uploaded — wasteful
CPU/IO against a growing, possibly 20GB+, local backlog. Added a `dashcam_hash_cache` SQLite table
(`modules/app_sql.py`, keyed by `relative_path`+`size`+`mtime`) so a file's hash is only computed
once; both the existing background uploader and the new pending-files endpoint benefit.

UI (`templates/dash.html` / `static/js/dash.js`): a new "Relay via phone" card on `/dash` lists
pending files with a per-file "Relay" button and a "Relay all pending" button (manual trigger only
— chosen deliberately over an automatic one, per explicit user preference), plus a "Stop relay"
button. Chunk downloads/uploads retry up to 3 times with backoff before surfacing an error, and a
resumed relay re-checks `/status` first so a previously-interrupted file picks up from its first
missing chunk instead of restarting.

**Tested**: a full local end-to-end simulation (no physical phone/network needed) drove the real
Pi `app.py` and real `server/app.py` Flask apps side by side (Postgres mocked at the `db` module
boundary only — all HTTP routing, chunking, auth, and CORS logic is real), reproducing the exact
sequence the phone's browser JS performs: list pending -> check remote `/status` -> download each
chunk from the Pi via `Range` requests (verified real `206 Partial Content` responses) -> upload
each chunk to the mock-backed server -> verify the reassembled file matches the original
byte-for-byte via SHA256 -> mark uploaded on the Pi -> confirm it drops out of the pending list.
Also verified: path-traversal attempts on `/api/dash/file/` correctly 404, the relay
heartbeat/stop lease correctly pauses/resumes `_upload_dashcam_files()`, and the sha256 cache
table is populated and reused across repeated `/api/dash/pending` calls.

**Deploy gotcha found**: the CORS fix in `server/app.py` was initially deployed to the Pi but not
rebuilt on the actual remote server (the server's Docker image bakes `app.py`/`modules/` in at
build time via `COPY`, so a plain `git pull` + container restart isn't enough — needs
`docker compose build api && docker compose up -d api`). Without it, the phone's cross-origin
`fetch()` calls to `yaris.ceaf.dev` were silently blocked by the browser's CORS policy, which
surfaces in JS as a generic, unhelpful `TypeError: Failed to fetch` — worth remembering as the
first thing to check if this error reappears after a future server-side change.

**Tested for real against a real phone and the real deployed server** (first end-to-end proof
beyond the local simulation above): the initial `/api/dash/pending` call took ~25 minutes for a
one-time full-backlog SHA256 hash of all 87 files (~28GB, no cache existed yet) — after that,
repeat calls returned in ~0.13s. The user then successfully relayed a real file from their phone
(home WiFi) partway through (10/~12 chunks) before stopping to check something important: **is
the relay actually faster, given the phone downloads from the Pi over the same congested radio
diagnosed as the bottleneck?** Measured directly from chunk-arrival timestamps on the server:
**~1.11MB/s (~8.9Mbit/s) for the phone-relay path — statistically the same as the ~9-10Mbit/s
measured for the Pi's own direct Cloudflare upload** earlier this session. **Conclusion: when the
phone is on the same home WiFi as the Pi, the relay provides no speed benefit** — the
download-to-phone leg pays the exact same slow-radio cost as a direct Pi upload would, and the
phone's reupload leg then goes out through the same home internet connection anyway, adding an
extra hop for no gain. The relay only has genuine value when the phone's *second* leg differs
meaningfully from the Pi's own path — e.g. cellular data with better upload than the home ISP, or
uploading after physically leaving home with files already grabbed locally. Stopped the test relay
(`POST /api/dash/relay/stop`) and confirmed the Pi's own background uploader resumed the same
file's chunked upload exactly where the phone left off (shared `/status`/`.upload_tmp` state on
the server made this seamless, no wasted re-transfer).

**Next idea (not yet built)**: the user wants a way to browse/preview individual dashcam clips on
the phone and select specific ones to view or upload (as opposed to bulk-relaying the entire
backlog) — useful for "just grab this one important clip" rather than a wholesale speed
optimization. Not scoped/designed yet.

**Update — tested the "away from home" hotspot+cellular scenario for real, found a real phone-OS
limitation (not a code bug)**: the user asked to test the specific scenario the relay feature was
actually designed for — phone connects to the **Pi's own hotspot** (not home WiFi) to download
pending files, then re-uploads using the phone's **cellular data**, bypassing the Pi's own (in
hotspot mode, nonexistent) internet connection entirely. Confirmed via code review this needs zero
changes: `/api/dash/relay/config` already hands the phone the public `https://yaris.ceaf.dev`
`SERVER_URL` (not a LAN-only address), so the re-upload leg was always going to be internet-bound
regardless of which physical network carries it.

Live-tested with a real dummy file (dashcam upload toggle paused first so the Pi's own uploader
couldn't grab it before the phone did): got `TypeError: Failed to fetch` trying to relay. Confirmed
via a direct test (loading `https://yaris.ceaf.dev/health` as a plain page navigation, not a
`fetch()`, ruling out CORS) that **the phone's own OS was declining to route internet-bound traffic
over cellular while still associated with the Pi's no-internet-gateway hotspot WiFi** — it only
succeeded once the phone auto-fell-back to the home WiFi network (which does have internet)
instead. This is a phone/OS-level "smart network switching" behavior (common on both iOS and
Android to avoid using a WiFi network with no internet gateway) that cannot be fixed in this
project's code — the architecture is sound and was already proven correct in the local end-to-end
simulation and the byte-for-byte reassembly tests earlier in this phase.

**Practical implication**: testing this exact scenario **at home is inherently confounded** — the
phone always has a known-good home WiFi it can silently fall back to, defeating the test. A true
validation needs either (a) actually being away from home with no known WiFi in range, forcing the
phone to use cellular for internet traffic, or (b) manually disabling the phone's WiFi (or its
"switch to mobile data automatically" setting) while connected to the Pi's hotspot, to force
cellular use for this one operation.

**Update — retested more rigorously, confirmed this is a fundamental architecture-level dead end,
not just an untested corner case.** Re-ran the same live test multiple times with longer/managed
"dead-man's-switch" windows (`yaris-netwatch.timer` explicitly stopped during each test run so it
wouldn't yank the Pi back to home WiFi mid-test — an earlier run's failure was actually caused by
netwatch reconnecting home WiFi under the phone, see Phase 6 update, not a phone-network issue).
With netwatch correctly disabled and the Pi confirmed to stay in AP mode for the whole test window,
still got `Failed to fetch`. Directly tested a plain page navigation (not `fetch()`) to
`https://yaris.ceaf.dev/health` with WiFi *fully turned off* (pure cellular) — **that also failed**,
while every other website loaded fine over the same cellular connection with WiFi off. This
narrows the cause precisely: **the browser/phone genuinely cannot reach the public internet at all
while `wlan0`/the phone's WiFi radio is associated with the Pi's hotspot, full stop** — this is not
a settings toggle problem (checked Pixel-specific "switch to mobile data automatically" options,
not present on this device/Android version) but a fundamental **OS-level "default network" routing
decision**: a device's default route for *all* IP traffic (including a web page's `fetch()` calls)
is a single, whole-device choice made by the OS's connectivity manager, not something a browser
page's JavaScript can override per-request. Real per-request network binding (e.g. Android's
`ConnectivityManager.bindProcessToNetwork()`) is an API only available to **native
apps**, not exposed to web pages at all, in any browser, on any phone. Given this, the phone-relay
feature as designed (a web page doing simultaneous WiFi-to-Pi + cellular-to-server transfers)
**cannot ever work while the phone is connected to the Pi's no-internet hotspot** — this isn't a
bug to fix, it's a hard platform ceiling.

**Decision: removed the phone-relay feature entirely** (was Phase 11, now reverted - see below).
Two ways to actually deliver on the original "upload from the road without the Pi's slow WiFi"
goal were discussed: (1) restructure as two distinct, non-simultaneous phases (fully download to
the phone's local storage first while on the Pi's hotspot, then have the user manually switch to
cellular, then upload from local storage) - no native app needed but a more awkward two-step user
flow, or (2) a real native Android app, which *can* hold both networks open at once via
`bindProcessToNetwork()` for a true simultaneous relay. Neither has been built - shelved for now
in favor of simplifying the dashcam retrieval UI (see the new phases below). The Pi's own
background uploader (plain WiFi/Ethernet, optionally SIM868 cellular fallback for tracking data
only) remains the only upload path; it's just slower when the Pi's own WiFi is weak, with no
workaround currently in place for that specific case.

**Side-fix from this test session**: found and fixed a real, unrelated bug in `yaris-netwatch.sh`
during this same test — see the Phase 6 update above.

### Phase 12: Remove phone-relay feature — ✅ DONE
Undo Phase 11 cleanly given the Phase 11 update above (fundamental browser platform limitation,
not fixable). Removed:
- `app.py`: `/api/dash/pending`, `/api/dash/file/<path>`, `/api/dash/relay/config`,
  `/api/dash/relay/heartbeat`, `/api/dash/relay/stop`, `/api/dash/relay/mark_uploaded` routes
  (and the now-unused `os`/`send_file`/`abort`/`safe_join` imports they needed).
- `modules/app_upload.py`: `list_pending_dashcam_files()`, `relay_heartbeat()`/`relay_stop()`/
  `is_relay_active()`, `RELAY_LEASE_SECONDS`/`_relay_lease_until`, and the `is_relay_active()`
  check inside `_upload_dashcam_files()`/`get_status()`.
- `templates/dash.html`: the "Relay via phone" card.
- `static/js/dash.js`: all relay JS (`relayOneFile`, `fetchChunkWithRetry`,
  `uploadChunkWithRetry`, heartbeat timers, pending-file list rendering, etc).
- `server/app.py`: the CORS `after_request` hook, added solely for the phone's cross-origin
  browser calls (the Pi's own Python `requests`-based client was never subject to CORS - that's
  a browser-only enforcement mechanism).
- Kept untouched, as planned: the chunked upload protocol itself
  (`/api/v1/dashcam/upload/chunk` + `/status`), `dashcam_hash_cache` table + `_hash_dashcam_file()`
  (still used by the Pi's own background uploader to avoid rehashing its 20GB+ backlog every
  loop), `DASHCAM_CHUNK_SIZE`/Cloudflare 100MB workaround - none of that is relay-specific.

**Verified**: `python3 -m py_compile` on all touched Python files, `node --check` on `dash.js`,
grepped all five touched files for any leftover `relay`/`RELAY` reference (none found). Smoke-
tested by actually importing `app.py` in a scratch venv and hitting it with Flask's test client:
app imports cleanly (GPS/OBD/SQL/upload startup all log their normal "no hardware" messages, no
crash), `app.url_map` confirms every relay route is gone while `/`, `/dash`, `/dtc`, and the
remaining dashcam/upload API routes are still registered, and both `/` and `/dash` render with
HTTP 200 (`/dash`'s HTML contains zero occurrences of "relay"). Deployed to both the Pi
(`git pull --ff-only` + `systemctl restart yaris-obd.service`, confirmed `active` and `/dash`
returns HTTP 200 with no "relay" text over the real network) and the remote server (needed a full
`docker compose build api && docker compose up -d api` on unraid, not just a restart, since the
CORS-hook removal is baked into the image at build time - confirmed `/health` still returns
`{"status":"ok"}` and the response no longer carries any `Access-Control-*` header).

**Update 2026-08-09**: Phase 13 (OBD hardening) was completed next; see its section below.
Phase 15 is the recommended next software phase because Phases 16/17 and the later list-based
features depend on its shared UI foundation. Phase 14 remains independent but requires a careful
hardware/filesystem deployment pass rather than ordinary application deployment.

### Phase 13: OBD real-hardware readiness (code hardening) — ✅ DONE
**Reordered from Phase 17 on 2026-08-09** because it was completed directly after Phase 12;
the phase numbers now follow actual execution order instead of leaving completed work below
several still-planned phases.
**Independent of everything else in this plan — only touches `app_obd.py`/`app_events.py`, no UI
dependency for the backend part, so this can be done anytime/in parallel with the phases above.**
Nothing has been installed or tested in the actual 2013 Yaris yet - `modules/app_obd.py` has
only ever been tested standalone on the Pi with nothing plugged into the OBD-II port (correctly
reports "no connection to car"). Per explicit user decision, this phase is **code hardening
only** - the user will handle the physical install/bring-up testing themselves. Planned fixes,
all in `modules/app_obd.py` / `modules/app_events.py`:
- **Add a genuine background reconnect loop.** Today, `connect_obd()`/`obd_connection()` only
  ever run once per browser tab's socket `connect` event (`app_events.py`'s
  `connect_event()`) - there is no periodic retry. In the real car, this means: if the page
  happens to load before the adapter/ignition is ready, the connection attempt fails once and
  never retries until another browser tab connects. Add a small polling loop (mirroring the
  pattern already proven in `app_gps.py`'s `_poll_loop()`) that periodically calls
  `connect_obd()` while disconnected, independent of any browser being open.
- **Catch exceptions from `connection.query()`.** Currently a communication hiccup
  (adapter timeout, malformed response, USB blip) would raise inside `obd_speed()`/`obd_rpm()`
  uncaught, silently killing that polling thread forever with no recovery - not just returning
  a null response, an actual crash. Wrap query calls, treat exceptions like a lost connection
  (stop polling, let the reconnect loop above pick it back up) instead of dying silently.
- **Add an `OBD_PROTOCOL` env override**, mirroring the existing `OBD_PORT` override pattern -
  in case the 2013 Yaris's ECU doesn't cleanly auto-negotiate via ELM327's `ATSP0` (auto) and
  needs an explicit protocol code forced (Toyotas this age are normally ISO 15765-4 CAN 11-bit
  500kbaud, but this has never been confirmed against this specific car).
- **Improve connection-failure diagnostics.** Distinguish and log clearly: no adapter detected
  at all (USB/port issue) vs adapter detected but ELM327 handshake failed vs handshake OK but no
  response from the car's ECU (car off/ignition not on) - currently all three collapse into the
  same generic "OBD connection failed" message, which will make first-real-car troubleshooting
  much harder than it needs to be.
- **Add a small connect-time diagnostic log** (ELM327 firmware version + negotiated protocol,
  both available from `python-obd`'s connection object) written once per successful connect -
  useful evidence to sanity-check against Toyota's expected protocol on the very first real test.
- **Expand the data pulled from the car beyond speed/RPM** (backend only - the dashboard cards
  displaying this land in Phase 23 once the redesign reaches them): coolant temperature, fuel
  level, throttle position, and DTC/MIL status (check-engine light on/off + stored trouble codes,
  reusing/wiring up the already-scaffolded but currently-empty `/dtc` page and `GET_DTC` command).
  **Odometer/mileage is not guaranteed** - the standard OBD-II mode 1 PID for total odometer
  (0xA6) was only added to the SAE J1979 spec in later revisions and is inconsistently supported
  across manufacturers/years; whether the 2013 Yaris's ECU exposes it at all can only be
  confirmed once tested against the real car (falls under "code hardening only" for now - add
  the query with graceful "unsupported" handling rather than assuming it will work).

**Implemented 2026-08-09**:
- Added an idempotent, browser-independent reconnect daemon started from `app.py`; it retries
  every 10 seconds and restarts any dead polling workers after reconnect.
- Serialized all python-obd queries so speed, RPM, and the expanded sensor worker cannot issue
  overlapping serial exchanges. Query exceptions and silent connection-state changes now close
  and discard the stale handle, emit disconnected state, and hand recovery back to the reconnect
  loop instead of permanently killing a worker.
- Added `OBD_PROTOCOL` support and distinct logs for no serial adapter, failed ELM327 handshake,
  and ELM327 connected but no ECU response. Successful connections log port, ELM327 firmware,
  negotiated protocol ID, and protocol name once.
- Added backend polling/emits for coolant temperature, fuel level, throttle position, MIL/DTC
  status, stored DTCs, and SAE PID `0xA6` odometer. Odometer uses a custom python-obd command with
  a null/unsupported result left as `None`; support is not assumed for the 2013 Yaris.
- Verified against the Pi's installed python-obd 0.7.3 API, plus focused mocked checks covering
  successful connection, forced protocol, ECU-off status, silent disconnect cleanup, query
  exceptions, and odometer decoding. The Pi currently has `/dev/ttyUSB0` present but is not
  connected to the car, so actual PID availability and negotiated Toyota protocol still require
  the user-planned first in-car test.
- Committed as `8d7909d`, pushed, and deployed to the Pi. After restart the service remained
  `active`, `/` returned HTTP 200, and the log showed four autonomous reconnect attempts with the
  correct "ELM327 handshake succeeded, but the car ECU did not respond" diagnosis and no traceback.

### Phase 14: Power-loss resilience & watchdog — 🟡 PARTIAL (watchdog done; filesystem/shutdown deferred)
**New phase, not code-dependent on anything else — OS/hardware-level, can be done anytime,
ideally before the Pi lives in the car full-time.** A car cuts power abruptly at every key-off;
nothing in the project currently protects against that, and repeated unclean power loss is a
classic cause of SD card corruption on embedded Pi projects. Two complementary angles:
- **Filesystem hardening**: convert the Pi's root filesystem to read-only with a writable overlay
  (`overlayroot`/`raspi-config`'s built-in overlay FS option), so an abrupt power cut can no longer
  corrupt the OS itself. App data that must persist (`db/sql/`, `dash/videos/`, logs) needs to
  live on an explicitly writable partition/bind-mount carved out of the overlay, since those
  still need real writes.
- **Clean shutdown before power loss**: either (a) a small delayed-shutdown power module/circuit
  (holds 5V for a few seconds after ignition-off so the Pi can run `shutdown -h now` cleanly), or
  (b) if the SIM868 HAT or another ADC/GPIO source can sense ignition/voltage state, a software
  voltage-drop watcher that triggers a clean shutdown proactively. Exact approach TBD - depends
  on what's easiest to wire into the existing power setup.
- **Hardware watchdog**: enable the Pi's built-in `bcm2835_wdt` watchdog (via `systemd`'s
  `RuntimeWatchdogSec`/`watchdog-device` support) so a fully hung app or kernel gets a hard reboot
  automatically instead of sitting dead in the car until someone notices.

**Implemented 2026-08-09**:
- Added `deploy/yaris-watchdog.conf` with systemd's hardware watchdog set to 10 seconds and
  `deploy/install_watchdog.sh` to install, activate, and verify it. Deployed to the Pi; confirmed
  `/dev/watchdog0` is the Broadcom BCM2835 watchdog and systemd reports
  `RuntimeWatchdogUSec=10s`.
- Rebooted the Pi and verified the watchdog setting persisted, systemd returned to `running`,
  `yaris-obd.service` returned to `active`, and `/` returned HTTP 200 after startup.
- **Explicitly deferred by user decision**: read-only root/filesystem hardening and clean-shutdown
  hardware. The current 59GB root partition fills the SD card and contains 28GB of dashcam data,
  so overlay root needs a separate persistent writable storage decision first; shrinking the live
  mounted root filesystem is not safe. Clean shutdown likewise needs delayed-power or
  ignition-sense hardware before software can implement it.

### Phase 15: UI design-system foundation — ✅ DONE
**New phase, pulled forward from the old "do the whole redesign at the end" plan.** Building the
Phase 16/17 file browsers (and later the trip-history/Pi-health/settings pages) against the
current old admin-template style just means rebuilding the same UI twice once the full redesign
(Phase 23) happens. Instead, do a *small, scoped* slice of the redesign right now: a shared page
layout/shell plus one reusable list/table component (sortable columns, per-row action buttons,
delete-with-confirmation pattern, empty-state messaging) that every subsequent list-based feature
in this plan (dashcam browser, video browser, trip history, settings) can be built on directly,
instead of writing bespoke list/delete UI code from scratch each time. Not a full redesign of
every existing page yet - just establishes the pattern once so nothing built after this point
needs to be redone in Phase 23.

**Implemented 2026-08-09**:
- Added a shared Jinja shell (`templates/base.html`) for navigation, top bar, footer, common
  styles/scripts, page titles, and per-route active navigation. Converted `/`, `/dash`, and
  `/dtc` to extend it, removing their duplicated shell markup while preserving page content and
  page-specific assets.
- Added a reusable data-table macro (`templates/components.html`) and generic
  `static/js/data-list.js` behavior for dynamic empty states, numeric/text sorting, accessible
  sort direction, and capture-phase confirmation that can stop destructive row handlers before
  they execute. `static/yaris.css` supplies the shared list/action styling.
- Used the empty `/dtc` page as the first component consumer without pulling its Phase 23 OBD
  wiring forward. Also stopped loading dashboard-only `yaris.js` there and guarded its optional
  `#obd-error` target after browser testing exposed an existing null dereference.
- Verified Jinja rendering and one active nav item on all three routes, JavaScript syntax, and
  live Pi HTTP 200 responses. Browser-tested all pages at 1280px and 390px with no horizontal
  overflow; injected sample rows confirmed ascending sort, `aria-sort`, empty-state switching,
  and cancellation of a destructive handler.

### Phase 16: Dashcam USB file browser (list + delete) — ✅ DONE
Replace the current all-or-nothing "Import videos" flow's blind spot (no visibility into what's
actually on the camera) with a real file browser for the mounted USB dashcam device, built on the
Phase 15 shared list component:
- New endpoint(s) in `modules/app_dash.py` to list every video file on the mounted dashcam
  (relative path, size, modified time) without copying anything - reuses the existing
  `find_video_files()` walk, pointed at the dashcam's mount point instead of the local
  destination folder.
- Delete support: individual file delete and a "delete all" bulk action, each requiring the
  mount to be briefly remounted read-write (currently always `ro`) - only for the duration of
  the delete call, then back to read-only (or unmounted) afterward, to minimize risk of
  filesystem corruption from an unclean power loss while writable.
- UI: a `/dash` card showing total file count + total size on the dashcam, a scrollable file
  list with a delete button per row, a "Delete all" button behind a confirmation dialog (real
  data loss risk - no undo).
- Explicitly deferred/out of scope: no in-browser preview/playback of files still on the
  dashcam itself (only after import - see Phase 17). No changes to the USB
  select-mode/data-communication toggle behavior - the user replugs power manually to return the
  camera to recording mode, as they already do.

**Implemented 2026-08-09**:
- Added list, single-delete, and delete-all backend operations using the existing USB-only storage
  detection. Each list/import/delete operation shares one lock so mount and file operations cannot
  race; listings include relative path, byte size, modified timestamp, count, and total size.
- Deletes accept only recognized video extensions, resolve and re-check every target beneath the
  selected mount, and reject absolute paths, traversal, missing files, and symlink escapes.
- Delete operations mount or remount the dashcam read-write only for the operation, then unmount
  or restore it read-only. If unmount fails, the cleanup path immediately attempts a read-only
  remount and reports a cleanup failure if neither safety action succeeds.
- Added `/api/dash/files` and `/api/dash/file` routes plus a `/dash` browser built on Phase 15's
  shared component: file/size/time sorting, totals, refresh, per-row confirmed delete, confirmed
  delete-all, empty/no-device states, and error feedback.
- Verified metadata, traversal rejection, one/all deletion, non-video preservation, import
  regression behavior, and cleanup-failure reporting against temporary filesystem fixtures.
  Deployed to the Pi; the live API correctly returned no storage devices while ignoring the
  attached PL2303 OBD adapter, and the responsive browser UI passed no-device and mocked populated
  list/sort/confirmation/delete tests.
- **Real-hardware deletion verification (2026-08-09)**: connected the 29.7GB FAT dashcam and listed
  all 87 original videos (29,615,111,758 bytes) through the live API. The first controlled delete
  exposed that FAT files were root-owned despite the read-write remount, so unprivileged
  `os.remove()` failed; added a privileged `sudo rm -- <validated-path>` fallback only after the
  existing extension, realpath, and mount-boundary checks pass. Created and deleted a tiny
  disposable `YARIS_OBD_DELETE_TEST.MP4` twice through the real API. Both cycles returned exactly
  to the original 87-file count and byte total, left every real video untouched, unmounted
  `/mnt/dashcam`, and kept the service active. Bulk delete uses the same validated per-file removal
  path but was not run against the real card because it would intentionally erase all footage.
- **Selected import enhancement and real-hardware verification (2026-08-09)**: added per-row
  Import buttons, checkboxes, Select all/Clear selection, and Import selected so importing is no
  longer all-or-nothing. The backend validates every selected path with the same mount-boundary
  protections as deletion, de-duplicates selections, preserves relative folders, tolerates a
  stale/missing file while importing the remaining valid selection, and keeps the existing
  same-size skip behavior. Before committing, paused background dashcam uploads, created a
  disposable `YARIS_OBD_IMPORT_TEST.MP4` on the physical card, selected it in the real mobile-width
  UI, and confirmed exactly one file copied with identical contents. A row-level repeat correctly
  skipped the existing copy; Select all selected all 88 visible rows and Clear selection reset the
  action. Removed both disposable copies, restored uploads, and verified the card returned to its
  original 87 files/29,615,111,758 bytes unmounted. The refined UI was then re-tested against a
  real existing video: it reported one skip, cleared the checkbox, and re-enabled all controls.

### Phase 17: Local imported-video browser + player — ✅ DONE
Same file-browser treatment for videos already imported to the Pi's own disk
(`dash/videos/`, populated by `import_dashcam()`), built on the Phase 15 shared list component:
- List endpoint (reuses `find_video_files()` against `app_dash.DEFAULT_DESTINATION`) showing
  relative path, size, modified time, and upload status (cross-check against
  `app_sql.dashcam_file_marked_uploaded()` - so it's visible at a glance which imported clips
  have already reached the remote server and which are still pending the Pi's own background
  uploader).
- Delete support: **local Pi copy only** - remote server copies are intentionally left alone
  as the safe backup/archive (explicit user decision - see 2026-08-09 discussion). No
  "also delete from server" option is being built for now.
- In-browser video player: since the dashcam records **MP4** (confirmed with the user), this
  is a plain HTML5 `<video>` element - no server-side transcoding needed. Reuses the same
  Range-request-capable file-serving pattern already proven in the (now-removed) relay's
  `/api/dash/file/<path>` endpoint (needed for correct seeking/scrubbing, not just playback).
- UI: a `/dash` card (or a dedicated sub-page if the list gets long) with a file list, a
  "Play" action that opens the video inline, and a delete button per row with a confirmation
  prompt.
- **QOL: video thumbnails.** Scrolling a list of `VID_00234.MOV`-style filenames to find a
  specific clip is rough. Grab one low-res frame per file via `ffmpeg` (cheap, done once per file
  and cached, not on every page load) so the browser shows an actual thumbnail per row instead of
  just a filename.
- **QOL: retention/auto-cleanup policy.** Once a file is confirmed uploaded to the remote server,
  it's safe to free local space. Add an optional automatic cleanup (configurable via the Phase 21
  Settings page - e.g. "delete local copies older than N days once uploaded", off by default)
  so the Pi's limited storage doesn't need to be babysat manually as the dashcam backlog grows.

**Implemented and real-hardware verified 2026-08-09**:
- Added local-video list, playback, thumbnail, and local-only delete APIs. Listing reports path,
  size, modified time, and exact upload status without rehashing: one SQLite join matches the
  existing `(relative path, size, mtime) -> SHA256` cache against confirmed uploads.
- Added a second `/dash` browser using the Phase 15 table component, with lazy thumbnails, sortable
  file/size/time/upload columns, Uploaded/Pending badges, inline HTML5 video playback, confirmed
  "Delete local" actions, totals, and retention status. Remote copies are never deleted.
- Playback uses Flask conditional file serving; a real `Range: bytes=0-1023` request returned 206,
  the exact `Content-Range`, and 1,024 bytes. Browser-tested a real H.264 1080p MOV at mobile width:
  the player reached readyState 4 with its duration loaded and no media error.
- Added cached ffmpeg JPEG thumbnails plus `deploy/install_video_tools.sh`; installed ffmpeg 5.1.9
  on the Pi. Real clips exposed and fixed the edge case where ffmpeg exits 0 without output at a
  late seek, so generation now retries frame zero and verifies a non-empty file. A normal 366MB
  clip generated in ~4.1s and its cached request returned in ~81ms. Generation uses a cooperative
  gevent semaphore/poll loop so cold thumbnails do not freeze the web server.
- Added off-by-default `local_video_retention_days` support. The upload loop checks it after sync;
  only files older than N days whose exact current content is confirmed uploaded are eligible.
  The Phase 21 Settings page will expose the control; `0`/unset remains disabled. Temporary
  filesystem tests confirmed disabled retention deletes nothing and enabled retention preserves
  pending files.
- Replaced a long-held upload/delete lock with per-path in-use/deleting state and reference counts:
  delete now returns HTTP 409 while that exact file is uploading or generating a thumbnail instead
  of blocking gevent and freezing every request.
- Deployed uncommitted code before committing and tested against the real 87-file/28GB Pi library:
  all 87 reported uploaded, total bytes matched 29,615,111,758, and retention reported off. Copied
  a tiny real clip to `YARIS_OBD_LOCAL_TEST.MOV`, verified it appeared Pending, played it in the
  browser, and deleted it through the confirmed UI action; the player closed and the library
  returned to exactly 87 uploaded files. Background uploads were paused for the disposable test,
  restored afterward, and the service remained active.

### Phase 18: Trip history & replay — ✅ DONE
Added a derived, idempotently rebuilt `trips` index over the original non-destructive `tracking`
stream, plus `/api/trips`, bounded `/api/trip/<id>` route responses, and a responsive `/history`
page using the Phase 15 sortable list and the existing offline Leaflet bundle. Trip rows and replay
details show distance in kilometres/miles and average/maximum speed in km/h and mph; speed metrics
prefer OBD wheel speed and fall back to plausible GPS-derived segment speed. The map replay includes
a timestamped scrubber and play/pause timeline that moves a vehicle-position marker through every
recorded point while showing its date, time, and recorded speed.

**Motion-boundary decision after auditing the real Pi data:** the logger records every five seconds
for as long as GPS has a fix, including while parked, so timestamp gaps alone are not trip
boundaries. OBD wheel speed above zero is the primary motion signal. When OBD is absent or stuck at
zero, three sustained GPS displacements of at least eight metres at normal logger cadence act as
the fallback; isolated jumps, implausible jumps, malformed coordinates, stationary jitter, clock
rollback, and five-minute motion gaps are handled explicitly. Trip summaries contain start/end
time and coordinates, Haversine distance, duration, and point count. Route responses retain both
endpoints, filter implausible GPS outliers, and downsample only above 2,000 points. OBD samples are
used only while their last successful query is fresh, preventing a disconnected adapter's stale
last-known speed from creating a false trip. Rebuilds process an overlap after the persisted
high-water mark plus the latest trip rather than loading the unbounded full history every time.

**Implemented and verified 2026-08-09:**
- Audited the live database before choosing the heuristic: 5,132 initial rows covering about 25.5
  hours, no null coordinates, all stored speeds zero, only 114 five-decimal coordinate positions,
  and one old NUL-prefixed latitude. Future coordinates are normalized before insertion; invalid
  snapshots are logged and rejected instead of poisoning trip calculations.
- Replaced unconditional five-second database writes with motion-aware persistence. Sensors are
  still checked every five seconds, but rows are written only for the first fix, fresh OBD motion,
  at least eight metres of accumulated GPS displacement, or a one-minute stationary heartbeat.
  The Pi normally runs only while the car has power, so this is not presented as a full-day parked
  storage problem; it reduces needless writes during bench testing and long idling while retaining
  enough stopped points for traffic pauses in the replay timeline.
- Nine focused detector tests cover sustained GPS fallback, parked jitter/malformed coordinates,
  inactivity splitting, OBD wheel-speed detection and slow-distance calculation, clock rollback,
  route outlier filtering, and route downsampling. SQLite incremental-build integration was also
  exercised against a disposable database, and five logger tests cover first-fix, parked,
  wheel-speed, GPS-fallback, and heartbeat decisions. A copy of the real database
  produced zero false trips across 5,179 parked rows.
- Deployed the uncommitted implementation to the Pi and built the schema against the real database
  without modifying any original tracking rows. Inserted an eight-point disposable GPS route while
  the service was stopped; the live API detected one 0.078km/35-second trip and returned all eight
  ordered replay points. Restored the exact pre-test database afterward, confirmed zero synthetic
  rows, `PRAGMA quick_check=ok`, zero derived trips, and an active service.
- Browser-tested `http://yaris.local/history` through the deployed mDNS/port-80 redirect at a
  390x844 mobile viewport. Empty state, populated sortable row, replay action, offline route
  polyline following every route point, start/end markers, metric/imperial distance and speed
  summary, and no-horizontal-overflow behavior passed. Offline tiles
  now attach only after selecting a route, avoiding irrelevant missing-tile requests around the
  generic empty-map center.
- With user approval, removed the accumulated stationary bench backlog after verification:
  5,454 local SQLite rows and 2,907 remotely synced Postgres rows, all with zero/null speed and no
  detected trip. Preserved a dated pre-cleanup SQLite backup under `/home/yaris/backups/`, vacuumed
  and integrity-checked SQLite, vacuum-analyzed Postgres, and left unrelated settings, upload
  markers, DTCs, and video data untouched. The deployed motion-aware logger then proved two
  consecutive stationary heartbeats were 60.02 seconds apart rather than five seconds.

### Phase 19: Pi health monitoring card — ✅ DONE
Added a dependency-free `/api/health` endpoint and responsive dashboard card reporting CPU
temperature, usable disk pressure/free bytes, RAM usage, human-readable uptime, and Raspberry Pi
firmware power/throttling flags. Data comes from thermal sysfs, `/proc`, `shutil.disk_usage`, and
`vcgencmd get_throttled`; individual unavailable sources are reported explicitly without hiding
the remaining readings. The endpoint exposes structured current and historical flags for Phase
20 alerts, plus warning/degraded/ok status. The browser refreshes every 30 seconds.

**Implemented and verified 2026-08-09:**
- Five focused tests cover memory parsing, missing memory fields, current/historical throttling
  bits, malformed firmware output, and ext4 reserved-block-aware disk pressure. Local non-Pi
  validation correctly reports thermal/firmware sources as unavailable while retaining disk,
  memory, and uptime instead of failing the endpoint.
- Removed the old browser-triggered thread that emitted uptime once per second indefinitely; the
  health poll now provides uptime cheaply with the rest of the system snapshot.
- Deployed uncommitted code to the real Pi. The API returned complete data with no source errors:
  about 47-49°C, 19-20% RAM, 59% usable disk pressure with 22.8GB free, and roughly two hours
  uptime during testing. SQLite/app service behavior remained active.
- Real hardware monitoring immediately found an actionable condition: `vcgencmd get_throttled`
  alternated between `0x50005` (current undervoltage and CPU throttling, both previously occurred)
  and `0x50000` (historical occurrences after the current condition cleared). The dashboard
  prominently distinguishes current warnings from prior events. The Pi power source/cable should
  be corrected before sustained in-car use.
- Browser-tested the real dashboard through `http://yaris.local/` at 390x844 and 1280x900:
  all four metrics, warning state, timestamp, responsive columns, and 30-second data path rendered
  without overflow or new console errors. Also fixed disconnected OBD speed showing `undefined`
  and deferred offline map tiles until a real GPS fix, avoiding irrelevant placeholder requests.

### Phase 20: Alerts/notifications — ✅ DONE
Added an offline-first notification system. A shared bell in the top bar on every web page shows
active local alerts with no modem, SIM card, or internet connection. Optional ntfy delivery mirrors
the same conditions only when explicitly enabled and configured; it defaults off until Phase 21
exposes its topic URL, optional access token, and thresholds.

Monitored conditions:
- MIL/check-engine state and DTC codes.
- Pi filesystem pressure and the physical dashcam USB filesystem, checked read-only and cached for
  five minutes to avoid repeated mounts.
- Pending tracking-row or local dashcam-video upload backlog.
- Pi CPU temperature and Raspberry Pi firmware undervoltage/throttling flags.
- GPS fix loss for a configured duration while fresh OBD wheel speed confirms movement.

The alert engine runs independently every minute and persists active state, first-active time,
last attempt/success, delivery errors, recovery state, and last value in SQLite. It sends once on a
transition, repeats only after the configured cooldown, retries failed deliveries after five
minutes, and sends a recovery notification after a delivered condition clears. GPS cold starts
must remain lost for the full duration before alerting, and stale MIL/DTC values are suppressed
after OBD communication stops. Firmware current flags alert immediately; new historical bits
remain visible for five minutes, and cleared bits are reconciled so the same fault can alert again
after reboot. Failed transient deliveries retry every minute during that window. The bell
explicitly reports a dead/stale monitor instead of silently showing old conditions.
`/api/alerts/status` exposes sanitized diagnostics without the ntfy URL, token, or request errors
that could contain a private topic URL.

**Implemented and verified 2026-08-09:**
- Thirteen focused alert tests cover ntfy request shape/authentication/error redaction, transition
  deduplication, cooldown repeats, recovery, failed-delivery retry, disabled delivery, GPS
  duration/reset, firmware-event recurrence, diagnostic freshness, and dashcam-usage caching. The
  combined targeted suite is 33 tests.
- Deployed uncommitted code and verified the alert thread, state table, status API, and service on
  the Pi. `enabled=false` and `configured=false` produced no external attempt while local alerts
  continued working.
- Tested the ntfy transport against a temporary HTTP receiver bound only to `127.0.0.1` on the Pi:
  path, UTF-8 body, title, priority, tags, and bearer Authorization header all arrived correctly;
  no third-party request or real token was used.
- The real read-only physical-dashcam check found 96.1% usage/1.23GB free, unmounted afterward, and
  the original library remained exactly 87 files/29,615,111,758 bytes. The live bell correctly
  showed that storage alert alongside the real Pi undervoltage/throttling alert.
- Browser-tested the real local bell/dropdown at 390x844 on the dashboard and trip-history page:
  badge counts, alert titles/details, no-alert state, synthetic monitor-failure state, shared-shell
  availability, and no-horizontal-overflow behavior passed with no new console errors.

### Phase 21: Settings page — ✅ DONE
**New phase - depends on Phase 15's shared components, and benefits from Phases 17/20 existing**
so there's real configuration to consolidate. Several settings currently only exist as
SSH-edited files or scattered individual toggles (dashcam upload pause switch on `/dash`,
cellular APN in `db/upload_creds.env`, hotspot SSID/password via `nmcli` directly) or don't exist
yet (retention days from Phase 17, alert webhook/thresholds from Phase 20). Add one dedicated
Settings page consolidating anything reasonably safe to expose over the local-only web UI:
upload toggle, cellular APN, local video retention policy, alert webhook URL + thresholds. WiFi
SSID/password changes likely stay out of scope (higher-risk to expose over a UI that could lock
the Pi out of its own network if fat-fingered) unless it turns out to be easy/safe to add later.

**Implemented and hardware-tested 2026-08-09.** Added a dedicated local `/settings` page and
validated `/api/settings` endpoint backed by the existing SQLite settings table. It consolidates
the dashcam-upload toggle, uploaded-video retention days, SIM868 cellular enable/APN, optional
ntfy enable/URL/token, and alert temperature/disk/backlog/GPS-loss/repeat thresholds. Updates are
validated as a complete group and committed atomically; invalid requests leave the prior
configuration untouched. Cellular enable/APN reads persisted settings live, so those changes do
not need an app restart. Sensitive ntfy URL/token values are write-only through the API: the page
only reports whether each is configured and supports explicit removal, so saved secrets are never
sent back to browser JavaScript. Local Web UI alerts remain active independently of ntfy.

When cellular is configured, ntfy delivery now uses the SIM868's small-body HTTP client as the
primary transport with the notification title, priority, tags, and optional bearer token.
WiFi/Ethernet is the fallback if the modem path fails. The generalized modem POST path rejects
quotes/newlines in AT-command parameters before touching hardware. This is mock/Pi-code tested but
remains explicitly unverified over a real carrier until a SIM is available; Phase 25 must prove
authenticated HTTPS, fallback, retry, and recovery behavior or remove ntfy entirely if the SIM868
cannot deliver it reliably.

Moved the dashcam-upload control from `/dash` into Settings rather than keeping duplicate controls,
while retaining pending-row, last-success/path, and last-error upload health in the new page.
WiFi credentials, remote-server device/API credentials, cellular username/password, and other
lockout/high-risk configuration remain file/SSH managed by design. Focused settings/alert tests
pass locally and on the Pi; the real Pi rejected an invalid update with HTTP 400 while preserving
its exact configuration. The live page loaded through `yaris.local` with the Pi's existing
defaults and shared alert/navigation shell. Browser testing at 390x844 found and fixed an initial
top-bar overlap by adding a dedicated bottom mobile save action; the real save then completed with
no horizontal overflow or console errors.

### Phase 22: Phone-hotspot internet uplink — ✅ DONE
Add a no-SIM internet path where the Pi joins a user's phone hotspot as a WiFi client and uses the
phone's cellular data for outbound synchronization. This must work with an ordinary iPhone or
Android hotspot and support tracking rows, alert events/state, software/server checks, and large
dashcam-video uploads without requiring a modem SIM.

The Pi currently creates its own hotspot for the local Web UI, so this phase must explicitly solve
the single-radio networking tradeoff rather than assuming it can be an access point and hotspot
client simultaneously:
- Add a Settings workflow to save/replace/remove a phone-hotspot SSID and password without exposing
  the saved password back to browser JavaScript. Validate before applying and provide a recovery
  path so a bad credential cannot permanently lock the user out of `yaris.local`.
- Prefer a deterministic NetworkManager design: when the configured phone hotspot is available,
  connect as a client for internet access; when it is unavailable or association fails, restore the
  Pi-hosted hotspot automatically within a bounded timeout. Preserve the existing boot/network
  watchdog behavior and document the exact priority/retry rules.
- Because a Pi 3 single WiFi interface cannot reliably host and join separate networks at once,
  test and document how the user reaches the local Web UI while connected to the phone hotspot.
  Prefer `http://yaris.local/` on the shared phone LAN when the phone permits peer traffic. If
  iOS/Android client isolation or mDNS behavior prevents that, keep the automatic Pi-hotspot
  fallback and present clear current-mode/status guidance rather than promising simultaneous access.
- Surface current uplink mode, connected SSID without password, IP address, internet reachability,
  last transition/error, and whether large video upload is permitted. Never mistake association
  with the phone for proven internet reachability.
- Keep dashcam video off SIM868 cellular, but allow it through the phone hotspot's WiFi path.
  Tracking, server alert synchronization, and other queued data must resume idempotently after the
  phone hotspot appears. Existing cellular-first ntfy behavior remains separate.
- Hardware-test with at least one real phone: cold boot with hotspot present/absent, hotspot
  enable/disable while running, wrong credentials, phone leaving/re-entering range, internet loss
  while still associated, automatic Pi-hotspot recovery, local `yaris.local` access, tracking
  catch-up, and a disposable chunked video upload. Record phone/OS limitations before committing.

**Implemented and hardware-tested 2026-08-09 with a real Pixel hotspot.**
- Added a root-owned `yaris-uplink` NetworkManager helper, systemd timer/wait service, dispatcher
  transition recorder, restricted sudoers entries, and Settings APIs/UI. Priority is
  `phone-hotspot` (20), `home-wifi` (10), then the Pi `Hotspot` (-10). A non-disruptive scan checks
  that the phone SSID is actually visible before replacing a working home connection.
- The intended single-phone workflow is **save credentials first → select Wait for hotspot → enable
  the phone hotspot**. The wait service gives the user ten seconds before dropping the Pi AP, scans
  for up to two minutes, and always restarts the periodic watcher. If the phone never appears or
  later disappears, NetworkManager/helper recovery selects home WiFi when available and otherwise
  restores the Pi hotspot. Multiple dead-man tests independently forced home WiFi recovery.
- Phone credentials go from browser to the privileged helper over stdin and into an atomic
  root-only (`0600`) NetworkManager keyfile. Passwords never enter SQLite, command arguments,
  status files, HTML/JavaScript responses, or read APIs. The UI exposes only configured state and
  SSID, supports replace/remove, and preserves in-progress edits during status polling.
- Status reports actual connection/mode/IP, NetworkManager-proven internet connectivity, whether
  large uploads are currently possible, last transition/error, and the last phone-assigned direct
  address. The non-secret transition record persists across reboot under `/var/lib/yaris-obd`.
- Pi-hosted hotspot testing initially found that the real phone could open `10.42.0.1` but not
  `yaris.local` even though Avahi correctly joined/published on `wlan0`. Added a NetworkManager
  shared-dnsmasq unicast fallback for `yaris.local`; the repeated dead-man test then passed.
- Pixel tethering has a different limitation: the hotspot-host phone can reach the Pi by its DHCP
  address, but relays neither mDNS (`yaris.local`) nor the DHCP hostname (`yaris`). Settings
  therefore retains and links the last direct phone-network address with a DHCP-change warning.
  The tested Pixel repeatedly assigned the same address, but the implementation does not assume it
  is permanent.
- Proved cold boot both with Pixel absent (selected home WiFi) and present (selected Pixel directly,
  Web UI/service started, and the server received a fresh heartbeat). Turning Pixel off repeatedly
  restored home WiFi in about 16 seconds. The same fallback code is covered for restoring the Pi
  hotspot when home WiFi is unavailable.
- Proved real tracking uploads through Pixel data against remote Postgres. The first disposable
  1 MiB video exposed repeated TLS EOF failures over tethering despite small requests working;
  setting only the phone profile MTU to 1400 fixed the path, after which a second 1 MiB disposable
  video reached the real server over Pixel data. Both named local/server files and exact database
  upload/hash rows were removed afterward, and dashcam uploads were restored enabled.
- The future Phase 24 alert-event synchronization queue will automatically use this normal IP
  uplink; Phase 22 does not prematurely implement that server schema/API.

### Phase 23: Finish web UI redesign — ✅ DONE
**Completed 2026-08-10 (final commit c685ef2).** Full responsive redesign of the Pi web UI,
including two UX polish passes and multiple bug-fix rounds.

**What was done:**
- Replaced the SB Admin 2-style fixed left sidebar with a purpose-built layout:
  - **Desktop (≥768 px):** dark fixed sidebar (`var(--sidebar-width) = 14 rem`) with collapsible
    icon-only mode (`4 rem`). State persisted via `localStorage`. Logo is a functional home link.
  - **Mobile (<768 px):** no sidebar. Full-width content with a sticky **bottom tab bar** (5 items:
    Dashboard, Dashcam, History, DTC, Settings). Logo shown in the topbar.
- Removed every placeholder element, inline style hacks, and stub dropdowns.
- Dashboard shows **6 OBD stat cards** wired to live socket events; RPM shows `—` (not `- rpm`)
  when disconnected.
- GPS map and coordinates merged into one card (lat/lon in card footer).
- **Speed history chart** (Chart.js): gradient fill, 60-point rolling window, session max/avg/count,
  NOW+MAX badges in header, auto-expands on desktop, collapsed by default on mobile.
- **Overview stats card** (new `/api/stats` endpoint): trips, total km, drive time, all-time max
  speed, videos on Pi, active DTC count (green/red) — displayed in a 3-column bottom row alongside
  GPS and speed history.
- DTC page fully wired to `/api/dtc` (MIL indicator + table, live refresh).
- `yaris.css` completely rewritten: CSS custom properties, sidebar, bottom nav, OBD cards,
  `border-start-*` utilities, mobile padding fix (removed Bootstrap `py-4 !important` conflict).
- `theme.js` rewritten: localStorage-backed collapse toggle.
- **Dashcam mobile**: thumbnail/size/date/upload columns hidden on mobile; action buttons icon-only
  (`<i>` only, no text); `data_table` macro supports optional column `class`; action buttons wrapped
  in `<div class="data-list-actions">` inside `<td>` (fixes desktop row layout).
- **Trip history mobile**: date shortened to `"9 Aug 18:00"` on <768 px; imperial units wrapped in
  `d-none d-sm-inline` spans; Replay button icon-only (`▶`).
- **Notification badge**: repositioned with `top/right` CSS offset (was clipped by `translate-middle`).
- Footer updated to `© vroom vroom yaris de course 2023`.
- Synthetic test trip in DB: 50 points, Old Port → Plateau Montreal, 3 km, 8 min, 43.5 km/h max.
- All 63 tests pass. Verified on Pi at 390×844 (mobile) and 1280×800 (desktop).

### Phase 23.5: UI polish, dashcam improvements, odometer, WiFi upgrade — ✅ DONE
**Completed 2026-08-10.** A focused multi-session polish and hardware upgrade pass on top of Phase 23.

**Dashcam & video:**
- Async dashcam import — import now fires in a background thread, returns `{status:"started"}` immediately, and the UI polls `/api/dash/import/status` every 3 s. Eliminates mobile timeout and `_device_lock` blocking all routes during copy.
- Per-file Convert button on MOV/non-MP4 files. `/api/local-video/convert` POST with `force=True` bypasses the auto-convert toggle.
- Real ffmpeg progress via `-progress pipe:1` — reads `out_time_ms` lines in a thread, computes `pct = out_time_s / duration_s * 100`. Old `elapsed/duration` estimate was wrong for RPi3 (encodes slower than realtime).
- Bitrate set to `-b:v 4M -b:a 128k` (~100 MB per 3-min clip); previous default was unusably small.
- `.converting.mp4` temp files excluded from file list and upload queue.
- Thumbnail patching during conversion — `refreshConvertingOnly()` updates badge text in-place without a full re-render; full render only when file count changes. Fixes thumbnail-reload flicker every 10 s.
- Mobile converting badge: spinner + % shown under thumbnail on mobile (`dash-thumb-meta`, `dash-thumb-converting`).
- Mobile checkbox column now visible in dashcam file list.
- Download button on Pi video files (`/api/local-video/file?download=1`, `as_attachment=True`).

**Odometer & trips:**
- `_log_loop` passes `app_obd.odometer` to `insert_tracking()` — OBD odometer now stored in every tracking row.
- `START_ODOMETER` / `END_ODOMETER` columns added to trips table (auto-migrated).
- `build_trip_summaries()` reads odometer from first/last tracking row of each trip; `list_trips()` falls back to cumulative distance sum for legacy trips with NULL odometer.
- Overview dashboard: `odometer_km` (MAX from tracking) and `video_count` now included in `/api/stats`.
- Trip history table: new "Odometer" column showing `start → end km`.

**Settings:**
- Separate upload toggles: "Upload trips & GPS logs" and "Upload dashcam videos (WiFi only)" are now independent switches. `TRIPS_UPLOAD_SETTING_KEY` added to `app_upload.py`.

**Map & location:**
- Recenter (⊕) button on dashboard GPS card and history route map.
- Offline reverse geocoding via `reverse_geocoder` Python library (~20 MB KD-tree, no internet needed). Exposed as `/api/geocode?lat=&lon=` — browser never calls any external service.
- When internet is available (phone hotspot), `/api/geocode` tries Nominatim first for street-level accuracy, falls back to `reverse_geocoder` for city/region. Results cached with `functools.lru_cache(512)` keyed at 4-decimal precision (~11 m).
- Dashboard GPS card footer: location name shown below lat/lon, updated on first fix and after moving >50 m.
- Trip history scrubber: location name shown in timeline panel, debounced 600 ms while scrubbing.
- GPS coords parsed as `float` before `toFixed()` — was crashing silently on string input from socket.

**Trip history UX:**
- Imperial units removed from `formatDistance` and `formatSpeed` (km / km/h only).
- Replay button hidden on mobile (`d-none d-sm-table-cell`); tapping anywhere on the trip row triggers replay instead (`data-trip-id` on `<tr>`, `cursor: pointer`).
- `data_table` macro supports optional `class` on column headers.

**WiFi upgrade — TP-Link Archer T3U (RTL8812BU):**
- RTL8812BU DKMS driver built and installed on Pi (`morrownr/88x2bu-20210702`, kernel 6.12.96+rpt-rpi-v8).
- `INTERFACE = "wlan1"` in `yaris_uplink.py`; `install_netwatch.sh` rebinds home-wifi, Hotspot, phone-hotspot to wlan1.
- `resolve_interface()` checks `/sys/class/net/wlan1` at startup — falls back to `wlan0` automatically if dongle is absent or driver fails. `INTERFACE` is module-level so all functions use the correct iface.
- `restore_fallback()` extended: if wlan1 primary fails, also tries `home-wifi-fb` and Hotspot on wlan0.
- `install_netwatch.sh` creates `home-wifi-fb` profile on wlan0 with `autoconnect=no` and lower priority — NM uses it only when explicitly activated by `restore_fallback()`.
- `dhcp-client-id=yaris-obd` in phone-hotspot keyfile — phone DHCP assigns a consistent lease by client-id, giving the Pi a stable IP on the phone hotspot network.
- Status JSON now includes `interface` field for diagnostics.
- SSH alias updated to new wlan1 IP (192.168.50.213).

### Phase 24: Remote trip + dashcam portal with shared web UI — ✅ DONE
**Completed 2026-08-10.** Extended the existing `server/` Flask deployment from an upload-only API into a full read-only browser portal backed by Postgres tracking data and the uploaded dashcam archive.

#### What was built

**Database (server/sql/)**
- `migrate_phase24.sql`: adds `users` (bcrypt browser auth) and `trips` (server-side computed trip index) tables
- `init.sql` updated to include both tables so fresh deployments don't need a separate migration step
- `trips` table: `(device_id, start_time)` unique key, full summary stats, start/end coordinates

**DB layer (server/modules/db.py)**
- `get_user_by_id()`, `get_user_by_username()`, `create_user()` — browser auth helpers
- `compute_trips_for_device()` — splits raw `tracking_points` into trip segments (gap > 5 min = new trip), filters out <50m blips, computes distance (haversine), duration, avg/max speed, upserts into `trips` table
- `list_trips()`, `get_trip_route()` — portal trip API helpers returning same JSON shape as Pi `/api/trips`/`/api/trip/<id>`
- `list_dashcam_files()`, `get_dashcam_file()` — dashcam browse/stream helpers
- `get_overview_stats()` — aggregated counts/totals for the Overview page

**Auth (server/modules/user_auth.py)**
- Flask-Login based browser session auth, completely separate from device `X-Api-Key` auth
- `User` class, `load_user()` loader, `check_password()`, `hash_password()`, `require_login` decorator
- `login_manager.login_view = "login_page"` — unauthenticated requests redirected to `/login`

**Application (server/app.py)**
- Added Flask-Login init, `SECRET_KEY` from env
- `_geocode_cached()` — same Nominatim→reverse_geocoder offline fallback as Pi, LRU cached
- `/login` GET/POST with CSRF token (stored in session, validated on POST)
- `/logout`
- Portal pages: `GET /` → Overview, `GET /history` → Trip history (reuses Pi `history.js`), `GET /dashcam` → Dashcam archive
- Portal JSON API: `/api/v1/portal/stats`, `/api/v1/portal/trips`, `/api/v1/portal/trip/<id>`, `/api/v1/portal/dashcam`, `/api/v1/portal/dashcam/stream/<id>` (Range-request streaming), `/api/v1/portal/recompute-trips`
- Pi history.js compatibility aliases: `/api/trips`, `/api/trip/<id>`, `/api/geocode`
- Auto-recompute trips in background thread after each tracking batch upload
- Path traversal guard on dashcam stream: rejects any storage path that escapes `STORAGE_DIR`

**Templates (server/templates/)**
- `base.html` — shared shell matching Pi visual style; "Remote server" badge in sidebar and mobile topbar; navigation has Overview / Trip history / Dashcam archive only (no Pi-hardware nav items)
- `login.html` — standalone login page with CSRF token, flash messages
- `index.html` — Overview page: 6 stat cards (trips, distance, drive time, top speed, videos, odometer), recent trips table, recent dashcam clips table
- `history.html` — identical layout to Pi history page; reuses `history.js` + `data-list.js` unchanged
- `dashcam.html` — file list table (name, size, uploaded date, download button) + inline video player with seek/Range support
- `components.html` — same `data_table` macro as Pi (needed by history.html)

**Scripts (server/scripts/)**
- `create_user.py` — create or update browser portal user: `docker compose exec server python scripts/create_user.py admin yourpassword`
- `recompute_trips.py` — recompute trips for all devices: `docker compose exec server python scripts/recompute_trips.py`

**Infrastructure**
- `server/requirements.txt`: added `Flask-Login`, `bcrypt`, `reverse_geocoder`
- `server/Dockerfile`: added `ffmpeg`, updated COPY paths to use repo-root build context (`context: ..`), copies shared `static/` assets
- `server/docker-compose.yml`: build `context: ..`, `dockerfile: server/Dockerfile`; added `SECRET_KEY` and `DEVICE_NAME` env vars; added `migrate_phase24.sql` mount (for existing deployments)
- `server/.env.example`: added `SECRET_KEY`, `DEVICE_NAME` entries with generation instructions

#### Bug fixes and UI polish (post-deploy)
- `/api/trips` was returning a bare array; `history.js` expected `{trips:[...]}` wrapper → fixed
- `formatSpeed(null)`, `formatDistance(null)`, `trip.odo_start_km.toFixed(0)` crashed on null data → null guards added in `static/js/history.js`
- Map tiles loaded grey: server has no offline tile cache → `history.js` now reads `data-tile-url` attribute from `#history-map` div; Pi template uses `/static/vendor/tiles/{z}/{x}/{y}.png`, server template uses OSM `https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png`
- Geocode returned `name` key; `history.js` expected `label` → server geocode now returns `label`
- Delete buttons: Bootstrap Modal `.getOrCreate()` was silently failing (async timing) → replaced with synchronous `window.confirm()` defined in `base.html` as `window.confirmDelete`
- Trip row delete button was wrapping to a new line → added `text-nowrap` to cell, removed whitespace between buttons in template
- Sidebar flash on page navigation: `theme.js` was restoring collapsed state with CSS transitions active → added transition suppression (set `transition: none` on sidebar + contentWrapper + toggleBtn before applying state, re-enable after two `requestAnimationFrame` calls)
- Sidebar toggle arrow direction wrong when collapsed: CSS already applies `transform: rotate(180deg)` to `.app-sidebar.collapsed .sidebar-collapse-btn`; the JS was also swapping the icon class (`fa-chevron-left` ↔ `fa-chevron-right`), causing a double-flip → removed JS icon class swap entirely, CSS handles it
- Added `.app-sidebar.collapsed .sidebar-env-badge { display: none }` and `.sidebar-user-text { opacity: 0; width: 0 }` CSS in `base.html` so badge and username hide correctly when sidebar is collapsed

#### Deployment steps (existing unraid stack)
1. `git pull` on unraid (or however you deploy the server)
2. Run migration: `docker compose exec db psql -U yarisobd -d yarisobd -f /docker-entrypoint-initdb.d/migrate_phase24.sql`
3. Add `SECRET_KEY` and `DEVICE_NAME` to `.env`
4. Rebuild: `docker compose up -d --build`
5. Create portal user: `docker compose exec api python scripts/create_user.py admin yourpassword`
6. Run initial trip computation: `docker compose exec api python scripts/recompute_trips.py`
7. Browse to `https://your-cloudflare-tunnel-url/` → login page


### Phase 24.5: Pre-car-test code audit & bug fixes — ✅ DONE
**Completed 2026-08-10.** Full read-only audit of all Pi modules, JS files, and server code. 63 tests pass.

#### Bugs found and fixed

| File | Severity | Bug | Fix |
|---|---|---|---|
| `modules/app_alerts.py` | MEDIUM | `videos["files"]` crashed if `list_local_videos()` returned `None` or missing key | `.get("files", [])` with `or {}` fallback |
| `modules/app_cellular.py` | MEDIUM | `read_resp.encode(errors="replace")` called `.encode()` on an already-str return value, changing return type to bytes | Removed `.encode()` — `body = read_resp` directly |
| `modules/app_settings.py` | MEDIUM | `apn` was uninitialized when `isinstance(apn, str)` check failed, causing `NameError`/wrong value stored | Initialize `apn = ""` in the else branch |
| `modules/app_upload.py` | LOW | `import json as _json` was inside a `ConnectionError` except handler — runs on every fallback attempt | Moved `import json` to top of file |
| `server/app.py` | HIGH | `int(request.args.get("limit"))` had no try/except — non-numeric query param crashed the trips and dashcam list endpoints with `ValueError` | Wrapped in `try/except (ValueError, TypeError)` |
| `server/app.py` | MEDIUM | `os.path.commonpath()` for path traversal safety raised `ValueError` on cross-root paths (Windows paths in DB), crashing the video stream endpoint | Wrapped in `try/except ValueError` |
| `tests/test_app_settings.py` | — | `valid_payload()` was missing `trips_upload_enabled` and `video_convert_enabled` (added in Phase 23.5 but test not updated), causing a pre-existing test failure | Added missing fields to `valid_payload()` |

#### Bugs confirmed NOT present (audit false positives)
- `app_obd.py` speed/rpm lock: `obd_speed_lock` already wraps the entire `while` loop including the write — no actual race
- `server/modules/db.py` DSN: Audit saw masked credential (view tool hides `password=`) — actual f-string is correct
- `app_sql.py` `state[0]` null crash: Code already has `if state is not None` check before indexing
- `app_cellular.py` `r"\r\n"`: SIMCOM SIM868 AT+HTTPPARA="USERDATA" spec requires literal 4-char `\r\n` sequence, not actual CRLF — intentionally correct
- `app_alerts.py` Authorization header: View tool masked `Bearer {token}` as `******` — actual code is correct

### Phase 24.6: Yaris 2013 OBD compatibility analysis & hardening — ✅ DONE
**Completed 2026-08-10.** Full compatibility analysis of every OBD PID against the 2013 Toyota Yaris (1.5L 1NZ-FE). Two real bugs fixed, deployed to Pi.

#### PID compatibility verdict

| PID | Command | OBD-II mandatory? | 2013 Yaris | Null-safe in code? |
|-----|---------|------------------|-----------|-------------------|
| 0x0D | SPEED | ✅ Yes | ✅ Yes | ✅ Yes |
| 0x0C | RPM | ✅ Yes | ✅ Yes | ✅ Yes |
| 0x05 | COOLANT_TEMP | ✅ Yes | ✅ Yes | ✅ Yes |
| 0x11 | THROTTLE_POS | ✅ Yes | ✅ Yes | ✅ Yes |
| 0x2F | FUEL_LEVEL | ⚠️ Optional | ✅ Likely | ✅ Yes |
| 0x01 | STATUS/MIL | ✅ Yes | ✅ Yes | ✅ Yes |
| Mode 3 | GET_DTC | ✅ Yes | ✅ Yes | ✅ Yes |
| 0xA6 | ODOMETER | ❌ Non-standard | ⚠️ Likely unsupported | ✅ Fixed (was broken) |

**Odometer note:** PID 0xA6 is a Mode 01 extended PID added in SAE J1979 DA (2014). The 2013 Yaris predates this spec; Toyota's ECU exposure of 0xA6 is inconsistent even on newer models. Expect `odometer = None` permanently — the UI handles this gracefully. The risk was in the decoder raising an exception that killed the whole OBD session; now fixed.

#### Protocol & connection
- **Protocol:** `OBD_PROTOCOL` env var defaults to `None` → python-obd sends `ATSP0` (auto-detect) → ELM327 will settle on ISO 15765-4 CAN 11-bit 500kbps, which is what the 2013 Yaris uses. ✅
- **OBD_PROTOCOL=6 hint:** After first confirmed working connection on the real car, uncomment `#Environment=OBD_PROTOCOL=6` in `deploy/yaris-obd.service` to skip the 3–5s auto-probe on every ignition-on.
- **Port:** auto-detects `/dev/ttyUSB0` (first USB serial device). ELM327 USB adapters enumerate there. Can override with `OBD_PORT` env var. ✅
- **Reconnect cycle:** ~10s (poll interval) + ~3–5s (ELM327 auto-probe) + ~2s (first poll) = up to ~17s from ignition-on to first live data. Acceptable.

#### Motion detection & trip logic
- **Primary:** OBD speed (PID 0x0D, polled every 2s) via `get_fresh_speed(max_age=5)`.
- **Fallback:** GPS haversine displacement when OBD speed is stale/unavailable.
- **During OBD warmup:** `speed_updated_at is None` → `get_fresh_speed()` returns `None` → GPS fallback activates. No false trip start — idle GPS drift rows spaced 60s apart are filtered by `MIN_GPS_MOTION_SAMPLES=3`.
- **Trips are post-computed** from raw `tracking` rows — no real-time open/close state machine. Power cut mid-trip is safe; all committed rows are recovered on restart. ✅

#### Bugs fixed

| File | Severity | Bug | Fix |
|---|---|---|---|
| `modules/app_obd.py` `_decode_odometer` | HIGH | Raised `ValueError` on unexpected byte count from unsupported/garbage 0xA6 response. Exception propagated through `_query()` and triggered `_mark_connection_lost()`, killing the entire OBD session on 2013 Yaris | Wrapped in `try/except Exception`, return `None` on any error |
| `modules/app_obd.py` `connect_obd` | MEDIUM | All sensor globals retained stale values from the previous ignition cycle — dashboard briefly shows wrong readings on reconnect | Reset all sensor globals to `None`/`[]` at start of each new connection |
| `deploy/yaris-obd.service` | INFO | No hint for pinning OBD protocol after initial test | Added commented `#Environment=OBD_PROTOCOL=6` with instructions |

### Phase 24.7: Final pre-car-test polish & map fixes — ✅ DONE
**Completed 2026-08-10.** Final round of fixes before car test.

#### Changes
- **`scripts/diagnose.sh`** — new diagnostic & recovery script. Run on Pi over SSH: `bash scripts/diagnose.sh` (read-only) or `bash scripts/diagnose.sh --fix` (auto-fix). Checks: service state, OBD USB adapter, SIM868 UART, disk/DB health, SQLite integrity, upload backlog, recent log errors, Python env, Pi throttling/temp/memory. Prints a cheatsheet of recovery commands. Exit code = number of critical issues.
- **`app.py`** — removed duplicate `app_gps` import; removed debug `print("hello ...")` that spammed the log on every index page load.
- **Server portal map tiles** — switched from `tile.openstreetmap.org` (rate-limited/blocked the Cloudflare tunnel egress IP) to **CARTO Voyager** (`basemaps.cartocdn.com`) which is free, CDN-backed, and reliable. Map now loads correctly on `yaris.ceaf.dev`.
- **Map zoom limits** — `history.js` now reads `data-min-zoom`/`data-max-zoom`/`data-tile-subdomains` from the `#history-map` div. Pi template caps to zoom 8–15 (matches offline cached tile range). Server template allows full 3–19. Prevents grey tiles when zooming outside cached range.
- **`tiles.addTo(map)` at init** — tiles were previously only added lazily on first trip click; now added immediately so the map background shows on page load.

### Phase 24.8: Deploy onboarding docs + uplink interface consistency — ✅ DONE
**Completed 2026-08-10.** Tightened the fresh-install path and aligned network-status plumbing with the Phase 23 wlan1-first architecture.

#### Changes
- **`deploy/readme.md`** — added a complete fresh-Pi install guide with:
  - required packages (`network-manager`, `nftables`, `avahi-daemon`, `dnsmasq-base`, Python tooling),
  - required NetworkManager profiles (`home-wifi`, `Hotspot`) with concrete `nmcli` examples,
  - deploy script run order (`install_service`, `install_port_redirect`, `install_netwatch`, `install_video_tools`, optional watchdog),
  - post-install verification and update/recovery commands.
- **Quick copy/paste bootstrap** — added a "Quick install" block at the top of `deploy/readme.md` for faster onboarding when provisioning a fresh Pi.
- **Root README deploy link** — `readme.md` now explicitly points to `deploy/readme.md` for clean Pi setup, instead of requiring PLAN deep-reading for first-time install.
- **Network status interface fix (`modules/app_network.py`)** — status endpoint previously hardcoded `wlan0`, which could misreport real uplink state after the T3U migration. It now resolves interface at runtime (`wlan1` primary, `wlan0` fallback) and reports the active interface in the response.
- **Dispatcher hook fix (`deploy/90-yaris-uplink`)** — previously ignored wlan1 events; now listens for both `wlan0` and `wlan1` so uplink transition recording remains accurate on current hardware.
- **README hardware photo** — added a current hardware build image (`pics/yaris-obd-hardware.jpg`) at the bottom of `readme.md`; saved without EXIF metadata and resized for lighter page load.

**Roadmap ordering note (2026-08-11; updated 2026-08-13):** the late phases are intentionally sequenced as
Phase 25.5a–25.5d (configuration/sync/status work) -> Phase 25 (real-car acceptance gate) ->
Phase 26 (road-test bug fixes) -> Phase 27 (feature additions from FEATURES.md) ->
Phase 28 (Android, in progress) -> Phase 29 (round-2 road-test bug fixes, 2026-08-13).

### Phase 25.5a: Device configuration sync backend — ✅ COMPLETE (connection-driven server sync)
**Server API and Pi sync thread implemented.** Server stores device config; Pi fetches when internet available and applies changes without restart.

#### Implemented (Phase 25.5a)
- **Database migrations** (`server/sql/migrate_phase25_5a.sql`):
  - Add `owner_name`, `car_model`, `device_name`, `avatar_path` to Postgres `devices` table
  - Add `config_version` and `config_updated_at` for change detection
  
- **Server API endpoints** (`server/modules/device_config.py`):
  - `GET /api/device-config` (device auth): Pi fetches config from server
  - `POST /api/device-config` (user auth): Portal/admin updates device config
  - Validation: 1-100 chars for names, 1-255 for paths
  - Config version incremented on any change (cache invalidation signal)
  
- **Server database helper** (`server/modules/db.py`):
  - `get_device_by_id()`: retrieve device with all config fields
  
- **Pi sync thread** (`modules/app_device_sync.py`):
  - Connectivity-driven: syncs every 5min when online, backoff to 60sec when offline
  - Fetches config from `/api/device-config` endpoint (device_key auth)
  - Detects changes via config_version
  - Stores synced config in SQLite
  - Reuses the existing upload credentials from `db/upload_creds.env`
    (`YARIS_SERVER_URL` + `YARIS_DEVICE_NAME` + `YARIS_API_KEY`;
    `YARIS_DEVICE_KEY` also accepted as an override for the key)
  - Ready for hot-reload on config change (future: signal handler)
  
- **Pi startup** (`app.py`):
  - `app_device_sync.start_sync()` starts background thread at boot

#### How it works
1. **At Pi startup**: loads device config from `config/device.json` (no network needed)
2. **When internet available**: sync thread wakes up every 5min and fetches from server
3. **If config changed** (new config_version): SQLite updated, next request reads new values
4. **If internet lost**: Pi continues with last-synced config from database
5. **User can still edit locally**: Settings page updates database; if sync enabled, server values override on next sync

#### Trade-offs
- Eventual consistency: changes on server may take 5min to reach Pi
- No hot-reload yet (requires app restart or signal handler to apply avatar/name changes immediately)
- Config version only increments on change; no periodic re-sync (efficient for offline car)

#### Testing
- Server endpoints accessible via `/api/device-config`
- Pi thread starts on boot (check logs for connectivity attempts)
- Sync disabled if `YARIS_SERVER_URL`, `YARIS_DEVICE_NAME`, or an API key
  (`YARIS_API_KEY` / `YARIS_DEVICE_KEY`) is not set
- **2026-08-10 access verification**: SSH access confirmed to Pi at `yaris@192.168.50.213`
  (new WiFi card/subnet) and to unraid via `ssh unraid` (`root@192.168.10.10`).
  Follow-up deployment succeeded the same day: Pi fast-forwarded from `d7b9bb6` to `cd004fb`,
  service restarted cleanly, and unraid rebuilt the API stack after manually applying
  `migrate_phase25_5a.sql`.

### Phase 25.5b: Remote portal device settings + hot-reload — ✅ COMPLETE
**The remote editor and Pi-side hot-reload are now wired.** The server portal can
edit the device identity that the Pi sync thread consumes, and the running Pi UI
updates its display-facing values without a manual restart once sync lands.

#### Implemented (Phase 25.5b so far)
- **Remote portal page**: `/settings/device` renders a login-protected editor for
  `owner_name`, `car_model`, `device_name`, and `avatar_path`
- **Portal JSON endpoint**: `GET /api/v1/portal/device-config` returns the current
  config for the configured single-device deployment
- **Navigation**: desktop sidebar + mobile bottom nav now expose **Device settings**
- **Portal UX**: live avatar preview, config-version display, dirty-state save
  button, and inline sync/explanation text
- **Pi hot-reload**:
  - `Account.reload_runtime_config()` and `Car.reload_runtime_config()` refresh
    display fields from the live SQLite-backed device config
  - local `/api/device-config` saves now reload the in-process `account`/`car`
    objects immediately
  - server sync now invokes a registered reload callback after writing new config
    locally, so next-page renders reflect synced values without restarting Flask

#### Validation
- Added runtime reload tests for `Account`, `Car`, and the sync callback path
- Existing suite still passes (`67 passed`)
- **Live 2026-08-10 verification**:
  - deployed the portal page to unraid and confirmed `/settings/device` routes to login (`302`)
  - fixed two real sync blockers on the Pi: reuse `db/upload_creds.env` and send
    the server's required `X-Device-Name` / `X-Api-Key` headers
  - enabled **Sync with server** on the Pi, changed `owner_name` on the server,
    waited through the sync thread's 60-second offline/backoff window, and
    observed both `GET /api/device-config` and the rendered topbar username on
    the Pi update to `quoije-sync-test`, then revert cleanly to `quoije`

---

### Phase 25.5c: Multi-device provisioning + force-sync UX — ✅ COMPLETE
Extend the current single-device portal page into a real multi-device management
surface: register devices from the server, mint/rotate API keys, help the Pi
consume those credentials without SSH-heavy manual steps, and expose an honest
**force sync** action that works with the system's pull-based/offline-first
architecture.

#### Goals
- **Multi-device server view**: stop assuming one `DEVICE_NAME` in the portal;
  show all registered devices and let the user select which one is being edited
- **Device registration from the portal**: create a new device row and return a
  one-time raw API key directly in the browser UI (same behavior as
  `server/scripts/create_device.py`, but web-managed)
- **Credential handoff to the Pi**: make it possible to put `YARIS_SERVER_URL`,
  `YARIS_DEVICE_NAME`, and `YARIS_API_KEY` onto the Pi from the app UI rather
  than editing `db/upload_creds.env` manually over SSH
- **Force sync button**: let the user request an immediate config pull without
  pretending the cloud server can magically push into an offline car

#### Proposed implementation
1. **Server: device management model + portal**
   - Add a device list page/card in the remote portal, with:
     - registered devices table
     - "Add device" action
     - "Rotate API key" action
     - per-device "Edit settings" action
   - Replace the single-device lookup helper with:
     - list devices
     - fetch one device by id
     - update one device's identity/config
   - Keep raw API keys **write-only/one-time reveal**:
     - show only immediately after create/rotate
     - never store raw keys server-side
     - only store hash as today

2. **Server: browser APIs**
   - Add login-protected endpoints for:
     - list devices
     - create device + return one-time API key
     - rotate device API key + return one-time API key
     - get/update selected device config
     - request force sync for a selected device
   - Device settings UI should populate from the selected device row, not from a
     single hardcoded portal `DEVICE_NAME`

3. **Pi: credential management UI**
   - Add a local Settings section for remote server credentials:
     - server URL
     - device name
     - API key
     - connection test button
   - Persist those values in the existing gitignored
     `db/upload_creds.env` (or an equivalent sqlite-backed secret store if we
     later choose to migrate away from env files)
   - After saving:
     - reload upload client / sync client config
     - allow enabling **Sync with server** only when creds test cleanly

4. **Force sync semantics**
   - **Constraint**: the server cannot directly reach a Pi behind hotspot/NAT,
     and the car is often offline. So a server-side force sync button cannot be
     a true push.
   - Proposed behavior:
     - server sets a per-device `force_sync_requested_at` flag/version bump
     - Pi checks that flag during normal online polling
     - local Pi UI also gets a **Sync now** button that triggers an immediate
       fetch against the server using the saved credentials
   - Result:
     - from server: "sync on next online contact"
     - from Pi: "sync immediately if internet is currently available"

#### Data model changes
- Add server columns such as:
  - `last_config_sync_at`
  - `last_config_sync_version`
  - `force_sync_requested_at`
  - `api_key_rotated_at`
- Optionally add a lightweight pairing/claim model later if copying raw API keys
  manually into the Pi UI feels too clunky

#### UI/UX shape
- **Remote portal**
  - left: device list / selector
  - right: settings form for selected device
  - actions: Add device, Rotate key, Copy creds, Request sync
  - status badges: last seen, last config version, last sync requested, maybe
    last successful contact
- **Pi local app**
  - server credentials card in Settings
  - Test connection button
  - Sync now button
  - clear warning when local edits are locked because server sync is enabled

#### Security / operational notes
- Never display a stored API key after the creation/rotation response is gone
- Keep key rotation explicit; rotating a key should warn that the old Pi config
  will stop authenticating until updated
- Avoid logging raw keys in Flask logs, JS errors, or shell output captured in
  committed docs
- Because the Pi is usually offline, all flows must degrade gracefully when the
  remote portal requests work the Pi cannot act on yet

#### Suggested execution order
1. Server multi-device DB/API refactor
2. Remote portal device list + register/rotate key UI
3. Pi local server-credentials UI + connection test
4. Pi/local immediate **Sync now** button
5. Server "request sync" flag + per-device status indicators
6. End-to-end validation:
   - create device on server
   - copy/paste creds into Pi UI
   - test connection
   - enable sync
   - change device identity on server
   - verify Pi receives it
   - rotate key and verify old key stops working until Pi updated

#### Open constraint to keep in mind
- If you want **zero manual copy/paste** for API key provisioning, that becomes
  a pairing problem (claim code / QR / short-lived token). That is doable, but
  it is a bigger scope than simply exposing key create/copy fields in the two
  existing UIs. My recommendation is to ship copy/paste provisioning first, then
  add a pairing flow only if it still feels too annoying.

#### Implementation progress (2026-08-10)
- **Server-side multi-device API/UI**
  - Added device listing, create-device, rotate-key, per-device config, and
    request-sync endpoints in the remote portal
  - Reworked `/settings/device` into a multi-device management page with a
    registered-device list, selected-device editor, one-time API-key reveal, and
    copy-credentials helper
- **Server schema**
  - Added Phase 25.5c migration for `last_config_sync_at`,
    `last_config_sync_version`, `force_sync_requested_at`, and
    `api_key_rotated_at`
  - Added device sync acknowledgement endpoint so the Pi can report when a
    config version was actually applied
- **Pi-side provisioning**
  - Added local Settings UI/API for `YARIS_SERVER_URL`, `YARIS_DEVICE_NAME`, and
    `YARIS_API_KEY`
  - Added **Test connection** and **Sync now** actions in the Pi app
  - Reused the existing gitignored `db/upload_creds.env` store instead of
    inventing a second credential location
- **Runtime behavior fixes**
  - Upload client now reads credentials dynamically instead of only at import
  - Device sync thread can restart after credentials are added later
  - Enabling **Sync with server** now validates that remote credentials exist

#### Validation
- Local tests expanded to `72 passed`
- Deployed Phase 25.5c to unraid and applied `migrate_phase25_5c.sql`
- Deployed Phase 25.5c to the Pi at `192.168.50.213`; `yaris-obd.service`
  restarted cleanly
- Pi local API checks:
  - `GET /api/remote-config` shows the saved remote server credentials state
  - `POST /api/remote-config/test` succeeds with authenticated device creds
  - `POST /api/remote-config/sync-now` succeeds (`no_change` when already current)
- Server sync-ack metadata verified live: `last_config_sync_version` matched the
  current `config_version` for `yaris-2013`, and `last_config_sync_at` populated
- Temporary create/delete validation of the new device-provisioning helper
  succeeded on the live server (`phase25c-validate-temp`)

#### Remaining polish ideas (not blockers)
- Exercise the authenticated browser flow for create/rotate/request-sync in the
  live portal UI itself, not just the backing APIs/helpers
- If copy/paste provisioning feels too clunky in practice, add a later QR /
  claim-code pairing flow
- Main portal overview/history/dashcam pages are **not** multi-device yet:
  the data model already separates rows by `device_id`, but those pages still
  read from the single configured `DEVICE_NAME`. Today that means the new
  multi-device management page can manage several devices, but the normal trip
  history view still shows one configured car/device at a time until a later
  portal-wide device selector/filter is implemented.

### Phase 25.5d: GPS dashboard visibility / live-status debugging — ✅ COMPLETE
- Add an explicit Pi dashboard GPS status indicator (thread alive / live fix / stale fix / no fix)
- Expose a local `/api/gps/status` endpoint so GPS health can be checked without relying on
  socket events alone
- Verify on the real Pi whether GPS is actually still producing fixes or whether the issue is
  now hardware/module/fix availability rather than a front-end live-update regression
- **Live Pi check 2026-08-10**: after deploying the new endpoint, the real Pi reported
  `thread_alive=true` but `has_fix=false`, `latitude=null`, `longitude=null`, and
  `fix_age_seconds=null`. That means the local app/socket path is still running, but there is
  currently no GNSS fix to stream into the dashboard.
- **Follow-up recheck 2026-08-10**: the Pi later recovered a live fix without further code
  changes. `/api/gps/status` returned `thread_alive=true`, `has_fix=true`,
  `latitude=45.544198`, `longitude=-73.536626`, and a fresh `fix_age_seconds` (~2.7s), which
  confirms the live-update path still works and the earlier issue was temporary lack of GNSS fix,
  not a dashboard/socket regression.
- **Dashboard fit polish 2026-08-10**: tightened the Pi dashboard layout so the GPS and Overview
  cards no longer stretch to the speed graph's full height, shortened the speed graph on
  Pi-sized screens, and reduced excess map/footer whitespace that previously forced unnecessary
  scrolling.
- **Next local UX follow-up 2026-08-10**: split the bulky Pi-health block out of the dashboard
  into a dedicated local **Status** page and expand it into a broader runtime diagnostics view:
  local `systemd` service states, server reachability/auth status, Wi-Fi uplink state, GPS fix
  state, SIM868/cellular-HAT responsiveness, upload backlog, and OBD adapter presence.
- **Implemented + verified 2026-08-10**: added a new local `/status` page plus `/api/status`
  aggregator. The dashboard no longer carries the bulky Pi-health block; instead the new page
  shows Pi health, remote-server reachability/auth, upload backlog, Wi-Fi/uplink mode, GPS fix
  status, SIM868 UART/modem responsiveness, OBD presence, and key `systemd` units
  (`yaris-obd.service`, `NetworkManager.service`, `yaris-netwatch.timer`,
  `yaris-uplink-wait.service`). Deployed to the real Pi (`192.168.50.213`) and confirmed the
  page/API served live values: server reachable+authenticated, uplink on Wi-Fi with internet,
  GPS thread alive with live fix, SIM868 responding on UART, OBD adapter present, and the local
  services reporting expected active/inactive states.
- **Next status-page polish 2026-08-10**: add quick action buttons directly on `/status` for
  server test, sync-now, uplink reconnect, and app-service restart so the Pi can be diagnosed and
  nudged from one screen instead of bouncing back to Settings/SSH.
- **Implemented + verified 2026-08-10**: `/status` now includes quick actions for server test,
  sync-now, uplink reconnect, and app restart. The first three reuse the existing local APIs; app
  restart adds a minimal sudo-backed `systemctl restart --no-block yaris-obd.service` path,
  installed by `deploy/install_service.sh` via `/etc/sudoers.d/yaris-status-actions`. Verified on
  the real Pi by deploying the sudoers update, loading the new buttons, exercising
  `/api/remote-config/test` (200/ok), `/api/remote-config/sync-now` (200/no_change), and
  `/api/status/restart-app` (202/restart_scheduled), then confirming `yaris-obd.service`
  returned to `active` and `/api/status` recovered with live values. The uplink-reconnect button
  was left unclicked during verification because this Pi currently has a saved phone hotspot and
  the action intentionally may switch networks/drop the active SSH session.
- **Dashboard card order tweak 2026-08-10**: swapped the local dashboard layout so the Overview
  card now appears before the Live GPS map. Deployed to the real Pi and confirmed the rendered
  `/` HTML places `Overview` before `Live GPS`.

---
### Phase 25: Pre-install Yaris 2013 compatibility test — ✅ DONE (owner road test; full checklist not followed)
**Owner conducted the in-car road test and checked for bugs. Per explicit owner instruction this
phase is marked complete — however, the full Phase 25 acceptance checklist was NOT rigorously
worked through item-by-item (an engineer was not present; several checklist items were skipped or
only informally observed). Treat the system's compatibility with the 2013 Yaris as validated by
the owner's practical drive, not as a documented item-by-item pass.** Bugs surfaced by the road
test are cataloged in `BUGS.md` and tracked for fixing in Phase 26. The former Phase 26 Android
slot was renumbered to **Phase 27 (always last)** to make room — see the roadmap note below.
**Added 2026-08-09; intentionally immediately before the Android app.** Bench tests, synthetic
routes, attached-hardware tests, and stationary Pi runs cannot prove the complete system works
when temporarily connected to and powered by the actual 2013 Toyota Yaris. This phase is the
pre-installation acceptance gate for the complete core Pi/web/server system after Phase 24. It is
verification and targeted correction—not permanent installation or another redesign—and the
system is not considered ready to install until the results are recorded here. Permanent mounting,
cable routing, trim work, and final installation are explicitly outside the roadmap and will be
handled by the owner after this phase passes.

Prepare a temporary, removable test setup before driving:
- ~~Connect the Pi, SIM868 HAT/antennas, USB OBD adapter, dashcam, and representative intended power
  source without permanently mounting or routing anything. Secure every temporary component and
  cable so it cannot interfere with pedals, steering, airbags, visibility, occupants, or loose-item
  safety during the controlled drive; remove the setup after testing.~~ — ✅ **DONE** (owner set up
  the temporary test rig).
- ~~Resolve the Phase 19/20 undervoltage/throttling warning before the test; confirm
  `vcgencmd get_throttled` shows no current power fault at idle and under load with the temporary
  representative power setup. This validates electrical compatibility, not final cable routing.~~ —
  ⚠️ **OPEN — NOT DONE.** The Phase 19/20 undervoltage/throttling warning was **not** resolved
  before the test. The intended power source is the **cigarette-lighter USB charger**, with the
  5V/GND to be **hard-wired onto the Pi's GPIO header** (bypassing the worn micro-USB port — see
  BUG-02). Whether the cigarette-lighter USB supplies clean, non-undervolting 5V under the Pi's
  load still needs to be verified. **Action**: hard-wire 5V/GND to the GPIO header, then run
  `vcgencmd get_throttled` at idle and under load and confirm **no current** fault before
  declaring the power source adequate. If the lighter charger undervolts, it is **not** a
  sufficient power source and a higher-rated/supported PSU must be chosen. (Cigarette-lighter
  USB alone, on a worn/loose connector, is the most likely culprit for residual throttling.)
- ~~Back up the Pi SQLite database and record baseline row/trip/video/upload counts on the Pi and
  remote server. Use a uniquely timestamped test window so generated records can be reconciled
  without deleting unrelated history.~~ — ⚠️ **SKIPPED** (owner: "didn't do it but don't very
  care"). Acceptable to skip the pre-test backup/baseline, but be aware there is now **no
  pre-test record** to reconcile test drive rows against, and no pre-drive SQLite backup exists.
- ~~Verify the service, watchdog, storage mounts, hotspot, `http://yaris.local/`, alert engine, GPS
  antenna view, dashcam recording mode, free space, and remote server health before moving.~~ —
  🟡 **PARTIAL.** Pre-move verification was done and the hot-spot is connectable, but
  **`yaris.local` does not resolve** on the phone → tracked as **BUG-01**. Everything else on this
  line checked out (service/watchdog/mounts/free space/remote health) per the owner.
- ~~Use a second person for UI/log observations while the vehicle is moving, or record diagnostics
  for review afterward. The driver must not operate or troubleshoot the UI while driving.~~ —
  ⛔ **SCRATCHED** (owner: "will do it solo"). **Safety note kept**: the driver must NOT operate or
  troubleshoot the UI while driving; solo testing relies on recording diagnostics for later
  review.

Run a controlled route containing ignition-off/on, stationary idle, low-speed streets, normal road
speed, several complete stops, turns, a short GPS-obstructed segment if safely available, and a
clean trip end:
- **OBD/ECU:** confirm the ELM327 reaches `CAR_CONNECTED`, record negotiated Toyota protocol and
  adapter firmware, and verify real speed/RPM/coolant/fuel/throttle/MIL/DTC values. Compare wheel
  speed against the instrument cluster at several safe speeds. Record unsupported PIDs (especially
  odometer PID A6) as capabilities, not failures, and verify reconnect after ignition cycling and a
  deliberate USB disconnect/reconnect. — ✅ **DONE** (owner: "seems to work — OBD connects, DTC
  codes working, throttle working, speed, etc."). Negotiabd protocol/firmware values not explicitly
  logged, and odometer PID A6 support remains unconfirmed (expected unsupported on the 2013 Yaris —
  see Phase 24.6).
- **GPS and motion logging:** confirm fixes while stationary and moving, route coordinates on the
  offline map, OBD-speed-first motion detection, GPS displacement fallback when OBD is deliberately
  unavailable, one-minute idle heartbeat behavior, and no false trip from stationary GPS drift. —
  ✅ **DONE** (owner: "seems to work for the trip"). **Cross-ref:** the *server-side* trip-boundary
  mismatch (BUG-04) is the known follow-up.
- **Trip boundary/replay:** after the inactivity timeout, verify exactly one trip (or the expected
  number for deliberate ignition gaps), correct start/end time and coordinates, plausible
  distance/average/max speed, km/mi and km/h/mph values, route polyline, start/end markers, timeline
  scrubber, play/pause marker position, and stopped portions of the route. —
  🟡 **PARTIAL / bug reported.** Local replay observed OK, but the server-side trip-splitting and
  replay issues are tracked in **BUG-04** (server merges two trips) and **BUG-05** (server replay
  `speed_source`), with **BUG-06** (timezone shift affecting times) also relevant.
- **Local web UI:** test the final Phase 23 interface on the actual phone over the Pi/phone network at
  `http://yaris.local/`; verify the explicit **In-car / Local Pi** identity, dashboard telemetry,
  DTC page, trip history, dashcam library, settings, alerts, responsive layout, reconnect behavior,
  and no new browser console/API errors. —
  🟡 **PARTIAL / bug reported.** Reaching the UI over the Pi hotspot is blocked by `yaris.local`
  not resolving on the phone → **BUG-01**. Other local UI pages work once reachable.
- **Dashcam workflow:** create a known short real clip, switch the camera safely into USB storage
  mode, detect/list it without confusing the OBD adapter, import only that selected file, verify
  thumbnail/playback/Range seeking, preserve the source until checks finish, and confirm the camera
  returns to recording afterward. Test deletion only with the disposable clip. —
  ⏳ **DEFERRED** (owner: "will test later stages"). Not yet exercised as part of this test; remains
  a follow-up for the Phase 26 bug-fix/verification pass.
- **Alerts:** safely exercise conditions that can be simulated without damaging hardware (upload
  backlog, temporary GPS loss while OBD reports motion, and disposable threshold overrides).
  Confirm local bell transition/recovery/cooldown behavior and offline alert-state reconciliation
  with the Phase 24 server after connectivity returns. Do not intentionally overheat, undervolt,
  fill storage, or trigger a real MIL. With a real SIM, explicitly determine whether the SIM868
  transport can reliably deliver ntfy notifications while WiFi/Ethernet is unavailable, including
  authentication, timeout, retry, and recovery. If that cannot be made reliable on the actual
  modem/network, remove the ntfy feature and its settings rather than retain a transport that fails
  in the only situation where it is useful. —
  🟡 **PARTIAL / done in part.** Owner reports alerts "seem to work for engine and DTC codes +
  power". The SIM868 **ntfy path remains unverified** (no real SIM on the modem yet, so the
  "determine whether the SIM868 transport can reliably deliver ntfy" sub-item is still open).
- **Power/reliability:** perform multiple normal ignition/power cycles, confirm automatic boot,
  service/watchdog startup, SIM868 power-on, OBD/GPS recovery, no SQLite corruption, and no current
  undervoltage/throttling. The deferred Phase 14 clean-shutdown limitation must be explicitly
  accepted or resolved with representative delayed-power/ignition-sense hardware before declaring
  the design ready for permanent installation. —
  🟡 **PARTIAL — Pi logs checked 2026-08-10 (at home, car not connected).** Verified from the live
  Pi:
  - `yaris-obd.service` **active**; uptime 2:48h with **no crash/restart** in the current boot
    window (systemd auto-restart + the 10s hardware watchdog did not need to fire since startup).
  - `vcgencmd get_throttled` → **`0x50000` = NO current undervoltage right now** (only the
    historical "occurred" bits 0x40000/0x10000 remain set). Confirms the Pi is not *currently*
    undervolting on whatever power is connected at home, but does not prove the in-car
    cigarette-lighter source yet (re-check under load in the car after hard-wiring — see the open
    power item above and BUG-02).
  - OBD reconnect loop logging as designed ("handshake succeeded / ECU did not respond" = car not
    on/not connected here). Uploads syncing fine ("sent N tracking rows").
  - No SQLite corruption or tracebacks in the log tail.
  **Still open**: a real multi-ignition-cycle verification *in the car* (boot recovery, watchdog
  restart under power loss, OBD/GPS recovery, no corruption) with the final hard-wired power
  source has not been demonstrated. The Phase 14 clean-shutdown limitation remains **accepted**
  (delayed-power / ignition-sense hardware not yet present).

Verify the complete remote path after connectivity returns:
- Confirm every test-window tracking row uploads exactly once and the selected dashcam clip resumes
  or completes without corruption; compare counts, timestamps, hashes, and byte sizes locally and
  remotely.
- On the Phase 24 portal, verify the explicit **Remote server** identity, authenticated
  vehicle-scoped trip summary/replay, route and timeline agreement with the local Pi, archived clip
  thumbnail/playback/Range seeking/download, synchronized alert history/current state without
  duplicate transitions after an offline interval, and absence of local-only OBD/cellular/Pi
  controls. —
  🟡 **PARTIAL.** Owner can connect to the remote server; the portal is reachable. The main
  blocker is the **Pi hotspot** reachability issue (**BUG-01**), which affects exercising the
  portal from the in-car phone. Server-side trip/replay/time issues (BUG-04/05/06) also remain.
- The Android companion is intentionally outside this core acceptance gate and remains the final,
  low-priority Phase 27. Its own hotspot/cellular binding, trip display, and disposable clip relay
  require separate real-phone/in-car acceptance when that phase is implemented.

Acceptance requires a written result for every item, captured negotiated protocol/PID support,
measured trip/count/hash comparisons, relevant logs/screenshots, no unexplained tracebacks or data
loss, and cleanup of every disposable record/file while preserving the real drive. Any defect found
must be fixed, redeployed uncommitted, and the affected in-car test repeated before the final commit
and **ready-to-install** sign-off. Afterward, the owner handles permanent installation outside the
phase roadmap.

---

### Phase 26: Road-test bug fixes — ✅ DONE (13 bugs resolved)
**Completed 2026-08-11; BUG-13 resolved 2026-08-12.** Targeted bug-fix pass for 13 defects
surfaced by the Phase 25 in-car road test and follow-up. Every item tracked in
[BUGS.md](./BUGS.md), fixed, deployed, and verified by owner. BUGS.md Open section is
now empty — all 13 bugs in Resolved.

**Implemented and verified 2026-08-11:**

- **BUG-01 — WiFi/hotspot unreachable from phone while cell data is on**: Dual-radio setup —
  wlan0 is permanent AP ("yaris 911" at 10.42.0.1, password "yaris911!"), wlan1 handles
  internet uplink only. `yaris_uplink.py` never touches wlan0. Phone-hotspot auto-connects on
  wlan1 within ~60s of appearing, falls back to home-wifi when it disappears. Settings page has
  in-car access card with hotspot URL, copy button, and cell-data warning. Tested on iOS and
  Android: `yaris.local` resolves, AP stays up with cell data active, phone-hotspot uplink
  verified end-to-end.
- **BUG-02 — Pi micro-USB power port physically damaged**: Hard-wired 5V/GND directly to GPIO
  header (red to Pin 2, black to Pin 6), bypassing the worn micro-USB connector. `vcgencmd
  get_throttled` now returns `0x0` — no undervoltage, no throttling. Pi runs at full 1.2 GHz
  instead of downclocked ~600 MHz. GPS fix acquired (45.544°N, 73.537°W, 7 satellites).
  System noticeably more responsive.
- **BUG-03 — DTC/MIL codes never persisted** (`e2f2314`): `save_dtc_state()` persists MIL +
  codes each OBD poll to SQLite. `/api/dtc` falls back to `get_persisted_dtc_state()` when
  OBD is offline. Cleared codes tracked via IS_SET timestamps.
- **BUG-04 — Server merges two trips into one** (`96fb897`): Replaced server-side time-gap
  splitting with Pi's motion-aware logic — OBD > 1 km/h primary, GPS >= 8m × 3 consecutive
  fallback, 5-min inactivity gap. Server trips cleared and recomputed; 17 trips matched on
  both Pi and server.
- **BUG-05 — Server route replay speed_source always "gps"** (`e2f2314`): Fixed to
  `"obd" if speed > 0 else "gps"` matching Pi logic. Docker rebuild verified.
- **BUG-06 — Server timestamps shifted (naive local time stored as UTC)** (`7ce2a1e` +
  migration): New `_to_tz_aware()` helper in `app_upload.py` converts naive SQLite timestamps
  to TZ-aware ISO before upload (`2026-08-10 19:27:30` → `2026-08-10T19:27:30-04:00`). One-time
  SQL migration on Unraid shifted 3,593 tracking rows +4h EDT, cleared deleted-trip markers,
  recomputed 43 trips from corrected data. Server trip times now match real-world clock
  (Aug 10 trips at 7:27 PM / 7:58 PM — was 3:27 PM / 3:58 PM).
- **BUG-07 — Trip replay on portal sometimes does nothing (intermittent freeze)** (`fa93384`):
  Full rewrite of `static/js/history.js` — unified AbortController cancels fetch + all
  downstream rendering; 300ms debounce prevents double-tap stacking; loading spinner on map +
  `.table-active` row highlight give clear feedback; geocode preload starts before map
  rendering so cache is warm; `signal.aborted` guards after every async step (route build,
  fitBounds, timeline init). Play/pause timeline preserved. Same JS file works on both Pi
  and server with tile config via `data-*` attributes. Verified on both platforms.
- **BUG-08 — No trip deletion on local Pi** (`c5c478b`): `deletable` macro, `window.confirmDelete`,
  `DELETE /api/trip/<id>` endpoint, configurable `data-delete-url` in shared `history.js`.
  Raw tracking preserved. Server-side: `deleted_trip_markers` table prevents deleted trips
  from reappearing on recompute, batch-delete with checkboxes via `POST /api/v1/portal/trips/delete`.
  Trip ID column added on server (`data-show-id="1"`), checkbox column server-only via
  `data-batch-delete-url`. Both now share same 6-column layout.
- **BUG-09 — Server trip scrubber geocoding is slow**: `preloadGeocodes()` batch-fetches all
  unique coords (concurrency 2) on trip load with AbortController. `geocodePoint()` checks
  in-memory cache first. Verified on both Pi and server Docker container.
- **BUG-10 — GPS-only false trips when car not connected**: `_car_ever_connected` flag in
  `app_obd.py` — set to `True` on first successful ECU connection. `build_trip_summaries()`
  returns empty list when car was never OBD-connected. `refresh_trips()` preserves existing
  trips when no OBD history (was deleting all trips on restart). Pi bench-test shows 0 trips.
- **BUG-11 — Dashboard GPS "Stale fix" while Status page shows "Live fix"** (`9733036`):
  `renderGpsStatus()` now reads `status.latitude`/`status.longitude` from the `/api/gps/status`
  API response (not just socket events). Status page now uses the same 30s staleness threshold
  (`fix_age_seconds > 30` → "Stale fix" badge) instead of calling any fix "Live". Both pages
  agree on fix status. Deployed to Pi.
- **BUG-12 — Dashboard OBD shows "—" while Status page shows correct OBD state** (`91281b1`,
  `a4a92cc`): Added `/api/obd/status` endpoint returning adapter presence, port, and car
  connection state. Dashboard calls `refreshObdStatus()` on page load + every 15s. Socket
  events still update in real-time. Side fix: `renderGpsStatus()` crashed on `status.latitude
  .toFixed is not a function` — API returns lat/lon as strings, fixed with `Number()` wrapper
  (`a4a92cc`). Socket `gps_location` event now passes lat/lon to `renderGpsStatus` so both
  API and Socket paths display coordinates correctly.
- **BUG-13 — Server Web UI shows raw coords instead of geocoded location** (`8309819`,
  2026-08-12): Server `_geocode_cached()` returned `None` → `/api/geocode` returned HTTP 204
  with no body. Frontend JS couldn't parse 204 → showed raw (x,y) fallback. Fixed by always
  returning `{label}` with coordinate-string last-resort fallback (matching Pi behavior),
  aligning Nominatim params (`jsonv2`, `accept-language=en`) and reverse_geocoder `mode=1`
  between server and Pi. Also hardened Pi against empty reverse_geocoder label.
- **BUG-14 — Geocode slow/unreliable during trip scrubbing** (`0ddfe39`, 2026-08-12):
  Nominatim was the root bottleneck — 1 req/sec rate limit, ~4s timeouts, often blocked
  from Docker/Cloudflare IPs. Replaced entirely with reverse_geocoder (offline Python lib,
  instant city/province lookup). JS `preload()` switched from N individual GETs to single
  `POST /api/geocode/batch`. Geocoding is now instant, zero network, always works.

**Result**: All 14 bugs closed. Pi runs at full 1.2 GHz with clean power. GPS has fix. Dual-radio
AP stays up regardless of uplink state. Server trip times are correct. Dashboard and Status page
agree on OBD and GPS state. Geocode is instant and always works (offline reverse_geocoder).
Unraid now uses SSH deploy key for git operations. BUGS.md Open section is empty.

---

### Phase 27: Feature additions — ✅ DONE (4 shipped, 3 candidates remain)
**Completed 2026-08-12.** Four features shipped; remaining 3 are low-priority ideas
tracked in FEATURES.md.

**Shipped:**
- **FEAT-03 — Manual odometer baseline** (2026-08-11, extended 2026-08-12): Settings
  "Odometer baseline (km)" field; dashboard shows baseline + trip distance. Extended
  2026-08-12 with a **simulated odometer in trip history**: replay scrubber shows an
  "Odometer" line below speed, and the trip-list column uses a continuous formula
  `baseline + Σ(prior trips) + within-trip distance` (raw OBD value when PID 0xA6 is
  reported). Fixed a latent bug where the Pi's list_trips fallback ignored the baseline
  and the server hardcoded `odo_start_km/end_km = None`. Changed: `modules/app_trip.py`,
  `modules/app_sql.py`, `server/modules/db.py`, `static/js/history.js`,
  `templates/history.html`, `server/templates/history.html`.
- **FEAT-06 — Trip ID column in history** (2026-08-11)
- **FEAT-07 — Phone-hotspot auto-connect docs** (2026-08-11)
- **FEAT-05 — DTC history sync to server** (2026-08-12): Pi pushes DTC snapshots
  via `POST /api/v1/dtc/batch`. Server diffs: insert/update/auto-clear. Portal
  `/dtc` page with code, description, MIL, status, timestamps, delete. Skips
  upload when OBD disconnected. Changed: `modules/app_upload.py`,
  `server/sql/migrate_phase27_dtc.sql`, `server/modules/db.py`, `server/app.py`,
  `server/templates/dtc.html`, `server/templates/base.html`.

**Remaining candidates (3):**
1. **FEAT-01** — Parallel/concurrent chunked dashcam uploads (medium)
2. **FEAT-02** — QR / claim-code device pairing flow (medium)
3. **FEAT-04** — Replace SIM868 2G GSM with 4G LTE modem (large)

### Phase 28: Native Android companion app — 🚧 IN PROGRESS (28.1 ✅, 28.2 ✅, 28.3 🚧, 28.4–28.7 pending)
**Renumbered from former Phase 26 on 2026-08-10 to make room for the road-test bug-fix phase.
Renumbered to Phase 28 on 2026-08-11 to make room for Phase 27 feature additions. Planning
completed 2026-08-11. Detailed implementation plan in [android/plan.md](android/plan.md).**

**28.1 — Scaffold + networking gate ✅ DONE (2026-08-12)**
- Project created in `android/` (Kotlin + Compose, Gradle KTS, version catalog).
- OkHttp + Retrofit networking layer with `PiApi` and `ServerApi` interfaces.
- `NetworkBinder.kt` discovers WiFi + cellular networks.
- `DualNetworkTestScreen` with simultaneous "Test Both" button.
- **Gate passed on real Pixel (2026-08-12):** Android OS routes `10.42.0.1` through WiFi
  and `yaris.ceaf.dev` through cellular automatically — per-socket `Network.bindSocket()`
  not needed. Both networks work simultaneously.

**28.2 — Dashcam browser + WiFi download ✅ DONE (2026-08-12)**
- `PiApi` extended: `listVideos`, `getVideoFile` (streaming), `getThumbnail`, `deleteVideo`.
- DTOs: `VideoListResponse`, `VideoFile`, `ConvertProgress`.
- `DashcamScreen`: `LazyColumn` with thumbnails (Coil), file size, date, upload status.
- `DashcamViewModel`: Retrofit-backed video list + download to phone storage + delete local copy.
- Downloads via `WorkManager` foreground service: `DownloadWorker` with notification +
  `setProgressAsync()`, `ActiveDownload` state via `LiveData`. Survives screen lock/background.
  FGS type `FOREGROUND_SERVICE_TYPE_DATA_SYNC` for Android 14+.
- Bottom nav: Dashcam tab + Test tab.
- **Verified:** video list loads from real Pi over "yaris 911" hotspot on the owner's Pixel.

**28.3 — Cellular upload 🚧 IN PROGRESS (2026-08-12)**
- `ServerApi.kt`: Retrofit chunked-upload endpoints (`/api/v1/dashcam/upload/status` + `.../chunk`).
- `UploadWorker.kt`: WorkManager foreground worker — SHA256 → `/status` check → 40MB chunks,
  skipping already-received chunks. Byte-level progress via okio `ForwardingSink` (real bytes
  sent, not chunk count), debug logging, robust chunk reads, re-upload guard.
- `DownloadsScreen`: per-file upload status + inline progress bar; in-app test file generator.

#### Why native (not web, not PWA)
Revives the phone-relay goal (Phase 11/12) properly this time. The Phase 11 update proved
conclusively that a browser page **cannot** hold a WiFi connection to the Pi's no-internet
hotspot and a cellular connection to the internet open simultaneously — the OS's connectivity
manager picks one default route for all IP traffic, and no browser JS API can override that
decision. A native Android app, however, can call
`ConnectivityManager.bindProcessToNetwork()` (or use `Network` objects directly) to pin
*specific* sockets to *specific* network interfaces independently — WiFi-bound socket to the
Pi for local API calls, cellular-bound socket to `yaris.ceaf.dev` for uploads. This is the
only way to achieve the true simultaneous dual-network relay the original Phase 11 intended.

#### Goals (what the app does)

1. **Dashcam file relay (primary goal).** Browse dashcam clips already imported to the Pi
   (same data as the Phase 17 local video browser), select files, download them over the
   Pi's hotspot WiFi to the phone's local storage, then upload to the remote server over
   the phone's cellular data — all without ever leaving the Pi's hotspot. The Pi's weak
   BCM43438 WiFi is only used for the short-hop download to the phone; the internet-bound
   leg goes over the phone's much faster LTE/5G radio.

2. **Live car dashboard (companion feature).** Surface live GPS position, speed, RPM,
   coolant temp, fuel level, throttle, and MIL/DTC status pulled from the Pi's existing
   REST API (`/api/gps/status`, `/api/obd/status`, `/api/stats`) while connected to the
   hotspot. Gives the passenger (or driver when parked) a real-time view of what the car
   is doing — the app is the "check on the car" companion, not just a file-transfer tool.

3. **Trip history browser (companion feature).** List and replay trips from the Pi's local
   SQLite (via `/api/trips`, `/api/trip/<id>`) — same data as the Phase 18 `/history` web
   page, just rendered natively. Offline map tiles not needed on the phone (Google Maps or
   MapLibre can provide the basemap).

4. **Remote-server direct access (stretch goal).** When away from the car (no Pi hotspot
   in range), talk directly to the Phase 24 remote portal's REST API
   (`https://yaris.ceaf.dev/api/v1/portal/...`) for trip history and dashcam archive
   browsing — so the app is useful even when not in the car.

#### Architecture decisions

- **Language:** Kotlin (modern Android standard, null safety, coroutines for async).
  Minimum SDK: API 26 (Android 8.0, covers 95%+ of active devices). Target SDK: API 35.
- **Networking:** OkHttp + Retrofit for REST calls. Two separate `OkHttpClient` instances:
  one bound to the WiFi network (Pi hotspot, `10.42.0.0/24`), one bound to cellular
  (or default network for normal internet). `Network.bindSocket()` per-request via a
  custom `ConnectionSpec` / interceptor that pins the socket to the desired `Network`
  object.
- **Local storage:** Room (SQLite) for downloaded-file tracking, upload queue state, and
  cached trip data. Files stored in the app's private external storage directory
  (`getExternalFilesDir()`).
- **UI:** Jetpack Compose (modern declarative UI, less boilerplate than XML, easier to
  iterate). Material 3 design system. Dark theme by default (car-friendly at night).
- **Background uploads:** `WorkManager` for cellular uploads that continue even if the
  app is backgrounded or the phone is locked. Chunked upload protocol reuses the exact
  same Phase 10 40MB-chunk `/api/v1/dashcam/upload/chunk` + `/status` endpoints the Pi's
  own Python client already uses — zero new server endpoints needed for the upload path
  itself.
- **Map:** MapLibre GL (open-source MapLibre GL Native for Android, free tier, no API key
  required for basic usage — can use OSM raster tiles or self-hosted styles). Google Maps
  as a fallback option if MapLibre proves problematic.

#### Pi-side API endpoints (already exist — zero new endpoints needed)
The app reuses the existing Pi REST API surface with no changes:
- `GET /api/gps/status` — GPS fix, lat/lon, fix age (Phase 25.5d)
- `GET /api/obd/status` — adapter presence, car connected, port (BUG-12 fix)
- `GET /api/stats` — trips, distance, drive time, max speed, video count, DTC count, odometer (Phase 23)
- `GET /api/trips` — trip list (Phase 18)
- `GET /api/trip/<id>` — trip route points (Phase 18)
- `GET /api/dtc` — MIL status + stored DTC codes (BUG-03 fix)
- `GET /api/local-video/list` — imported video files with upload status, size, thumbnails (Phase 17)
- `GET /api/local-video/file?path=<relpath>` — stream/download a video file, Range-request support (Phase 17)
- `GET /api/local-video/thumbnail?path=<relpath>` — cached thumbnail JPEG (Phase 17)
- `DELETE /api/local-video/file?path=<relpath>` — delete local copy only (Phase 17)
- `GET /api/remote-config` — server URL + device name (Phase 25.5c)
- `GET /api/dash/relay/config` — **was removed in Phase 12** — if the app needs server credentials from the Pi, a new lightweight endpoint may be needed, but the app can also store its own copy of `YARIS_SERVER_URL`/`YARIS_DEVICE_NAME`/`YARIS_API_KEY` independently

#### Server-side API endpoints (already exist — zero new endpoints needed)
- `GET /api/v1/dashcam/upload/status?sha256=...` — which chunks already received (Phase 10)
- `POST /api/v1/dashcam/upload/chunk` — multipart chunk upload with index + total (Phase 10)
- `GET /api/v1/portal/trips` — trip list (Phase 24)
- `GET /api/v1/portal/trip/<id>` — trip route (Phase 24)
- `GET /api/v1/portal/dashcam` — dashcam file list (Phase 24)
- `GET /api/v1/portal/dashcam/stream/<id>` — video stream with Range support (Phase 24)

#### What needs NEW endpoints
- **None for the core relay flow.** The chunked upload protocol on the server and the
  local-video list/download endpoints on the Pi already cover everything.
- **Maybe:** `GET /api/app/config` on the Pi — a single endpoint returning server URL,
  device name, and API key in one JSON payload, so the app doesn't need the user to type
  credentials manually. Alternatively, the app could store its own independent copy of
  these credentials (simpler, no new Pi endpoint needed). Decision deferred to
  implementation.

#### Dev environment setup

1. **Android Studio** (latest stable, Ladybug or newer). Install on the dev machine
   (Fedora). SDK Manager: install API 35 platform + build tools, API 26+ system images
   for emulator.
2. **Emulator:** create an AVD with API 35, 4GB RAM, 2GB internal storage. No Play Store
   needed. The emulator can't test real WiFi/cellular dual-network binding (it only has
   one virtual network interface), so this is for UI/flow testing only.
3. **Real device:** a physical Android phone (the owner's Pixel) is required for
   dual-network testing. Enable Developer Options → USB debugging. The real-device test
   flow: connect phone to Pi hotspot → run app → verify it can simultaneously call
   `10.42.0.1/api/*` AND `yaris.ceaf.dev/api/*`.
4. **Project scaffold:** use the Android Studio "Empty Activity" template with Compose.
   Gradle with Kotlin DSL. Version catalog (`libs.versions.toml`) for dependency
   management.
5. **Repo location:** new `android/` directory at the repo root (sibling to `server/`,
   `modules/`, etc.). The Android project is part of the same git repo — keeps everything
   in one place, same as the server.

#### Implementation order (sub-phases)

**28.1 — Scaffold + networking proof-of-concept (smallest possible scope)**
- Create Android project in `android/`
- Add OkHttp + Retrofit, set up two `OkHttpClient` instances (WiFi-bound + cellular-bound)
- Build a single "Test connection" screen: one button calls `10.42.0.1/api/gps/status`
  over WiFi, one button calls `yaris.ceaf.dev/health` over cellular, both at the same time
- **Gate:** prove dual-network binding actually works on the real Pixel before building
  anything else. If `bindProcessToNetwork()` doesn't work as expected on the owner's
  specific Android version/device, the entire Phase 28 premise needs revisiting.

**28.2 — Dashcam browser + download**
- List imported videos from `GET /api/local-video/list`
- Show thumbnails, file sizes, upload status (same data as `/dash` web UI)
- Download selected files to phone local storage over WiFi
- Progress bar (the Pi's BCM43438 WiFi is slow — ~1.1MB/s — so a 300MB file takes ~5 min)
- Handle interruptions (download resumes where it left off via Range requests)

**28.3 — Cellular upload**
- Queue downloaded files for upload via `WorkManager`
- Chunked upload to `POST /api/v1/dashcam/upload/chunk` over cellular
- Check `/status` first to skip already-received chunks (same as Pi's Python client)
- Notification showing upload progress (so user can lock phone / switch apps)
- Mark file as uploaded on the Pi (`POST /api/local-video/mark-uploaded` — does this
  endpoint exist? Check — the relay's `mark_uploaded` was removed in Phase 12. May need
  a new lightweight endpoint or the app skips this and the Pi's own uploader just sees
  the file is already on the server next time it scans.)

**28.4 — Live dashboard**
- GPS position on a MapLibre map (updates every 2s from `/api/gps/status`)
- Speed, RPM, coolant temp, fuel level, throttle cards (from `/api/obd/status` +
  socket.io — or polling `/api/stats` every few seconds)
- MIL/DTC indicator (from `/api/dtc`)

**28.5 — Trip history**
- List trips from `/api/trips` with date, distance, duration, start/end locations
- Tap to view route on map with scrubber timeline (reuse the same data as Phase 18 web
  replay, just rendered natively)
- Geocode via the Pi's `/api/geocode` or the phone's own Android
  `Geocoder` (no network needed for the Pi's offline `reverse_geocoder` path)

**28.6 — Remote portal direct access (away from car)**
- Talk to `https://yaris.ceaf.dev/api/v1/portal/...` directly
- Browse trip history and dashcam archive from anywhere (not just when connected to Pi)
- Login with portal credentials (same as Phase 24 web portal login)

**28.7 — Polish + distribution**
- App icon, splash screen, dark theme polish
- Error handling: Pi unreachable, server unreachable, no cellular data, storage full
- APK signing + release build
- Distribution: APK sideload (simplest — host the APK on the Pi itself at
  `http://yaris.local/app/yaris-obd.apk` so the user just opens that URL on the phone
  to install/update). Play Store is out of scope (Google developer account fee, review
  process, updates lag behind development).

#### Open questions (resolve during 28.1)

1. **Dual-network binding on the owner's Pixel**: `bindProcessToNetwork()` behavior varies
   by Android version and manufacturer. Test on the real device before committing to the
   full build. If it doesn't work, fallback options: (a) download-then-switch-networks
   two-step flow (download from Pi on WiFi, prompt user to switch to cellular, then
   upload — clunky but always works), or (b) use the phone's hotspot instead of the Pi's
   (the phone hosts, the Pi connects as client — reverses the network topology and
   eliminates the dual-network problem entirely, but means the Pi is a WiFi client, not
   an AP).

2. **Credentials flow**: how does the app get `YARIS_SERVER_URL` + `YARIS_DEVICE_NAME` +
   `YARIS_API_KEY`? Options:
   - User types them in (simple, works, annoying)
   - App connects to Pi hotspot → calls a new `GET /api/app/provision` endpoint that
     returns the creds JSON (zero manual entry, but adds one Pi endpoint)
   - QR code on the Pi's Settings page (scan once, app stores forever)
   - App stores its own independent copy (user enters once in app settings, never touches
     the Pi's copy)

3. **Socket.io for real-time data**: the Pi's dashboard uses Socket.IO for live OBD/GPS
   updates. The app could use a Socket.IO Kotlin client (`io.socket:socket.io-client`)
   for push updates, or simply poll the REST endpoints every 2–5 seconds. Polling is
   simpler and the Pi is on the same local WiFi (low latency, trivial load). Decision:
   start with polling, only add Socket.IO if the polling feels laggy.

4. **Video playback on the phone**: after downloading a dashcam clip to the phone, the
   app should be able to play it. Android's `ExoPlayer` (Media3) handles H.264 MP4/MOV
   natively — same codec the dashcam records. No transcoding needed.

#### Explicitly out of scope

- **iOS app**: not planned. Android-only. If an iOS user wants the same features, they
  can use the web UI (with the Phase 11 limitation that they can't do simultaneous
  dual-network relay — download+upload would be two separate manual steps).
- **Play Store listing**: sideload only for now. Lowers the barrier to ship and iterate.
- **Background GPS tracking**: the Pi already does this. The app is a display/relay
  client, not a replacement tracker.
- **Push notifications from server**: the Pi's alert engine (Phase 20) handles
  ntfy-based push. The app doesn't need its own push channel.
- **Offline mode when neither Pi nor internet is available**: the app is a connected
  companion, not a standalone tool. Cached trip data from the last sync is a nice-to-have
  but not required for v1.

### UI Branding (2026-08-12) — ✅ DONE

**Sidebar & logo redesign:**
- **Sidebar background**: changed from dark navy (`#1a1a2e`) to dark gray (`#1c1c1e`).
- **Logo**: replaced `fa-car` icon + text with a styled **YARIS-OBD** wordmark using
  **Racing Sans One** (free Google Font, SIL OFL license, DMCA-safe). White italic text
  with a red (`#e63946`) underline accent on the dark sidebar. Iterated through
  "YARIS" → "yaris-obd" (lowercase) → "YARIS-OBD" (final all-caps with dash).
- **Applied to both**: Pi (`templates/base.html`) and remote server
  (`server/templates/base.html`) — sidebar brand + mobile topbar logo.
- **CSS**: new `.sidebar-brand-logo` class in `static/yaris.css` with `::after`
  pseudo-element for the red accent underline. Collapsed sidebar shrinks the logo and
  hides the underline.

**Mobile dark theme:**
- **Bottom nav**: white background → dark (`#1c1c1e`), white icons/text, subtle white
  top border, deeper shadow. Active item is pure white; inactive items are 55% opacity
  white (was gray `#6c757d` with blue active).
- **Mobile topbar**: white → dark on screens < 768px (no sidebar on mobile, so the
  topbar becomes the header). White text, white-outline buttons, semi-transparent
  form controls, white bell icon.
- **Mobile logo**: uses same `sidebar-brand-logo` class → Racing Sans One font + red
  underline accent, sized down to 1.2rem for the narrower mobile topbar.
- **Desktop topbar**: unchanged — remains white, contrasts with the dark sidebar.

**Deployed to production:** both Pi (`yaris-obd.service` restart) and Unraid server
(`docker compose up -d --build`) on 2026-08-12. Verified CSS served correctly on both.

### Phase 29: Road-test bug fixes (round 2) — 🚧 IN PROGRESS (2026-08-13)
**Owner completed a second road test on 2026-08-13.** Issues found during the drive are
logged as `BUG-XX` entries in [BUGS.md](./BUGS.md) (BUG-16 onward) and fixed + verified here.

- **BUG-16 (trip missing from Pi local history) — ✅ FIXED**: `connect_obd()` set
  `_car_ever_connected` without a `global` declaration, so the module-level flag was never
  set and local trip building was silently blocked since BUG-10. The flag is now persisted
  across reboots; also fixed a NULL-GPS speed-only row crash in the trip builder. Verified the
  2026-08-13 drive now appears locally.
- **BUG-17 (trip timestamps wrong) — OPEN**: no RTC → clock drifts without internet.
- **BUG-18 (no RTC) — OPEN**: plan set — DS3231 RTC on I²C (desolder charging circuit, CR2032)
  + SIM868 CR1220 RTC backup; NTP online, RTC/GNSS fallback offline.

Open items from the drive — fill in as reported:
- BUG-17 / BUG-18 (clock) pending.

## Working Notes / Environment Gotchas

- **Pi credentials/SSH**: `ssh yaris@<pi-ip>`. Confirm current IP before connecting (varies by
network — check locally, not recorded here) — usually reachable via `yaris.local` too (mDNS,
see Phase 6). **Current verified LAN IP (2026-08-10): `192.168.50.213`** after the TP-Link
Archer T3U WiFi change.
- **Local dev repo**: work happens on a separate dev machine, not on the Pi. Push to
`origin/shameless-aislop`, the repo's current default branch (repo is public).
- **unraid server SSH**: `ssh root@192.168.10.10` (alias `unraid`). Real git checkout lives at
`/mnt/user/app/git/yaris-obd` (not `/mnt/user/appdata/...`, which is only the docker-managed data
volume for the `db`/`dashcam` containers). **Re-verified 2026-08-10**: alias works and lands in
`/root`. Deploy with
`cd /mnt/user/app/git/yaris-obd && git pull --ff-only && cd server && docker compose up -d --build`.
Container names: `yaris-obd-db` (Postgres 17), `yaris-obd-api`. For existing databases, also run
the mounted migration scripts manually, e.g.
`docker compose exec db psql -U yarisobd -d yarisobd -f /docker-entrypoint-initdb.d/migrate_phase25_5a.sql`.
**Important 2026-08-10**: this checkout had a long-diverged local `shameless-aislop`
(`ahead 136, behind 146`), so `git pull --ff-only` failed. To avoid destroying local history, the
Phase 25.5a deploy was performed from a fresh temporary clone while reusing the same `.env` and
Docker data. Reconcile or replace the old checkout before relying on it for future ff-only deploys.
- **Pi deploy loop**: `ssh yaris@<pi-ip>` (alias `yarisobd`) ->
`cd ~/app/yaris-obd && git pull --ff-only` -> `sudo systemctl restart yaris-obd.service`
(passwordless sudo confirmed via `yaris ALL=(ALL) NOPASSWD: ALL`). App stdout/stderr goes to
`~/app/yaris-obd/yaris-obd.log` (not journald - the systemd unit redirects both streams there),
so use `tail`/`grep` on that file, not `journalctl`, to see application-level prints (upload
progress, OBD/GNSS errors, etc.) - `journalctl -u yaris-obd.service` only shows service
start/stop/crash lifecycle events. **Update 2026-08-09**: plain `print()` calls were silently
invisible in that log for months because Python fully-buffers stdout when it's not a TTY (a
file, in this case) — `logging`-based lines (e.g. the `obd` library's own logger, which uses
stderr) showed up fine, but bare `print()` diagnostics from `app_gps.py` never did. Fixed by
adding `Environment=PYTHONUNBUFFERED=1` to the systemd unit
(`deploy/yaris-obd.service`) — **remember to also copy the updated unit file to
`/etc/systemd/system/yaris-obd.service` and run `sudo systemctl daemon-reload`** on the Pi, a
plain `git pull` + restart does NOT pick up systemd unit file changes (the live
`/etc/systemd/system/` copy is independent of the repo's `deploy/` copy).
- **Pi deploy loop, requirements.txt changes**: after any `requirements.txt` change (e.g. adding
`RPi.GPIO`), also run `~/app/yaris-obd/venv/bin/pip install -r requirements.txt` on the Pi — a
`git pull` alone doesn't install new dependencies into the venv.
- **Pi memory budget**: ~900MB usable RAM, no swap. Any code path that might buffer a whole
dashcam-sized file (hundreds of MB) in memory is unsafe on this hardware - stream, don't buffer
(see Phase 10).
- **Git push auth**: if `git push` fails with a credential/auth error, run `gh auth login`
(device-code browser flow) then `gh auth setup-git` to wire git to use the `gh` CLI's stored
credentials — this fixed push auth issues in this environment without needing SSH keys or PATs.
- **Pi environment gap**: Debian Bookworm blocks system-wide `pip install` (PEP 668,
"externally-managed-environment"). Use a venv for the real app dependency install (Phase 5)
rather than `--break-system-packages` everywhere. (Resolved in Phase 5 — venv at
`/home/yaris/app/yaris-obd/venv`.)
- **Pi → GitHub auth for `git pull` (private repo)**: the Pi's pre-existing
`~/.ssh/id_ed25519.pub` is an *inbound* key only (private half is on the dev machine) — can't
be reused for the Pi's own outbound git auth. Generated a dedicated keypair on the Pi
(`~/.ssh/yaris_deploy_key`), added it as a GitHub **read-only deploy key** on this repo, and
added an `~/.ssh/config` entry pinning `github.com` to that key. `git@github.com:...` (SSH
remote) then works standalone on the Pi, no PAT/interactive login needed.
- **piwheels**: pip on the Pi automatically uses `piwheels.org` as an extra index (Raspberry Pi
OS default `pip.conf`) — this is why `gevent` (which normally needs a C compile) installed in
seconds from a prebuilt ARM wheel instead of taking many minutes to build from source.
- **python-obd note**: `obd.OBD()` (or with `portstr=`) performs the ELM327 handshake at
connection time; if there's no car on the other end, it raises rather than silently returning
a dead handle — `app_obd.py`'s `connect_obd()` already catches this and sets `connection = None`.