Document phone-relay hard limitation, plan 4 new phases
Retested the hotspot+cellular phone-relay scenario more rigorously (netwatch explicitly disabled during tests so it wouldn't yank the Pi back to home WiFi mid-test - an earlier failed run turned out to be caused by that, not a phone issue). Confirmed with WiFi fully off (pure cellular) that requests to the public server still failed while associated with the Pi's hotspot, and that this is a fundamental OS-level 'single default network for the whole device' limitation - not something a web page's JS can work around, only a native app could (bindProcessToNetwork). Documented the decision to remove the feature. Added Phase 12 (remove phone-relay), Phase 13 (dashcam USB file browser: list+delete), Phase 14 (local imported-video browser + HTML5 player), Phase 15 (OBD code hardening ahead of first real car test - background reconnect loop, exception handling, protocol override, better diagnostics). Co-authored-by: Copilot <[email protected]>
This commit is contained in:
Mathieu Moran
@@ -478,7 +478,7 @@ data connection when WiFi/Ethernet is unreachable. Design notes:
|
||||
a plain-HTTP router port-forward path, avoiding the need for the modem's TLS stack to validate
|
||||
a cert chain).
|
||||
|
||||
### Phase 11: Phone-relay dashcam upload — ✅ DONE
|
||||
### Phase 11: Phone-relay dashcam upload — ⛔ REMOVED (see update below - hit a hard platform limitation)
|
||||
Root cause investigation into "why is the transfer from the Pi to the server so slow?" (real
|
||||
dashcam backlog draining at only ~350MB/4-5min) found the bottleneck is the **Pi's own onboard
|
||||
BCM43438 WiFi chip** (2.4GHz-only, congested/interfered channel) — confirmed via `iperf3`
|
||||
@@ -604,11 +604,132 @@ phone always has a known-good home WiFi it can silently fall back to, defeating
|
||||
validation needs either (a) actually being away from home with no known WiFi in range, forcing the
|
||||
phone to use cellular for internet traffic, or (b) manually disabling the phone's WiFi (or its
|
||||
"switch to mobile data automatically" setting) while connected to the Pi's hotspot, to force
|
||||
cellular use for this one operation. Not yet re-tested under either condition.
|
||||
cellular use for this one operation.
|
||||
|
||||
**Update — retested more rigorously, confirmed this is a fundamental architecture-level dead end,
|
||||
not just an untested corner case.** Re-ran the same live test multiple times with longer/managed
|
||||
"dead-man's-switch" windows (`yaris-netwatch.timer` explicitly stopped during each test run so it
|
||||
wouldn't yank the Pi back to home WiFi mid-test — an earlier run's failure was actually caused by
|
||||
netwatch reconnecting home WiFi under the phone, see Phase 6 update, not a phone-network issue).
|
||||
With netwatch correctly disabled and the Pi confirmed to stay in AP mode for the whole test window,
|
||||
still got `Failed to fetch`. Directly tested a plain page navigation (not `fetch()`) to
|
||||
`https://yaris.ceaf.dev/health` with WiFi *fully turned off* (pure cellular) — **that also failed**,
|
||||
while every other website loaded fine over the same cellular connection with WiFi off. This
|
||||
narrows the cause precisely: **the browser/phone genuinely cannot reach the public internet at all
|
||||
while `wlan0`/the phone's WiFi radio is associated with the Pi's hotspot, full stop** — this is not
|
||||
a settings toggle problem (checked Pixel-specific "switch to mobile data automatically" options,
|
||||
not present on this device/Android version) but a fundamental **OS-level "default network" routing
|
||||
decision**: a device's default route for *all* IP traffic (including a web page's `fetch()` calls)
|
||||
is a single, whole-device choice made by the OS's connectivity manager, not something a browser
|
||||
page's JavaScript can override per-request. Real per-request network binding (e.g. Android's
|
||||
`ConnectivityManager.bindProcessToNetwork()`) is an API only available to **native
|
||||
apps**, not exposed to web pages at all, in any browser, on any phone. Given this, the phone-relay
|
||||
feature as designed (a web page doing simultaneous WiFi-to-Pi + cellular-to-server transfers)
|
||||
**cannot ever work while the phone is connected to the Pi's no-internet hotspot** — this isn't a
|
||||
bug to fix, it's a hard platform ceiling.
|
||||
|
||||
**Decision: removed the phone-relay feature entirely** (was Phase 11, now reverted - see below).
|
||||
Two ways to actually deliver on the original "upload from the road without the Pi's slow WiFi"
|
||||
goal were discussed: (1) restructure as two distinct, non-simultaneous phases (fully download to
|
||||
the phone's local storage first while on the Pi's hotspot, then have the user manually switch to
|
||||
cellular, then upload from local storage) - no native app needed but a more awkward two-step user
|
||||
flow, or (2) a real native Android app, which *can* hold both networks open at once via
|
||||
`bindProcessToNetwork()` for a true simultaneous relay. Neither has been built - shelved for now
|
||||
in favor of simplifying the dashcam retrieval UI (see the new phases below). The Pi's own
|
||||
background uploader (plain WiFi/Ethernet, optionally SIM868 cellular fallback for tracking data
|
||||
only) remains the only upload path; it's just slower when the Pi's own WiFi is weak, with no
|
||||
workaround currently in place for that specific case.
|
||||
|
||||
**Side-fix from this test session**: found and fixed a real, unrelated bug in `yaris-netwatch.sh`
|
||||
during this same test — see the Phase 6 update above.
|
||||
|
||||
### Phase 12: Remove phone-relay feature — PLANNED
|
||||
Undo Phase 11 cleanly given the Phase 11 update above (fundamental browser platform limitation,
|
||||
not fixable). Remove:
|
||||
- `app.py`: `/api/dash/pending`, `/api/dash/file/<path>`, `/api/dash/relay/config`,
|
||||
`/api/dash/relay/heartbeat`, `/api/dash/relay/stop`, `/api/dash/relay/mark_uploaded` routes.
|
||||
- `modules/app_upload.py`: `list_pending_dashcam_files()`, `_hash_dashcam_file()`/hash-cache
|
||||
plumbing if unused elsewhere, `relay_heartbeat()`/`relay_stop()`/`is_relay_active()`,
|
||||
`RELAY_LEASE_SECONDS`, and the `is_relay_active()` check inside `_upload_dashcam_files()`.
|
||||
- `templates/dash.html` / `static/js/dash.js`: the "Relay via phone" card and all its JS
|
||||
(`relayOneFile`, `fetchChunkWithRetry`, `uploadChunkWithRetry`, heartbeat timers, etc).
|
||||
- `server/app.py`: the CORS `after_request` hook was added solely for the phone's cross-origin
|
||||
browser calls — remove it too (the Pi's own Python `requests`-based client was never
|
||||
subject to CORS, that's a browser-only enforcement mechanism).
|
||||
- Keep untouched: the chunked upload protocol itself (`/api/v1/dashcam/upload/chunk` +
|
||||
`/status`), `dashcam_hash_cache` table (still used by the Pi's own background uploader to
|
||||
avoid rehashing its 20GB+ backlog every loop), `DASHCAM_CHUNK_SIZE`/Cloudflare 100MB
|
||||
workaround - none of that is relay-specific.
|
||||
|
||||
### Phase 13: Dashcam USB file browser (list + delete) — PLANNED
|
||||
Replace the current all-or-nothing "Import videos" flow's blind spot (no visibility into what's
|
||||
actually on the camera) with a real file browser for the mounted USB dashcam device:
|
||||
- New endpoint(s) in `modules/app_dash.py` to list every video file on the mounted dashcam
|
||||
(relative path, size, modified time) without copying anything - reuses the existing
|
||||
`find_video_files()` walk, pointed at the dashcam's mount point instead of the local
|
||||
destination folder.
|
||||
- Delete support: individual file delete and a "delete all" bulk action, each requiring the
|
||||
mount to be briefly remounted read-write (currently always `ro`) - only for the duration of
|
||||
the delete call, then back to read-only (or unmounted) afterward, to minimize risk of
|
||||
filesystem corruption from an unclean power loss while writable.
|
||||
- UI: a `/dash` card showing total file count + total size on the dashcam, a scrollable file
|
||||
list with a delete button per row, a "Delete all" button behind a confirmation dialog (real
|
||||
data loss risk - no undo).
|
||||
- Explicitly deferred/out of scope: no in-browser preview/playback of files still on the
|
||||
dashcam itself (only after import - see Phase 14). No changes to the USB
|
||||
select-mode/data-communication toggle behavior - the user replugs power manually to return the
|
||||
camera to recording mode, as they already do.
|
||||
|
||||
### Phase 14: Local imported-video browser + player — PLANNED
|
||||
Same file-browser treatment for videos already imported to the Pi's own disk
|
||||
(`dash/videos/`, populated by `import_dashcam()`):
|
||||
- List endpoint (reuses `find_video_files()` against `app_dash.DEFAULT_DESTINATION`) showing
|
||||
relative path, size, modified time, and upload status (cross-check against
|
||||
`app_sql.dashcam_file_marked_uploaded()` - so it's visible at a glance which imported clips
|
||||
have already reached the remote server and which are still pending the Pi's own background
|
||||
uploader).
|
||||
- Delete support: **local Pi copy only** - remote server copies are intentionally left alone
|
||||
as the safe backup/archive (explicit user decision - see 2026-08-09 discussion). No
|
||||
"also delete from server" option is being built for now.
|
||||
- In-browser video player: since the dashcam records **MP4** (confirmed with the user), this
|
||||
is a plain HTML5 `<video>` element - no server-side transcoding needed. Reuses the same
|
||||
Range-request-capable file-serving pattern already proven in the (now-removed) relay's
|
||||
`/api/dash/file/<path>` endpoint (needed for correct seeking/scrubbing, not just playback).
|
||||
- UI: a `/dash` card (or a dedicated sub-page if the list gets long) with a file list, a
|
||||
"Play" action that opens the video inline, and a delete button per row with a confirmation
|
||||
prompt.
|
||||
|
||||
### Phase 15: OBD real-hardware readiness (code hardening) — PLANNED
|
||||
Nothing has been installed or tested in the actual 2013 Yaris yet - `modules/app_obd.py` has
|
||||
only ever been tested standalone on the Pi with nothing plugged into the OBD-II port (correctly
|
||||
reports "no connection to car"). Per explicit user decision, this phase is **code hardening
|
||||
only** - the user will handle the physical install/bring-up testing themselves. Planned fixes,
|
||||
all in `modules/app_obd.py` / `modules/app_events.py`:
|
||||
- **Add a genuine background reconnect loop.** Today, `connect_obd()`/`obd_connection()` only
|
||||
ever run once per browser tab's socket `connect` event (`app_events.py`'s
|
||||
`connect_event()`) - there is no periodic retry. In the real car, this means: if the page
|
||||
happens to load before the adapter/ignition is ready, the connection attempt fails once and
|
||||
never retries until another browser tab connects. Add a small polling loop (mirroring the
|
||||
pattern already proven in `app_gps.py`'s `_poll_loop()`) that periodically calls
|
||||
`connect_obd()` while disconnected, independent of any browser being open.
|
||||
- **Catch exceptions from `connection.query()`.** Currently a communication hiccup
|
||||
(adapter timeout, malformed response, USB blip) would raise inside `obd_speed()`/`obd_rpm()`
|
||||
uncaught, silently killing that polling thread forever with no recovery - not just returning
|
||||
a null response, an actual crash. Wrap query calls, treat exceptions like a lost connection
|
||||
(stop polling, let the reconnect loop above pick it back up) instead of dying silently.
|
||||
- **Add an `OBD_PROTOCOL` env override**, mirroring the existing `OBD_PORT` override pattern -
|
||||
in case the 2013 Yaris's ECU doesn't cleanly auto-negotiate via ELM327's `ATSP0` (auto) and
|
||||
needs an explicit protocol code forced (Toyotas this age are normally ISO 15765-4 CAN 11-bit
|
||||
500kbaud, but this has never been confirmed against this specific car).
|
||||
- **Improve connection-failure diagnostics.** Distinguish and log clearly: no adapter detected
|
||||
at all (USB/port issue) vs adapter detected but ELM327 handshake failed vs handshake OK but no
|
||||
response from the car's ECU (car off/ignition not on) - currently all three collapse into the
|
||||
same generic "OBD connection failed" message, which will make first-real-car troubleshooting
|
||||
much harder than it needs to be.
|
||||
- **Add a small connect-time diagnostic log** (ELM327 firmware version + negotiated protocol,
|
||||
both available from `python-obd`'s connection object) written once per successful connect -
|
||||
useful evidence to sanity-check against Toyota's expected protocol on the very first real test.
|
||||
|
||||
## Working Notes / Environment Gotchas
|
||||
|
||||
- **Pi credentials/SSH**: `ssh yaris@<pi-ip>`. Confirm current IP before connecting (varies by
|
||||
|
||||
Reference in New Issue
Block a user