diff --git a/PLAN.md b/PLAN.md index ed6ba96..06fd57c 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1412,8 +1412,8 @@ including two UX polish passes and multiple bug-fix rounds. **Roadmap ordering note (2026-08-11):** the late phases are intentionally sequenced as Phase 25.5a–25.5d (configuration/sync/status work) -> Phase 25 (real-car acceptance gate) -> -Phase 26 (road-test bug fixes) -> Phase 27 (feature additions from FEATURES.md) -> -Phase 28 (Android, always last). +Phase 26 (road-test bug fixes) -> Phase 27 (feature additions from FEATURES.md, in progress) -> +Phase 28 (Android, planning in parallel with Phase 27). ### Phase 25.5a: Device configuration sync backend — ✅ COMPLETE (connection-driven server sync) **Server API and Pi sync thread implemented.** Server stores device config; Pi fetches when internet available and applies changes without restart. @@ -1963,28 +1963,219 @@ agree on OBD and GPS state. BUGS.md Open section is empty. readme.md project sta 5. **FEAT-08** — Predictable IP on phone hotspot (small) 6. **FEAT-09** — mDNS on phone hotspot (small) -### Phase 28: Native Android companion app — 🚧 PLANNED (low priority, always last) -**Renumbered from former Phase 26 on 2026-08-10 to make room for the road-test bug-fix phase. Renumbered to Phase 28 on 2026-08-11 to make room for Phase 27 feature additions.** -Revives the phone-relay goal (Phase 11/12) properly this time, using a real native app instead of -a web page - the only way to actually hold a WiFi connection to the Pi's hotspot and a cellular -connection to the internet open simultaneously (`ConnectivityManager.bindProcessToNetwork()`, -unavailable to browser JS - see the Phase 11 update). Explicitly **not urgent** and intentionally -kept at the end of the roadmap, after all web/server work and the Phase 25 core in-car acceptance -test are done. Permanent vehicle installation remains owner-handled and is not an implementation -phase. Scope, at a high level (not yet designed in detail): -- **Dashcam file transfer**: browse/select/delete dashcam clips already imported to the Pi (same -data as the Phase 17 local video browser, just a native client instead of/alongside the web UI), -download them to the phone, and upload to the remote server over cellular while the phone stays -on the Pi's hotspot for the download leg - the true simultaneous dual-network relay the web -version could never deliver. -- **Trip/location info**: surface live GPS position, speed, and trip history (Phase 18) pulled -from the Pi while connected to its hotspot (reusing the existing `/api/gps`-style data feeds), -so the app is useful as a real "check on the car" companion, not just a file-transfer tool. -- Not yet scoped: distribution (sideload vs Play Store), whether it talks to the Pi directly over -the hotspot only or also to the remote server directly for trip history when away from the car, -and how much of the existing REST API surface can be reused as-is vs needs new -mobile-friendly endpoints. To be designed once Phases 12-24 (web-side cleanup + hardening, -redesign, remote portal, and core in-car acceptance) are done. +### Phase 28: Native Android companion app — 🚧 PLANNING (low priority, always last) +**Renumbered from former Phase 26 on 2026-08-10 to make room for the road-test bug-fix phase. +Renumbered to Phase 28 on 2026-08-11 to make room for Phase 27 feature additions. Planning +begun 2026-08-11 alongside Phase 27 execution.** + +#### Why native (not web, not PWA) +Revives the phone-relay goal (Phase 11/12) properly this time. The Phase 11 update proved +conclusively that a browser page **cannot** hold a WiFi connection to the Pi's no-internet +hotspot and a cellular connection to the internet open simultaneously — the OS's connectivity +manager picks one default route for all IP traffic, and no browser JS API can override that +decision. A native Android app, however, can call +`ConnectivityManager.bindProcessToNetwork()` (or use `Network` objects directly) to pin +*specific* sockets to *specific* network interfaces independently — WiFi-bound socket to the +Pi for local API calls, cellular-bound socket to `yaris.ceaf.dev` for uploads. This is the +only way to achieve the true simultaneous dual-network relay the original Phase 11 intended. + +#### Goals (what the app does) + +1. **Dashcam file relay (primary goal).** Browse dashcam clips already imported to the Pi + (same data as the Phase 17 local video browser), select files, download them over the + Pi's hotspot WiFi to the phone's local storage, then upload to the remote server over + the phone's cellular data — all without ever leaving the Pi's hotspot. The Pi's weak + BCM43438 WiFi is only used for the short-hop download to the phone; the internet-bound + leg goes over the phone's much faster LTE/5G radio. + +2. **Live car dashboard (companion feature).** Surface live GPS position, speed, RPM, + coolant temp, fuel level, throttle, and MIL/DTC status pulled from the Pi's existing + REST API (`/api/gps/status`, `/api/obd/status`, `/api/stats`) while connected to the + hotspot. Gives the passenger (or driver when parked) a real-time view of what the car + is doing — the app is the "check on the car" companion, not just a file-transfer tool. + +3. **Trip history browser (companion feature).** List and replay trips from the Pi's local + SQLite (via `/api/trips`, `/api/trip/`) — same data as the Phase 18 `/history` web + page, just rendered natively. Offline map tiles not needed on the phone (Google Maps or + MapLibre can provide the basemap). + +4. **Remote-server direct access (stretch goal).** When away from the car (no Pi hotspot + in range), talk directly to the Phase 24 remote portal's REST API + (`https://yaris.ceaf.dev/api/v1/portal/...`) for trip history and dashcam archive + browsing — so the app is useful even when not in the car. + +#### Architecture decisions + +- **Language:** Kotlin (modern Android standard, null safety, coroutines for async). + Minimum SDK: API 26 (Android 8.0, covers 95%+ of active devices). Target SDK: API 35. +- **Networking:** OkHttp + Retrofit for REST calls. Two separate `OkHttpClient` instances: + one bound to the WiFi network (Pi hotspot, `10.42.0.0/24`), one bound to cellular + (or default network for normal internet). `Network.bindSocket()` per-request via a + custom `ConnectionSpec` / interceptor that pins the socket to the desired `Network` + object. +- **Local storage:** Room (SQLite) for downloaded-file tracking, upload queue state, and + cached trip data. Files stored in the app's private external storage directory + (`getExternalFilesDir()`). +- **UI:** Jetpack Compose (modern declarative UI, less boilerplate than XML, easier to + iterate). Material 3 design system. Dark theme by default (car-friendly at night). +- **Background uploads:** `WorkManager` for cellular uploads that continue even if the + app is backgrounded or the phone is locked. Chunked upload protocol reuses the exact + same Phase 10 40MB-chunk `/api/v1/dashcam/upload/chunk` + `/status` endpoints the Pi's + own Python client already uses — zero new server endpoints needed for the upload path + itself. +- **Map:** MapLibre GL (open-source MapLibre GL Native for Android, free tier, no API key + required for basic usage — can use OSM raster tiles or self-hosted styles). Google Maps + as a fallback option if MapLibre proves problematic. + +#### Pi-side API endpoints (already exist — zero new endpoints needed) +The app reuses the existing Pi REST API surface with no changes: +- `GET /api/gps/status` — GPS fix, lat/lon, fix age (Phase 25.5d) +- `GET /api/obd/status` — adapter presence, car connected, port (BUG-12 fix) +- `GET /api/stats` — trips, distance, drive time, max speed, video count, DTC count, odometer (Phase 23) +- `GET /api/trips` — trip list (Phase 18) +- `GET /api/trip/` — trip route points (Phase 18) +- `GET /api/dtc` — MIL status + stored DTC codes (BUG-03 fix) +- `GET /api/local-video/list` — imported video files with upload status, size, thumbnails (Phase 17) +- `GET /api/local-video/file?path=` — stream/download a video file, Range-request support (Phase 17) +- `GET /api/local-video/thumbnail?path=` — cached thumbnail JPEG (Phase 17) +- `DELETE /api/local-video/file?path=` — delete local copy only (Phase 17) +- `GET /api/remote-config` — server URL + device name (Phase 25.5c) +- `GET /api/dash/relay/config` — **was removed in Phase 12** — if the app needs server credentials from the Pi, a new lightweight endpoint may be needed, but the app can also store its own copy of `YARIS_SERVER_URL`/`YARIS_DEVICE_NAME`/`YARIS_API_KEY` independently + +#### Server-side API endpoints (already exist — zero new endpoints needed) +- `GET /api/v1/dashcam/upload/status?sha256=...` — which chunks already received (Phase 10) +- `POST /api/v1/dashcam/upload/chunk` — multipart chunk upload with index + total (Phase 10) +- `GET /api/v1/portal/trips` — trip list (Phase 24) +- `GET /api/v1/portal/trip/` — trip route (Phase 24) +- `GET /api/v1/portal/dashcam` — dashcam file list (Phase 24) +- `GET /api/v1/portal/dashcam/stream/` — video stream with Range support (Phase 24) + +#### What needs NEW endpoints +- **None for the core relay flow.** The chunked upload protocol on the server and the + local-video list/download endpoints on the Pi already cover everything. +- **Maybe:** `GET /api/app/config` on the Pi — a single endpoint returning server URL, + device name, and API key in one JSON payload, so the app doesn't need the user to type + credentials manually. Alternatively, the app could store its own independent copy of + these credentials (simpler, no new Pi endpoint needed). Decision deferred to + implementation. + +#### Dev environment setup + +1. **Android Studio** (latest stable, Ladybug or newer). Install on the dev machine + (Fedora). SDK Manager: install API 35 platform + build tools, API 26+ system images + for emulator. +2. **Emulator:** create an AVD with API 35, 4GB RAM, 2GB internal storage. No Play Store + needed. The emulator can't test real WiFi/cellular dual-network binding (it only has + one virtual network interface), so this is for UI/flow testing only. +3. **Real device:** a physical Android phone (the owner's Pixel) is required for + dual-network testing. Enable Developer Options → USB debugging. The real-device test + flow: connect phone to Pi hotspot → run app → verify it can simultaneously call + `10.42.0.1/api/*` AND `yaris.ceaf.dev/api/*`. +4. **Project scaffold:** use the Android Studio "Empty Activity" template with Compose. + Gradle with Kotlin DSL. Version catalog (`libs.versions.toml`) for dependency + management. +5. **Repo location:** new `android/` directory at the repo root (sibling to `server/`, + `modules/`, etc.). The Android project is part of the same git repo — keeps everything + in one place, same as the server. + +#### Implementation order (sub-phases) + +**28.1 — Scaffold + networking proof-of-concept (smallest possible scope)** +- Create Android project in `android/` +- Add OkHttp + Retrofit, set up two `OkHttpClient` instances (WiFi-bound + cellular-bound) +- Build a single "Test connection" screen: one button calls `10.42.0.1/api/gps/status` + over WiFi, one button calls `yaris.ceaf.dev/health` over cellular, both at the same time +- **Gate:** prove dual-network binding actually works on the real Pixel before building + anything else. If `bindProcessToNetwork()` doesn't work as expected on the owner's + specific Android version/device, the entire Phase 28 premise needs revisiting. + +**28.2 — Dashcam browser + download** +- List imported videos from `GET /api/local-video/list` +- Show thumbnails, file sizes, upload status (same data as `/dash` web UI) +- Download selected files to phone local storage over WiFi +- Progress bar (the Pi's BCM43438 WiFi is slow — ~1.1MB/s — so a 300MB file takes ~5 min) +- Handle interruptions (download resumes where it left off via Range requests) + +**28.3 — Cellular upload** +- Queue downloaded files for upload via `WorkManager` +- Chunked upload to `POST /api/v1/dashcam/upload/chunk` over cellular +- Check `/status` first to skip already-received chunks (same as Pi's Python client) +- Notification showing upload progress (so user can lock phone / switch apps) +- Mark file as uploaded on the Pi (`POST /api/local-video/mark-uploaded` — does this + endpoint exist? Check — the relay's `mark_uploaded` was removed in Phase 12. May need + a new lightweight endpoint or the app skips this and the Pi's own uploader just sees + the file is already on the server next time it scans.) + +**28.4 — Live dashboard** +- GPS position on a MapLibre map (updates every 2s from `/api/gps/status`) +- Speed, RPM, coolant temp, fuel level, throttle cards (from `/api/obd/status` + + socket.io — or polling `/api/stats` every few seconds) +- MIL/DTC indicator (from `/api/dtc`) + +**28.5 — Trip history** +- List trips from `/api/trips` with date, distance, duration, start/end locations +- Tap to view route on map with scrubber timeline (reuse the same data as Phase 18 web + replay, just rendered natively) +- Geocode via the Pi's `/api/geocode` or the phone's own Android + `Geocoder` (no network needed for the Pi's offline `reverse_geocoder` path) + +**28.6 — Remote portal direct access (away from car)** +- Talk to `https://yaris.ceaf.dev/api/v1/portal/...` directly +- Browse trip history and dashcam archive from anywhere (not just when connected to Pi) +- Login with portal credentials (same as Phase 24 web portal login) + +**28.7 — Polish + distribution** +- App icon, splash screen, dark theme polish +- Error handling: Pi unreachable, server unreachable, no cellular data, storage full +- APK signing + release build +- Distribution: APK sideload (simplest — host the APK on the Pi itself at + `http://yaris.local/app/yaris-obd.apk` so the user just opens that URL on the phone + to install/update). Play Store is out of scope (Google developer account fee, review + process, updates lag behind development). + +#### Open questions (resolve during 28.1) + +1. **Dual-network binding on the owner's Pixel**: `bindProcessToNetwork()` behavior varies + by Android version and manufacturer. Test on the real device before committing to the + full build. If it doesn't work, fallback options: (a) download-then-switch-networks + two-step flow (download from Pi on WiFi, prompt user to switch to cellular, then + upload — clunky but always works), or (b) use the phone's hotspot instead of the Pi's + (the phone hosts, the Pi connects as client — reverses the network topology and + eliminates the dual-network problem entirely, but means the Pi is a WiFi client, not + an AP). + +2. **Credentials flow**: how does the app get `YARIS_SERVER_URL` + `YARIS_DEVICE_NAME` + + `YARIS_API_KEY`? Options: + - User types them in (simple, works, annoying) + - App connects to Pi hotspot → calls a new `GET /api/app/provision` endpoint that + returns the creds JSON (zero manual entry, but adds one Pi endpoint) + - QR code on the Pi's Settings page (scan once, app stores forever) + - App stores its own independent copy (user enters once in app settings, never touches + the Pi's copy) + +3. **Socket.io for real-time data**: the Pi's dashboard uses Socket.IO for live OBD/GPS + updates. The app could use a Socket.IO Kotlin client (`io.socket:socket.io-client`) + for push updates, or simply poll the REST endpoints every 2–5 seconds. Polling is + simpler and the Pi is on the same local WiFi (low latency, trivial load). Decision: + start with polling, only add Socket.IO if the polling feels laggy. + +4. **Video playback on the phone**: after downloading a dashcam clip to the phone, the + app should be able to play it. Android's `ExoPlayer` (Media3) handles H.264 MP4/MOV + natively — same codec the dashcam records. No transcoding needed. + +#### Explicitly out of scope + +- **iOS app**: not planned. Android-only. If an iOS user wants the same features, they + can use the web UI (with the Phase 11 limitation that they can't do simultaneous + dual-network relay — download+upload would be two separate manual steps). +- **Play Store listing**: sideload only for now. Lowers the barrier to ship and iterate. +- **Background GPS tracking**: the Pi already does this. The app is a display/relay + client, not a replacement tracker. +- **Push notifications from server**: the Pi's alert engine (Phase 20) handles + ntfy-based push. The app doesn't need its own push channel. +- **Offline mode when neither Pi nor internet is available**: the app is a connected + companion, not a standalone tool. Cached trip data from the last sync is a nice-to-have + but not required for v1. ## Working Notes / Environment Gotchas diff --git a/modules/app_device_sync.py b/modules/app_device_sync.py index 4e40566..693a3fa 100644 --- a/modules/app_device_sync.py +++ b/modules/app_device_sync.py @@ -122,8 +122,8 @@ def _store_synced_config(config): app_sql.set_setting("device_avatar_path", config.get("avatar_path", "img/yaris.png")) app_sql.set_setting("device_config_version", str(config.get("config_version", 0))) app_sql.set_setting("device_config_synced_at", str(int(time.time()))) - if config.get("odometer_baseline") is not None: - app_sql.set_setting("odometer_baseline", str(config["odometer_baseline"])) + # odometer_baseline is Pi→Server only — never overwrite the Pi's + # local value with whatever the server has (which could be stale). def _on_config_changed():