fix: disabling sync via settings page no longer returns spurious error 400

2026-08-12 00:02:54 -04:00
parent 0fea9718bc
commit 79222c8bb5
3 changed files with 407 additions and 5 deletions
@@ -1963,10 +1963,11 @@ agree on OBD and GPS state. BUGS.md Open section is empty. readme.md project sta
 5. **FEAT-08** — Predictable IP on phone hotspot (small)
 6. **FEAT-09** — mDNS on phone hotspot (small)

-### Phase 28: Native Android companion app — 🚧 PLANNING (low priority, always last)
+### Phase 28: Native Android companion app — 🚧 IN PROGRESS (low priority, always last)
 **Renumbered from former Phase 26 on 2026-08-10 to make room for the road-test bug-fix phase.
 Renumbered to Phase 28 on 2026-08-11 to make room for Phase 27 feature additions. Planning
-begun 2026-08-11 alongside Phase 27 execution.**
+completed 2026-08-11; sub-phase 28.1 scaffold begins. Detailed implementation plan in
+[android/plan.md](android/plan.md).**

 #### Why native (not web, not PWA)
 Revives the phone-relay goal (Phase 11/12) properly this time. The Phase 11 update proved
@@ -0,0 +1,398 @@
+# yaris-obd Android Companion App — Implementation Plan
+
+Native Android (Kotlin + Jetpack Compose) companion app for the yaris-obd Pi system.
+Primary goal: dashcam file relay (download from Pi over hotspot WiFi → upload to remote
+server over cellular, simultaneously) — the one thing the web UI cannot do (see main
+PLAN.md Phase 11/12 for the platform limitation backstory).
+
+This file is the ground-level implementation plan. The high-level architecture, API
+surface, and scope decisions live in the main `PLAN.md` (Phase 28). This file tracks
+day-to-day progress, per-sub-phase implementation details, and working notes specific
+to the Android codebase.
+
+## Dev environment
+
+- **IDE:** Android Studio (Ladybug or newer) on Fedora dev machine
+- **Language:** Kotlin 2.x, JDK 17 (Android Studio bundled)
+- **Build:** Gradle with Kotlin DSL, version catalog (`gradle/libs.versions.toml`)
+- **Emulator:** API 35 AVD, 4GB RAM, 2GB storage — UI/flow testing only (single virtual
+  NIC, can't test dual-network binding)
+- **Real device:** owner's Pixel, USB debugging enabled — required for dual-network tests
+- **Repo:** `android/` at project root, part of the same `yaris-obd` git repo
+
+## Project scaffold
+
+```
+android/
+├── plan.md                          # this file
+├── .gitignore                       # Android-specific ignores
+├── build.gradle.kts                 # root build file
+├── settings.gradle.kts              # project settings + version catalog
+├── gradle/
+│   └── libs.versions.toml           # dependency version catalog
+├── gradle.properties                # JVM args, AndroidX, Compose compiler
+├── gradlew / gradlew.bat            # Gradle wrapper
+├── app/
+│   ├── build.gradle.kts             # app module build
+│   ├── proguard-rules.pro           # R8/ProGuard rules
+│   └── src/
+│       ├── main/
+│       │   ├── AndroidManifest.xml
+│       │   ├── java/com/yarisobd/app/
+│       │   │   ├── YarisApp.kt              # Application class
+│       │   │   ├── MainActivity.kt          # single-activity host
+│       │   │   ├── di/                       # manual DI or Hilt modules
+│       │   │   ├── data/
+│       │   │   │   ├── local/               # Room DB, DAOs, entities
+│       │       │   │   ├── remote/           # Retrofit services, DTOs
+│       │       │   │   └── repository/       # repository layer
+│       │       │   ├── domain/
+│       │       │   │   └── model/            # domain models
+│       │       │   ├── ui/
+│       │       │   │   ├── theme/            # Material 3 theme, colors, typography
+│       │       │   │   ├── navigation/       # NavHost, route definitions
+│       │       │   │   ├── dashboard/        # live car dashboard screen
+│       │       │   │   ├── dashcam/          # file browser + download screen
+│       │       │   │   ├── upload/           # upload queue + progress screen
+│       │       │   │   ├── trips/            # trip list + route replay screen
+│       │       │   │   ├── settings/         # credentials + preferences screen
+│       │       │   │   └── components/       # shared composables
+│       │       │   └── util/
+│       │       │       └── NetworkBinder.kt   # dual-network socket binding
+│       │       └── res/
+│       │           ├── values/               # strings, colors, themes
+│       │           ├── drawable/             # icons, splash
+│       │           └── mipmap/               # app launcher icons
+│       └── test/                             # unit tests
+│       └── androidTest/                      # instrumented tests
+```
+
+## Sub-phases
+
+### 28.1 — Scaffold + networking proof-of-concept ⬜ NOT STARTED
+**Gate: dual-network binding must work on the real Pixel before anything else is built.**
+
+#### 28.1.1 — Project creation
+- [ ] Create Android project in `android/` via Android Studio (Empty Activity + Compose)
+- [ ] Kotlin DSL build files with version catalog
+- [ ] `.gitignore` for Android (`*.iml`, `.gradle/`, `build/`, `local.properties`, `.idea/`)
+- [ ] Verify project builds and runs on emulator (blank screen)
+- [ ] Commit: `init: android project scaffold`
+
+#### 28.1.2 — Networking layer
+- [ ] Add dependencies: OkHttp 4.x, Retrofit 2.x, Kotlinx Serialization (or Moshi)
+- [ ] Create `NetworkBinder.kt` — utility class that:
+  - On startup, discovers available `Network` objects via `ConnectivityManager`
+  - Identifies the WiFi network matching `10.42.0.0/24` (Pi hotspot subnet)
+  - Identifies the cellular network (`TRANSPORT_CELLULAR`)
+  - Exposes `bindToWifi(): OkHttpClient` and `bindToCellular(): OkHttpClient` factories
+  - Falls back gracefully if one network is unavailable
+- [ ] Create Retrofit service interfaces:
+  - `PiApi.kt` — endpoints on `http://10.42.0.1` (or dynamically resolved)
+  - `ServerApi.kt` — endpoints on `https://yaris.ceaf.dev`
+- [ ] Wire up a manual test screen with two buttons + status text:
+  - "Test Pi" → calls `/api/gps/status` on WiFi-bound client, shows response
+  - "Test Server" → calls `/health` on cellular-bound client, shows response
+- [ ] **Real-device gate test:**
+  - Connect phone to Pi hotspot ("yaris 911")
+  - Verify cellular data is still active (browse any website while on hotspot)
+  - Press "Test Pi" → should return GPS status JSON
+  - Press "Test Server" → should return `{"status":"ok"}`
+  - Press both rapidly → both should succeed
+  - If either fails: investigate `bindProcessToNetwork()` behavior on this Pixel/Android version
+- [ ] Commit: `feat: dual-network proof-of-concept`
+
+#### 28.1.3 — Credential storage
+- [ ] Add EncryptedSharedPreferences for server URL, device name, API key
+- [ ] First-launch screen: text fields for server URL, device name, API key
+- [ ] "Test connection" button on credentials screen (calls `/health` on server)
+- [ ] Persist credentials after successful test
+- [ ] Commit: `feat: encrypted credential storage + first-launch flow`
+
+### 28.2 — Dashcam browser + download ⬜ NOT STARTED
+**Depends on: 28.1 (networking layer + credentials).**
+
+#### 28.2.1 — Pi API data layer
+- [ ] Retrofit interface for Pi endpoints:
+  - `GET /api/local-video/list` → `List<LocalVideoFile>` (relative_path, size, mtime, sha256, uploaded, thumbnail_url)
+  - `GET /api/local-video/file?path={relpath}` → `ResponseBody` (streaming, supports Range)
+  - `GET /api/local-video/thumbnail?path={relpath}` → `ResponseBody` (JPEG bytes)
+- [ ] Data classes for API responses (`LocalVideoFile`, etc.)
+- [ ] Repository: `VideoRepository` wrapping Pi API calls + Room cache
+- [ ] Commit: `feat: Pi video API data layer`
+
+#### 28.2.2 — Video file list UI
+- [ ] Compose screen: LazyColumn of video files
+- [ ] Each row: thumbnail (coil3 async image loader), filename, size, upload status badge
+- [ ] Sort controls: by name, size, date
+- [ ] Selection: checkboxes + "Select all" / "Clear"
+- [ ] Pull-to-refresh (re-fetches `/api/local-video/list`)
+- [ ] Empty state: "No imported videos on the Pi"
+- [ ] Error state: "Cannot reach Pi" with retry button
+- [ ] Commit: `feat: video file list screen`
+
+#### 28.2.3 — WiFi download
+- [ ] Room entity: `DownloadedFile` (relative_path, sha256, local_uri, size, downloaded_at)
+- [ ] Room DAO: insert, query, delete downloaded files
+- [ ] Download manager: OkHttp `Call` with Range header support, writes to app external storage
+- [ ] Progress tracking: `Flow<DownloadProgress>` emitting bytes downloaded / total
+- [ ] UI: download button per file → progress bar → "Downloaded" checkmark
+- [ ] Resume: if download interrupted, resume from last byte (Range: bytes=X-)
+- [ ] Commit: `feat: WiFi download with progress + resume`
+
+### 28.3 — Cellular upload ⬜ NOT STARTED
+**Depends on: 28.2 (files on phone local storage).**
+
+#### 28.3.1 — Server upload data layer
+- [ ] Retrofit interface for server chunked upload:
+  - `GET /api/v1/dashcam/upload/status?sha256={hash}` → `UploadStatus`
+  - `POST /api/v1/dashcam/upload/chunk` → multipart (file chunk, sha256, chunk_index, total_chunks)
+- [ ] Data classes: `UploadStatus` (already_uploaded, received_chunk_indices, total_chunks)
+- [ ] Commit: `feat: server chunked upload API`
+
+#### 28.3.2 — Upload worker
+- [ ] WorkManager `CoroutineWorker`:
+  - Input: file local URI, sha256, relative_path
+  - Step 1: check `/status` for already-received chunks
+  - Step 2: read file in 40MB chunks (matching `DASHCAM_CHUNK_SIZE`)
+  - Step 3: POST each missing chunk via cellular-bound OkHttp client
+  - Step 4: verify server reports `already_uploaded` after last chunk
+  - Step 5: delete local file copy on success (optional, configurable)
+- [ ] Foreground service notification: "Uploading VID_00234.MOV — 3/8 chunks (37%)"
+- [ ] Retry on failure: WorkManager backoff policy (exponential, max 3 attempts)
+- [ ] Commit: `feat: WorkManager cellular upload worker`
+
+#### 28.3.3 — Upload queue UI
+- [ ] Screen: list of downloaded files waiting to upload
+- [ ] Per-file: status (queued / uploading / done / failed), progress bar, retry button
+- [ ] "Upload all" button → enqueue all pending files
+- [ ] Observation: watch WorkManager `WorkInfo` via `LiveData`/`Flow` for real-time status
+- [ ] Commit: `feat: upload queue screen`
+
+### 28.4 — Live dashboard ⬜ NOT STARTED
+**Depends on: 28.1 (networking layer). Independent of 28.2/28.3.**
+
+#### 28.4.1 — Data layer
+- [ ] Retrofit interface additions to `PiApi`:
+  - `GET /api/gps/status` → `GpsStatus` (has_fix, lat, lon, fix_age_seconds, thread_alive)
+  - `GET /api/obd/status` → `ObdStatus` (adapter_present, port, car_connected)
+  - `GET /api/stats` → `Stats` (speed, rpm, coolant_temp, fuel_level, throttle_pos, mil_on, dtc_count, odometer_km)
+- [ ] Polling scheduler: coroutine `while(true) { delay(2000); fetch() }` scoped to ViewModel
+- [ ] Commit: `feat: dashboard data polling`
+
+#### 28.4.2 — Dashboard UI
+- [ ] GPS card: MapLibre MapView centered on last fix with car marker
+- [ ] Speed card: large number (km/h), OBD or GPS source label
+- [ ] RPM card: gauge or number with color scale (green → yellow → red)
+- [ ] Engine stats row: coolant temp (°C), fuel level (%), throttle (%)
+- [ ] MIL/DTC indicator: check-engine icon (green/grey = no MIL, yellow/red = MIL on)
+  with DTC code count badge
+- [ ] Connection status bar: Pi WiFi icon (green = connected), cellular icon (green = online)
+- [ ] Auto-refresh every 2s, pause when app is backgrounded
+- [ ] Commit: `feat: live dashboard screen`
+
+### 28.5 — Trip history ⬜ NOT STARTED
+**Depends on: 28.1. Independent of 28.2–28.4.**
+
+#### 28.5.1 — Data layer
+- [ ] Retrofit interface additions:
+  - `GET /api/trips` → `List<TripSummary>`
+  - `GET /api/trip/{id}` → `TripRoute` (list of route points with lat, lon, speed, timestamp)
+  - `GET /api/geocode?lat={}&lon={}` → `GeocodeResult` (label)
+- [ ] Room cache: store trip list + last-viewed route for offline viewing
+- [ ] Commit: `feat: trip history data layer`
+
+#### 28.5.2 — Trip list UI
+- [ ] LazyColumn of trips: date, start time, distance (km), duration, start→end locations
+- [ ] Tap → navigates to route replay
+- [ ] Sort: by date (default: newest first)
+- [ ] Commit: `feat: trip list screen`
+
+#### 28.5.3 — Route replay UI
+- [ ] MapLibre MapView with route polyline + start/end markers
+- [ ] Timeline scrubber (slider) with timestamp + speed label
+- [ ] Play/pause button: animates marker along route
+- [ ] Geocoded location name below scrubber (debounced while scrubbing)
+- [ ] Trip stats header: date, distance, duration, avg speed, max speed
+- [ ] Commit: `feat: route replay screen`
+
+### 28.6 — Remote portal direct access ⬜ NOT STARTED
+**Depends on: 28.1 (credentials). Independent of 28.2–28.5. Stretch goal.**
+
+#### 28.6.1 — Auth + data layer
+- [ ] Retrofit interface for server portal:
+  - `POST /login` → session cookie (or token)
+  - `GET /api/v1/portal/stats` → overview stats
+  - `GET /api/v1/portal/trips` → trip list
+  - `GET /api/v1/portal/trip/{id}` → trip route
+  - `GET /api/v1/portal/dashcam` → dashcam file list
+  - `GET /api/v1/portal/dashcam/stream/{id}` → video stream
+- [ ] Session persistence: store cookie/token in EncryptedSharedPreferences
+- [ ] Commit: `feat: server portal data layer`
+
+#### 28.6.2 — Portal screens
+- [ ] Reuse trip list + route replay composables from 28.5 (same data shape)
+- [ ] Reuse dashcam list + player composables (server serves same JSON shape as Pi)
+- [ ] Login screen (if not already authenticated)
+- [ ] "Connected to car" vs "Connected to server" mode indicator
+- [ ] Commit: `feat: remote portal screens`
+
+### 28.7 — Polish + distribution ⬜ NOT STARTED
+**Depends on: 28.1–28.6 (whichever subset ships).**
+
+#### 28.7.1 — App polish
+- [ ] App icon (adaptive icon: foreground + background layers)
+- [ ] Splash screen (Android 12+ SplashScreen API)
+- [ ] Dark theme verified on all screens (default, car-friendly)
+- [ ] Error handling audit: every network call has a user-visible error state + retry
+- [ ] Accessibility: content descriptions on icons, reasonable touch targets (≥48dp)
+- [ ] Commit: `feat: app polish`
+
+#### 28.7.2 — Build + distribution
+- [ ] Release build signing key (keystore generated, not committed)
+- [ ] ProGuard/R8 rules verified (Retrofit, OkHttp, Coil, MapLibre keep rules)
+- [ ] APK output: `app/build/outputs/apk/release/app-release.apk`
+- [ ] Self-hosted distribution:
+  - Copy APK to Pi: `scp app-release.apk [email protected]:~/app/yaris-obd/static/app/`
+  - Add a lightweight Pi route: `GET /app` → serves `static/app/index.html` with
+    download link + install instructions
+  - Phone flow: connect to Pi hotspot → open `http://yaris.local/app` → download APK →
+    install (user must enable "Install unknown apps" once)
+- [ ] Commit: `feat: release build + self-hosted distribution`
+
+## Working notes
+
+### Dual-network binding reference
+
+The key mechanism — from the Android docs and the Phase 11 investigation:
+
+```kotlin
+val connectivityManager = context.getSystemService<ConnectivityManager>()
+
+// Find the WiFi network (connected to Pi hotspot)
+val wifiNetwork: Network? = connectivityManager.allNetworks
+    .firstOrNull { network ->
+        val caps = connectivityManager.getNetworkCapabilities(network)
+        caps?.hasTransport(NetworkCapabilities.TRANSPORT_WIFI) == true &&
+        caps?.hasCapability(NetworkCapabilities.NET_CAPABILITY_INTERNET) == false
+        // Pi hotspot has no internet — we WANT this network for local API calls
+    }
+
+// Find the cellular network
+val cellNetwork: Network? = connectivityManager.allNetworks
+    .firstOrNull { network ->
+        val caps = connectivityManager.getNetworkCapabilities(network)
+        caps?.hasTransport(NetworkCapabilities.TRANSPORT_CELLULAR) == true
+    }
+
+// Create OkHttp clients bound to specific networks
+fun OkHttpClient.bindToNetwork(network: Network?): OkHttpClient {
+    if (network == null) return this
+    return newBuilder()
+        .socketFactory(network.socketFactory)
+        .dns { network.getAllByName(it) }
+        .build()
+}
+```
+
+The two clients then share nothing — different socket factories, different DNS resolvers —
+fully independent network paths.
+
+### Pi hotspot info
+
+- SSID: "yaris 911" (WPA2-PSK)
+- Subnet: `10.42.0.0/24`
+- Pi IP: `10.42.0.1`
+- mDNS: `yaris.local` (may not resolve on phone hotspot — documented limitation)
+- App should use IP `10.42.0.1` directly, not rely on mDNS
+
+### Chunked upload protocol (reused from Phase 10)
+
+- Chunk size: 40MB (well under Cloudflare's 100MB free-tier limit)
+- `/status?sha256=...` returns `{already_uploaded: true}` or `{received_chunk_indices: [0,1,3], total_chunks: 8}`
+- `/chunk` POST: multipart with `file` (binary chunk), `sha256`, `chunk_index` (0-based), `total_chunks`
+- Server reassembles when all chunks received, verifies SHA256, records in Postgres
+- Entirely deduplicated by SHA256 — safe to re-upload chunks, server ignores duplicates
+
+### Dependencies (tentative)
+
+```toml
+[versions]
+kotlin = "2.1.0"
+compose-bom = "2025.01.00"
+okhttp = "4.12.0"
+retrofit = "2.11.0"
+kotlinx-serialization = "1.7.3"
+room = "2.6.1"
+workmanager = "2.10.0"
+coil = "3.0.4"
+maplibre = "11.5.0"
+hilt = "2.51.1"
+
+[libraries]
+# Compose
+compose-bom = { group = "androidx.compose", name = "compose-bom", version.ref = "compose-bom" }
+compose-material3 = { group = "androidx.compose.material3", name = "material3" }
+compose-navigation = { group = "androidx.navigation", name = "navigation-compose" }
+
+# Network
+okhttp = { group = "com.squareup.okhttp3", name = "okhttp", version.ref = "okhttp" }
+retrofit = { group = "com.squareup.retrofit2", name = "retrofit", version.ref = "retrofit" }
+retrofit-kotlinx-serialization = { group = "com.squareup.retrofit2", name = "converter-kotlinx-serialization", version.ref = "retrofit" }
+
+# Local storage
+room-runtime = { group = "androidx.room", name = "room-runtime", version.ref = "room" }
+room-ktx = { group = "androidx.room", name = "room-ktx", version.ref = "room" }
+room-compiler = { group = "androidx.room", name = "room-compiler", version.ref = "room" }
+
+# Background work
+workmanager = { group = "androidx.work", name = "work-runtime-ktx", version.ref = "workmanager" }
+
+# Images
+coil-compose = { group = "io.coil-kt.coil3", name = "coil-compose", version.ref = "coil" }
+
+# Maps
+maplibre = { group = "org.maplibre.gl", name = "android-sdk", version.ref = "maplibre" }
+
+# Security
+security-crypto = { group = "androidx.security", name = "security-crypto", version = "1.1.0-alpha06" }
+
+# DI (optional — manual DI is fine for this scope; add Hilt only if boilerplate gets heavy)
+hilt-android = { group = "com.google.dagger", name = "hilt-android", version.ref = "hilt" }
+hilt-compiler = { group = "com.google.dagger", name = "hilt-compiler", version.ref = "hilt" }
+```
+
+### Real-device testing workflow
+
+1. Build debug APK: `./gradlew assembleDebug`
+2. Install on Pixel: `adb install app/build/outputs/apk/debug/app-debug.apk`
+3. Connect phone to Pi hotspot
+4. Confirm cellular data toggle is ON (Android should show both WiFi and LTE icons)
+5. Run the app → check dual-network test screen
+6. For upload tests: temporarily pause the Pi's own background uploader
+   (Settings → "Upload dashcam videos" toggle OFF) so the phone's upload isn't racing
+   the Pi's own uploader for the same files
+
+### Environment gotchas
+
+- **Emulator networking**: the Android emulator has its own virtual router. Accessing
+  `10.42.0.1` from the emulator goes through the host machine's network stack — it won't
+  reach the Pi unless the host is also on the Pi's WiFi hotspot. Prefer real-device testing
+  for any network-dependent code.
+- **Pixel "switch to mobile data automatically"**: some Android versions have a setting
+  that auto-disables WiFi if it detects no internet. On the Pi hotspot (which has no
+  internet), this could silently drop the WiFi connection. The Phase 11 testing already
+  confirmed this is a hard platform limitation for the web UI; the native app's
+  `bindProcessToNetwork()` might bypass it. If the OS still kills the WiFi association,
+  the fallback is asking the user to enable "Keep WiFi on when no internet" in Developer
+  Options.
+- **WorkManager + doze**: on Android 6+, WorkManager jobs may be deferred when the device
+  is in Doze mode. For uploads that the user explicitly triggered, consider using a
+  foreground service instead of (or in addition to) WorkManager so uploads continue
+  immediately.
+- **40MB chunk memory**: the OkHttp `MultipartBody` will buffer each 40MB chunk in memory
+  before sending. On modern phones with 6GB+ RAM this is fine, but if it becomes an issue,
+  switch to a streaming multipart approach (OkHttp's `RequestBody` as `okio` pipe).
+- **SHA256 of large files**: computing SHA256 of a 300MB file on a phone takes a few
+  seconds. Do it once when the download completes, store the result in Room, and reuse
+  for upload. Don't recompute on every upload attempt.
@@ -33,12 +33,15 @@ def update_device_config(body, errors):
    # Check if sync is enabled; if so, only allow sync-related updates
    sync_enabled = _get_sync_enabled()
    if sync_enabled:
-        # User cannot edit these fields if syncing from server
-        if any(k in body for k in ["owner_name", "car_model", "avatar_path"]):
-            errors["error"] = "Cannot edit device config while server sync is enabled. Server is source of truth."
        # Allow toggling sync_enabled off
        if "sync_enabled" in body and not body["sync_enabled"]:
            _set_sync_enabled(False)
+            # Toggling off is always allowed — don't return error even if
+            # other locked fields are in the body (the JS always sends them).
+            return errors
+        # User cannot edit locked fields while syncing from server.
+        if any(k in body for k in ["owner_name", "car_model", "avatar_path"]):
+            errors["error"] = "Cannot edit device config while server sync is enabled. Server is source of truth."
        return errors

    # Not syncing: allow all edits