Nominatim was slow (rate-limited, 1 req/sec, ~4s timeouts) and unreliable
from Docker/Cloudflare (blocked or throttled). It was the root cause of
location labels failing to appear during trip scrubbing.
- server/app.py: _geocode_cached now uses reverse_geocoder exclusively
(offline, instant, city/province level). Removed Nominatim HTTP calls,
ThreadPoolExecutor, concurrent.futures import.
- app.py (Pi): same — reverse_geocoder imported at module level,
Nominatim + urllib.request removed.
- Batch endpoint simplified to sequential loop with lru_cache + dedup.
No network dependency at all — geocoding is instant and always works.
reverse_geocoder provides city/province labels (e.g. 'Montreal, Quebec')
which is the right granularity for trip scrubbing.
JS preload(): replaced N individual GET /api/geocode (concurrency 2) with
a single POST /api/geocode/batch — one HTTP roundtrip instead of up to
N/2 sequential ones. schedLoc debounce reduced 600→200ms.
Server geocode_batch(): ThreadPoolExecutor(max_workers=4) parallelises
Nominatim/reverse_geocoder calls instead of sequential loop. Deduplicates
coords before submitting futures. lru_cache absorbs duplicates within batch.
Pi geocode_batch_route(): deduplicate coords, removed stale 'if label:'
guard (label always non-empty after BUG-13 fix).
Root cause: server _geocode_cached() returned None → /api/geocode
returned HTTP 204 with no body. JS frontend treated 204 as success,
r.json() on empty body threw → showed raw-coordinate fallback.
Server fixes:
- _geocode_cached() now always returns a non-empty string (coordinate
fallback when both Nominatim and reverse_geocoder fail)
- /api/geocode always returns HTTP 200 with {label: '...'} (never 204)
- /api/geocode/batch always includes every coord key in result dict
- Nominatim call: jsonv2 + accept-language=en (aligned with Pi)
- reverse_geocoder: mode=1 (aligned with Pi)
Pi alignment:
- _geocode_cached() now checks for empty reverse_geocoder label before
returning, falling through to coordinate string (same guard as server)
- _portal_device_id() reads from session, falls back to DEVICE_NAME env var
- GET/POST /api/v1/portal/select-device for selector data + set
- Dropdown in server base.html topbar, JS populates from API, page reloads on change
- All portal pages (overview, trips, dashcam, DTC, device settings) now use selected device
- Server: dtc_events table (migrate_phase27_dtc.sql), db.py helpers
(upsert_dtc_snapshot, list_dtc_events, delete_dtc_event)
- Server API: POST /api/v1/dtc/batch (device auth), GET/DELETE /api/v1/portal/dtc
- Server portal: /dtc page with code, description, MIL, status, timestamps, delete
- Server nav: DTC history in sidebar + mobile bottom nav
- Pi: _upload_dtc_snapshot() in app_upload.py — uploads when snapshot changes,
skips when OBD disconnected or unchanged
- add portal API and UI action to delete non-default devices
- protect the configured portal default device from deletion
- document that trip/history pages still scope to the single DEVICE_NAME
Co-authored-by: Copilot <[email protected]>
- add multi-device portal device management, key rotation, and sync request APIs
- add Pi settings UI for remote server URL, device name, and API key
- add local test-connection and sync-now actions
- make upload and config sync reuse dynamic shared credentials
- add Phase 25.5c migration and docs updates
Co-authored-by: Copilot <[email protected]>
- Add database migration (migrate_phase25_5a.sql):
* Add owner_name, car_model, device_name, avatar_path columns to devices table
* Add config_version and config_updated_at for cache invalidation
- Add server/modules/device_config.py:
* get_device_config(device_id): retrieve config for Pi sync
* update_device_config(device_id, data): update config with validation
- Add server API endpoints:
* GET /api/device-config (authenticated with device_key, Pi fetches config)
* POST /api/device-config (authenticated as user, admin updates config)
- Enhance server/modules/db.py:
* Add get_device_by_id() function to retrieve all device config fields
Phase 25.5a enables server-side device identity management and connection-driven sync.
Pi will fetch from this endpoint when internet available, keeping config in sync.
Co-authored-by: Copilot <[email protected]>
- app_alerts.py: null guard on list_local_videos() result (.get('files', []))
- app_cellular.py: remove erroneous .encode(errors='replace') on str response body
- app_settings.py: initialize apn='' when isinstance check fails (NameError guard)
- app_upload.py: move json import to top of file instead of inside except handler
- server/app.py: wrap int(request.args.get('limit')) in try/except ValueError
- server/app.py: wrap os.path.commonpath() in try/except ValueError (cross-root paths)
- tests/test_app_settings.py: add missing trips_upload_enabled/video_convert_enabled
to valid_payload() (fields added in Phase 23.5 but test not updated)
All 63 tests pass.
Co-authored-by: Copilot <[email protected]>
- history.js reads tile URL from data-tile-url attribute on #history-map div,
falling back to local offline tiles (Pi behavior unchanged)
- server history.html sets data-tile-url to live OSM tiles
- maxZoom raised to 19 (OSM supports it, local tiles were limited to 15)
- server geocode returns 'label' key to match what history.js/yaris.js expect
Co-authored-by: Copilot <[email protected]>
- /api/trips now returns {trips:[...]} to match what history.js expects (result.trips)
- odo_start_km null guard in history.js (server trips have no odometer data)
- require_login returns 401 JSON for /api/* paths instead of HTML redirect
Co-authored-by: Copilot <[email protected]>
- GET/POST /account/password with CSRF token and bcrypt re-hash
- 'Password' link in sidebar next to Logout
- change_password.html template
Co-authored-by: Copilot <[email protected]>
Removes the phone-relay feature entirely per the Phase 11 finding
that it hit a fundamental phone-OS limitation (a browser page can't
route WiFi and cellular traffic simultaneously, no code fix possible)
- see PLAN.md for the full test writeup.
- app.py: dropped /api/dash/pending, /api/dash/file/<path>,
/api/dash/relay/config, /api/dash/relay/heartbeat,
/api/dash/relay/stop, /api/dash/relay/mark_uploaded routes and
their now-unused imports (os, send_file, abort, safe_join).
- modules/app_upload.py: dropped list_pending_dashcam_files(),
relay_heartbeat()/relay_stop()/is_relay_active(),
RELAY_LEASE_SECONDS/_relay_lease_until, and the relay-lease check
inside _upload_dashcam_files()/get_status(). Kept the chunked
upload protocol, dashcam_hash_cache/_hash_dashcam_file() (still
used by the Pi's own background uploader), and DASHCAM_CHUNK_SIZE -
none of that was relay-specific.
- templates/dash.html: removed the 'Relay via phone' card.
- static/js/dash.js: removed all relay JS (relayOneFile,
fetchChunkWithRetry, uploadChunkWithRetry, heartbeat timers,
pending-file list rendering).
- server/app.py: removed the CORS after_request hook, added solely
for the phone's cross-origin browser calls (the Pi's own Python
requests-based client was never subject to CORS).
- readme.md: dropped the phone-relay feature bullet, updated the
project-status blurb to explain the removal + point at the planned
native Android app as the real fix for that use case.
- PLAN.md: marked Phase 12 done with a verification writeup.
Verified: py_compile on all touched Python files, node --check on
dash.js, grepped all touched files for any leftover relay/RELAY
reference (none), and smoke-tested by importing app.py + hitting it
with Flask's test client - app starts cleanly, every relay route is
gone from url_map, and both / and /dash still render HTTP 200 with
zero 'relay' occurrences in /dash's HTML.
Co-authored-by: Copilot <[email protected]>
Root-caused the slow-upload complaint to the Pi's own congested 2.4GHz
WiFi radio (confirmed via iperf3 - same degraded throughput even on
pure local LAN traffic, ruling out Cloudflare/ISP). Since the Pi 3 B
has no 5GHz radio, add a way to bypass it for the internet-bound leg:
a phone opens /dash and manually relays pending dashcam files straight
to the remote server over its own network, reusing the exact same
chunked upload protocol the Pi's client already uses.
- server/app.py: add CORS headers (after_request) so the phone's
browser can call the remote server cross-origin from a page served
by the Pi.
- app.py: new endpoints - GET /api/dash/pending (list not-yet-uploaded
files), GET /api/dash/file/<path> (Range-request-capable download,
path-traversal safe), GET /api/dash/relay/config (hands the phone
the device credentials), POST /api/dash/relay/heartbeat|stop (lease-
based pause of the Pi's own background uploader, self-heals if the
phone's tab closes uncleanly), POST /api/dash/relay/mark_uploaded.
- modules/app_upload.py: list_pending_dashcam_files(), relay lease
helpers, _upload_dashcam_files() now pauses while a relay lease is
active.
- modules/app_sql.py: new dashcam_hash_cache table so a file's sha256
is only computed once (keyed by path+size+mtime) instead of being
recomputed on every loop iteration/pending-list call - benefits the
existing background uploader too.
- templates/dash.html / static/js/dash.js: new "Relay via phone" card
- per-file and "relay all" buttons (manual trigger only, per user
preference), downloads+uploads one 40MB slice at a time so the
phone never holds a whole file in memory, retries chunks up to 3x.
- PLAN.md/readme.md/server/README.md updated (Phase 11).
Tested via a full local end-to-end simulation driving the real Pi and
server Flask apps side by side (Postgres mocked only at the db module
boundary - all HTTP/chunking/auth/CORS logic is real): verified Range
downloads (206), byte-for-byte reassembly via SHA256, path-traversal
rejection, and the relay lease correctly pausing/resuming the
background uploader. Not yet tested against a real phone/live network.
Confirmed against the real deployed server: Cloudflare Tunnel's free
tier caps request bodies at ~100MB and returns a plain 413 - a direct
LAN upload of the same 250MB test file succeeds, the identical request
through yaris.ceaf.dev does not. Real dashcam clips average ~320MB, so
no dashcam video could ever complete an upload through the tunnel as
it stood, regardless of the earlier memory-streaming fix.
Server: new /api/v1/dashcam/upload/chunk (accepts one chunk at a time,
keyed by relative_path+sha256+chunk_index/total_chunks, reassembles and
verifies the sha256 once every chunk has arrived) and
/api/v1/dashcam/upload/status (reports which chunk indices a given
upload already has, or "already_uploaded" if it's done) endpoints.
Chunks are staged under STORAGE_DIR/.upload_tmp/<device>/<sha256>/ and
cleaned up after reassembly; a stale partial set left by a differently
-chunked earlier attempt is detected via a small marker file and wiped
before restarting. The original single-shot /upload endpoint is left
in place unchanged.
Client (modules/app_upload.py): _upload_dashcam_files() now calls
_upload_video_chunked(), which checks /status first (so a prior
partial upload resumes instead of restarting from byte 0) and then
POSTs any missing 40MB chunks (DASHCAM_CHUNK_SIZE, comfortably under
Cloudflare's cap) via the same streaming MultipartEncoder approach as
before.
Co-authored-by: Copilot <[email protected]>
The upload loop previously always slept the full 30s UPLOAD_INTERVAL
between batches, even when a full batch was just sent (meaning more
rows were likely still queued). With GPS/OBD rows logged every 5s
locally, an offline/stale device could build up a large backlog that
took a very long time to drain at 100 rows/30s.
- Bump TRACKING_BATCH_SIZE 100 -> 500 (cuts round-trips per backlog).
- Only sleep the full UPLOAD_INTERVAL once actually caught up (a
short batch came back); while a full batch keeps coming back, drain
back-to-back with a short CATCHUP_INTERVAL (1s) instead.
- server/app.py: add a sane MAX_TRACKING_BATCH_POINTS (5000) cap on
the tracking/batch endpoint as a defensive guard against a runaway
client, independent of the Pi's own batch size.
Verified against the real deployed server: a simulated 1250-row
backlog now drains in 3 batches (~0.35s server round-trip time) instead
of 13 batches spread across ~6.5 minutes of idle sleeping.
Co-authored-by: Copilot <[email protected]>
Mathieu Moran