BUG-03: DTC/MIL codes were never persisted; lost on OBD disconnect or reboot.
- Add new dtc table schema (code, description, MIL, first/last seen, IS_SET)
and a migration that rebuilds the original unused NAME/STATUS/DTC_ID table.
- Persist MIL + codes to SQLite on each successful OBD poll (_persist_dtc_state).
- /api/dtc falls back to the last-persisted snapshot when OBD is offline/stale.
- Fix get_overview_stats to count the real dtc table (was querying non-existent
dtc_codes table).
- Add cleared/since tracking via IS_SET + FIRST_SEEN_AT/LAST_SEEN_AT.
BUG-05: server trip replay hardcoded speed_source:'gps'; mirror Pi's
'obd if speed>0 else gps' logic in server get_trip_route.
Adds tests/test_dtc_persistence.py (7 tests).
- add /api/gps/status for thread/fix-age visibility
- surface GPS status in Pi health and live GPS cards
- add tests for app_gps status snapshots
Co-authored-by: Copilot <[email protected]>
- add multi-device portal device management, key rotation, and sync request APIs
- add Pi settings UI for remote server URL, device name, and API key
- add local test-connection and sync-now actions
- make upload and config sync reuse dynamic shared credentials
- add Phase 25.5c migration and docs updates
Co-authored-by: Copilot <[email protected]>
- load server sync creds from db/upload_creds.env like uploads do
- fall back from YARIS_DEVICE_KEY to existing YARIS_API_KEY
- document shared credential flow for server-synced identity
- add test coverage for sync credential fallback
Co-authored-by: Copilot <[email protected]>
- use app_network status for device sync connectivity checks
- mount migrate_phase25_5a.sql in server docker compose
- document manual server upgrade migration steps
- record verified Pi and unraid SSH access details in PLAN
Co-authored-by: Copilot <[email protected]>
- Created app_device_sync.py: background sync thread with connectivity-driven scheduling (5min online, 60sec backoff when offline)
- Server /api/device-config endpoints: GET for Pi device auth, POST for admin user auth
- Database migration adds config_version, owner_name, car_model, device_name, avatar_path to devices table
- Pi syncs config from server on network availability; offline-first design (loads from local JSON at startup)
- Settings UI fields read-only when server sync enabled
All 63 tests passing. Ready for testing with actual server connection.
Co-authored-by: Copilot <[email protected]>
- Add app_device_config.py module for managing device configuration (owner_name, car_model, avatar_path, device_name)
- Add sync_enabled flag to control whether Pi syncs config from server
- When sync is enabled, device config fields are read-only (server is source of truth)
- When sync is disabled, user can edit device config via Settings page
- Add /api/device-config endpoint (GET to retrieve, POST to update)
- Add Device identity section to settings.html with form fields for all config values
- Update settings.js to load/save device config with sync toggle
- Gracefully handles both config file (at startup) and database (live updates via settings UI)
- All 63 tests passing
Phase 25.5: Offline-first device config with eventual consistency. Config loads from JSON at startup (no network needed), can be edited via settings if not syncing, and will sync from server on next connectivity when Phase 25.5a server endpoints are implemented.
Co-authored-by: Copilot <[email protected]>
- Create config/device.json with owner_name, car_model, device_name, avatar_path
- Update app_account.py to load owner_name and avatar_path from config
- Update app_car.py to load car_model and device_name from config
- Update base.html template to use avatar_path from config instead of hardcoded
- Add graceful fallback to hardcoded defaults if config file missing
- Document in deploy/readme.md and readme.md how to customize device identity
- Update PLAN.md Phase 25.5 to reflect current implementation and future server sync
Benefits:
- No code edits needed to customize Pi (edit JSON file only)
- Offline-first: all values loaded from local file at startup
- Connection-driven sync planned for Phase 25.5a (sync when internet available, not on timer)
- Server becomes source of truth after Phase 25.5a implementation
All 63 tests passing. Database filename still based on owner_name, with migration warning in docs.
Co-authored-by: Copilot <[email protected]>
Counterfeit ELM327 chips (common in cheap PL2303 USB adapters) often report
bad/zero voltage even when the OBD connection is working. python-obd's
check_voltage=True default rejects these. Add OBD_SKIP_VOLTAGE_CHECK=1 env
var to pass check_voltage=False to obd.OBD(), documented in yaris-obd.service.
Also document Phase 24.6 Yaris 2013 compatibility analysis in PLAN.md.
Co-authored-by: Copilot <[email protected]>
- _decode_odometer: catch all exceptions and return None instead of raising.
PID 0xA6 (odometer) is not guaranteed on 2013 Toyota Yaris — a garbage
response or unsupported PID previously raised ValueError through python-obd's
decoder, which app_obd._query() caught and treated as a lost connection,
killing the entire OBD session. Now it gracefully returns None.
- connect_obd: reset all sensor globals (rpm, coolant_temp, fuel_level,
throttle_position, mil_on, dtc_codes, odometer, timestamps) on each new
connection so the UI doesn't display stale readings from the previous
ignition cycle while fresh polls are completing.
- yaris-obd.service: add commented OBD_PROTOCOL=6 hint for post-confirmation
pinning (eliminates 3-5s auto-probe delay after first confirmed connection).
Co-authored-by: Copilot <[email protected]>
- app_alerts.py: null guard on list_local_videos() result (.get('files', []))
- app_cellular.py: remove erroneous .encode(errors='replace') on str response body
- app_settings.py: initialize apn='' when isinstance check fails (NameError guard)
- app_upload.py: move json import to top of file instead of inside except handler
- server/app.py: wrap int(request.args.get('limit')) in try/except ValueError
- server/app.py: wrap os.path.commonpath() in try/except ValueError (cross-root paths)
- tests/test_app_settings.py: add missing trips_upload_enabled/video_convert_enabled
to valid_payload() (fields added in Phase 23.5 but test not updated)
All 63 tests pass.
Co-authored-by: Copilot <[email protected]>
- ffmpeg -progress pipe:1 streams out_time_ms lines; we parse them in the
reading loop so % = out_time_s / duration_s (actual encode position)
- find_video_files() skips *.converting.mp4 temp files so they are never
listed, uploaded, or shown in the UI
Co-authored-by: Copilot <[email protected]>
- OBD odometer now logged to tracking rows on every GPS snapshot
- trips table: START_ODOMETER/END_ODOMETER columns (migration)
- build_trip_summaries: picks odometer from first/last tracking row
- list_trips: uses real OBD odo, falls back to cumulative distance
- overview stats: odometer_km (max seen from OBD), video_count fixed
- history page: Odometer column shows start→end km per trip
- app_upload: TRIPS_UPLOAD_SETTING_KEY, is_trips_upload_enabled()
- _upload_tracking_batch: skips when trips upload is off
- settings: separate 'Upload trips & GPS' toggle from dashcam toggle
Co-authored-by: Copilot <[email protected]>
File size ratio was wrong since output bitrate != input bitrate.
Now uses ffprobe to get video duration, then elapsed/duration for %.
Co-authored-by: Copilot <[email protected]>
- New POST /api/local-video/convert route triggers queue_convert(force=True)
- queue_convert() now returns True/False/None and accepts force param
to bypass the auto-convert toggle (manual always works)
- Convert button shown for non-MP4, non-converting files in video list
- Clicking spins the button and refreshes list after 1s to show badge
Co-authored-by: Copilot <[email protected]>
- _convert_bg now runs in real threading.Thread (not gevent greenlet)
so ffmpeg proc.wait() doesn't block gevent event loop
- _device_lock released before copy loop; only held for mount/unmount
so /api/dash/files and other routes don't block during long copies
- Date now shows time (Aug 9, 10:30 PM) on mobile video rows
- Spacing added above converting spinner badge
Co-authored-by: Copilot <[email protected]>
- Import now returns immediately with {status: 'started'}
- Backend runs copy in a daemon thread, tracks job state in _import_job
- New GET /api/dash/import/status endpoint for polling
- UI polls every 3s, shows progress bar until done
- Page reload mid-import resumes polling automatically
- Prevents double-import with 'busy' guard
Co-authored-by: Copilot <[email protected]>
- yaris.css: on mobile (<576px) hide file/size/date/upload cols, show
filename inside thumb cell, icon-only buttons, dash-actions-td nowrap
- local-videos.js: filename in thumb cell for mobile; Converting badge with
spinner; auto-refresh every 10s while any file is converting
- app_video.py: CONVERT_SETTING, is_convert_enabled(), queue_convert(),
_convert_bg() — h264_v4l2m2m + aac faststart, deletes original on success
- app_dash.py: call queue_convert() after each successful copy on import
- app_settings.py: expose video_convert_enabled in get/save settings
- settings.html: toggle for 'Convert imported videos for web playback'
Co-authored-by: Copilot <[email protected]>
- app_sql.py: add get_overview_stats() with single SQL query (trip count,
total km, duration, max speed, active DTC count)
- app.py: add /api/stats route (was missing after scp overwrote the file)
- local-videos.js: remove leftover old closePlayer/playVideo/renderFiles
duplicate block that still had scrollIntoView and references to the old
panel elements; new modal-based code is the only path now
Co-authored-by: Copilot <[email protected]>
- app.py: replace send_file with manual Range-aware chunked generator;
returns 206 on partial requests so browsers can seek/buffer properly.
Thumbnail route returns 202 while ffmpeg is generating.
- app_video.py: thumbnail generation moved to background gevent greenlet
(_generate_thumbnail_bg). get_thumbnail_path returns immediately with
RuntimeError('generating') if not cached; ffmpeg only ever runs once
per video (result cached in dash/thumbnails/).
- dash.html: replaced bottom card player with Bootstrap modal (modal-xl,
centered, dark bg) — video pops up in place regardless of scroll position.
- local-videos.js: rewritten to use Bootstrap modal; thumbnail loader
polls with fetch() and retries on 202 every 2s (up to 15x) instead of
setting img.src directly (which can't detect 202).
Co-authored-by: Copilot <[email protected]>
Prefer a configured phone hotspot for outbound synchronization, preserve secure write-only credentials, and recover through home WiFi or the local Pi hotspot.
Co-authored-by: Copilot <[email protected]>
Removes the phone-relay feature entirely per the Phase 11 finding
that it hit a fundamental phone-OS limitation (a browser page can't
route WiFi and cellular traffic simultaneously, no code fix possible)
- see PLAN.md for the full test writeup.
- app.py: dropped /api/dash/pending, /api/dash/file/<path>,
/api/dash/relay/config, /api/dash/relay/heartbeat,
/api/dash/relay/stop, /api/dash/relay/mark_uploaded routes and
their now-unused imports (os, send_file, abort, safe_join).
- modules/app_upload.py: dropped list_pending_dashcam_files(),
relay_heartbeat()/relay_stop()/is_relay_active(),
RELAY_LEASE_SECONDS/_relay_lease_until, and the relay-lease check
inside _upload_dashcam_files()/get_status(). Kept the chunked
upload protocol, dashcam_hash_cache/_hash_dashcam_file() (still
used by the Pi's own background uploader), and DASHCAM_CHUNK_SIZE -
none of that was relay-specific.
- templates/dash.html: removed the 'Relay via phone' card.
- static/js/dash.js: removed all relay JS (relayOneFile,
fetchChunkWithRetry, uploadChunkWithRetry, heartbeat timers,
pending-file list rendering).
- server/app.py: removed the CORS after_request hook, added solely
for the phone's cross-origin browser calls (the Pi's own Python
requests-based client was never subject to CORS).
- readme.md: dropped the phone-relay feature bullet, updated the
project-status blurb to explain the removal + point at the planned
native Android app as the real fix for that use case.
- PLAN.md: marked Phase 12 done with a verification writeup.
Verified: py_compile on all touched Python files, node --check on
dash.js, grepped all touched files for any leftover relay/RELAY
reference (none), and smoke-tested by importing app.py + hitting it
with Flask's test client - app starts cleanly, every relay route is
gone from url_map, and both / and /dash still render HTTP 200 with
zero 'relay' occurrences in /dash's HTML.
Co-authored-by: Copilot <[email protected]>
Root-caused a "GPS not working" report to the SIM868 module chip being
fully powered off (the HAT's 5V rail can be live while the chip itself
stays off until PWRKEY is triggered) - confirmed by getting zero AT
response at any baud rate while the app's GPS thread had the serial
port open normally.
Discovered this specific HAT revision has the "P4" GPIO power-on mod
wire installed (Waveshare FAQ / GSM_PWRKEY.rar script), letting PWRKEY
be triggered via physical pin 7 (BCM GPIO4) instead of a manual button
press. Verified live: pulsing it low for 4s then high brought the
module from zero AT response to responding immediately.
modules/app_gps.py: added _module_responds()/_pulse_pwrkey()/
_ensure_module_powered(), wired into _poll_loop() before the existing
GNSS power-on check. Checks AT first and only pulses PWRKEY if the
module isn't already responding, since PWRKEY is a toggle (pulsing an
already-on module would turn it back off).
Also fixed a related logging gap found while debugging: print()
diagnostics from app_gps.py were invisible in yaris-obd.log for
months because Python fully-buffers stdout when it isn't a TTY.
deploy/yaris-obd.service now sets PYTHONUNBUFFERED=1.
requirements.txt: added RPi.GPIO.
PLAN.md: documented both findings and the Pi deploy gotchas (systemd
unit file changes need a manual copy + daemon-reload; requirements.txt
changes need a manual venv pip install - neither is picked up by a
plain git pull + restart).
Co-authored-by: Copilot <[email protected]>
Root-caused the slow-upload complaint to the Pi's own congested 2.4GHz
WiFi radio (confirmed via iperf3 - same degraded throughput even on
pure local LAN traffic, ruling out Cloudflare/ISP). Since the Pi 3 B
has no 5GHz radio, add a way to bypass it for the internet-bound leg:
a phone opens /dash and manually relays pending dashcam files straight
to the remote server over its own network, reusing the exact same
chunked upload protocol the Pi's client already uses.
- server/app.py: add CORS headers (after_request) so the phone's
browser can call the remote server cross-origin from a page served
by the Pi.
- app.py: new endpoints - GET /api/dash/pending (list not-yet-uploaded
files), GET /api/dash/file/<path> (Range-request-capable download,
path-traversal safe), GET /api/dash/relay/config (hands the phone
the device credentials), POST /api/dash/relay/heartbeat|stop (lease-
based pause of the Pi's own background uploader, self-heals if the
phone's tab closes uncleanly), POST /api/dash/relay/mark_uploaded.
- modules/app_upload.py: list_pending_dashcam_files(), relay lease
helpers, _upload_dashcam_files() now pauses while a relay lease is
active.
- modules/app_sql.py: new dashcam_hash_cache table so a file's sha256
is only computed once (keyed by path+size+mtime) instead of being
recomputed on every loop iteration/pending-list call - benefits the
existing background uploader too.
- templates/dash.html / static/js/dash.js: new "Relay via phone" card
- per-file and "relay all" buttons (manual trigger only, per user
preference), downloads+uploads one 40MB slice at a time so the
phone never holds a whole file in memory, retries chunks up to 3x.
- PLAN.md/readme.md/server/README.md updated (Phase 11).
Tested via a full local end-to-end simulation driving the real Pi and
server Flask apps side by side (Postgres mocked only at the db module
boundary - all HTTP/chunking/auth/CORS logic is real): verified Range
downloads (206), byte-for-byte reassembly via SHA256, path-traversal
rejection, and the relay lease correctly pausing/resuming the
background uploader. Not yet tested against a real phone/live network.
Confirmed against the real deployed server: Cloudflare Tunnel's free
tier caps request bodies at ~100MB and returns a plain 413 - a direct
LAN upload of the same 250MB test file succeeds, the identical request
through yaris.ceaf.dev does not. Real dashcam clips average ~320MB, so
no dashcam video could ever complete an upload through the tunnel as
it stood, regardless of the earlier memory-streaming fix.
Server: new /api/v1/dashcam/upload/chunk (accepts one chunk at a time,
keyed by relative_path+sha256+chunk_index/total_chunks, reassembles and
verifies the sha256 once every chunk has arrived) and
/api/v1/dashcam/upload/status (reports which chunk indices a given
upload already has, or "already_uploaded" if it's done) endpoints.
Chunks are staged under STORAGE_DIR/.upload_tmp/<device>/<sha256>/ and
cleaned up after reassembly; a stale partial set left by a differently
-chunked earlier attempt is detected via a small marker file and wiped
before restarting. The original single-shot /upload endpoint is left
in place unchanged.
Client (modules/app_upload.py): _upload_dashcam_files() now calls
_upload_video_chunked(), which checks /status first (so a prior
partial upload resumes instead of restarting from byte 0) and then
POSTs any missing 40MB chunks (DASHCAM_CHUNK_SIZE, comfortably under
Cloudflare's cap) via the same streaming MultipartEncoder approach as
before.
Co-authored-by: Copilot <[email protected]>
First real-hardware test of dashcam retrieval kept crashing the whole
yaris-obd service (repeated SIGKILL/OOM on the Pi 3's ~900MB RAM) a few
minutes into every run, aborting in-flight file copies. Root cause:
_upload_dashcam_files() used requests.post(files={"file": f}), which
builds the entire multipart/form-data body in memory before sending -
fine for small payloads, but this dashcam's real clips average ~320MB
each (up to a couple hundred more with overhead), so every dashcam
upload attempt alone could push the process well past 300-600MB, and
the Pi's OOM killer stepped in immediately.
Reproduced in isolation: uploading a 300MB file the old way spiked the
client process from 28MB to 628MB RSS; switching to requests-toolbelt's
MultipartEncoder (which streams the file in chunks as the request body
is sent, instead of materializing it up front) kept it flat at ~29MB
for the same upload, with an identical byte-for-byte result on the
receiving end. No server-side changes needed - the wire format is
unchanged, still ordinary multipart/form-data.
Added requests-toolbelt to requirements.txt.
Co-authored-by: Copilot <[email protected]>
mount_device() used a bare os.makedirs() to create the mount point,
but the app runs as an unprivileged user while /mnt is root-owned -
surfaced as a real PermissionError during the first actual hardware
test. Create the mount point via sudo mkdir -p instead, consistent
with the existing sudo mount/umount calls in this module.
Co-authored-by: Copilot <[email protected]>
First real-hardware test revealed the connected dashcam's SD card ships
factory-formatted with no partition table - lsblk reports it as a bare
"disk" with an fstype set directly, never as a child "part". The
previous detection logic only matched type=="part", so it silently
missed this real device entirely.
list_usb_storage_partitions() now also matches whole-disk USB devices
that have no children and an fstype set directly on them, in addition
to the existing partition-table case. Verified against a fixture built
from this dashcam's actual `lsblk -J` output.
Co-authored-by: Copilot <[email protected]>
Mathieu Moran