- INTERFACE = 'wlan1' in yaris_uplink.py
- install_netwatch.sh binds home-wifi, Hotspot, phone-hotspot to wlan1
- Disables wlan0 (onboard) via nmcli device set managed no
- WLAN variable at top of script for easy revert to wlan0 if needed
Co-authored-by: Copilot <[email protected]>
- /api/local-video/file?download=1 serves file as attachment
- Download link (green DL button) added to each row in local-videos.js
- Dashcam USB files skipped: no persistent mount for direct serve;
use Import then download from Pi Videos
Co-authored-by: Copilot <[email protected]>
- ffmpeg -progress pipe:1 streams out_time_ms lines; we parse them in the
reading loop so % = out_time_s / duration_s (actual encode position)
- find_video_files() skips *.converting.mp4 temp files so they are never
listed, uploaded, or shown in the UI
Co-authored-by: Copilot <[email protected]>
- OBD odometer now logged to tracking rows on every GPS snapshot
- trips table: START_ODOMETER/END_ODOMETER columns (migration)
- build_trip_summaries: picks odometer from first/last tracking row
- list_trips: uses real OBD odo, falls back to cumulative distance
- overview stats: odometer_km (max seen from OBD), video_count fixed
- history page: Odometer column shows start→end km per trip
- app_upload: TRIPS_UPLOAD_SETTING_KEY, is_trips_upload_enabled()
- _upload_tracking_batch: skips when trips upload is off
- settings: separate 'Upload trips & GPS' toggle from dashcam toggle
Co-authored-by: Copilot <[email protected]>
File size ratio was wrong since output bitrate != input bitrate.
Now uses ffprobe to get video duration, then elapsed/duration for %.
Co-authored-by: Copilot <[email protected]>
Instead of full re-render every 10s, patch converting badges in-place.
Only does a full renderFiles() when file count changes (conversion done)
or on the final poll when all conversions finish.
Co-authored-by: Copilot <[email protected]>
- New POST /api/local-video/convert route triggers queue_convert(force=True)
- queue_convert() now returns True/False/None and accepts force param
to bypass the auto-convert toggle (manual always works)
- Convert button shown for non-MP4, non-converting files in video list
- Clicking spins the button and refreshes list after 1s to show badge
Co-authored-by: Copilot <[email protected]>
- _convert_bg now runs in real threading.Thread (not gevent greenlet)
so ffmpeg proc.wait() doesn't block gevent event loop
- _device_lock released before copy loop; only held for mount/unmount
so /api/dash/files and other routes don't block during long copies
- Date now shows time (Aug 9, 10:30 PM) on mobile video rows
- Spacing added above converting spinner badge
Co-authored-by: Copilot <[email protected]>
- Import now returns immediately with {status: 'started'}
- Backend runs copy in a daemon thread, tracks job state in _import_job
- New GET /api/dash/import/status endpoint for polling
- UI polls every 3s, shows progress bar until done
- Page reload mid-import resumes polling automatically
- Prevents double-import with 'busy' guard
Co-authored-by: Copilot <[email protected]>
- yaris.css: split mobile rules by list ID — video-list shows thumb+actions,
dashcam-file-list shows dash-col-file+actions; thead display:none !important
- yaris.js: check data !== '-' before appending rpm/°C/% units so disconnected
state shows — instead of '- rpm'
Co-authored-by: Copilot <[email protected]>
Inline style.width on td was overriding the mobile media query width rule.
Use dash-thumb-td class for desktop (9rem) and mobile (5rem !important).
Co-authored-by: Copilot <[email protected]>
Bootstrap d-flex sets display:flex !important which beats the HTML hidden
attribute. Switch to style.display='none'/'flex' for both status bars.
Co-authored-by: Copilot <[email protected]>
- yaris.css: use .dash-row class on all rows; only td:first-child and
.dash-actions-td visible on mobile (<576px); buttons never overflow
- dash.js: add dash-row class + dash-actions-td wrapper div to dashcam
file rows; import status moved to card footer at top with spinner + ETA
- local-videos.js: add dash-row class to video rows
- dash.html: import status bar (spinner + ETA + result) in device card
at top; removed old footer import status from files card
Co-authored-by: Copilot <[email protected]>
Replace column-hiding approach (still caused 691px overflow) with full
table reflow: on <576px each tr becomes a flex row, all tds hidden except
first (thumb+name) and .dash-actions-td (buttons). Buttons can never
overflow because the row is constrained to viewport width.
Co-authored-by: Copilot <[email protected]>
- yaris.css: on mobile (<576px) hide file/size/date/upload cols, show
filename inside thumb cell, icon-only buttons, dash-actions-td nowrap
- local-videos.js: filename in thumb cell for mobile; Converting badge with
spinner; auto-refresh every 10s while any file is converting
- app_video.py: CONVERT_SETTING, is_convert_enabled(), queue_convert(),
_convert_bg() — h264_v4l2m2m + aac faststart, deletes original on success
- app_dash.py: call queue_convert() after each successful copy on import
- app_settings.py: expose video_convert_enabled in get/save settings
- settings.html: toggle for 'Convert imported videos for web playback'
Co-authored-by: Copilot <[email protected]>
- local-videos.js: action buttons go in a div.data-list-actions inside td
so td stays a table cell (not flex); thumbnail td gets fixed width 9rem
- app.py: video route now pipes ffmpeg with -c copy -movflags
frag_keyframe+empty_moov so browsers get a fragmented MP4 stream that
starts playing immediately without needing the moov atom at the end
Co-authored-by: Copilot <[email protected]>
- app_sql.py: add get_overview_stats() with single SQL query (trip count,
total km, duration, max speed, active DTC count)
- app.py: add /api/stats route (was missing after scp overwrote the file)
- local-videos.js: remove leftover old closePlayer/playVideo/renderFiles
duplicate block that still had scrollIntoView and references to the old
panel elements; new modal-based code is the only path now
Co-authored-by: Copilot <[email protected]>
- app.py: replace send_file with manual Range-aware chunked generator;
returns 206 on partial requests so browsers can seek/buffer properly.
Thumbnail route returns 202 while ffmpeg is generating.
- app_video.py: thumbnail generation moved to background gevent greenlet
(_generate_thumbnail_bg). get_thumbnail_path returns immediately with
RuntimeError('generating') if not cached; ffmpeg only ever runs once
per video (result cached in dash/thumbnails/).
- dash.html: replaced bottom card player with Bootstrap modal (modal-xl,
centered, dark bg) — video pops up in place regardless of scroll position.
- local-videos.js: rewritten to use Bootstrap modal; thumbnail loader
polls with fetch() and retries on 202 every 2s (up to 15x) instead of
setting img.src directly (which can't detect 202).
Co-authored-by: Copilot <[email protected]>
Phase 23 section updated with full list of changes from all polish passes:
Overview stats card, speed chart rework, mobile dashcam table fixes,
trip history mobile layout, notification badge fix, footer text.
README project status updated to reflect Phases 1-23 complete.
Co-authored-by: Copilot <[email protected]>
- Replace 8-point scatter chart with proper 60-point line chart
- Blue gradient fill (rgba 35%→2%), tension 0.4 smooth curve
- No legend (obvious), no X axis grid, soft Y axis gridlines
- Y axis auto-scales: max(60, sessionMax*1.2) rounded to 10s
- Ticks: '0 km/h' labels on Y, time strings on X (max 6)
- chart.update('none') for live updates (no animation stutter)
- Speed history card header now shows current speed + session max
even when the graph is collapsed
- Footer bar inside expanded graph: session avg, point count, last reading time
- makeGradient() refreshed on every update in case canvas was resized
- obd_rpm now appends ' rpm' to the displayed value
Co-authored-by: Copilot <[email protected]>
Dashboard:
- OBD stat cards more compact (py-2, h6 health values, 'OBD' label fits)
- obd-val class prevents Disconnected wrapping
- Speed graph collapsed by default with toggle chevron button
- GPS map + coordinates merged into single card with footer coords
- Pi health rows use h6 (less tall), alert py-2
Dashcam:
- Device status is now a compact single-line bar
- Files on dashcam section is collapsible (hidden by default)
- Import status moved into the dashcam files card footer
- Imported videos on Pi moved to top, open by default
- Column headers trimmed (Select→empty, Actions→empty, Preview→empty)
Trip history:
- col-12 on mobile so list stacks above the map
- Shorter column labels (Distance→km, Duration→Time, Max speed→Max)
- history-map height reduced for mobile (22rem on small screens)
Settings:
- Cellular + ntfy merged into one collapsible card
- Alert thresholds collapsible (advanced)
- Phone hotspot instructions shortened
- Upload retention uses inline row layout (compact)
- All cards use shadow-sm, smaller headers
CSS:
- obd-val: word-break + smaller font for OBD status values
- collapse-chevron: smooth rotation transition
- #map height 18rem (was 20rem)
- history-map 32rem desktop / 22rem mobile (was 36rem)
Co-authored-by: Copilot <[email protected]>
- Replace SB Admin 2 sidebar with a clean fixed sidebar (desktop)
and a sticky bottom tab navigation (mobile, ≤767px)
- Sidebar collapses to icon-only mode with state persisted in localStorage
- Logo is a functional home link on all viewports
- Mobile: full-width content, bottom nav replaces sidebar
- Dashboard: 6 OBD stat cards (status, speed, RPM, coolant, throttle, fuel)
wired to live socket events; removed all placeholder buttons/dropdowns
- GPS info section cleaned up (lat/lon display instead of X/Y paragraphs)
- DTC page: live /api/dtc wiring — MIL indicator + DTC table populated from API
- New /api/dtc route added to app.py (mil + codes)
- yaris.js: added obd_coolant_temp / obd_fuel_level / obd_throttle listeners
- yaris.css: full rewrite — layout variables, sidebar, bottom nav, OBD cards,
border-start utilities, mobile padding for bottom nav
Co-authored-by: Copilot <[email protected]>
Prefer a configured phone hotspot for outbound synchronization, preserve secure write-only credentials, and recover through home WiFi or the local Pi hotspot.
Co-authored-by: Copilot <[email protected]>
Recorded Phase 12's deployment step (Pi restart + unraid server
docker compose build/up since server/app.py's CORS removal is baked
into the image at build time - a plain restart wouldn't pick it up)
and its verification (both confirmed on real infrastructure: Pi's
/dash returns HTTP 200 with no relay text, server's /health has no
Access-Control-* headers anymore).
Added a handoff note for whoever picks this up next: Phase 17 (OBD
hardening) was the next phase in progress, but no code was written
yet - only read app_events.py/app_gps.py for context and started
(but didn't finish, due to a tool permission interruption) installing
the python-obd package locally to check its actual API surface
before writing the DTC/protocol/diagnostics code. Reset that todo
back to pending to reflect no actual progress was made. Phases 13
and 14 are equally ready to start (no blocking deps) if a different
starting point makes more sense.
Co-authored-by: Copilot <[email protected]>
Removes the phone-relay feature entirely per the Phase 11 finding
that it hit a fundamental phone-OS limitation (a browser page can't
route WiFi and cellular traffic simultaneously, no code fix possible)
- see PLAN.md for the full test writeup.
- app.py: dropped /api/dash/pending, /api/dash/file/<path>,
/api/dash/relay/config, /api/dash/relay/heartbeat,
/api/dash/relay/stop, /api/dash/relay/mark_uploaded routes and
their now-unused imports (os, send_file, abort, safe_join).
- modules/app_upload.py: dropped list_pending_dashcam_files(),
relay_heartbeat()/relay_stop()/is_relay_active(),
RELAY_LEASE_SECONDS/_relay_lease_until, and the relay-lease check
inside _upload_dashcam_files()/get_status(). Kept the chunked
upload protocol, dashcam_hash_cache/_hash_dashcam_file() (still
used by the Pi's own background uploader), and DASHCAM_CHUNK_SIZE -
none of that was relay-specific.
- templates/dash.html: removed the 'Relay via phone' card.
- static/js/dash.js: removed all relay JS (relayOneFile,
fetchChunkWithRetry, uploadChunkWithRetry, heartbeat timers,
pending-file list rendering).
- server/app.py: removed the CORS after_request hook, added solely
for the phone's cross-origin browser calls (the Pi's own Python
requests-based client was never subject to CORS).
- readme.md: dropped the phone-relay feature bullet, updated the
project-status blurb to explain the removal + point at the planned
native Android app as the real fix for that use case.
- PLAN.md: marked Phase 12 done with a verification writeup.
Verified: py_compile on all touched Python files, node --check on
dash.js, grepped all touched files for any leftover relay/RELAY
reference (none), and smoke-tested by importing app.py + hitting it
with Flask's test client - app starts cleanly, every relay route is
gone from url_map, and both / and /dash still render HTTP 200 with
zero 'relay' occurrences in /dash's HTML.
Co-authored-by: Copilot <[email protected]>
Reordered/renumbered the roadmap based on dependency analysis:
- Phase 12 (remove relay) stays first - pure cleanup, no deps.
- New Phase 13: power-loss resilience + hardware watchdog - OS-level,
independent, should land before the Pi lives in the car full-time
(cars cut power abruptly at every key-off, a real SD-corruption risk
nothing in the project currently addresses).
- New Phase 14: UI design-system foundation - pulls forward a small
slice of the old end-of-roadmap redesign (shared layout + reusable
list/table component) so the dashcam/video browsers, trip history,
and settings pages are all built on it once instead of being built
in the old template and then rebuilt during the redesign later.
- Phases 15/16 (dashcam + local video browsers) now explicitly build
on Phase 14's foundation. Folded two more QOL ideas directly into
Phase 16: ffmpeg thumbnails per video, and an optional
upload-confirmed local retention/auto-cleanup policy.
- Phase 17 (OBD hardening) explicitly marked independent/parallel-
frins to other phases - backend-only work, no UI dependency.
- New Phase 18: trip history + map replay (trips table + GPS-gap
boundary detection + list/replay UI reusing the offline Leaflet map).
- New Phase 19: Pi health monitoring card (CPU temp/disk/uptime/RAM).
- New Phase 20: alerts/notifications via a simple outbound webhook
(ntfy.sh/Telegram) for DTC-on, storage full, backlog growing,
overheating, lost GPS fix.
- New Phase 21: consolidated Settings page (upload toggle, cellular
APN, retention policy, alert config) - reduces routine SSH-editing.
- Phase 22: renamed 'Finish web UI redesign' - now scoped as the
remaining cleanup after Phase 14's foundation + all functional
phases land, rather than one big redesign at the very end.
- Phase 23: native Android app, unchanged content, still last/low
priority per explicit user instruction.
Updated the session's todo tracking table to match the new phase
IDs/order and added todo_deps reflecting the same dependency chain.
Co-authored-by: Copilot <[email protected]>
Formatting-only pass, no information changed:
- Added missing checkmarks to Phase 7 and Phase 9 headers (were plain
'DONE' text while every other completed phase uses '✅ DONE').
- Added a status icon to the un-numbered SIM868 cellular fallback
section ('⚠️ built, UNTESTED').
- Added a consistent '🚧 PLANNED' icon to Phases 12-17, which had no
status icon at all.
- Fixed a stale cross-reference in the Phase 6 update ('see Phase 9
update below' -> 'see Phase 11 update below' - the phone-relay
content it points to has always lived under Phase 11).
Co-authored-by: Copilot <[email protected]>
Phase 16: complete web UI redesign - the current template is full of
never-replaced placeholder cruft (stub buttons, empty /dtc page); do
one real design pass once the functional phases land rather than
keep bolting onto the old admin-template style.
Phase 17: native Android companion app - the real fix for the
phone-relay goal that a web page could never deliver (true
simultaneous WiFi-to-Pi + cellular-to-server via
bindProcessToNetwork), plus a trip/location info view. Explicitly
deprioritized to the end of the roadmap per user - not urgent.
Also expanded Phase 15 (OBD hardening) scope to pull richer dashboard
data (coolant temp, fuel level, throttle, DTC/MIL status) alongside
speed/RPM, noting odometer/mileage support is not guaranteed on this
vehicle/year and needs real-car confirmation.
Co-authored-by: Copilot <[email protected]>
Retested the hotspot+cellular phone-relay scenario more rigorously
(netwatch explicitly disabled during tests so it wouldn't yank the Pi
back to home WiFi mid-test - an earlier failed run turned out to be
caused by that, not a phone issue). Confirmed with WiFi fully off
(pure cellular) that requests to the public server still failed while
associated with the Pi's hotspot, and that this is a fundamental
OS-level 'single default network for the whole device' limitation -
not something a web page's JS can work around, only a native app could
(bindProcessToNetwork). Documented the decision to remove the feature.
Added Phase 12 (remove phone-relay), Phase 13 (dashcam USB file
browser: list+delete), Phase 14 (local imported-video browser +
HTML5 player), Phase 15 (OBD code hardening ahead of first real
car test - background reconnect loop, exception handling, protocol
override, better diagnostics).
Co-authored-by: Copilot <[email protected]>
Live-tested the away-from-home hotspot+cellular relay scenario. Found the
netwatch reconnect bug (documented in Phase 6 update) during this test.
Confirmed the relay architecture itself needs no code changes - the
TypeError: Failed to fetch was a phone-OS network-switching behavior
(won't route internet traffic over cellular while connected to a
no-internet WiFi), not a CORS/code issue - ruled out via a plain page
navigation test. Testing this fully requires being genuinely away from
home or manually forcing cellular use on the phone.
Co-authored-by: Copilot <[email protected]>
Real-hardware finding: netwatch repeatedly reported 'home WiFi not found'
4 times over 13min while sitting right next to the router. Root cause:
a plain 'nmcli con up home-wifi' straight out of AP mode relies on NM's
scan cache for wlan0, which is stale/empty coming out of AP mode (the
radio can't scan while acting as an AP). A single con-up attempt didn't
wait for a fresh scan to land before giving up.
Fix: explicitly drop the AP connection first to free the radio, force a
fresh scan with a settle delay, and retry the connect attempt (longer
30s timeout) up to 2 times before conceding and restoring the hotspot.
Bumped the systemd unit's TimeoutStartSec to 180s to give the retry loop
enough headroom (was defaulting to 90s).
Co-authored-by: Copilot <[email protected]>
Root-caused a "GPS not working" report to the SIM868 module chip being
fully powered off (the HAT's 5V rail can be live while the chip itself
stays off until PWRKEY is triggered) - confirmed by getting zero AT
response at any baud rate while the app's GPS thread had the serial
port open normally.
Discovered this specific HAT revision has the "P4" GPIO power-on mod
wire installed (Waveshare FAQ / GSM_PWRKEY.rar script), letting PWRKEY
be triggered via physical pin 7 (BCM GPIO4) instead of a manual button
press. Verified live: pulsing it low for 4s then high brought the
module from zero AT response to responding immediately.
modules/app_gps.py: added _module_responds()/_pulse_pwrkey()/
_ensure_module_powered(), wired into _poll_loop() before the existing
GNSS power-on check. Checks AT first and only pulses PWRKEY if the
module isn't already responding, since PWRKEY is a toggle (pulsing an
already-on module would turn it back off).
Also fixed a related logging gap found while debugging: print()
diagnostics from app_gps.py were invisible in yaris-obd.log for
months because Python fully-buffers stdout when it isn't a TTY.
deploy/yaris-obd.service now sets PYTHONUNBUFFERED=1.
requirements.txt: added RPi.GPIO.
PLAN.md: documented both findings and the Pi deploy gotchas (systemd
unit file changes need a manual copy + daemon-reload; requirements.txt
changes need a manual venv pip install - neither is picked up by a
plain git pull + restart).
Co-authored-by: Copilot <[email protected]>
Retook pics/ui.png to show the current dashboard, including the
Dashcam nav item added in later phases. Also switched the readme's
image link from a hardcoded old-commit GitHub URL to a relative path
so it won't go stale again.
Co-authored-by: Copilot <[email protected]>
Deployed and tested the phone-relay feature end-to-end for real:
found and fixed a deploy gap (CORS fix wasn't rebuilt into the actual
remote server's Docker image, only pulled on the Pi - surfaced as a
generic "TypeError: Failed to fetch" in the browser). After fixing,
measured real throughput from server-side chunk timestamps: ~1.11MB/s
via phone relay (home WiFi) vs ~9-10Mbit/s for the Pi's own direct
upload - statistically the same. Confirms the relay provides no speed
benefit when the phone is on the same home WiFi as the Pi, since the
download-to-phone leg pays the same congested-radio cost either way.
Documented this finding plus a follow-up idea (per-clip browse/select
upload from the phone, not yet designed).
Root-caused the slow-upload complaint to the Pi's own congested 2.4GHz
WiFi radio (confirmed via iperf3 - same degraded throughput even on
pure local LAN traffic, ruling out Cloudflare/ISP). Since the Pi 3 B
has no 5GHz radio, add a way to bypass it for the internet-bound leg:
a phone opens /dash and manually relays pending dashcam files straight
to the remote server over its own network, reusing the exact same
chunked upload protocol the Pi's client already uses.
- server/app.py: add CORS headers (after_request) so the phone's
browser can call the remote server cross-origin from a page served
by the Pi.
- app.py: new endpoints - GET /api/dash/pending (list not-yet-uploaded
files), GET /api/dash/file/<path> (Range-request-capable download,
path-traversal safe), GET /api/dash/relay/config (hands the phone
the device credentials), POST /api/dash/relay/heartbeat|stop (lease-
based pause of the Pi's own background uploader, self-heals if the
phone's tab closes uncleanly), POST /api/dash/relay/mark_uploaded.
- modules/app_upload.py: list_pending_dashcam_files(), relay lease
helpers, _upload_dashcam_files() now pauses while a relay lease is
active.
- modules/app_sql.py: new dashcam_hash_cache table so a file's sha256
is only computed once (keyed by path+size+mtime) instead of being
recomputed on every loop iteration/pending-list call - benefits the
existing background uploader too.
- templates/dash.html / static/js/dash.js: new "Relay via phone" card
- per-file and "relay all" buttons (manual trigger only, per user
preference), downloads+uploads one 40MB slice at a time so the
phone never holds a whole file in memory, retries chunks up to 3x.
- PLAN.md/readme.md/server/README.md updated (Phase 11).
Tested via a full local end-to-end simulation driving the real Pi and
server Flask apps side by side (Postgres mocked only at the db module
boundary - all HTTP/chunking/auth/CORS logic is real): verified Range
downloads (206), byte-for-byte reassembly via SHA256, path-traversal
rejection, and the relay lease correctly pausing/resuming the
background uploader. Not yet tested against a real phone/live network.
- PLAN.md: mark Phase 8 (dashcam retrieval) verified on real hardware, add
Phase 10 documenting the OOM/multipart and Cloudflare body-size fixes plus
the chunked/resumable upload redesign, both verified end-to-end against
real dashcam footage. Update architecture decision #3 (Mongo -> Postgres
supersession), add unraid/Pi deploy notes and memory-budget gotcha.
- readme.md: refresh feature list (remote upload, chunked dashcam uploads,
toggle) and project status (all 10 phases done, only cellular fallback
untested).
- server/README.md: document the new chunked upload endpoints and the
Cloudflare body-size-limit rationale behind them.
Confirmed against the real deployed server: Cloudflare Tunnel's free
tier caps request bodies at ~100MB and returns a plain 413 - a direct
LAN upload of the same 250MB test file succeeds, the identical request
through yaris.ceaf.dev does not. Real dashcam clips average ~320MB, so
no dashcam video could ever complete an upload through the tunnel as
it stood, regardless of the earlier memory-streaming fix.
Server: new /api/v1/dashcam/upload/chunk (accepts one chunk at a time,
keyed by relative_path+sha256+chunk_index/total_chunks, reassembles and
verifies the sha256 once every chunk has arrived) and
/api/v1/dashcam/upload/status (reports which chunk indices a given
upload already has, or "already_uploaded" if it's done) endpoints.
Chunks are staged under STORAGE_DIR/.upload_tmp/<device>/<sha256>/ and
cleaned up after reassembly; a stale partial set left by a differently
-chunked earlier attempt is detected via a small marker file and wiped
before restarting. The original single-shot /upload endpoint is left
in place unchanged.
Client (modules/app_upload.py): _upload_dashcam_files() now calls
_upload_video_chunked(), which checks /status first (so a prior
partial upload resumes instead of restarting from byte 0) and then
POSTs any missing 40MB chunks (DASHCAM_CHUNK_SIZE, comfortably under
Cloudflare's cap) via the same streaming MultipartEncoder approach as
before.
Co-authored-by: Copilot <[email protected]>
First real-hardware test of dashcam retrieval kept crashing the whole
yaris-obd service (repeated SIGKILL/OOM on the Pi 3's ~900MB RAM) a few
minutes into every run, aborting in-flight file copies. Root cause:
_upload_dashcam_files() used requests.post(files={"file": f}), which
builds the entire multipart/form-data body in memory before sending -
fine for small payloads, but this dashcam's real clips average ~320MB
each (up to a couple hundred more with overhead), so every dashcam
upload attempt alone could push the process well past 300-600MB, and
the Pi's OOM killer stepped in immediately.
Reproduced in isolation: uploading a 300MB file the old way spiked the
client process from 28MB to 628MB RSS; switching to requests-toolbelt's
MultipartEncoder (which streams the file in chunks as the request body
is sent, instead of materializing it up front) kept it flat at ~29MB
for the same upload, with an identical byte-for-byte result on the
receiving end. No server-side changes needed - the wire format is
unchanged, still ordinary multipart/form-data.
Added requests-toolbelt to requirements.txt.
Co-authored-by: Copilot <[email protected]>
mount_device() used a bare os.makedirs() to create the mount point,
but the app runs as an unprivileged user while /mnt is root-owned -
surfaced as a real PermissionError during the first actual hardware
test. Create the mount point via sudo mkdir -p instead, consistent
with the existing sudo mount/umount calls in this module.
Co-authored-by: Copilot <[email protected]>
First real-hardware test revealed the connected dashcam's SD card ships
factory-formatted with no partition table - lsblk reports it as a bare
"disk" with an fstype set directly, never as a child "part". The
previous detection logic only matched type=="part", so it silently
missed this real device entirely.
list_usb_storage_partitions() now also matches whole-disk USB devices
that have no children and an fstype set directly on them, in addition
to the existing partition-table case. Verified against a fixture built
from this dashcam's actual `lsblk -J` output.
Co-authored-by: Copilot <[email protected]>
modules/app_cellular.py: uses the SIM868's built-in AT-command HTTP(S)
client (AT+HTTPINIT/HTTPPARA/HTTPACTION/HTTPREAD) instead of a real
ppp0 interface, since the module's GNSS (polled continuously by
app_gps.py) and its GSM/GPRS radio share the same physical UART
(/dev/serial0) - a pppd/chat dial-up would take over that port
entirely and starve GNSS polling. Staying in AT-command mode lets
cellular HTTP calls interleave safely through the same serial lock.
Only carries small JSON tracking batches, never dashcam video -
cellular is far too slow/limited for that, and the modem's HTTP AT
interface has too small an internal buffer anyway. Dashcam uploads
remain WiFi/Ethernet-only, unchanged.
modules/app_upload.py: tracking-batch upload always tries WiFi/
Ethernet first; only on a connection-level failure (not an HTTP error
status, which means the server was reached) does it retry over
cellular. Exposes last_network_path/cellular_fallback_configured in
/api/upload/status; UI shows which path the last batch used.
Feature-flagged off by default via new YARIS_CELLULAR_ENABLED/
YARIS_CELLULAR_APN/_APN_USER/_APN_PASS keys in db/upload_creds.env
(all blank by default) - has zero effect on the existing WiFi upload
path unless explicitly configured. Regression-tested against the real
deployed server that normal WiFi uploads are unaffected while
cellular stays unconfigured.
*** UNTESTED against real cellular hardware - no SIM card was
*** available. See docstrings in app_cellular.py and PLAN.md for
*** exactly what needs validation once a SIM card is available.
Co-authored-by: Copilot <[email protected]>
Reflects the completed deployment: server running on unraid, exposed
at https://yaris.ceaf.dev via Cloudflare Tunnel, Pi-side upload client
live and backlog-drain tuned.
Co-authored-by: Copilot <[email protected]>
The upload loop previously always slept the full 30s UPLOAD_INTERVAL
between batches, even when a full batch was just sent (meaning more
rows were likely still queued). With GPS/OBD rows logged every 5s
locally, an offline/stale device could build up a large backlog that
took a very long time to drain at 100 rows/30s.
- Bump TRACKING_BATCH_SIZE 100 -> 500 (cuts round-trips per backlog).
- Only sleep the full UPLOAD_INTERVAL once actually caught up (a
short batch came back); while a full batch keeps coming back, drain
back-to-back with a short CATCHUP_INTERVAL (1s) instead.
- server/app.py: add a sane MAX_TRACKING_BATCH_POINTS (5000) cap on
the tracking/batch endpoint as a defensive guard against a runaway
client, independent of the Pi's own batch size.
Verified against the real deployed server: a simulated 1250-row
backlog now drains in 3 batches (~0.35s server round-trip time) instead
of 13 batches spread across ~6.5 minutes of idle sleeping.
Co-authored-by: Copilot <[email protected]>
- app_sql.py: add UPLOADED column + migration on tracking table, add
settings and dashcam_uploads tables, add helpers for unsent-row
queries, marking rows uploaded, and dashcam upload dedupe tracking.
- app_upload.py: new background upload client. Periodically batches
unsent tracking rows to POST /api/v1/tracking/batch and uploads new
dashcam files (sha256-verified) to POST /api/v1/dashcam/upload, with
exponential backoff on failure. Exposes get_status() and a
dashcam-upload enable/disable toggle backed by the settings table.
- app.py: start the upload client thread at startup, add
GET /api/upload/status and POST /api/upload/dashcam_toggle routes.
- dash.html/dash.js: add a "Remote server upload" card with upload
status and a toggle switch to pause dashcam video uploads while
trip/GPS log uploads keep running (per user request - useful on a
slow/metered connection).
- db/upload_creds.env: new gitignore-exempted template (mirrors
db/creds.env) for YARIS_SERVER_URL/YARIS_DEVICE_NAME/YARIS_API_KEY.
- requirements.txt: add requests.
Tested against the real deployed unraid server: tracking batch upload
+ dedupe, dashcam upload + dedupe, and toggle-off correctly pausing
dashcam uploads while tracking uploads continue. Also verified the
full Flask app imports/runs cleanly and the new API routes + dash.html
toggle render correctly in a headless browser.
Co-authored-by: Copilot <[email protected]>
Squashed rewrite of the project revival work (replaces prior history on this
branch to remove personal data that had been accidentally committed - real
LAN IP, GPS coordinates, home WiFi SSID, and phone model). See PLAN.md for
full phase-by-phase details. Summary:
- Phase 2: adapt app_obd.py for a USB OBD-II adapter (was Bluetooth-only),
auto-detect /dev/ttyUSB*, fix several pre-existing crash bugs.
- Phase 3/4: make the web UI work fully offline (vendored Socket.IO/Leaflet/
Chart.js/FontAwesome, offline map tile downloader script, live GPS marker
tracking fix).
- Phase 5: systemd deployment/autostart on the Pi.
- Phase 6: WiFi hotspot mode via NetworkManager (auto-switch between home
WiFi and a local AP for in-car web UI access), mDNS hostname + port 80
redirect so the UI is reachable at a plain hostname with no port number.
- Phase 7: fix app_db.py/app_sql.py bugs, wire up local SQLite trip logging.
- Phase 8: brand-agnostic USB dashcam file retrieval (lsblk transport-based
detection, works with any dashcam brand/folder layout).
- Rewritten readme.md with a proper project overview.
Co-authored-by: Copilot <[email protected]>
