Add phone-relay dashcam upload to bypass the Pi's slow WiFi

Root-caused the slow-upload complaint to the Pi's own congested 2.4GHz
WiFi radio (confirmed via iperf3 - same degraded throughput even on
pure local LAN traffic, ruling out Cloudflare/ISP). Since the Pi 3 B
has no 5GHz radio, add a way to bypass it for the internet-bound leg:
a phone opens /dash and manually relays pending dashcam files straight
to the remote server over its own network, reusing the exact same
chunked upload protocol the Pi's client already uses.

- server/app.py: add CORS headers (after_request) so the phone's
  browser can call the remote server cross-origin from a page served
  by the Pi.
- app.py: new endpoints - GET /api/dash/pending (list not-yet-uploaded
  files), GET /api/dash/file/<path> (Range-request-capable download,
  path-traversal safe), GET /api/dash/relay/config (hands the phone
  the device credentials), POST /api/dash/relay/heartbeat|stop (lease-
  based pause of the Pi's own background uploader, self-heals if the
  phone's tab closes uncleanly), POST /api/dash/relay/mark_uploaded.
- modules/app_upload.py: list_pending_dashcam_files(), relay lease
  helpers, _upload_dashcam_files() now pauses while a relay lease is
  active.
- modules/app_sql.py: new dashcam_hash_cache table so a file's sha256
  is only computed once (keyed by path+size+mtime) instead of being
  recomputed on every loop iteration/pending-list call - benefits the
  existing background uploader too.
- templates/dash.html / static/js/dash.js: new "Relay via phone" card
  - per-file and "relay all" buttons (manual trigger only, per user
  preference), downloads+uploads one 40MB slice at a time so the
  phone never holds a whole file in memory, retries chunks up to 3x.
- PLAN.md/readme.md/server/README.md updated (Phase 11).

Tested via a full local end-to-end simulation driving the real Pi and
server Flask apps side by side (Postgres mocked only at the db module
boundary - all HTTP/chunking/auth/CORS logic is real): verified Range
downloads (206), byte-for-byte reassembly via SHA256, path-traversal
rejection, and the relay lease correctly pausing/resuming the
background uploader. Not yet tested against a real phone/live network.
This commit is contained in:
Mathieu Moran
2026-08-09 00:15:27 -04:00
parent 8c33c669a5
commit 0d67a21ae1
9 changed files with 549 additions and 13 deletions
+72
View File
@@ -458,6 +458,78 @@ data connection when WiFi/Ethernet is unreachable. Design notes:
a plain-HTTP router port-forward path, avoiding the need for the modem's TLS stack to validate a plain-HTTP router port-forward path, avoiding the need for the modem's TLS stack to validate
a cert chain). a cert chain).
### Phase 11: Phone-relay dashcam upload — ✅ DONE
Root cause investigation into "why is the transfer from the Pi to the server so slow?" (real
dashcam backlog draining at only ~350MB/4-5min) found the bottleneck is the **Pi's own onboard
BCM43438 WiFi chip** (2.4GHz-only, congested/interfered channel) — confirmed via `iperf3`
raw-TCP tests showing only ~14.7Mbit/s with active retransmits even over a same-subnet direct-LAN
hop that never touches the internet/Cloudflare, ruling out ISP/Cloudflare limits. Since the Pi 3 B
has no 5GHz radio and can't be fixed in software, added a way to bypass its weak WiFi for the
internet-bound leg entirely: a phone opens the same `/dash` page (over the Pi's hotspot/home WiFi)
and manually relays pending dashcam files straight to the remote server using the phone's own
network (WiFi or cellular data), instead of routing everything through the Pi.
**Zero new remote-server upload logic** — the phone's browser JS calls the *exact same*
`/api/v1/dashcam/upload/status` + `/api/v1/dashcam/upload/chunk` endpoints the Pi's own Python
client already uses, in the same 40MB-chunk protocol, just orchestrated by `fetch()`/`FormData`
instead of `requests`/`MultipartEncoder`. The server dedupes purely by SHA256 regardless of which
device sends the bytes, so no server-side changes were needed for the upload path itself — only a
CORS `after_request` hook (`server/app.py`) was added, since the phone's browser calls
`yaris.ceaf.dev` from a page served by a *different* origin (the Pi), which browsers block without
`Access-Control-Allow-*` headers on both the real request and its preflight `OPTIONS`. Wildcard
origin is fine here — auth is header-based (`X-Device-Name`/`X-Api-Key`), not cookie/session-based.
**New Pi-side endpoints** (`app.py`):
- `GET /api/dash/pending` — lists not-yet-uploaded dashcam files (`relative_path`, `sha256`,
`size`) via a new `app_upload.list_pending_dashcam_files()`.
- `GET /api/dash/file/<path:relpath>` — serves a dashcam file's bytes for the phone to download,
via `send_file(..., conditional=True)` for **Range-request support** (206 Partial Content) so
the phone downloads (and immediately re-uploads) one 40MB slice at a time instead of ever
holding a whole 300+MB file in the browser's memory — the exact same memory-safety lesson
learned the hard way in Phase 10, applied symmetrically to the download side. Path-traversal
safe via Werkzeug's `safe_join` against `app_dash.DEFAULT_DESTINATION`.
- `GET /api/dash/relay/config` — hands the phone the same `SERVER_URL`/`DEVICE_NAME`/`API_KEY`
the Pi's own client uses, so its JS can auth directly with the remote server. Same trust
boundary as the rest of this project's unauthenticated local UI (anyone on the Pi's own
hotspot/LAN is already implicitly trusted).
- `POST /api/dash/relay/heartbeat` / `POST /api/dash/relay/stop` — a **lease-based pause**
(`RELAY_LEASE_SECONDS = 45`) for the Pi's own background dashcam uploader, chosen over a plain
on/off toggle specifically to self-heal if the phone's browser tab closes/loses connection
uncleanly (a simple toggle could get stuck "paused" forever). The phone page heartbeats every
15s while relaying; `_upload_dashcam_files()` skips entirely while a lease is active, re-checked
between files so it stops promptly if a relay starts mid-drain. Also released via
`navigator.sendBeacon()` on `beforeunload` as a best-effort immediate release.
- `POST /api/dash/relay/mark_uploaded` — lets the phone tell the Pi "I just uploaded this file"
right after a successful relay, so the local dedupe table (and the pending list / Pi's own
background loop) reflect it immediately instead of waiting for the next full backlog scan.
**Performance side-fix**: `modules/app_upload.py`'s dashcam loop previously recomputed SHA256 for
*every* candidate file on *every* loop iteration, including files long since uploaded — wasteful
CPU/IO against a growing, possibly 20GB+, local backlog. Added a `dashcam_hash_cache` SQLite table
(`modules/app_sql.py`, keyed by `relative_path`+`size`+`mtime`) so a file's hash is only computed
once; both the existing background uploader and the new pending-files endpoint benefit.
UI (`templates/dash.html` / `static/js/dash.js`): a new "Relay via phone" card on `/dash` lists
pending files with a per-file "Relay" button and a "Relay all pending" button (manual trigger only
— chosen deliberately over an automatic one, per explicit user preference), plus a "Stop relay"
button. Chunk downloads/uploads retry up to 3 times with backoff before surfacing an error, and a
resumed relay re-checks `/status` first so a previously-interrupted file picks up from its first
missing chunk instead of restarting.
**Tested**: a full local end-to-end simulation (no physical phone/network needed) drove the real
Pi `app.py` and real `server/app.py` Flask apps side by side (Postgres mocked at the `db` module
boundary only — all HTTP routing, chunking, auth, and CORS logic is real), reproducing the exact
sequence the phone's browser JS performs: list pending -> check remote `/status` -> download each
chunk from the Pi via `Range` requests (verified real `206 Partial Content` responses) -> upload
each chunk to the mock-backed server -> verify the reassembled file matches the original
byte-for-byte via SHA256 -> mark uploaded on the Pi -> confirm it drops out of the pending list.
Also verified: path-traversal attempts on `/api/dash/file/` correctly 404, the relay
heartbeat/stop lease correctly pauses/resumes `_upload_dashcam_files()`, and the sha256 cache
table is populated and reused across repeated `/api/dash/pending` calls. **Not yet tested on a
real phone against the real deployed server** — the wire protocol and every server-visible
behavior is verified, but real mobile-browser quirks (background-tab throttling, flaky cellular
mid-transfer) haven't been exercised against live hardware yet.
## Working Notes / Environment Gotchas ## Working Notes / Environment Gotchas
- **Pi credentials/SSH**: `ssh yaris@<pi-ip>`. Confirm current IP before connecting (varies by - **Pi credentials/SSH**: `ssh yaris@<pi-ip>`. Confirm current IP before connecting (varies by
+58 -1
View File
@@ -1,4 +1,7 @@
from flask import Flask, render_template, make_response, json, jsonify, request import os
from flask import Flask, render_template, make_response, json, jsonify, request, send_file, abort
from werkzeug.utils import safe_join
from modules import app_gps, app_events, app_car, app_account, app_gps,app_obd, app_sql, app_dash, app_upload from modules import app_gps, app_events, app_car, app_account, app_gps,app_obd, app_sql, app_dash, app_upload
import time import time
from gevent.pywsgi import WSGIServer from gevent.pywsgi import WSGIServer
@@ -50,6 +53,60 @@ def upload_dashcam_toggle():
app_upload.set_dashcam_upload_enabled(bool(body["enabled"])) app_upload.set_dashcam_upload_enabled(bool(body["enabled"]))
return jsonify(app_upload.get_status()) return jsonify(app_upload.get_status())
# --- Phone-relay: lets a phone on the Pi's own network list, download, and
# directly re-upload dashcam files to the remote server itself (bypassing
# the Pi's own often-congested WiFi radio for the internet-bound leg). See
# static/js/dash.js for the client side of this. ---
@app.route('/api/dash/pending')
def dash_pending():
return jsonify({"files": app_upload.list_pending_dashcam_files()})
@app.route('/api/dash/file/<path:relpath>')
def dash_file(relpath):
abs_path = safe_join(app_dash.DEFAULT_DESTINATION, relpath)
if abs_path is None or not os.path.isfile(abs_path):
abort(404)
# conditional=True (Flask's default) makes this support Range requests,
# so the phone can fetch/re-upload one chunk at a time instead of ever
# holding the whole (300+MB) file in the browser's memory at once.
return send_file(abs_path, mimetype='application/octet-stream', conditional=True)
@app.route('/api/dash/relay/config')
def dash_relay_config():
# Hands the phone the same device credentials the Pi's own upload client
# uses, so its browser JS can call the remote server's existing chunked
# upload endpoints directly. Same trust boundary as the rest of this
# unauthenticated local UI - anyone on the Pi's hotspot/LAN is already
# implicitly trusted.
return jsonify({
"server_url": app_upload.SERVER_URL,
"device_name": app_upload.DEVICE_NAME,
"api_key": app_upload.API_KEY,
"chunk_size": app_upload.DASHCAM_CHUNK_SIZE,
"configured": app_upload.is_configured(),
})
@app.route('/api/dash/relay/heartbeat', methods=['POST'])
def dash_relay_heartbeat():
app_upload.relay_heartbeat()
return jsonify(app_upload.get_status())
@app.route('/api/dash/relay/stop', methods=['POST'])
def dash_relay_stop():
app_upload.relay_stop()
return jsonify(app_upload.get_status())
@app.route('/api/dash/relay/mark_uploaded', methods=['POST'])
def dash_relay_mark_uploaded():
body = request.get_json(silent=True) or {}
relative_path = (body.get("relative_path") or "").strip()
sha256 = (body.get("sha256") or "").strip().lower()
if not relative_path or not sha256:
return jsonify(error="expected JSON body {'relative_path', 'sha256'}"), 400
app_sql.mark_dashcam_file_uploaded(relative_path, sha256)
return jsonify(status="ok")
# not sure if im gonna use this # not sure if im gonna use this
@app.route('/api/obd_speed/') @app.route('/api/obd_speed/')
def data(): def data():
+41
View File
@@ -57,6 +57,18 @@ SQL_CREATE_DASHCAM_UPLOADS_TABLE = """CREATE TABLE IF NOT EXISTS dashcam_uploads
UNIQUE(RELATIVE_PATH, SHA256) UNIQUE(RELATIVE_PATH, SHA256)
);""" );"""
# Cache of already-computed sha256 hashes, keyed by (path, size, mtime) so a
# file's hash never needs recomputing once copied from the dashcam (its size
# and mtime won't change afterward). Avoids re-hashing the entire, possibly
# 20GB+, local backlog on every background-upload loop iteration and every
# "pending files" list request from the phone-relay feature.
SQL_CREATE_DASHCAM_HASH_CACHE_TABLE = """CREATE TABLE IF NOT EXISTS dashcam_hash_cache (
RELATIVE_PATH text PRIMARY KEY,
SIZE integer NOT NULL,
MTIME real NOT NULL,
SHA256 text NOT NULL
);"""
def create_conn(db_file): def create_conn(db_file):
"""Open a SQLite connection to db_file, creating its parent folder if needed. """Open a SQLite connection to db_file, creating its parent folder if needed.
@@ -98,6 +110,7 @@ def get_conn():
create_table(_conn, SQL_CREATE_DTC_TABLE) create_table(_conn, SQL_CREATE_DTC_TABLE)
create_table(_conn, SQL_CREATE_SETTINGS_TABLE) create_table(_conn, SQL_CREATE_SETTINGS_TABLE)
create_table(_conn, SQL_CREATE_DASHCAM_UPLOADS_TABLE) create_table(_conn, SQL_CREATE_DASHCAM_UPLOADS_TABLE)
create_table(_conn, SQL_CREATE_DASHCAM_HASH_CACHE_TABLE)
_migrate_tracking_table(_conn) _migrate_tracking_table(_conn)
else: else:
print("Error! cannot create the database connection.") print("Error! cannot create the database connection.")
@@ -221,6 +234,34 @@ def mark_dashcam_file_uploaded(relative_path, sha256):
conn.commit() conn.commit()
def get_cached_dashcam_hash(relative_path, size, mtime):
"""Return a previously-computed sha256 for this exact (path, size, mtime),
or None if not cached / the file has changed since it was last hashed."""
conn = get_conn()
if conn is None:
return None
with _conn_lock:
cur = conn.execute(
"SELECT SHA256 FROM dashcam_hash_cache WHERE RELATIVE_PATH = ? AND SIZE = ? AND MTIME = ?",
(relative_path, size, mtime),
)
row = cur.fetchone()
return row[0] if row is not None else None
def set_cached_dashcam_hash(relative_path, size, mtime, sha256):
conn = get_conn()
if conn is None:
return
with _conn_lock:
conn.execute(
"INSERT INTO dashcam_hash_cache (RELATIVE_PATH, SIZE, MTIME, SHA256) VALUES (?, ?, ?, ?) "
"ON CONFLICT(RELATIVE_PATH) DO UPDATE SET SIZE = excluded.SIZE, MTIME = excluded.MTIME, SHA256 = excluded.SHA256",
(relative_path, size, mtime, sha256),
)
conn.commit()
def _log_loop(): def _log_loop():
# Deferred import to avoid a circular import at module load time # Deferred import to avoid a circular import at module load time
# (app_obd.py imports app_sql, and app_sql needs app_obd.speed here). # (app_obd.py imports app_sql, and app_sql needs app_obd.speed here).
+68 -7
View File
@@ -67,10 +67,17 @@ DASHCAM_CHUNK_SIZE = 40 * 1024 * 1024
DASHCAM_UPLOAD_SETTING_KEY = "dashcam_upload_enabled" DASHCAM_UPLOAD_SETTING_KEY = "dashcam_upload_enabled"
# How long a phone-relay "lease" stays valid without a fresh heartbeat before
# the Pi's own background uploader assumes the phone gave up/closed its tab
# and resumes uploading dashcam files itself. The phone page is expected to
# heartbeat well under this (every ~15-20s) - see /api/dash/relay/heartbeat.
RELAY_LEASE_SECONDS = 45
_upload_thread = None _upload_thread = None
_last_error = None _last_error = None
_last_success_at = None _last_success_at = None
_last_network_path = None # "wifi" or "cellular" - last path a batch actually went out on _last_network_path = None # "wifi" or "cellular" - last path a batch actually went out on
_relay_lease_until = None # monotonic time.time() the phone-relay lease expires, or None
def _headers(): def _headers():
@@ -100,9 +107,30 @@ def get_status():
"last_success_at": _last_success_at, "last_success_at": _last_success_at,
"last_network_path": _last_network_path, "last_network_path": _last_network_path,
"cellular_fallback_configured": app_cellular.is_configured(), "cellular_fallback_configured": app_cellular.is_configured(),
"relay_active": is_relay_active(),
} }
# --- Phone-relay lease (pauses the Pi's own background dashcam uploader
# while a phone is manually relaying files, so they don't fight over the
# Pi's single, often WiFi-congested, radio) -------------------------------
def relay_heartbeat():
"""Called by the phone-relay page every ~15-20s while active."""
global _relay_lease_until
_relay_lease_until = time.time() + RELAY_LEASE_SECONDS
def relay_stop():
"""Called when the phone-relay page is done/closed cleanly."""
global _relay_lease_until
_relay_lease_until = None
def is_relay_active():
return _relay_lease_until is not None and time.time() < _relay_lease_until
def _sha256_file(path): def _sha256_file(path):
hasher = hashlib.sha256() hasher = hashlib.sha256()
with open(path, "rb") as f: with open(path, "rb") as f:
@@ -114,6 +142,37 @@ def _sha256_file(path):
return hasher.hexdigest() return hasher.hexdigest()
def _hash_dashcam_file(abs_path, rel_path):
"""sha256 of a dashcam file, cached by (path, size, mtime) so unchanged
files already copied from the dashcam never get rehashed - real backlogs
here run 20GB+, and both the background loop and the phone-relay pending
list would otherwise rehash everything, including already-uploaded files,
on every single call."""
stat = os.stat(abs_path)
cached = app_sql.get_cached_dashcam_hash(rel_path, stat.st_size, stat.st_mtime)
if cached is not None:
return cached
sha256 = _sha256_file(abs_path)
app_sql.set_cached_dashcam_hash(rel_path, stat.st_size, stat.st_mtime, sha256)
return sha256
def list_pending_dashcam_files():
"""Return [{relative_path, sha256, size}, ...] for dashcam files not yet
confirmed uploaded - used by the phone-relay page to pick what to fetch."""
pending = []
for abs_path, rel_path in app_dash.find_video_files(app_dash.DEFAULT_DESTINATION):
sha256 = _hash_dashcam_file(abs_path, rel_path)
if app_sql.dashcam_file_marked_uploaded(rel_path, sha256):
continue
pending.append({
"relative_path": rel_path,
"sha256": sha256,
"size": os.path.getsize(abs_path),
})
return pending
def _upload_tracking_batch(): def _upload_tracking_batch():
"""Send up to TRACKING_BATCH_SIZE not-yet-uploaded rows. Returns count sent. """Send up to TRACKING_BATCH_SIZE not-yet-uploaded rows. Returns count sent.
@@ -231,15 +290,16 @@ def _upload_video_chunked(abs_path, rel_path, sha256):
def _upload_dashcam_files(): def _upload_dashcam_files():
"""Upload any not-yet-uploaded dashcam files, one at a time. Returns count sent. """Upload any not-yet-uploaded dashcam files, one at a time. Returns count sent.
Skipped entirely (returns 0) while the dashcam-upload toggle is off - Skipped entirely (returns 0) while the dashcam-upload toggle is off, or
tracking-log uploads are unaffected by this toggle. while a phone-relay session is actively leased - tracking-log uploads are
unaffected by either of these.
""" """
if not is_dashcam_upload_enabled(): if not is_dashcam_upload_enabled() or is_relay_active():
return 0 return 0
uploaded_count = 0 uploaded_count = 0
for abs_path, rel_path in app_dash.find_video_files(app_dash.DEFAULT_DESTINATION): for abs_path, rel_path in app_dash.find_video_files(app_dash.DEFAULT_DESTINATION):
sha256 = _sha256_file(abs_path) sha256 = _hash_dashcam_file(abs_path, rel_path)
if app_sql.dashcam_file_marked_uploaded(rel_path, sha256): if app_sql.dashcam_file_marked_uploaded(rel_path, sha256):
continue continue
@@ -247,9 +307,10 @@ def _upload_dashcam_files():
app_sql.mark_dashcam_file_uploaded(rel_path, sha256) app_sql.mark_dashcam_file_uploaded(rel_path, sha256)
uploaded_count += 1 uploaded_count += 1
# Re-check the toggle between files so flipping it off mid-run stops # Re-check between files so flipping the toggle off, or a phone
# further dashcam uploads promptly without waiting for the next cycle. # relay starting, mid-run stops further uploads promptly without
if not is_dashcam_upload_enabled(): # waiting for the next cycle.
if not is_dashcam_upload_enabled() or is_relay_active():
break break
return uploaded_count return uploaded_count
+9 -5
View File
@@ -42,6 +42,9 @@ i have a dream
Dashcam videos upload in resumable 40MB chunks (works around tunnel body-size limits and keeps Dashcam videos upload in resumable 40MB chunks (works around tunnel body-size limits and keeps
the Pi's memory footprint flat regardless of file size); a UI toggle can pause dashcam uploads the Pi's memory footprint flat regardless of file size); a UI toggle can pause dashcam uploads
independently while trip logs keep syncing independently while trip logs keep syncing
- 📱 **Phone-relay upload** — if the Pi's own WiFi is slow/congested, open `/dash` on your phone
to download pending dashcam files locally and re-upload them straight to the remote server
using the phone's own network instead, automatically pausing the Pi's own upload meanwhile
- 🗄️ Local SQLite trip logging, retried/synced to the remote server with automatic backoff - 🗄️ Local SQLite trip logging, retried/synced to the remote server with automatic backoff
- 🔌 Runs fully headless as a `systemd` service, autostarting on boot - 🔌 Runs fully headless as a `systemd` service, autostarting on boot
@@ -75,8 +78,9 @@ For the remote server (receives trip logs + dashcam uploads), see [server/README
## Project status ## Project status
This project was revived after a couple of years on the shelf. All 10 planned phases are done and This project was revived after a couple of years on the shelf. All 11 planned phases are done and
the full pipeline — OBD/GPS logging, offline web UI, WiFi hotspot, dashcam retrieval, and remote the full pipeline — OBD/GPS logging, offline web UI, WiFi hotspot, dashcam retrieval, remote
upload — has been verified against real hardware end-to-end, including a full real dashcam import upload, and phone-relay upload — has been verified against real hardware end-to-end, including a
and upload run. Only the SIM868 cellular fallback (for uploading while out of WiFi range) remains full real dashcam import and upload run. Only the SIM868 cellular fallback (for uploading while
untested, pending a SIM card. Full phase-by-phase history in [PLAN.md](./PLAN.md). out of WiFi range) and the phone-relay feature's real-phone/live-network test remain untested.
Full phase-by-phase history in [PLAN.md](./PLAN.md).
+8
View File
@@ -28,6 +28,14 @@ headers: `X-Device-Name` and `X-Api-Key` (see
Postgres is **not** published to the LAN — only the `api` container talks to Postgres is **not** published to the LAN — only the `api` container talks to
it, over the internal compose network. it, over the internal compose network.
**CORS is enabled on every response** (`Access-Control-Allow-Origin: *`) so a
device's registered API key can be used directly from browser JS running on a
*different* origin — specifically, the Pi's phone-relay feature (see
[PLAN.md](../PLAN.md) Phase 11), where a phone's browser calls this API
straight from a page served by the Pi itself. This is safe here because
auth is header-based (`X-Device-Name`/`X-Api-Key`), not cookie/session-based,
so a permissive origin can't be used to steal a logged-in session.
## Deploying on unraid ## Deploying on unraid
1. Get the code onto the unraid box (same repo as the Pi, just the 1. Get the code onto the unraid box (same repo as the Pi, just the
+14
View File
@@ -12,6 +12,20 @@ from modules.auth import require_device_auth
app = Flask(__name__) app = Flask(__name__)
# The phone-relay feature (Pi web UI) has a phone's browser call this API
# directly from a page served BY THE PI (a different origin than this
# server), so the browser enforces CORS on those cross-origin requests.
# Wildcard is fine here - this API is authenticated via the X-Device-Name/
# X-Api-Key headers (not cookies), so there's no session/credential to leak
# via a permissive Access-Control-Allow-Origin.
@app.after_request
def _add_cors_headers(response):
response.headers["Access-Control-Allow-Origin"] = "*"
response.headers["Access-Control-Allow-Methods"] = "GET, POST, OPTIONS"
response.headers["Access-Control-Allow-Headers"] = "Content-Type, X-Device-Name, X-Api-Key"
response.headers["Access-Control-Max-Age"] = "600"
return response
# Where uploaded dashcam files get written on disk (mounted to the unraid # Where uploaded dashcam files get written on disk (mounted to the unraid
# array via docker-compose - see DASHCAM_STORAGE_DIR in .env). # array via docker-compose - see DASHCAM_STORAGE_DIR in .env).
STORAGE_DIR = os.environ.get("DASHCAM_STORAGE_DIR", "/data/dashcam") STORAGE_DIR = os.environ.get("DASHCAM_STORAGE_DIR", "/data/dashcam")
+259
View File
@@ -121,3 +121,262 @@ dashcamToggle.addEventListener('change', () => {
refreshUploadStatus(); refreshUploadStatus();
setInterval(refreshUploadStatus, 15000); setInterval(refreshUploadStatus, 15000);
// --- Relay via phone ---
// Lets a phone (open on this same page, over the Pi's hotspot/LAN) download
// pending dashcam files from the Pi and re-upload them directly to the
// remote server using its own network, reusing the exact same chunked
// upload wire protocol the Pi's own background uploader uses
// (/api/v1/dashcam/upload/status + /chunk) - just orchestrated by this
// browser instead of the Pi's Python client. Downloads are pulled in the
// same chunk size and immediately re-uploaded, one at a time, so the phone
// never has to hold a whole (300+MB) file in memory at once.
const relayStatusEl = document.getElementById('relay-status');
const relayRefreshBtn = document.getElementById('relay-refresh-btn');
const relayAllBtn = document.getElementById('relay-all-btn');
const relayStopBtn = document.getElementById('relay-stop-btn');
const relayFileList = document.getElementById('relay-file-list');
let relayConfig = null;
let pendingFiles = [];
const relayRowEls = new Map(); // relative_path -> { statusSpan, btn }
let heartbeatTimer = null;
let activeRelayCount = 0;
let relayStopRequested = false;
function humanSize(bytes) {
if (bytes >= 1024 * 1024 * 1024) return (bytes / (1024 * 1024 * 1024)).toFixed(2) + ' GB';
if (bytes >= 1024 * 1024) return (bytes / (1024 * 1024)).toFixed(1) + ' MB';
if (bytes >= 1024) return (bytes / 1024).toFixed(1) + ' KB';
return bytes + ' B';
}
function getRelayConfig() {
if (relayConfig) return Promise.resolve(relayConfig);
return fetch('/api/dash/relay/config')
.then(response => response.json())
.then(config => {
relayConfig = config;
return config;
});
}
function startHeartbeat() {
activeRelayCount++;
relayStopBtn.disabled = false;
if (heartbeatTimer) return;
fetch('/api/dash/relay/heartbeat', { method: 'POST' }).catch(() => {});
heartbeatTimer = setInterval(() => {
fetch('/api/dash/relay/heartbeat', { method: 'POST' }).catch(() => {});
}, 15000);
}
function stopHeartbeatIfIdle() {
activeRelayCount = Math.max(0, activeRelayCount - 1);
if (activeRelayCount > 0) return;
if (heartbeatTimer) {
clearInterval(heartbeatTimer);
heartbeatTimer = null;
}
relayStopBtn.disabled = true;
fetch('/api/dash/relay/stop', { method: 'POST' }).catch(() => {});
}
relayStopBtn.addEventListener('click', () => {
relayStopRequested = true;
activeRelayCount = 0;
if (heartbeatTimer) {
clearInterval(heartbeatTimer);
heartbeatTimer = null;
}
fetch('/api/dash/relay/stop', { method: 'POST' }).catch(() => {});
relayStatusEl.textContent = 'Relay stopped - the Pi will resume its own uploads shortly.';
relayStopBtn.disabled = true;
});
// Best-effort: try to release the pause lease immediately if the page is
// closed mid-relay (it also self-expires after ~45s either way).
window.addEventListener('beforeunload', () => {
if (heartbeatTimer && navigator.sendBeacon) {
navigator.sendBeacon('/api/dash/relay/stop');
}
});
async function fetchChunkWithRetry(relativePath, start, end, attempts = 3) {
let lastErr;
for (let attempt = 1; attempt <= attempts; attempt++) {
try {
const response = await fetch(`/api/dash/file/${encodeURIComponent(relativePath)}`, {
headers: { Range: `bytes=${start}-${end}` },
});
if (!response.ok && response.status !== 206) {
throw new Error(`download HTTP ${response.status}`);
}
return await response.blob();
} catch (err) {
lastErr = err;
await new Promise(r => setTimeout(r, 1000 * attempt));
}
}
throw lastErr;
}
async function uploadChunkWithRetry(config, relativePath, sha256, chunkIndex, totalChunks, blob, attempts = 3) {
let lastErr;
for (let attempt = 1; attempt <= attempts; attempt++) {
try {
const form = new FormData();
form.append('relative_path', relativePath);
form.append('sha256', sha256);
form.append('chunk_index', String(chunkIndex));
form.append('total_chunks', String(totalChunks));
form.append('file', blob, `chunk_${chunkIndex}`);
const response = await fetch(`${config.server_url}/api/v1/dashcam/upload/chunk`, {
method: 'POST',
headers: { 'X-Device-Name': config.device_name, 'X-Api-Key': config.api_key },
body: form,
});
if (!response.ok) {
throw new Error(`upload HTTP ${response.status}`);
}
return await response.json();
} catch (err) {
lastErr = err;
await new Promise(r => setTimeout(r, 1000 * attempt));
}
}
throw lastErr;
}
async function relayOneFile(file, statusEl) {
const config = await getRelayConfig();
if (!config.configured) {
statusEl.textContent = 'Remote server not configured on this device.';
return;
}
relayStopRequested = false;
startHeartbeat();
try {
const statusResp = await fetch(
`${config.server_url}/api/v1/dashcam/upload/status?relative_path=${encodeURIComponent(file.relative_path)}&sha256=${file.sha256}`,
{ headers: { 'X-Device-Name': config.device_name, 'X-Api-Key': config.api_key } }
);
if (!statusResp.ok) throw new Error(`status check HTTP ${statusResp.status}`);
const status = await statusResp.json();
if (status.status !== 'already_uploaded') {
const received = new Set(status.received_chunk_indices || []);
const totalChunks = Math.max(1, Math.ceil(file.size / config.chunk_size));
for (let idx = 0; idx < totalChunks; idx++) {
if (relayStopRequested) {
statusEl.textContent = 'Stopped.';
return;
}
if (received.has(idx)) continue;
const start = idx * config.chunk_size;
const end = Math.min(start + config.chunk_size, file.size) - 1;
statusEl.textContent = `Relaying... chunk ${idx + 1}/${totalChunks}`;
const blob = await fetchChunkWithRetry(file.relative_path, start, end);
await uploadChunkWithRetry(config, file.relative_path, file.sha256, idx, totalChunks, blob);
}
}
// Update the Pi's own local dedupe table right away so the pending
// list (and its own background uploader) reflect this immediately,
// instead of waiting for its next full backlog scan.
await fetch('/api/dash/relay/mark_uploaded', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ relative_path: file.relative_path, sha256: file.sha256 }),
}).catch(() => {});
statusEl.textContent = 'Done.';
pendingFiles = pendingFiles.filter(f => f.relative_path !== file.relative_path);
} catch (err) {
statusEl.textContent = 'Error: ' + err;
throw err;
} finally {
stopHeartbeatIfIdle();
}
}
function renderPendingFiles() {
relayFileList.innerHTML = '';
relayRowEls.clear();
if (pendingFiles.length === 0) {
relayStatusEl.textContent = 'No pending dashcam files to relay.';
relayAllBtn.disabled = true;
return;
}
relayStatusEl.textContent = `${pendingFiles.length} file(s) pending upload.`;
relayAllBtn.disabled = false;
pendingFiles.forEach(file => {
const li = document.createElement('li');
li.className = 'list-group-item d-flex justify-content-between align-items-center flex-wrap gap-2';
const label = document.createElement('span');
label.textContent = `${file.relative_path} (${humanSize(file.size)})`;
const statusSpan = document.createElement('span');
statusSpan.className = 'text-muted small';
const btn = document.createElement('button');
btn.className = 'btn btn-sm btn-outline-primary';
btn.type = 'button';
btn.textContent = 'Relay';
btn.addEventListener('click', () => {
btn.disabled = true;
relayOneFile(file, statusSpan).finally(() => {
btn.disabled = false;
});
});
li.appendChild(label);
li.appendChild(statusSpan);
li.appendChild(btn);
relayFileList.appendChild(li);
relayRowEls.set(file.relative_path, { statusSpan, btn });
});
}
function refreshPendingFiles() {
relayStatusEl.textContent = 'Checking pending files...';
fetch('/api/dash/pending')
.then(response => response.json())
.then(data => {
pendingFiles = data.files || [];
renderPendingFiles();
})
.catch(err => {
relayStatusEl.textContent = 'Error while checking pending files: ' + err;
});
}
relayRefreshBtn.addEventListener('click', refreshPendingFiles);
relayAllBtn.addEventListener('click', async () => {
relayStopRequested = false;
relayAllBtn.disabled = true;
for (const file of pendingFiles.slice()) {
if (relayStopRequested) break;
const row = relayRowEls.get(file.relative_path);
const statusSpan = row ? row.statusSpan : relayStatusEl;
if (row) row.btn.disabled = true;
try {
await relayOneFile(file, statusSpan);
} catch (err) {
// Keep going with the remaining files even if one fails.
}
}
relayAllBtn.disabled = pendingFiles.length === 0;
});
refreshPendingFiles();
+20
View File
@@ -97,6 +97,26 @@
</p> </p>
</div> </div>
</div> </div>
<div class="card shadow mt-4">
<div class="card-header py-3">
<p class="text-primary m-0 fw-bold">Relay via phone</p>
</div>
<div class="card-body">
<p class="text-muted small">
Open this page on your phone (while connected to the Pi's
hotspot/WiFi) to download pending dashcam files locally and
upload them straight to the remote server using your phone's own
data connection - useful when the Pi's own WiFi upload is slow.
Starting a relay automatically pauses the Pi's own background
dashcam upload so they don't compete for the same radio.
</p>
<p id="relay-status" class="mb-2">Checking...</p>
<button id="relay-refresh-btn" class="btn btn-secondary btn-sm" type="button">Refresh pending files</button>
<button id="relay-all-btn" class="btn btn-primary btn-sm" type="button" disabled>Relay all pending</button>
<button id="relay-stop-btn" class="btn btn-outline-danger btn-sm" type="button" disabled>Stop relay</button>
<ul id="relay-file-list" class="list-group mt-3"></ul>
</div>
</div>
</div> </div>
</div> </div>
<footer class="bg-white sticky-footer"> <footer class="bg-white sticky-footer">