Squashed rewrite of the project revival work (replaces prior history on this branch to remove personal data that had been accidentally committed - real LAN IP, GPS coordinates, home WiFi SSID, and phone model). See PLAN.md for full phase-by-phase details. Summary: - Phase 2: adapt app_obd.py for a USB OBD-II adapter (was Bluetooth-only), auto-detect /dev/ttyUSB*, fix several pre-existing crash bugs. - Phase 3/4: make the web UI work fully offline (vendored Socket.IO/Leaflet/ Chart.js/FontAwesome, offline map tile downloader script, live GPS marker tracking fix). - Phase 5: systemd deployment/autostart on the Pi. - Phase 6: WiFi hotspot mode via NetworkManager (auto-switch between home WiFi and a local AP for in-car web UI access), mDNS hostname + port 80 redirect so the UI is reachable at a plain hostname with no port number. - Phase 7: fix app_db.py/app_sql.py bugs, wire up local SQLite trip logging. - Phase 8: brand-agnostic USB dashcam file retrieval (lsblk transport-based detection, works with any dashcam brand/folder layout). - Rewritten readme.md with a proper project overview. Co-authored-by: Copilot <[email protected]>
331 lines
24 KiB
Markdown
331 lines
24 KiB
Markdown
# yaris-obd Revival Plan
|
|
|
|
This document tracks the plan to revive and finish this project: a Raspberry Pi 3-based
|
|
OBD-II + GPS car tracker for a 2013 Toyota Yaris, hosting a local web UI over a self-broadcast
|
|
WiFi hotspot, with dashcam file retrieval and remote upload of itinerary/dashcam data.
|
|
|
|
Keep this file up to date as phases complete or plans change, so any future session/agent can
|
|
pick up the project without needing prior chat history.
|
|
|
|
## Hardware
|
|
|
|
- **Board**: Raspberry Pi 3 Model B Rev 1.2, 906MB RAM. Confirmed too weak to run an AI coding
|
|
agent — all agent-assisted dev work happens on a separate local machine; the Pi is only used
|
|
for hardware bring-up/testing/deployment via SSH (`yaris@<pi-ip>`).
|
|
- **GPS/GSM**: Waveshare **SIM868 GSM/GPRS/GNSS/Bluetooth HAT**. Also has unused cellular
|
|
(GSM/GPRS) data capability, earmarked as a fallback internet uplink (see Remote Server phase).
|
|
- **OBD-II adapter**: a cheap unbranded USB adapter using a **PL2303** USB-to-serial chip,
|
|
presumed ELM327-compatible (works with the `python-obd` library). Originally the project used
|
|
a Bluetooth OBD adapter; this has been switched to USB.
|
|
- **Car**: 2013 Toyota Yaris.
|
|
|
|
### SIM868 HAT bring-up notes (important gotchas)
|
|
- UART routing is a 3-position jumper: **A** = USB↔SIM868 (via onboard CP2102), **B** = Pi
|
|
GPIO↔SIM868, **C** = USB↔Pi console. Must be on **B** for the Pi to talk to the module over
|
|
its own GPIO UART.
|
|
- Pi-side UART setup required: `enable_uart=1` and `dtoverlay=disable-bt` in
|
|
`/boot/firmware/config.txt` (frees the full PL011 UART — `/dev/ttyAMA0`/`/dev/serial0` — from
|
|
the onboard Bluetooth controller), `raspi-config nonint do_serial_cons 1` (stops a login shell
|
|
from claiming the serial port), and disable the `hciuart` systemd service.
|
|
- **The SIM868 module chip itself needs its PWRKEY button pressed and held (~1-2s) to power on**
|
|
— this is separate from the board's 5V/PWR LED, which can be lit while the module chip is
|
|
fully off. AT commands get zero response if the module chip isn't powered, even with perfect
|
|
UART wiring/config. This is a manual step needed on every power cycle unless a P4
|
|
GPIO-power-on hardware mod (per Waveshare FAQ) is done — not pursued so far.
|
|
- Confirmed AT command baud rate: **115200**.
|
|
- GNSS workflow: `AT+CGNSPWR?` to check GNSS power state, `AT+CGNSPWR=1` to power on,
|
|
`AT+CGNSINF` to poll for a fix. Response format:
|
|
`+CGNSINF: run,fix,utc_time,lat,lon,alt,speed,course,fixmode,reserved1,hdop,pdop,vdop,reserved2,gps_sat_view,gnss_sat_used,gnss_sat_view,cn0,hpa,vpa`
|
|
— field 1 (`fix`) must be `'1'` for a valid fix.
|
|
- **The GPS/GNSS antenna is separate from the GSM antenna** — both need to be physically
|
|
connected to their respective u.FL connectors on the HAT. A real fix was obtained only after
|
|
discovering the GPS antenna wasn't plugged in (only GSM was).
|
|
- Confirmed real fix obtained near a window: ~80s cold-start TTFF, 5-7 satellites, HDOP ~1.2-1.6.
|
|
|
|
## Software Architecture Decisions
|
|
|
|
1. **Offline maps**: Pre-cache/download offline map tiles for the car's usual driving area so
|
|
the live map works fully in the car with no internet (not just numeric GPS readout).
|
|
Tooling: MapTiler Cloud free tier (raster XYZ tiles, `streets-v2` style) — one-time download
|
|
on the dev machine via `scripts/download_map_tiles.py`, output shipped to the Pi as static
|
|
PNGs. Bulk-downloading directly from `tile.openstreetmap.org` was ruled out — it violates the
|
|
OSM tile usage policy. Bounding box is a required CLI arg (the user's own driving area, not
|
|
committed here), zoom 8-15 by default (~14k tiles for a city-plus-suburbs-sized area, well
|
|
under MapTiler's 100k/month free quota).
|
|
2. **WiFi hotspot mode**: Auto-switch — the Pi joins home WiFi when in range (for
|
|
syncing/updates), and falls back to broadcasting its own AP otherwise. **Important scope
|
|
clarification**: the hotspot is *only* for local web-UI viewing on a phone/laptop while
|
|
driving — it is not part of the upload path. Remote-server uploads happen independently via
|
|
home WiFi (when in range) or the SIM868 cellular fallback (decision #5) — the phone connecting
|
|
to the hotspot never relays/carries upload traffic.
|
|
3. **Local/remote persistence**: Keep SQLite locally (reliable local write path per trip) +
|
|
MongoDB Atlas for remote sync when online. Fix existing bugs rather than replace the design.
|
|
4. **Remote server**: Self-hosted — a custom server/API (e.g. on a home server or VPS) receives
|
|
itinerary and dashcam uploads, rather than 3rd-party managed cloud services.
|
|
5. **Remote upload uplink**: Use the SIM868's GSM/GPRS cellular connection as a fallback uplink
|
|
for uploads when no WiFi is available (e.g. while driving), not just "sync when on home WiFi."
|
|
|
|
## Phases
|
|
|
|
### Phase 1: Real GPS integration — ✅ DONE
|
|
`modules/app_gps.py` rewritten to read real SIM868 GNSS data via serial AT commands
|
|
(`/dev/serial0` @ 115200 baud) instead of hardcoded fake coordinates. Background daemon thread
|
|
polls `AT+CGNSINF` every 2s, parses fix data, emits `gps_location` via socketio only on an actual
|
|
fix. Verified against real hardware (see bring-up notes above).
|
|
|
|
### Phase 2: OBD fixes for USB adapter — ✅ DONE
|
|
`modules/app_obd.py` fixed (commit "Fix app_obd.py bugs and adapt for USB OBD-II adapter",
|
|
pushed to `origin/shameless-aislop`):
|
|
- `connection` is never left undefined on failed connect (was a `NameError` crash).
|
|
- Added `_find_obd_port()`/`connect_obd()`: auto-detects `/dev/ttyUSB*` (the PL2303 adapter),
|
|
honors an `OBD_PORT` env override, falls back to python-obd's own port auto-scan (still covers
|
|
the old `/dev/rfcomm*` Bluetooth path).
|
|
- Fixed stale-response bug: `obd_speed()`/`obd_rpm()` now re-query every loop iteration instead
|
|
of reusing one response forever.
|
|
- Fixed `obd_rpm()` using the wrong lock, never updating the global `rpm`, and emitting to the
|
|
wrong socketio event (`obd_speed` instead of `obd_rpm`).
|
|
- Fixed `kill_obd()` being a no-op — now uses `threading.Event` flags actually checked in the
|
|
polling loops.
|
|
- Fixed inverted thread-start logic (threads were starting on failed connection, not success).
|
|
- Fixed `obm_error` → `obd_error` typo (frontend listens for `obd_error`).
|
|
- Added `obd` (python-obd) to `requirements.txt`.
|
|
- **Still untested**: a real connection to the actual car (adapter has only been tested
|
|
standalone on the Pi so far — correctly reports "no connection to car" with nothing attached).
|
|
|
|
### Phase 3: Web UI must work fully offline — ✅ DONE
|
|
The whole point of the project is a Pi hotspot with **no internet in the car**, but the current
|
|
UI silently depended on the internet for core functionality:
|
|
- `templates/index.html`/`dash.html`/`dtc.html` loaded Google Fonts, FontAwesome, Leaflet JS/CSS,
|
|
and the Socket.IO client from public CDNs — none of this loads offline, breaking layout, the
|
|
websocket connection itself, and the map entirely.
|
|
- `static/js/yaris.js` loaded live map tiles from `tile.openstreetmap.org` (also needs internet).
|
|
- A local `static/js/chart.min.js` existed but was unused — `index.html` loaded Chart.js from
|
|
`cdn.jsdelivr.net` instead (turned out the local copy was a stale Chart.js **v2.8.0** while
|
|
`yaris.js`'s config uses v3/v4-style `scales: {x:{...}, y:{...}}` syntax — that mismatch is
|
|
exactly why the CDN was used instead. Replaced the local file with a current v4.5.0 UMD build).
|
|
|
|
**What was done**:
|
|
- Vendored Socket.IO client (4.8.1, matches Flask-SocketIO 5.6.1's Engine.IO v4 protocol) under
|
|
`static/vendor/socketio/`, and Leaflet (1.9.4 JS/CSS + marker/layers images) under
|
|
`static/vendor/leaflet/`.
|
|
- FontAwesome 5.12.0 was already vendored locally at `static/fonts/` (self-consistent, just never
|
|
referenced) — repointed all three templates to it instead of the CDN.
|
|
- Dropped the Google Fonts (Nunito) `<link>` entirely rather than self-hosting 12 font-weight
|
|
files — Bootstrap's `--bs-font-sans-serif` variable already falls back cleanly to system fonts
|
|
(`-apple-system, Segoe UI, Roboto, ...`) after Nunito, so this is a clean no-cost simplification.
|
|
- Replaced `static/js/chart.min.js` with a current Chart.js v4.5.0 UMD build (auto-registering,
|
|
same drop-in behavior as the old CDN `<script src="https://cdn.jsdelivr.net/npm/chart.js">`).
|
|
- Offline map tiles: added `scripts/download_map_tiles.py`, a dev-machine-only one-time tile
|
|
downloader using the MapTiler Cloud free tier (raster XYZ, `streets-v2` style, 256px tiles).
|
|
Reads `MAPTILER_KEY` from an env var/CLI arg only — the key itself is never committed or
|
|
deployed, only the resulting static PNG tiles are. Bounding box is a required CLI arg (your own
|
|
driving area's coordinates - not committed/hardcoded) at zoom 8-15 (~14k tiles for a
|
|
city-plus-suburbs-sized area, well under MapTiler's 100k tiles/month free quota). Output goes to
|
|
`static/vendor/tiles/{z}/{x}/{y}.png` (gitignored — regenerate locally, rsync to the Pi as part
|
|
of deployment, don't bloat the git repo with binary tile data).
|
|
- **Tiles populated**: user signed up for a free MapTiler key and the full zoom 8-15 download
|
|
ran to completion — 14,241 downloaded, 24 skipped (out-of-bounds), 0 failed (14,265 tiles
|
|
total, ~285MB). Rsynced to the Pi's `static/vendor/tiles/` (tile counts verified identical
|
|
on both sides). Live Pi instance re-confirmed serving HTTP 200 with the complete offline
|
|
tile set.
|
|
- Updated `static/js/yaris.js`'s Leaflet init to point at `/static/vendor/tiles/{z}/{x}/{y}.png`
|
|
(`minZoom`/`maxZoom` 8/15 to match). The initial map center is just a generic placeholder (not
|
|
a real location) - it re-centers automatically to the real live GPS fix once one comes in, and
|
|
the marker is exposed as `carMarker` so Phase 4 can move it on live fixes.
|
|
- Added a repo-root `.gitignore` (Python caches, venvs, `.vscode/`, `static/vendor/tiles/`,
|
|
`*.env` except the already-tracked `db/creds.env` placeholder) — there wasn't one before, which
|
|
is why stray `__pycache__`/`.pyc` files kept showing up in source control.
|
|
- **Verified**: ran the Flask app in a local venv, confirmed all three pages (`/`, `/dtc`, `/dash`)
|
|
render with **zero external URLs** in the served HTML, every local static asset (JS, CSS, fonts,
|
|
Leaflet images) returns HTTP 200, and all vendored JS files pass a Node.js syntax check.
|
|
|
|
### Phase 4: Map/GPS display bug — ✅ DONE
|
|
`yaris.js`'s Leaflet marker (`carMarker`) was never moved on incoming `gps_location` socket
|
|
events — only two text spans (`gpsLocationX`/`Y`) got updated. Fixed: `carMarker.setLatLng(...)`
|
|
is now called on every `gps_location` event, and the map re-centers (zoom 15) on the first fix
|
|
only (`mapCentered` flag), so the user isn't fighting the map re-centering under them while
|
|
panning/zooming during a trip.
|
|
|
|
**Verified**: also used this as an opportunity to test Phase 3 end-to-end — ran
|
|
`scripts/download_map_tiles.py` for real with the user's actual MapTiler API key (smoke-tested
|
|
zoom 8-10 first, confirmed idempotent re-run skips existing tiles, confirmed a downloaded tile
|
|
is a real map image of the target driving area), then started the full zoom 8-15
|
|
download. Took a **headless Chromium screenshot** of the running app's `/` page and confirmed:
|
|
sidebar/icons/fonts render correctly (FontAwesome working), the Speed Graph (Chart.js v4) renders,
|
|
and **the Leaflet map renders real offline map tiles with the marker in place** — full offline
|
|
end-to-end proof, not just "no CDN URLs in the HTML".
|
|
- Fixed (was noted here as out of scope, now addressed in Phase 8): `/dash` and `/dtc` used to
|
|
render near-identical body content (both showed "DTC Codes") — `dash.html` now has real
|
|
dashcam-specific content.
|
|
- **Full tile download completed** (see Phase 3 update above) and rsynced to the Pi — Phase
|
|
3/4's offline map is now 100% functional end-to-end on the actual deployed device, not just
|
|
the dev machine.
|
|
|
|
### Phase 5: Pi deployment / autostart — ✅ DONE
|
|
Set up a proper venv (`/home/yaris/app/yaris-obd/venv`), installed all requirements (all found
|
|
as prebuilt ARM wheels via piwheels — no slow source compiles needed, even for gevent).
|
|
Added `deploy/yaris-obd.service` (systemd unit: runs as the `yaris` user, `Restart=on-failure`,
|
|
5s `ExecStartPre` sleep to let USB/serial devices enumerate after boot, logs to
|
|
`yaris-obd.log`) and `deploy/install_service.sh` (idempotent one-time install helper — also
|
|
creates the venv if missing). Installed and enabled on the Pi (`systemctl enable` — survives
|
|
reboot).
|
|
|
|
**Also had to fix Pi → GitHub auth for `git pull`**: the repo is private, and the Pi's existing
|
|
`~/.ssh/id_ed25519.pub` turned out to be an *inbound* SSH key (its private half lives on the dev
|
|
machine, not the Pi) — useless for the Pi to authenticate outbound. Generated a dedicated
|
|
ed25519 keypair **on the Pi itself** (`~/.ssh/yaris_deploy_key`), registered it as a **read-only
|
|
deploy key** on the GitHub repo (least-privilege — the Pi only ever needs to pull), and added an
|
|
`~/.ssh/config` entry so git transparently uses it for `github.com`. Switched the Pi's `origin`
|
|
remote to SSH. `git pull` now works standalone on the Pi with no PAT/interactive auth needed.
|
|
|
|
**Verified end-to-end**: pulled latest code onto the Pi (was 5 commits stale), confirmed all
|
|
Python deps import cleanly, ran the app manually first (HTTP 200, all local static assets —
|
|
Leaflet/Socket.IO/Chart.js/FontAwesome — served correctly from the Pi itself), then installed +
|
|
started the systemd service and confirmed `active (running)` + `enabled`. **Tested crash
|
|
recovery**: sent SIGKILL to the running process, confirmed systemd auto-restarted it (through the
|
|
5s `ExecStartPre` delay) and the app was serving HTTP 200 again within ~10s, with zero manual
|
|
intervention.
|
|
|
|
### Phase 6: WiFi hotspot mode — ✅ DONE
|
|
Used NetworkManager (already the network manager in use, not hostapd/dnsmasq directly — those
|
|
were leftover/unused) since it natively supports AP mode via `nmcli`. Created two connection
|
|
profiles on the Pi: `home-wifi` (your real home WiFi network, `autoconnect-priority 10`) and
|
|
`Hotspot` (AP mode, `ipv4.method shared`, its own dedicated SSID/password set locally on the Pi -
|
|
not committed here, WPA2-PSK, `autoconnect-priority -10`). NM's own autoconnect logic handles
|
|
"leaving home WiFi range -> fall back to hotspot" automatically since `home-wifi` has higher
|
|
priority and NM always tries it first.
|
|
|
|
For the reverse direction (home WiFi back in range while sitting in AP mode) added
|
|
`deploy/yaris-netwatch.sh` + `.service`/`.timer`: since the Pi's onboard BCM43438 radio can't be
|
|
an AP and background-scan for client networks simultaneously, a systemd timer runs every 3
|
|
minutes, briefly attempts reconnecting to `home-wifi`, and restores the hotspot if home WiFi still
|
|
isn't in range. Installed via `deploy/install_netwatch.sh`.
|
|
|
|
**Root-caused and fixed a real bug during testing**: the hotspot's SSID broadcast worked
|
|
immediately, but its DHCP server initially failed (`dnsmasq: failed to create listening socket
|
|
for 10.42.0.1: Address already in use`) — a pre-existing **orphaned system-wide `dnsmasq.service`**
|
|
(noted as "likely unused" in earlier investigation) was already bound to the address space
|
|
NetworkManager's own internal dnsmasq needs for the hotspot. Confirmed the system dnsmasq's
|
|
config was genuinely empty/default (nothing in this project depends on it) and disabled it
|
|
(`systemctl disable --now dnsmasq`).
|
|
|
|
Also added easy access without needing to know the Pi's IP:
|
|
- **Renamed the Pi's hostname** from `yarisobd` to `yaris` — `avahi-daemon` (already installed
|
|
and running) now publishes it as `yaris.local` via mDNS.
|
|
- **Port 80 -> 5000 redirect** via nftables (`deploy/nftables-yaris.conf` +
|
|
`install_port_redirect.sh`, loaded through an `include` in `/etc/nftables.conf`), so the app is
|
|
reachable at plain `http://yaris.local/` (no port number) instead of `:5000`.
|
|
|
|
**Verified on real hardware, very carefully** since the SSH session used to do all this testing
|
|
itself runs over `wlan0`: used a "dead-man's-switch" pattern for every risky test — a detached
|
|
background command (`setsid nohup bash -c 'sleep N && nmcli con up home-wifi' & disown`) armed
|
|
*before* each hotspot activation, so the Pi automatically reverts to home WiFi (and SSH access
|
|
comes back) even if something goes wrong, no physical access needed. Confirmed: hotspot broadcasts
|
|
and a real phone gets a genuine DHCP lease from NetworkManager's own DHCP server and can load the
|
|
web UI; `yaris-netwatch` correctly detects AP mode and reconnects to home WiFi when in range (and
|
|
is a no-op when already on home WiFi); `http://yaris.local/` and `http://<pi-lan-ip>/` both return
|
|
HTTP 200 with no port number from another machine on the network. (Real hotspot SSID/password and
|
|
the Pi's LAN IP are intentionally not recorded in this file — see local, non-committed notes.)
|
|
|
|
### Phase 7: app_db.py / app_sql.py fixes — DONE
|
|
Neither file was called from `app.py` or anywhere else - orphaned from a prior iteration. Fixed:
|
|
- `app_db.py`: applied a prior-session stash fix - resolves `.env` path via `__file__` (was
|
|
cwd-relative), fixes `os.getenv()` calls to pass the actual key names (was called with zero
|
|
args, an immediate `TypeError` if ever invoked), added a clear `RuntimeError` guard if
|
|
credentials are still empty placeholders, returns `client[mDB]`.
|
|
- `app_sql.py`: rewritten - resolves `db/sql/` via `__file__` + `os.makedirs` (was cwd-relative),
|
|
fixed `create_conn()` closing the connection immediately and returning `None` (so table
|
|
creation never ran and every caller got a dead connection), added `get_conn()` (lazy singleton
|
|
+ table creation for both `tracking` and `dtc` tables), `insert_tracking()`, and a background
|
|
`start_logging()`/`_log_loop()` thread (5s interval) that snapshots GPS (`app_gps.LocationX/Y`)
|
|
+ OBD speed (`app_obd.speed`) into the local `tracking` table. Used a deferred (function-local)
|
|
import of `app_obd` inside `_log_loop()` to keep the pre-existing `app_obd`<->`app_sql`
|
|
top-level circular import safe/explicit.
|
|
- Also fixed a Python 3.12+/3.14 compat bug found during testing: `sqlite3.version` (the
|
|
sqlite3 *module's* version) was removed/deprecated - switched to `sqlite3.sqlite_version`
|
|
(the actual SQLite library version), which is what was intended.
|
|
- Wired `app_sql.start_logging()` into `app.py` alongside the other startup calls
|
|
(`app_gps.start_gps()`, `app_obd.OBD()`).
|
|
- `db/sql/` added to `.gitignore` (per-device local trip data, not source).
|
|
- Verified locally: `get_conn()` creates `db/sql/<account>_<car>.db` with both tables;
|
|
`insert_tracking()` writes a real row; `start_logging()` thread starts and logs
|
|
"SQL tracking-logger thread started, alive: True"; full `app.py` run confirms the thread
|
|
starts cleanly and (with no real GPS/OBD hardware on the dev machine) correctly logs read
|
|
errors instead of crashing; `app_db.get_database()` raises the new clear `RuntimeError` instead
|
|
of crashing, since `db/creds.env` still holds empty placeholder credentials (real Mongo
|
|
credentials are a user-provided follow-up, out of scope here).
|
|
|
|
### Phase 8: Dashcam retrieval (app_dash.py) — DONE (tested without physical hardware)
|
|
Rewrote `app_dash.py` to be brand/device-agnostic: detection via `lsblk -J`, matching partitions
|
|
whose parent disk reports `TRAN == "usb"` (this is what safely distinguishes a plugged-in
|
|
dashcam/USB stick from the Pi's own onboard storage, which is always `mmc` transport - no more
|
|
hardcoded `/dev/sda`). Walks the whole mounted filesystem for video file extensions
|
|
(`.mp4/.mov/.avi/.ts/.mkv`) instead of assuming a specific brand's folder layout (DCIM, MOVIE,
|
|
NORM/EVENT/PARKING, etc. all just work), preserves each file's relative sub-folder path when
|
|
copying (avoids filename collisions across dashcam event/date folders), and skips files already
|
|
copied (same relative path + size) so re-running an import is cheap. Mounts read-only to avoid
|
|
any risk of corrupting the dashcam's SD card. Dropped the unused/uninstalled `tqdm` CLI-progress
|
|
dependency (never actually in `requirements.txt`) since this now runs headless inside the always-
|
|
on web server, not interactively.
|
|
|
|
Wired into `app.py`: `GET /api/dash/devices` (list detected USB storage) and
|
|
`POST /api/dash/import` (run the mount->copy->unmount pipeline), both JSON. Fixed the pre-existing
|
|
`/dash` vs `/dtc` duplicate-template bug — `dash.html` now has real dashcam content (scan/import
|
|
buttons + live status) instead of a copy-pasted "DTC Codes" body. Added a dedicated `dash.js`
|
|
(not the shared `yaris.js`, which unconditionally initializes the `/index`-only map/chart and
|
|
would throw/crash on `/dash` and `/dtc` since those pages don't have those DOM elements - a
|
|
pre-existing bug, worked around by not depending on that shared file for this page's new JS).
|
|
Added `dash/videos/` to `.gitignore` (imported footage is per-device data, not source).
|
|
|
|
**Tested thoroughly without the physical dashcam** (user didn't have it on hand this session):
|
|
1. Unit-tested the `lsblk`-JSON detection/filtering logic against a realistic synthetic sample
|
|
(Pi's own `mmcblk0` boot+root partitions plus a simulated USB dashcam partition) - confirmed
|
|
it correctly finds only the USB partition and ignores the Pi's own storage.
|
|
2. Integration-tested the real mount -> find-videos -> copy (preserving structure) -> dedupe ->
|
|
unmount pipeline on the actual Pi using a loop-device-backed FAT32 image (`mkfs.vfat` +
|
|
`losetup`) populated with fake nested dashcam video files (`DCIM/100EVENT/...`,
|
|
`DCIM/100NORMAL/...`) - confirmed real mount/copy/unmount all work, confirmed a second import
|
|
run correctly skips already-copied files, confirmed non-video files (e.g. `thumbs.db`) are
|
|
ignored. All simulation artifacts (loop device, test image, temp files) cleaned up afterward.
|
|
3. Verified the full `/dash` page + both new API routes locally (Flask dev run + headless
|
|
browser check): auto-scan on page load correctly reports "No USB dashcam/storage device
|
|
detected" (accurate, since none is plugged into the dev machine), Import button correctly
|
|
stays disabled until a device is found, no JS console errors, no regression on `/` or `/dtc`.
|
|
|
|
**Still to verify once the physical dashcam is available**: plug it into the Pi, hit "Scan for
|
|
dashcam" on `/dash`, confirm it's detected and its real videos import correctly into
|
|
`dash/videos/` on the Pi. No code changes should be needed for that - only the transport (loop
|
|
device vs. real USB) differs, and the detection/copy logic is transport-agnostic by design.
|
|
|
|
### Phase 9: Remote server for itinerary/dashcam upload — ⏳ TODO
|
|
Per architecture decisions #4 and #5: build a self-hosted server/API to receive itinerary
|
|
(from the SQLite/Mongo pipeline) and dashcam file uploads, with the Pi using its home WiFi when
|
|
available and falling back to the SIM868's GSM/GPRS cellular connection otherwise.
|
|
|
|
## Working Notes / Environment Gotchas
|
|
|
|
- **Pi credentials/SSH**: `ssh yaris@<pi-ip>`. Confirm current IP before connecting (varies by
|
|
network — check locally, not recorded here) — usually reachable via `yaris.local` too (mDNS,
|
|
see Phase 6).
|
|
- **Local dev repo**: work happens on a separate dev machine, not on the Pi. Push to
|
|
`origin/shameless-aislop` (not yet merged to `main`) when work is ready.
|
|
- **Git push auth**: if `git push` fails with a credential/auth error, run `gh auth login`
|
|
(device-code browser flow) then `gh auth setup-git` to wire git to use the `gh` CLI's stored
|
|
credentials — this fixed push auth issues in this environment without needing SSH keys or PATs.
|
|
- **Pi environment gap**: Debian Bookworm blocks system-wide `pip install` (PEP 668,
|
|
"externally-managed-environment"). Use a venv for the real app dependency install (Phase 5)
|
|
rather than `--break-system-packages` everywhere. (Resolved in Phase 5 — venv at
|
|
`/home/yaris/app/yaris-obd/venv`.)
|
|
- **Pi → GitHub auth for `git pull` (private repo)**: the Pi's pre-existing
|
|
`~/.ssh/id_ed25519.pub` is an *inbound* key only (private half is on the dev machine) — can't
|
|
be reused for the Pi's own outbound git auth. Generated a dedicated keypair on the Pi
|
|
(`~/.ssh/yaris_deploy_key`), added it as a GitHub **read-only deploy key** on this repo, and
|
|
added an `~/.ssh/config` entry pinning `github.com` to that key. `[email protected]:...` (SSH
|
|
remote) then works standalone on the Pi, no PAT/interactive login needed.
|
|
- **piwheels**: pip on the Pi automatically uses `piwheels.org` as an extra index (Raspberry Pi
|
|
OS default `pip.conf`) — this is why `gevent` (which normally needs a C compile) installed in
|
|
seconds from a prebuilt ARM wheel instead of taking many minutes to build from source.
|
|
- **python-obd note**: `obd.OBD()` (or with `portstr=`) performs the ELM327 handshake at
|
|
connection time; if there's no car on the other end, it raises rather than silently returning
|
|
a dead handle — `app_obd.py`'s `connect_obd()` already catches this and sets `connection = None`.
|