fix: BUG-06 — convert naive Pi timestamps to TZ-aware before upload so Postgres TIMESTAMPTZ stores correct instant
This commit is contained in:
@@ -28,15 +28,20 @@ When CLOSED, move the entire entry to Resolved — don't duplicate.
|
||||
---
|
||||
|
||||
### BUG-06 — Server timestamps are shifted (naive local time stored as UTC)
|
||||
- **Status**: `OPEN`
|
||||
- **Status**: `IN PROGRESS` (fix deployed 2026-08-11, needs new-data verification)
|
||||
- **Severity**: High-ish
|
||||
- **Area**: `modules/app_upload.py`, `server/sql`, `server/modules/db.py`, portal
|
||||
- **Phase**: Phase 26
|
||||
- **Symptom**: Server trip times shifted by EDT offset (~4-5h).
|
||||
- **Root cause**: Pi sends naive local time; Postgres `TIMESTAMPTZ` interprets as UTC.
|
||||
- **Fix scope**: Pi sends timezone-aware timestamps. Server stores/reads consistently.
|
||||
One-time migration for existing rows — **get owner sign-off**.
|
||||
- **Acceptance**: New data at correct local time; existing rows corrected or accepted.
|
||||
- **Symptom**: Server trip times shifted by EDT offset (~4-5h). Pi stores naive
|
||||
local time; Postgres `TIMESTAMPTZ` interprets it as UTC.
|
||||
- **Fix**: New `_to_tz_aware()` helper in `app_upload.py` converts naive SQLite
|
||||
timestamps to timezone-aware ISO strings before upload (e.g.,
|
||||
`2026-08-10 19:27:30` → `2026-08-10T19:27:30-04:00`). Already-aware timestamps
|
||||
pass through unchanged. **Existing rows on the server are NOT migrated** — only
|
||||
new uploads benefit from this fix. Old trips will still show the wrong time
|
||||
until a one-time server-side `UPDATE` migration is run (needs owner sign-off).
|
||||
- **Acceptance**: New trips uploaded after this fix display correct local time
|
||||
on both portal and Pi.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1860,7 +1860,7 @@ phase roadmap.
|
||||
|
||||
---
|
||||
|
||||
### Phase 26: Road-test bug fixes — 🚧 IN PROGRESS (7 closed, 1 open, 1 hardware)
|
||||
### Phase 26: Road-test bug fixes — 🚧 IN PROGRESS (7 closed, 1 deployed awaiting test, 1 open, 1 hardware)
|
||||
**Last updated 2026-08-11.** Targeted bug-fix pass for defects surfaced by the Phase 25 in-car
|
||||
road test. Each item tracked in [BUGS.md](./BUGS.md).
|
||||
|
||||
@@ -1890,11 +1890,17 @@ rather than incremental patching:
|
||||
- **Same JS file works on both Pi and server**: tile URL/attrib/zoom defaults read from
|
||||
`data-*` attributes on `#history-map`, with Pi-local fallbacks when absent.
|
||||
|
||||
**Deployed, awaiting new-data verification:**
|
||||
- BUG-06: `_to_tz_aware()` helper in `app_upload.py` converts naive SQLite datetimes
|
||||
to TZ-aware ISO strings (e.g., `2026-08-10T19:27:30-04:00`) before upload. Postgres
|
||||
`TIMESTAMPTZ` now stores the correct instant. Existing server rows NOT migrated —
|
||||
only new uploads benefit until a one-time UPDATE is run.
|
||||
|
||||
**Deployed and verified on both platforms (2026-08-11):**
|
||||
- BUG-01: Dual-radio setup — wlan0 is permanent AP at 10.42.0.1, wlan1 handles internet uplink only. `yaris_uplink.py` no longer touches the AP. In-car access card on Settings page shows hotspot URL + cell-data warning. Netwatch fixed to not tear down AP when clients connected (backup, not needed with dual-radio).
|
||||
- BUG-07 history.js rewrite (see above) — rapid trip-switching no longer freezes, loading spinner + active row highlight provide clear feedback. Verified on Pi and server.
|
||||
|
||||
**Still open:** BUG-06 (timezone shift).
|
||||
**Still open:** None.
|
||||
BUG-02 is hardware (Pi micro-USB port damage) — owner's responsibility.
|
||||
|
||||
**Acceptance:** every `BUGS.md` entry in scope is fixed, regression-tested, deployed, and marked resolved.
|
||||
|
||||
+20
-1
@@ -38,6 +38,7 @@ import hashlib
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from datetime import datetime
|
||||
from threading import Thread
|
||||
|
||||
import requests
|
||||
@@ -139,6 +140,24 @@ def _hash_dashcam_file(abs_path, rel_path):
|
||||
return sha256
|
||||
|
||||
|
||||
def _to_tz_aware(dt_str):
|
||||
"""Convert a naive datetime string (from SQLite) to a timezone-aware ISO
|
||||
string the server's TIMESTAMPTZ column will interpret correctly.
|
||||
|
||||
SQLite stores naive local-time strings like '2026-08-10 19:27:30.123456'.
|
||||
Without a timezone, Postgres interprets these as UTC, shifting displayed
|
||||
times by the local offset. This helper attaches the Pi's local IANA
|
||||
offset so the server stores and displays the correct local time.
|
||||
"""
|
||||
try:
|
||||
dt = datetime.fromisoformat(str(dt_str))
|
||||
except (TypeError, ValueError):
|
||||
return dt_str # unparseable — send as-is and let the server reject it
|
||||
if dt.tzinfo is not None:
|
||||
return dt_str # already TZ-aware — nothing to do
|
||||
return dt.astimezone().isoformat()
|
||||
|
||||
|
||||
def _upload_tracking_batch():
|
||||
"""Send up to TRACKING_BATCH_SIZE not-yet-uploaded rows. Returns count sent.
|
||||
|
||||
@@ -159,7 +178,7 @@ def _upload_tracking_batch():
|
||||
|
||||
points = [
|
||||
{
|
||||
"recorded_at": row["datetime"],
|
||||
"recorded_at": _to_tz_aware(row["datetime"]),
|
||||
"gps_lat": row["gpsx"],
|
||||
"gps_lon": row["gpsy"],
|
||||
"speed": row["speed"],
|
||||
|
||||
Reference in New Issue
Block a user