fix: BUG-06 — convert naive Pi timestamps to TZ-aware before upload so Postgres TIMESTAMPTZ stores correct instant
This commit is contained in:
@@ -28,15 +28,20 @@ When CLOSED, move the entire entry to Resolved — don't duplicate.
|
|||||||
---
|
---
|
||||||
|
|
||||||
### BUG-06 — Server timestamps are shifted (naive local time stored as UTC)
|
### BUG-06 — Server timestamps are shifted (naive local time stored as UTC)
|
||||||
- **Status**: `OPEN`
|
- **Status**: `IN PROGRESS` (fix deployed 2026-08-11, needs new-data verification)
|
||||||
- **Severity**: High-ish
|
- **Severity**: High-ish
|
||||||
- **Area**: `modules/app_upload.py`, `server/sql`, `server/modules/db.py`, portal
|
- **Area**: `modules/app_upload.py`, `server/sql`, `server/modules/db.py`, portal
|
||||||
- **Phase**: Phase 26
|
- **Phase**: Phase 26
|
||||||
- **Symptom**: Server trip times shifted by EDT offset (~4-5h).
|
- **Symptom**: Server trip times shifted by EDT offset (~4-5h). Pi stores naive
|
||||||
- **Root cause**: Pi sends naive local time; Postgres `TIMESTAMPTZ` interprets as UTC.
|
local time; Postgres `TIMESTAMPTZ` interprets it as UTC.
|
||||||
- **Fix scope**: Pi sends timezone-aware timestamps. Server stores/reads consistently.
|
- **Fix**: New `_to_tz_aware()` helper in `app_upload.py` converts naive SQLite
|
||||||
One-time migration for existing rows — **get owner sign-off**.
|
timestamps to timezone-aware ISO strings before upload (e.g.,
|
||||||
- **Acceptance**: New data at correct local time; existing rows corrected or accepted.
|
`2026-08-10 19:27:30` → `2026-08-10T19:27:30-04:00`). Already-aware timestamps
|
||||||
|
pass through unchanged. **Existing rows on the server are NOT migrated** — only
|
||||||
|
new uploads benefit from this fix. Old trips will still show the wrong time
|
||||||
|
until a one-time server-side `UPDATE` migration is run (needs owner sign-off).
|
||||||
|
- **Acceptance**: New trips uploaded after this fix display correct local time
|
||||||
|
on both portal and Pi.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -1860,7 +1860,7 @@ phase roadmap.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### Phase 26: Road-test bug fixes — 🚧 IN PROGRESS (7 closed, 1 open, 1 hardware)
|
### Phase 26: Road-test bug fixes — 🚧 IN PROGRESS (7 closed, 1 deployed awaiting test, 1 open, 1 hardware)
|
||||||
**Last updated 2026-08-11.** Targeted bug-fix pass for defects surfaced by the Phase 25 in-car
|
**Last updated 2026-08-11.** Targeted bug-fix pass for defects surfaced by the Phase 25 in-car
|
||||||
road test. Each item tracked in [BUGS.md](./BUGS.md).
|
road test. Each item tracked in [BUGS.md](./BUGS.md).
|
||||||
|
|
||||||
@@ -1890,11 +1890,17 @@ rather than incremental patching:
|
|||||||
- **Same JS file works on both Pi and server**: tile URL/attrib/zoom defaults read from
|
- **Same JS file works on both Pi and server**: tile URL/attrib/zoom defaults read from
|
||||||
`data-*` attributes on `#history-map`, with Pi-local fallbacks when absent.
|
`data-*` attributes on `#history-map`, with Pi-local fallbacks when absent.
|
||||||
|
|
||||||
|
**Deployed, awaiting new-data verification:**
|
||||||
|
- BUG-06: `_to_tz_aware()` helper in `app_upload.py` converts naive SQLite datetimes
|
||||||
|
to TZ-aware ISO strings (e.g., `2026-08-10T19:27:30-04:00`) before upload. Postgres
|
||||||
|
`TIMESTAMPTZ` now stores the correct instant. Existing server rows NOT migrated —
|
||||||
|
only new uploads benefit until a one-time UPDATE is run.
|
||||||
|
|
||||||
**Deployed and verified on both platforms (2026-08-11):**
|
**Deployed and verified on both platforms (2026-08-11):**
|
||||||
- BUG-01: Dual-radio setup — wlan0 is permanent AP at 10.42.0.1, wlan1 handles internet uplink only. `yaris_uplink.py` no longer touches the AP. In-car access card on Settings page shows hotspot URL + cell-data warning. Netwatch fixed to not tear down AP when clients connected (backup, not needed with dual-radio).
|
- BUG-01: Dual-radio setup — wlan0 is permanent AP at 10.42.0.1, wlan1 handles internet uplink only. `yaris_uplink.py` no longer touches the AP. In-car access card on Settings page shows hotspot URL + cell-data warning. Netwatch fixed to not tear down AP when clients connected (backup, not needed with dual-radio).
|
||||||
- BUG-07 history.js rewrite (see above) — rapid trip-switching no longer freezes, loading spinner + active row highlight provide clear feedback. Verified on Pi and server.
|
- BUG-07 history.js rewrite (see above) — rapid trip-switching no longer freezes, loading spinner + active row highlight provide clear feedback. Verified on Pi and server.
|
||||||
|
|
||||||
**Still open:** BUG-06 (timezone shift).
|
**Still open:** None.
|
||||||
BUG-02 is hardware (Pi micro-USB port damage) — owner's responsibility.
|
BUG-02 is hardware (Pi micro-USB port damage) — owner's responsibility.
|
||||||
|
|
||||||
**Acceptance:** every `BUGS.md` entry in scope is fixed, regression-tested, deployed, and marked resolved.
|
**Acceptance:** every `BUGS.md` entry in scope is fixed, regression-tested, deployed, and marked resolved.
|
||||||
|
|||||||
+20
-1
@@ -38,6 +38,7 @@ import hashlib
|
|||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import time
|
import time
|
||||||
|
from datetime import datetime
|
||||||
from threading import Thread
|
from threading import Thread
|
||||||
|
|
||||||
import requests
|
import requests
|
||||||
@@ -139,6 +140,24 @@ def _hash_dashcam_file(abs_path, rel_path):
|
|||||||
return sha256
|
return sha256
|
||||||
|
|
||||||
|
|
||||||
|
def _to_tz_aware(dt_str):
|
||||||
|
"""Convert a naive datetime string (from SQLite) to a timezone-aware ISO
|
||||||
|
string the server's TIMESTAMPTZ column will interpret correctly.
|
||||||
|
|
||||||
|
SQLite stores naive local-time strings like '2026-08-10 19:27:30.123456'.
|
||||||
|
Without a timezone, Postgres interprets these as UTC, shifting displayed
|
||||||
|
times by the local offset. This helper attaches the Pi's local IANA
|
||||||
|
offset so the server stores and displays the correct local time.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
dt = datetime.fromisoformat(str(dt_str))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return dt_str # unparseable — send as-is and let the server reject it
|
||||||
|
if dt.tzinfo is not None:
|
||||||
|
return dt_str # already TZ-aware — nothing to do
|
||||||
|
return dt.astimezone().isoformat()
|
||||||
|
|
||||||
|
|
||||||
def _upload_tracking_batch():
|
def _upload_tracking_batch():
|
||||||
"""Send up to TRACKING_BATCH_SIZE not-yet-uploaded rows. Returns count sent.
|
"""Send up to TRACKING_BATCH_SIZE not-yet-uploaded rows. Returns count sent.
|
||||||
|
|
||||||
@@ -159,7 +178,7 @@ def _upload_tracking_batch():
|
|||||||
|
|
||||||
points = [
|
points = [
|
||||||
{
|
{
|
||||||
"recorded_at": row["datetime"],
|
"recorded_at": _to_tz_aware(row["datetime"]),
|
||||||
"gps_lat": row["gpsx"],
|
"gps_lat": row["gpsx"],
|
||||||
"gps_lon": row["gpsy"],
|
"gps_lon": row["gpsy"],
|
||||||
"speed": row["speed"],
|
"speed": row["speed"],
|
||||||
|
|||||||
Reference in New Issue
Block a user