Optimize backlog upload speed for tracking rows
The upload loop previously always slept the full 30s UPLOAD_INTERVAL between batches, even when a full batch was just sent (meaning more rows were likely still queued). With GPS/OBD rows logged every 5s locally, an offline/stale device could build up a large backlog that took a very long time to drain at 100 rows/30s. - Bump TRACKING_BATCH_SIZE 100 -> 500 (cuts round-trips per backlog). - Only sleep the full UPLOAD_INTERVAL once actually caught up (a short batch came back); while a full batch keeps coming back, drain back-to-back with a short CATCHUP_INTERVAL (1s) instead. - server/app.py: add a sane MAX_TRACKING_BATCH_POINTS (5000) cap on the tracking/batch endpoint as a defensive guard against a runaway client, independent of the Pi's own batch size. Verified against the real deployed server: a simulated 1250-row backlog now drains in 3 batches (~0.35s server round-trip time) instead of 13 batches spread across ~6.5 minutes of idle sleeping. Co-authored-by: Copilot <[email protected]>
This commit is contained in:
+14
-3
@@ -27,8 +27,9 @@ SERVER_URL = os.getenv("YARIS_SERVER_URL", "").rstrip("/")
|
||||
DEVICE_NAME = os.getenv("YARIS_DEVICE_NAME", "")
|
||||
API_KEY = os.getenv("YARIS_API_KEY", "")
|
||||
|
||||
UPLOAD_INTERVAL = 30 # seconds between upload attempts when idle/healthy
|
||||
TRACKING_BATCH_SIZE = 100
|
||||
UPLOAD_INTERVAL = 30 # seconds between upload attempts once caught up
|
||||
CATCHUP_INTERVAL = 1 # seconds between batches while draining a backlog
|
||||
TRACKING_BATCH_SIZE = 500
|
||||
BACKOFF_INITIAL = 5 # seconds, doubles on each consecutive failure
|
||||
BACKOFF_MAX = 300
|
||||
|
||||
@@ -160,7 +161,17 @@ def _upload_loop():
|
||||
_last_error = None
|
||||
_last_success_at = _dt.datetime.now().isoformat()
|
||||
backoff = BACKOFF_INITIAL
|
||||
time.sleep(UPLOAD_INTERVAL)
|
||||
|
||||
# A full batch means there's likely more backlog queued up (the
|
||||
# tracking table logs a new row every 5s, so a stale/offline
|
||||
# device can build up thousands of rows quickly) - keep draining
|
||||
# back-to-back instead of idling for the full interval. Only
|
||||
# settle into the slow/idle cadence once a batch comes back
|
||||
# short, i.e. we're actually caught up.
|
||||
if sent >= TRACKING_BATCH_SIZE:
|
||||
time.sleep(CATCHUP_INTERVAL)
|
||||
else:
|
||||
time.sleep(UPLOAD_INTERVAL)
|
||||
except requests.RequestException as e:
|
||||
_last_error = str(e)
|
||||
print(f"Upload failed ({e}), retrying in {backoff}s")
|
||||
|
||||
Reference in New Issue
Block a user