Compare commits

...
55 Commits
Author SHA1 Message Date
quoije a4d132f134 Harden worker cleanup + reduce cover-loading churn
build / Linux x86_64 (push) Successful in 3m30s
build / Windows x64 (Wine) (push) Successful in 4m22s
- Library/metadata workers: delete the worker on its own thread via
  worker.finished -> worker.deleteLater (bound) instead of letting Python GC
  delete it on the GUI thread after untracking.
- Cover templates: compute the transparent-window scan outside the lock so a
  slow first scan never holds the lock and stalls other cover threads.
- Store: larger cover/date batches (500/400/300) so the store loads 3449
  archives with far fewer QThread + ThreadPoolExecutor creations.
2026-09-14 22:33:49 -04:00
quoije 3a69f3f6a1 Fix scrape crash: don't run GUI slots on worker threads
build / Linux x86_64 (push) Successful in 3m4s
build / Windows x64 (Wine) (push) Successful in 3m44s
Connecting a signal to a Python lambda has no QObject receiver, so PySide6
treats it as a direct connection and runs the lambda on the emitting (worker)
thread. The scrape dialog connected its results signals to lambdas, so
_on_results/_on_scraped executed on the worker thread and called
accept()/done() there -> QThread::wait on itself and GUI widget mutation off
the GUI thread -> 'QThread: Destroyed while thread is still running' abort.

Connect to bound methods (queued to the GUI thread) and carry the generation
on the worker (read via sender().gen). Also replaced the thread.finished
lambdas in load_library/metadata with bound-method cleanup keyed by a
thread._worker attribute.
2026-09-14 21:54:59 -04:00
quoije 7381f2c39d Fix scrape crash + cover compose responsiveness
build / Windows x64 (Wine) (push) Successful in 6m16s
build / Linux x86_64 (push) Successful in 6m22s
- Cache the UMD/game cover templates AND their transparent-window rects
  (computed once, under a lock). The old code re-ran an O(w*h) Python pixel
  scan on every composited cover, stalling video cover loading and scraping
  for ~1s per cover with GIL contention.
- ScrapeDialog: ignore stale search/scrape results via a generation counter
  and cancel in-flight workers when a new search/scrape starts; clear threads
  on close.
- MetadataWorker (library details): cancel stale metadata fetches and delete
  finished workers/threads instead of leaking one per item selection.
2026-09-14 21:26:57 -04:00
quoije 8b24d391ac Harden Library tab thread cleanup (covers + items)
build / Windows x64 (Wine) (push) Successful in 9m0s
build / Linux x86_64 (push) Successful in 10m44s
Library covers used the same QThread+worker architecture as the store but
never deleted finished threads/loaders, so every reload leaked one QThread
and one CoverLoader. Library item loading leaked its QThread too.

CoverShowcase now removes + deleteLater's loaders on their finished signal
and deletes their threads on thread.finished; MainWindow.load_library deletes
the finished worker and thread instead of only untracking them.
2026-09-14 20:53:30 -04:00
quoije d94ae52e14 Fix store crash when filtering after load (loader thread cleanup)
build / Windows x64 (Wine) (push) Successful in 10m49s
build / Linux x86_64 (push) Successful in 13m4s
StorePage cover/date loaders were never removed from tracking once their
thread finished, and _populate/_begin_scan dropped the whole list while
threads were still running. That garbage-collected QObjects whose QThread
had already been deleted, leaving a dangling thread affinity that crashed
Qt (SIGSEGV) when searching the store after it fully loaded.

Loaders now remove themselves and deleteLater on their finished signal, and
the tracking list is kept until each thread winds down.
2026-09-14 20:36:59 -04:00
quoije d8a552c61e Client: keep PS1/theme covers+metadata local when a server is configured
build / Linux x86_64 (push) Successful in 2m54s
build / Windows x64 (Wine) (push) Successful in 3m40s
2026-09-14 20:15:26 -04:00
quoije bf1fa04f11 Document the new FastAPI server and WebUI
build / Linux x86_64 (push) Successful in 2m55s
build / Windows x64 (Wine) (push) Successful in 3m50s
2026-09-14 19:41:35 -04:00
quoije 7b7d31bdfa Rewrite server: FastAPI + auth (users/roles/tokens) + admin WebUI + server-authoritative covers/metadata 2026-09-14 19:40:58 -04:00
quoije 14b0b955f5 Add All filter to NoPayStation store source
build / Linux x86_64 (push) Successful in 3m28s
build / Windows x64 (Wine) (push) Successful in 4m14s
2026-09-14 19:07:40 -04:00
quoije b378214e9b Run transfer queue jobs concurrently (max 3) so downloads don't block each other
build / Linux x86_64 (push) Successful in 3m15s
build / Windows x64 (Wine) (push) Successful in 3m41s
2026-09-14 12:50:05 -04:00
quoije 0110b50a07 Download cover bundle through the transfer queue (fix QProgressDialog NameError) 2026-09-14 12:47:13 -04:00
quoije 76eb6a6ef4 Smooth store source transitions: cancel stale batches and batch grid updates 2026-09-14 00:49:50 -04:00
quoije 58d348859a Fix store FD exhaustion: run cover/date loaders one at a time 2026-09-14 00:38:37 -04:00
quoije 3359ecf131 Fix store lag: decode cached covers off the GUI thread
build / Linux x86_64 (push) Successful in 2m48s
build / Windows x64 (Wine) (push) Successful in 3m17s
2026-09-14 00:24:48 -04:00
quoije 38a3859a21 Unbundle covers: fetch psppt-covers.zip from repo on demand with user prompt
build / Linux x86_64 (push) Successful in 2m28s
build / Windows x64 (Wine) (push) Successful in 3m8s
2026-09-14 00:03:09 -04:00
quoije 96e876b8ec Harden shutdown: no segfault closing mid-load (thread cleanup + graceful Ctrl+C) 2026-09-13 23:49:17 -04:00
quoije 733e3c9089 Fix UI freeze on first cover load (decode+scale off the GUI thread)
build / Linux x86_64 (push) Failing after 8m22s
build / Windows x64 (Wine) (push) Canceled after 17m46s
2026-09-13 23:20:08 -04:00
quoije dc7d6c9720 Rebrand to psppt (psp-preserve -> psppt) 2026-09-13 23:08:22 -04:00
quoije 5c5bb6e5d6 fix line break 2026-09-13 22:57:08 -04:00
quoije 35db22e3c5 Add grrr meme to README (resized to 156x148) 2026-09-13 22:55:46 -04:00
quoije b0cdac5483 Put contents_directory on EXE (COLLECT reads it from the exe) 2026-09-13 22:33:04 -04:00
quoije f367bbe298 Clean /usr/bin install: contents dir 'lib' + install.sh
build / Linux x86_64 (push) Successful in 13m11s
build / Windows x64 (Wine) (push) Successful in 20m38s
2026-09-13 21:47:32 -04:00
quoije c61d8e9d3f Revert contents_directory (collides with exe name)
build / Linux x86_64 (push) Failing after 3m8s
build / Windows x64 (Wine) (push) Canceled after 0s
2026-09-13 21:43:28 -04:00
quoije 78f6d44c73 Fix frozen-build SSL certs, taskbar icon, and rename _internal to psp-preserve
build / Linux x86_64 (push) Failing after 7m17s
build / Windows x64 (Wine) (push) Canceled after 0s
- config: point SSL at certifi bundle when frozen (fixes store/NPS/TMDB SSL errors)
- gui: set app name + window icon for the taskbar
- spec: contents_directory=psp-preserve; bundle a .desktop file for Wayland launchers
2026-09-13 21:19:56 -04:00
quoije a7f90cd5a7 Runner image: use WineHQ wine-staging + embeddable Python (Wine 8.0 too old for Py3.12) 2026-09-13 21:13:30 -04:00
quoije c9575817fe Runner image: use full Python installer (bundles pip + OpenSSL) 2026-09-13 21:11:45 -04:00
quoije b576ff5747 Settings: add Storage section showing state dir, DB, cover cache, and user data paths 2026-09-13 21:07:23 -04:00
quoije 6510229b07 Add cover-grid sorting to Library and Store
Library: Title / Release date / Size / Region / Type / Serial
Store: Newest first / Title / Region / Size / In library first
2026-09-13 21:03:49 -04:00
quoije e61d91418e Runner image: add wine32 (winetricks regedit needs 32-bit Wine) 2026-09-13 20:52:42 -04:00
quoije a1c461f6aa Runner image: use embeddable Python (avoid 32-bit installer bootstrap) 2026-09-13 20:50:57 -04:00
quoije 985c8df2cc Runner image: add xauth (xvfb-run requirement) 2026-09-13 20:38:48 -04:00
quoije 6eab2eca9c Use upload-artifact v3 (Gitea lacks v4 artifact protocol)
build / Linux x86_64 (push) Successful in 13m34s
build / Windows x64 (Wine) (push) Canceled after 0s
2026-09-13 20:28:57 -04:00
quoije eac29c428d Add Wine-based runner image for Windows builds
- runner/Dockerfile: Debian + Wine + Windows CPython 3.12 + PyInstaller + act_runner
- runner/entrypoint.sh: self-register with the windows:host label
- workflow windows job now builds via winpython under Wine
2026-09-13 20:18:06 -04:00
quoije ce00e3f518 Add PyInstaller spec and Gitea Actions CI for Linux/Windows binaries
build / Linux x86_64 (push) Failing after 13m46s
build / Windows x64 (push) Canceled after 0s
- run.py: top-level entry point for PyInstaller (relative imports need it)
- psp-preserve.spec: onedir build bundling DATs, loader, cover bundles, icons
- requirements-build.txt: pinned build deps (PySide6 6.8.3 for Windows ICU fix)
- .gitea/workflows/build.yml: builds linux + windows artifacts on tags/manual
- gitignore build/ dist/
2026-09-13 20:09:55 -04:00
quoije c19717402c Polish settings, add matching defaults, drop pkg2zip, externalize infra config
- Rework Settings tab into grouped sections
- Add preferred-region matching and full-hash-by-default setting
- Replace combo-box square arrow with a proper chevron
- Fix UMD Refresh button (clicked bool was clearing the view)
- Remove pkg2zip.exe fallback (pure-Python decryptor is the primary path)
- Move LAN IP/share paths to .env; use example defaults in config
2026-09-13 20:03:44 -04:00
quoije 1ec2e81e15 Scan tab: remove duplicate Scan button, add redo-match-choice context menu 2026-09-13 19:31:28 -04:00
quoije bd9cf47ca9 Run saves scan in background so app startup doesn't freeze 2026-09-09 23:15:18 -04:00
quoije 6e06d1389c Background scrape/eject/refresh, PS1 PIC0 art polish, top-right logo, and updated screenshot 2026-09-09 21:25:34 -04:00
quoije 8c5f9b6ea0 Dark theme, scan auto-ingest, packaging prep, and README refresh 2026-09-09 18:17:23 -04:00
quoije 74164b33ec Update screenshot (5-column grid) 2026-09-09 12:28:18 -04:00
quoije 75db7a3faa Fill cover grid width in library and store (no dead gap) 2026-09-09 12:27:46 -04:00
quoije ed3cc9bc8a Update README screenshot 2026-09-08 23:59:34 -04:00
quoije e01013ce6a Fix DLC switch freeze: skip bulk cover scraping for themes/DLC 2026-09-08 23:55:51 -04:00
quoije 318a920873 Fix store scan signals running on worker thread (bound methods, not lambdas) 2026-09-08 23:27:29 -04:00
quoije ec9f5e6d8e Store: fix NPS refresh, scan race, thread cleanup, and matching perf 2026-09-08 23:18:23 -04:00
quoije be54ff00aa Re-enable cover/metadata scraping for themes and DLC 2026-09-08 23:04:24 -04:00
quoije 115cdd84b0 Store: show cached theme previews in grid and details panel 2026-09-08 22:56:58 -04:00
quoije 62ce4b1913 PSP Sync: show friendly theme/DLC names instead of raw filenames 2026-09-08 22:51:36 -04:00
quoije 46c675ce79 PSP Sync: list themes and DLC in the device inventory 2026-09-08 22:45:36 -04:00
quoije 42fac24cf1 Skip ISO.BIN.EDAT wrapper so DLC shows as a single library item 2026-09-08 22:15:36 -04:00
quoije 7cc5debfa6 Skip NoPayStation items with MISSING (delisted) pkg URLs 2026-09-08 22:00:56 -04:00
quoije c03f4619e6 Show theme/DLC names and embedded theme previews in the library 2026-09-08 17:32:11 -04:00
quoije 1be33be4ab Fix NPS store content-type dropdown fetching wrong list 2026-09-07 23:58:54 -04:00
quoije 2fed95da05 Skip cover/metadata scraping for NoPayStation themes and DLC 2026-09-07 23:53:55 -04:00
quoije c94bcdc993 Add NoPayStation themes, DLC and minis support 2026-09-07 23:34:56 -04:00
61 changed files with 4805 additions and 754 deletions
+14
View File
@@ -0,0 +1,14 @@
# Copy this file to `.env` and fill in your values. `.env` is git-ignored.
# Static LAN IP for the API container on Unraid's macvlan bridge (br0).
PSP_LAN_IP=192.168.10.100
# Host paths mounted into the container (Unraid shares).
PSP_STORE_HOST=/mnt/user/downloads/Sony - PlayStation Portable
PSP_STORE_VIDEOS_HOST=/mnt/user/psp/umd_videos_store
PSP_UMD_GAMES_HOST=/mnt/user/psp/umd_games
PSP_UMD_VIDEOS_HOST=/mnt/user/psp/umd_videos
# Initial admin account created on first boot (change the password after login).
PSP_ADMIN_USERNAME=admin
PSP_ADMIN_PASSWORD=change-me-now
+60
View File
@@ -0,0 +1,60 @@
name: build
on:
push:
tags: ["v*"]
workflow_dispatch:
jobs:
linux:
name: Linux x86_64
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install build dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-build.txt
- name: Build
run: pyinstaller --clean --noconfirm psppt.spec
- name: Package tarball
run: |
cp packaging/install.sh dist/psppt/
cp packaging/psppt.desktop dist/psppt/
cp img/app-icon.png dist/psppt/
tar -czf psppt-linux-x86_64.tar.gz -C dist psppt
- uses: actions/upload-artifact@v3
with:
name: psppt-linux-x86_64
path: psppt-linux-x86_64.tar.gz
windows:
name: Windows x64 (Wine)
runs-on: windows
steps:
- uses: actions/checkout@v4
- name: Install build dependencies
run: |
winpython -m pip install --upgrade pip
winpython -m pip install -r requirements-build.txt
- name: Build
run: winpython -m PyInstaller --clean --noconfirm psppt.spec
- name: Package zip
run: |
cd dist && zip -rq ../psppt-windows-x64.zip psppt
- uses: actions/upload-artifact@v3
with:
name: psppt-windows-x64
path: psppt-windows-x64.zip
+12 -1
View File
@@ -16,4 +16,15 @@ data/*.pbp
data/ps1_templates/
# nps tsv files
data/nps/*.tsv
data/nps/*.tsv
# Local environment overrides (LAN paths, secrets). See .env.example
.env
# PyInstaller output
build/
dist/
# Offline cover bundles (large; fetched from the project release on first run)
data/covers/*.zip
/psppt-covers.zip
+1 -1
View File
@@ -3,5 +3,5 @@
# Port-forward 80 and 443 on your router to this host.
hostname.com {
reverse_proxy psp-preserve:8000
reverse_proxy psppt:8000
}
+7 -3
View File
@@ -2,8 +2,11 @@ FROM python:3.13-slim
WORKDIR /app
# The API is stdlib-only (no pip install needed).
COPY requirements-server.txt ./
RUN pip install --no-cache-dir -r requirements-server.txt
COPY app ./app
COPY server ./server
ENV PSP_DATA_DIR=/data \
PSP_DATS_DIR=/data/dats \
@@ -11,8 +14,9 @@ ENV PSP_DATA_DIR=/data \
PSP_COVERS_DIR=/data/covers \
PSP_GAMES_ROOTS=/data/umd_games \
PSP_VIDEO_ROOTS=/data/umd_videos \
PSP_STORE_DIR=/data/store
PSP_STORE_DIR=/data/store \
PSP_STORE_VIDEOS_DIR=/data/store_videos
EXPOSE 8000
CMD ["python", "-m", "app.api", "--host", "0.0.0.0", "--port", "8000"]
CMD ["uvicorn", "server.main:app", "--host", "0.0.0.0", "--port", "8000"]
+203 -80
View File
@@ -1,17 +1,22 @@
# PSP Preservation Tool
# psppt
the name is terrible, i know.
![grrr](img/grrr.png)
![AI Generated](https://raw.githubusercontent.com/dotnetdave/ai-usage-badges/refs/heads/main/badges/png/1x/ai-generated.png)
Native Linux GUI (PySide6) for preserving PSP UMD **games** and **videos** as
`.iso` files. It scans your discs, verifies them against the full Redump catalog
(Redump DATs), reports what's missing, and retags/renames discs to a canonical
scheme. It also fetches cover art and metadata, syncs discs and save-data with a
real PSP, can pull missing discs from your own Redump 7z archive collection, and
converts PlayStation 1 ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew.
A cross-platform (Linux + Windows) desktop GUI — PySide6 — for preserving PSP
UMD **games** and **videos** as `.iso` files. It scans your discs, verifies them
against the Redump/No-Intro catalogs, reports what's missing, and retags/renames
discs to a canonical scheme. It fetches cover art and metadata, syncs discs and
save-data with a real PSP, can pull missing discs from a Redump 7z archive
collection, downloads PSN content from NoPayStation, and converts PlayStation 1
ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew.
![PSP Preservation Tool](img/qt-gui.png)
* Use default app theme for Qt.
![psppt](img/qt-gui.png)
* Dark theme with a blue accent — the Library tab in cover view. *
## Features
@@ -35,9 +40,9 @@ converts PlayStation 1 ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew.
## Requirements
- Linux + Python 3
- `PySide6` + `pycryptodome` (`pip install -r requirements.txt`)
- `7z` (p7zip) — used to peek/extract disc images from store archives
- Python 3.10+ on Linux or Windows
- `PySide6`, `curl_cffi`, `pycryptodome` (`pip install -r requirements.txt`)
- `7z` (p7zip / 7-Zip) — used to peek/extract disc images from store archives
- PPSSPP (optional) — to launch discs from the details panel
NoPayStation `.pkg` files are decrypted by the bundled **pure-Python
@@ -58,6 +63,77 @@ pip install -r requirements.txt
python3 -m app.main
```
## Building binaries
The GUI is packaged with PyInstaller into a self-contained folder. Build
requirements are pinned in `requirements-build.txt` (PySide6 6.8.3 on purpose —
see the note inside).
```bash
python3 -m venv .venv-build
. .venv-build/bin/activate
pip install -r requirements-build.txt
pyinstaller --clean --noconfirm psppt.spec
# output: dist/psppt/ → run dist/psppt/psppt
```
Notes:
- The build bundles the DATs, PS1 loader, cover templates, and index catalogs
(~80 MB total). The large offline cover zips (~850 MB) are **not** bundled —
the app fetches them on first run (see
[Offline bundles](#offline-bundles--auto-download)).
- `7z` (p7zip-full on Linux, 7-Zip on Windows) is a **runtime** dependency for
reading store archives — it is not bundled.
- User data (credentials, manual covers, PS1 templates, NPS cache, library DB,
cover cache) lives in the OS state dir, never next to the binary.
### CI (Gitea Actions)
`.gitea/workflows/build.yml` builds Linux and Windows artifacts on version-tag
pushes (`v*`) or on manual trigger (**Actions → build → Run workflow**).
Artifacts appear on the Actions run page and can be attached to a release.
Each job needs a [Gitea Actions runner](https://docs.gitea.com/usage/actions/act-runner)
registered with the matching label:
```bash
# Linux runner (Docker mode, run on your server)
./act_runner register \
--instance https://git.ceaf.dev \
--token <registration-token> \
--name linux-builder \
--labels ubuntu-latest:docker://node:20-bookworm
# Windows runner (host mode, run on a Windows machine)
act_runner.exe register \
--instance https://git.ceaf.dev \
--token <registration-token> \
--name win-builder \
--labels windows:host
```
The registration token is generated in Gitea under
**Site Administration → Actions → Runners → Create new runner**.
#### Windows builds without a Windows box (Wine runner)
The `windows` job targets a runner labelled `windows` that builds under Wine on
Linux. Build it from `runner/`:
```bash
docker build -f runner/Dockerfile -t psp-wine-runner .
docker run -d --name psp-wine-runner --restart unless-stopped \
-e GITEA_INSTANCE_URL=https://git.ceaf.dev \
-e GITEA_RUNNER_REGISTRATION_TOKEN=<registration-token> \
-v /mnt/user/appdata/psp-wine-runner:/data \
psp-wine-runner
```
The image ships Wine + Windows CPython 3.12 + PyInstaller, and registers itself
with the `windows:host` label. (If you have a real Windows machine instead,
register a native `act_runner.exe` with the same `windows` label.)
## First-time setup
1. Put your DAT files in `data/dats/`:
@@ -66,14 +142,15 @@ python3 -m app.main
Redump lacks — keeps serial-based matching working).
- videos: No-Intro `Sony - PlayStation Portable (UMD Video)` (Redump has
no UMD Video DAT).
2. Ingest them into the local SQLite library:
2. Start the app — any DATs missing from the library are ingested automatically
at startup (no manual step). You can also re-ingest on demand:
```bash
python3 -m app.ingest
```
3. Open the GUI and point **Settings → General → Games/Videos directory** at
your disc folders (or set `PSP_GAMES_ROOTS` / `PSP_VIDEO_ROOTS`).
3. Point **Settings → General → Games/Videos directory** at your disc folders
(or set `PSP_GAMES_ROOTS` / `PSP_VIDEO_ROOTS`).
4. (Optional) PS1 conversion — copy any PSN PS1 classic `EBOOT.PBP` to
`data/base.PBP` (it supplies Sony's encrypted loader). Then set the **PS1
directory**, **PS1 output directory** and **PS1 base.pbp** in
@@ -95,9 +172,10 @@ Discs are matched by hash first (most precise), then serial, then filename.
- `data/credentials.json` — cover/metadata API keys (IGDB, TheMovieDB,
ScreenScraper, Google Custom Search). Environment variables take precedence.
- `data/manual_covers/games|videos/` — your own cover images.
- `data/covers/` — checked-in assets: the UMD video/game cover templates,
`libretro-index.json` + `openvgdb-index.json` catalogs, `libretro-covers.zip` +
`openvgdb-covers.zip` offline cover bundles, and the legacy cover cache.
- `data/covers/` — checked-in assets: the UMD video/game cover templates and
the `libretro-index.json` + `openvgdb-index.json` catalogs. The large
`libretro-covers.zip` + `openvgdb-covers.zip` bundles are git-ignored and
distributed as `psppt-covers.zip` (see above).
- `data/base.PBP` — a PSN PS1 classic EBOOT (any game) supplying `STARTDAT`
+ a base `PARAM.SFO`/icon for PS1 conversion (git-ignored, copyrighted).
- `data/loader/DATA.PSP` + `data/loader/KEYS.BIN` — bundled plain-ELF POPS
@@ -105,30 +183,29 @@ Discs are matched by hash first (most precise), then serial, then filename.
- `data/ps1_templates/` — optional PS1 Classics art templates
(`ICON0.PNG` case frame, `PIC1.PNG` launch background); the app falls back
to drawn art when they're absent.
- `data/nps/` — NoPayStation TSV lists (cached here on first store scan; drop
`PSP_GAMES.tsv` / `PSX_GAMES.tsv` manually to work offline).
- `data/bin/pkg2zip.exe` — bundled `pkg2zip` binary (MIT, by mmozeiko) kept
only as an optional fallback decryptor; the app normally uses its own
pure-Python decryptor (`app/pkg.py`) and never needs Wine.
See `data/bin/ATTRIBUTION.md`.
- State dir (`~/.local/state/psp-preserve/` by default, or `$XDG_STATE_HOME/psp-preserve`):
- `data/nps/` — NoPayStation TSV lists (cached here on first store scan in dev;
a packaged build stores them under the state dir instead).
- State dir (`~/.local/state/psppt/` on Linux, `%LOCALAPPDATA%\psppt`
on Windows, or `$PSP_STATE_DIR`):
- `library.sqlite` — the catalog + library database.
- `covers/` — downloaded cover cache.
- `data/` — user-writable data in packaged builds (credentials, manual
covers, PS1 templates, NPS TSV cache).
SQLite is kept on a local disk because SQLite locking is unreliable over
network filesystems; the DATs, manual covers and credentials stay in `data/`
(which may be a network share).
network filesystems. Read-only resources (DATs, the POPS loader, icons) are
bundled with the app; user-writable data lives in the state dir.
## Configuration
Everything is driven by environment variables, with GUI settings (stored in
QSettings under `PSPPreserve`) taking precedence where noted.
QSettings under `psppt`) taking precedence where noted.
| Variable | Default | Purpose |
|---|---|---|
| `PSP_DATA_DIR` | `./data` | data root (DATs, credentials, manual covers) |
| `PSP_DATS_DIR` | `$PSP_DATA_DIR/dats` | DAT folder |
| `PSP_STATE_DIR` | `~/.local/state/psp-preserve` | local state root |
| `PSP_STATE_DIR` | `~/.local/state/psppt` | local state root |
| `PSP_DB_PATH` | `$PSP_STATE_DIR/library.sqlite` | SQLite library |
| `PSP_COVERS_DIR` | `$PSP_STATE_DIR/covers` | cover cache |
| `PSP_GAMES_ROOTS` | `/data/umd_games` | comma-separated game disc roots |
@@ -136,7 +213,7 @@ QSettings under `PSPPreserve`) taking precedence where noted.
| `PSP_PS1_ROOTS` | `/data/ps1` | comma-separated PS1 disc roots |
| `PSP_PS1_OUTPUT_DIR` | `/data/ps1_converted` | converted EBOOT.PBP output folder |
| `PSP_SCAN_WORKERS` | `1` | parallel hashing workers |
| `PSP_STORE_DIR` | `/media/downloads/Sony - PlayStation Portable` | local 7z store folder |
| `PSP_STORE_DIR` | `/data/store` | local 7z store folder |
| `PSP_STORE_SERVER` | `https://psp.example.com` | store server base URL |
| `PSP_STORE_TOKEN` | *(empty)* | store server bearer token |
| `PSP_ARCHIVE_ITEM` | `psp-redump-roms321.com` | archive.org item identifier |
@@ -144,11 +221,12 @@ QSettings under `PSPPreserve`) taking precedence where noted.
| `PSP_ARCHIVE_LOGGED_IN_USER` | *(empty)* | archive.org `logged-in-user` cookie |
| `PSP_NPS_DIR` | `<state>/nps` | NoPayStation output folder |
| `PSP_NPS_TSV_BASE` | `https://nopaystation.com/tsv/` | NoPayStation TSV base URL |
| `PSP_PKG2ZIP_PATH` | *(empty)* | pkg2zip binary path (else auto-detected) |
| `PSP_API_TOKEN` | *(empty)* | server API bearer token (unset = open) |
| `PSP_CREDENTIALS` | `$PSP_DATA_DIR/credentials.json` | credentials file |
| `PSP_USER_DATA_DIR` | `<state>/data` (packaged) / `./data` (dev) | user-writable data (credentials, manual covers, PS1 templates, NPS cache) |
| `PSP_CREDENTIALS` | `$PSP_USER_DATA_DIR/credentials.json` | credentials file |
| `PSP_LIBRETRO_COVERS_URL` | *(empty)* | direct URL for `libretro-covers.zip` (else derived from store server) |
| `PSP_OVGDB_COVERS_URL` | *(empty)* | direct URL for `openvgdb-covers.zip` (else derived from store server) |
| `PSP_COVERS_BUNDLE_URL` | Gitea `covers` release asset | combined `psppt-covers.zip` download URL |
Cover/metadata keys can also be set as env vars: `IGDB_CLIENT_ID`,
`IGDB_CLIENT_SECRET`, `TMDB_API_KEY`, `GOOGLE_API_KEY`, `GOOGLE_CX`,
@@ -181,12 +259,23 @@ Manual covers in `data/manual_covers/videos/` always win.
### Offline bundles & auto-download
The two bundles (`libretro-covers.zip`, `openvgdb-covers.zip`) are shipped in
`data/covers/`, so a fresh install works with **no internet and no API keys**.
If either is missing at startup, the app downloads it from the store server's
`/api/covers/<name>.zip` endpoint (see [Server setup](#server-setup)).
The two bundles (`libretro-covers.zip`, `openvgdb-covers.zip`) are packaged
into one `psppt-covers.zip` (~850 MB) and attached to the repo's `covers`
release — **not** bundled into the app binary.
Rebuild them yourself:
On first launch, if the bundle isn't installed, the app asks whether to
download it (with a progress bar). Covers keep working online via libretro /
OpenVGDB / ScreenScraper / IGDB without it — the bundle just makes them instant
and offline. "Don't ask again" is remembered, and you can re-download from
**Settings → Cover sources → Download offline cover bundle…**.
Build the bundle yourself:
```bash
python3 scripts/make-covers-bundle.py # -> psppt-covers.zip
```
Rebuild the inner zips:
```bash
python3 -m app.libretro # libretro-thumbnails -> libretro-covers.zip
@@ -235,7 +324,7 @@ Configure the folders in **Settings → General** (or `PSP_PS1_ROOTS` /
`PSP_PS1_OUTPUT_DIR`). Current limits: single-disc games only (no multi-disc
PBP); disc data is DEFLATE-compressed (level 1 by default, `--complevel 0-9`).
## Store — archive.org and local folder
## Store — four sources
The **Store** tab downloads missing discs from a Redump 7z archive collection
(`Name (Region) (tags).7z`, containing the `.iso`/`.cso` inside). It has three
@@ -274,78 +363,109 @@ skipped — and you'll get a clear error if *everything* in the item is private.
### 4. NoPayStation
The **NoPayStation** source lists PSN content from
[NoPayStation](https://nopaystation.com/) — **PSP games** (`PSP_GAMES.tsv`) and
**PS1 Classics** (`PSX_GAMES.tsv`), selectable from the content-type dropdown
next to the source selector.
[NoPayStation](https://nopaystation.com/), selectable from the content-type
dropdown next to the source selector:
- **PSP Games** (`PSP_GAMES.tsv`) — includes **minis** (also listed separately
in the `Type` column), PC Engine and NeoGeo releases.
- **PS1 Classics** (`PSX_GAMES.tsv`)
- **PSP Themes** (`PSP_THEMES.tsv`)
- **PSP DLC** (`PSP_DLCS.tsv`)
Requirements & behaviour:
1. `.pkg` files are decrypted by the built-in **pure-Python decryptor**
(`app/pkg.py`, powered by `pycryptodome` for speed) — no Wine and no
`pkg2zip` install needed. A bundled `pkg2zip.exe` is kept as a fallback
only, and only runs if Python decryption fails and Wine is present.
`pkg2zip` needed.
2. Set **NoPayStation output dir** in **Settings → Store** (default
`$PSP_STATE_DIR/nps`).
3. Downloading a PSP game writes `PSP/<TITLE_ID>/EBOOT.PBP` (plus
`PSP-KEY.EDAT` / `CONTENT.DAT` when the package ships them); a PS1 Classic
writes `PSX/<TITLE_ID>/EBOOT.PBP` (+ `DOCUMENT.DAT` and `KEYS.BIN`). Both
then show up in the **Library** (PS1 Classics under **PS1**, PSP games
under **Games**) and can be copied to the PSP like any other item.
4. The TSV lists are cached in `data/nps/` and refreshed on each store scan
3. Downloads land in the NPS folder and appear in the **Library**:
- PSP game → `PSP/<TITLE_ID>/EBOOT.PBP` (`psp_kind="eboot"`).
- PS1 Classic → `PSX/<TITLE_ID>/EBOOT.PBP` (+ `DOCUMENT.DAT`, `KEYS.BIN`).
- Theme → `THEME/<TITLE_ID>/<content_id>.ptf` (the packaged PTF is
EDAT-unwrapped to a raw theme file).
- DLC → `DLC/<GAME_ID>/<content_id>.edat` (installed as-is into the owning
game's folder).
4. Copying to the PSP puts games/PS1 under `/PSP/GAME/` (optionally nested in
a Category Lite folder), themes under `/PSP/THEME/`, and DLC under
`/PSP/GAME/<GAME_ID>/`.
5. The TSV lists are cached in `data/nps/` and refreshed on each store scan
(drop the `.tsv` files there manually to work offline).
> **DRM note:** NPS content is PSN-DRM'd. To *run* it on the PSP you need the
> [`npdrm_free`](https://github.com/qwikrazor87/npdrm_free) plugin (ARK and most
> CFW already ship it). The desktop app only downloads and decrypts; it doesn't
> strip DRM.
>
> **Covers/metadata:** themes and DLC have no cover art or game metadata of
> their own — the app scrapes them like games (IGDB/libretro/etc.), which can
> return unrelated results. Downloaded themes keep their embedded preview as
> the primary cover (used instead of a scraped image when available).
## Server setup
The catalog + store API is a single-file, stdlib-only HTTP server
(`app/api.py`) — no pip install needed on the server.
The server is a **FastAPI** app (`server/`) with an admin **WebUI** and a
JSON API. It owns the catalog, metadata, covers, and the game/video store
archives, and serves them to the desktop app. Users log in with a username +
password; the app holds a **per-user API token** (roles: `admin`, `editor`,
`user`).
Run it directly (dev):
```bash
PSP_DATA_DIR=/data \
PSP_DATS_DIR=/data/dats \
pip install -r requirements-server.txt
PSP_DB_PATH=/data/library.sqlite \
PSP_COVERS_DIR=/data/covers \
PSP_GAMES_ROOTS=/data/umd_games \
PSP_VIDEO_ROOTS=/data/umd_videos \
PSP_STORE_DIR=/data/store \
PSP_API_TOKEN=<long-random-secret> \
python3 -m app.api --host 0.0.0.0 --port 8000
PSP_STORE_VIDEOS_DIR=/data/store_videos \
PSP_ADMIN_USERNAME=admin \
PSP_ADMIN_PASSWORD=<change-me> \
uvicorn server.main:app --host 0.0.0.0 --port 8000
```
Endpoints:
On first boot it creates the `PSP_ADMIN_USERNAME` account (log in and change
the password). The admin WebUI lives at `/`:
- **Dashboard** — storage, catalog counts, recent downloads.
- **Games & Videos** — browse the catalog, edit metadata, upload covers.
- **Store** — upload/delete game and video archives.
- **Users** — create users, assign roles, reset passwords.
API endpoints:
| Endpoint | Description |
|---|---|
| `GET /api/summary` | release counts per system |
| `POST /api/auth/login` | username/password → API token |
| `GET /api/auth/me` | current user |
| `GET /api/users`, `POST`, `PATCH`, `DELETE` | user management *(admin)* |
| `GET /api/releases?system=&q=&limit=&offset=` | search the catalog |
| `GET /api/releases/<id>` | one release + its rom hashes |
| `GET /api/library?system=&refresh=` | your scanned library |
| `GET /api/cover/<id>` | cover image (fetched on demand) |
| `GET /api/covers/<name>.zip` | offline cover bundle (public) |
| `GET /api/store` | list store archives *(token)* |
| `GET /api/store/download/<name>` | stream one archive *(token)* |
| `GET /api/releases/<id>` | release detail + metadata + rom hashes |
| `PATCH /api/releases/<id>` | edit metadata *(editor+)* |
| `GET /api/cover/<id>` | cover image |
| `POST /api/cover/<id>` | upload/replace cover *(editor+)* |
| `GET /api/store?system=games|videos` | list store archives |
| `GET /api/store/download/<system>/<name>` | stream an archive *(user+)* |
| `POST /api/store/upload`, `DELETE /api/store/<system>/<name>` | manage archives *(editor+)* |
| `GET /api/stats` | dashboard numbers *(admin)* |
Token protection: set `PSP_API_TOKEN`; the client sends
`Authorization: Bearer <token>`. Leave it unset to allow anonymous access
(not recommended).
Authentication is `Authorization: Bearer <token>` for the API and a session
cookie for the WebUI.
### Docker (Unraid)
`docker-compose.yml` builds the API image and attaches it to Unraid's `br0`
macvlan bridge so it gets its own LAN IP (e.g. `192.168.1.100`, plain HTTP on 8000):
`docker-compose.yml` builds the server image and attaches it to Unraid's `br0`
macvlan bridge so it gets its own LAN IP:
```bash
# in /path/to/psp-preserve/ — adjust the volume paths for your shares
printf 'PSP_API_TOKEN=<long-random-secret>\n' > .env
# in /path/to/psppt/
cp .env.example .env # then edit: LAN IP, share paths, admin password
docker compose up -d --build
```
The compose file mounts your Unraid shares into `/data/store`, `/data/umd_games`
and `/data/umd_videos` (the store is mounted read-only).
The compose file reads the LAN IP and Unraid share paths from `.env` and mounts
them into `/data/store`, `/data/store_videos`, `/data/umd_games` and
`/data/umd_videos` (the store mounts are writable so the WebUI can manage
archives).
### TLS (optional)
@@ -356,17 +476,20 @@ ports 80/443 on your router to it.
### Client
In the GUI: **Settings → Store** → enter the **Server URL** (e.g.
`http://<server-ip>:8000` or `https://psp.example.com`) and the same **API
token**, then pick **Server** in the Store tab's source selector. Use
**Archive.org** or **Local folder** the same way.
In the GUI: **Settings → Store** → enter the **Server URL**, **Server
username**, and **Server password**, then **Save settings** (the app logs in
and stores the per-user token). Pick **Server** in the Store tab's source
selector. When a server is configured it is authoritative for covers and
metadata — if it's unreachable the app shows an error instead of falling back
to public sources. Use **Archive.org**, **Local folder**, or **NoPayStation**
the same way as before.
## PSP Sync
Connect a PSP over USB (or mount a UMD) and the **PSP Sync** tab detects it:
- **Games / UMD Videos / Homebrew-PSN / PKG** — tables of what's installed, with
delete-from-PSP (checkboxes and right-click).
- **Games / UMD Videos / Homebrew-PSN / Themes / DLC / PKG** — tables of what's
installed, with delete-from-PSP (checkboxes and right-click).
- **UMD** — reads `PARAM.SFO` from a mounted UMD, shows the title/serial, and
can dump the disc image into your library.
- **Sync from Library** — copy discs from your library onto the PSP, optionally
+1 -1
View File
@@ -1 +1 @@
"""PSP Preservation Tool application package."""
"""psppt application package."""
+2 -2
View File
@@ -251,7 +251,7 @@ def _library(conn, system=None, refresh=False):
class Handler(BaseHTTPRequestHandler):
server_version = "PSPPreserve/0.1"
server_version = "psppt/0.1"
def do_GET(self):
parsed = urlparse(self.path)
@@ -262,7 +262,7 @@ class Handler(BaseHTTPRequestHandler):
try:
if path == "/":
_json(self, {
"name": "PSP Preservation Tool API",
"name": "psppt API",
"endpoints": [
"/api/summary",
"/api/releases?system=games|videos&q=&limit=&offset=",
+56 -9
View File
@@ -1,8 +1,33 @@
"""Central configuration, driven by environment variables."""
import os
import sys
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parent.parent
# PyInstaller sets ``sys.frozen`` and unpacks bundled files to ``sys._MEIPASS``.
# Read-only resources (DATs, loader, icons) then live in the bundle; otherwise
# they live in the project checkout.
IS_FROZEN = getattr(sys, "frozen", False)
_BUNDLE_DIR = getattr(sys, "_MEIPASS", None)
PROJECT_ROOT = Path(_BUNDLE_DIR).resolve() if _BUNDLE_DIR else Path(__file__).resolve().parent.parent
# Frozen bundles don't ship the system CA store, so Python's urllib would fail
# TLS verification against the store server / NoPayStation / TMDB / archive.org.
# Point OpenSSL at the certifi bundle (a curl_cffi dependency) when frozen only;
# source runs keep using the OS CA store.
if IS_FROZEN:
try:
import certifi
_CA_BUNDLE = certifi.where()
os.environ.setdefault("SSL_CERT_FILE", _CA_BUNDLE)
os.environ.setdefault("REQUESTS_CA_BUNDLE", _CA_BUNDLE)
except Exception: # noqa: BLE001
pass
def resource_path(*parts):
"""Absolute path to a read-only bundled resource (works when frozen)."""
return PROJECT_ROOT.joinpath(*parts)
DATA_DIR = Path(os.environ.get("PSP_DATA_DIR", str(PROJECT_ROOT / "data")))
DATS_DIR = Path(os.environ.get("PSP_DATS_DIR", str(DATA_DIR / "dats")))
@@ -12,25 +37,38 @@ def _default_state_dir():
"""Local state directory (fast, reliable local disk).
SQLite locking is unreliable over network filesystems (SMB/FUSE), so the
library DB and the cover cache live on local storage while DATs, manual
covers and credentials stay in ``DATA_DIR`` (which may be a network share).
library DB and the cover cache live on local storage while read-only DATs
stay bundled with the app.
"""
if os.name == "nt":
base = os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or str(Path.home())
return Path(base) / "psppt"
xdg = os.environ.get("XDG_STATE_HOME")
base = Path(xdg) if xdg else Path.home() / ".local" / "state"
return base / "psp-preserve"
return base / "psppt"
STATE_DIR = Path(os.environ.get("PSP_STATE_DIR", str(_default_state_dir())))
DB_PATH = Path(os.environ.get("PSP_DB_PATH", str(STATE_DIR / "library.sqlite")))
COVERS_DIR = Path(os.environ.get("PSP_COVERS_DIR", str(STATE_DIR / "covers")))
# User-writable data (credentials, manual covers, PS1 templates, NPS TSV cache).
# In dev this is the checkout's ``data/``; when frozen the bundle is read-only,
# so it points at local state that survives app upgrades.
USER_DATA_DIR = Path(os.environ.get(
"PSP_USER_DATA_DIR", str(DATA_DIR if not IS_FROZEN else STATE_DIR / "data")))
LEGACY_DB_PATH = DATA_DIR / "library.sqlite"
LEGACY_COVERS_DIR = DATA_DIR / "covers"
def user_data_path(*parts):
"""Absolute path to a user-writable data file/folder (works when frozen)."""
return USER_DATA_DIR.joinpath(*parts)
# Redump 7z store (browsable in the Store tab). Env override; GUI setting wins.
STORE_ROOT = Path(os.environ.get("PSP_STORE_DIR", "/media/downloads/Sony - PlayStation Portable"))
STORE_ROOT = Path(os.environ.get("PSP_STORE_DIR", "/data/store"))
# Remote store server (HTTPS). Used by the Store tab as the primary source.
STORE_SERVER_URL = os.environ.get("PSP_STORE_SERVER", "https://psp.ceaf.dev")
STORE_SERVER_URL = os.environ.get("PSP_STORE_SERVER", "https://psp.example.com")
STORE_TOKEN = os.environ.get("PSP_STORE_TOKEN", "")
# Optional direct URLs for the offline cover bundle zips. When unset, the GUI
@@ -38,9 +76,21 @@ STORE_TOKEN = os.environ.get("PSP_STORE_TOKEN", "")
LIBRETRO_COVERS_URL = os.environ.get("PSP_LIBRETRO_COVERS_URL", "")
OVGDB_COVERS_URL = os.environ.get("PSP_OVGDB_COVERS_URL", "")
# Combined offline cover bundle (both inner zips) fetched on first run when the
# user opts in. Defaults to the project's Gitea release asset.
COVERS_BUNDLE_URL = os.environ.get(
"PSP_COVERS_BUNDLE_URL",
"https://git.ceaf.dev/quoije/psp-preservation-tool/releases/download/covers/psppt-covers.zip",
)
# archive.org item hosting the same Redump 7z set (identifier or full URL).
ARCHIVE_ITEM = os.environ.get("PSP_ARCHIVE_ITEM", "psp-redump-roms321.com")
# UMD video store: a second archive root (mirrors STORE_ROOT) and the
# archive.org item hosting UMD video rips.
STORE_VIDEOS_ROOT = Path(os.environ.get("PSP_STORE_VIDEOS_DIR", "/data/store_videos"))
ARCHIVE_VIDEOS_ITEM = os.environ.get("PSP_ARCHIVE_VIDEOS_ITEM", "the-psp-movie-collection-so-far_202607")
# Optional archive.org login cookies (for items whose files are "private" /
# login-required). Grab these from a logged-in browser session on archive.org.
ARCHIVE_LOGGED_IN_SIG = os.environ.get("PSP_ARCHIVE_LOGGED_IN_SIG", "")
@@ -51,9 +101,6 @@ ARCHIVE_LOGGED_IN_USER = os.environ.get("PSP_ARCHIVE_LOGGED_IN_USER", "")
NPS_OUTPUT_DIR = Path(os.environ.get("PSP_NPS_DIR", str(STATE_DIR / "nps")))
NPS_TSV_BASE = os.environ.get("PSP_NPS_TSV_BASE", "https://nopaystation.com/tsv/")
# pkg2zip binary used to decrypt/extract NPS .pkg files (blank = auto-detect).
PKG2ZIP_PATH = os.environ.get("PSP_PKG2ZIP_PATH", "")
# Scan roots (used by the scanner phase). Comma-separated in env.
GAMES_ROOTS = [Path(p) for p in os.environ.get("PSP_GAMES_ROOTS", "/data/umd_games").split(",") if p.strip()]
VIDEO_ROOTS = [Path(p) for p in os.environ.get("PSP_VIDEO_ROOTS", "/data/umd_videos").split(",") if p.strip()]
+164 -78
View File
@@ -19,7 +19,7 @@ import urllib.request
import zipfile
from pathlib import Path
from . import config, sources
from . import config, server_client, sources
from .db import get_connection
log = logging.getLogger("covers")
@@ -52,9 +52,21 @@ _IGDB_IMAGE = "https://images.igdb.com/igdb/image/upload/"
_TMDB_SEARCH = "https://api.themoviedb.org/3/search/movie"
_TMDB_IMAGE = "https://image.tmdb.org/t/p/"
_GOOGLE_CSE_API = "https://www.googleapis.com/customsearch/v1"
_MANUAL_DIR = config.DATA_DIR / "manual_covers"
_OVGDB_ZIP = config.DATA_DIR / "covers" / "openvgdb-covers.zip"
_LIBRETRO_ZIP = config.DATA_DIR / "covers" / "libretro-covers.zip"
_MANUAL_DIR = config.USER_DATA_DIR / "manual_covers"
# Offline cover bundles live in the writable cover dir (downloaded on demand),
# not the read-only app bundle. A source checkout keeps them in data/covers/.
_OVGDB_ZIP = config.COVERS_DIR / "openvgdb-covers.zip"
_LIBRETRO_ZIP = config.COVERS_DIR / "libretro-covers.zip"
_BUNDLE_ZIP = config.COVERS_DIR / "psppt-covers.zip"
def _bundle_zip_path(name):
"""Resolve a cover bundle zip: writable dir first, legacy checkout second."""
for base in (config.COVERS_DIR, config.LEGACY_COVERS_DIR):
p = base / name
if p.exists():
return p
return config.COVERS_DIR / name
_WIKI_SEARCH = "https://en.wikipedia.org/w/api.php"
_WIKI_SUMMARY = "https://en.wikipedia.org/api/rest_v1/page/summary/"
@@ -62,7 +74,7 @@ _WIKI_SUMMARY = "https://en.wikipedia.org/api/rest_v1/page/summary/"
_PNG_MAGIC = b"\x89PNG\r\n\x1a\n"
_JPEG_MAGIC = b"\xff\xd8\xff"
_MAX_HOPS = 3 # follow libretro symlink text bodies at most this many times
_UA = {"User-Agent": "psp-preserve/0.1 (https://github.com; local tool)"}
_UA = {"User-Agent": "psppt/0.1 (https://github.com; local tool)"}
_igdb_token_state = {"token": "", "expires": 0.0} # IGDB OAuth2 token cache
_igdb_lock = threading.Lock()
_igdb_last = 0.0 # monotonic clock of the last IGDB API call
@@ -158,48 +170,74 @@ def clear_cover_cache():
pass
def _download_file(url, dest, timeout=600):
"""Stream `url` to `dest` (atomic); returns True on success."""
def _download_file(url, dest, timeout=600, progress_cb=None, cancel_event=None):
"""Stream `url` to `dest` (atomic); returns True on success.
``progress_cb(done, total)`` is called with bytes so far (total is 0 when
the server doesn't report a content length). ``cancel_event`` aborts the
download when set.
"""
req = urllib.request.Request(url, headers=_UA)
with urllib.request.urlopen(req, timeout=timeout) as resp:
if resp.status != 200:
return False
total = int(resp.headers.get("Content-Length") or 0)
tmp = dest.with_name(dest.name + ".tmp")
done = 0
with open(tmp, "wb") as f:
while True:
if cancel_event is not None and cancel_event.is_set():
raise InterruptedError("download cancelled")
chunk = resp.read(1 << 20)
if not chunk:
break
f.write(chunk)
done += len(chunk)
if progress_cb is not None:
progress_cb(done, total)
os.replace(tmp, dest)
return True
def ensure_cover_bundles(server_url=""):
"""Download missing offline cover bundle zips from the store server.
def covers_bundle_present():
"""True when the offline cover bundle is installed (both inner zips)."""
return (_bundle_zip_path("libretro-covers.zip").exists()
and _bundle_zip_path("openvgdb-covers.zip").exists())
``server_url`` is the base URL (e.g. ``http://192.168.10.100:8000``); the
zips are fetched from ``<base>/api/covers/<name>.zip`` unless a direct URL
is set in config. Returns the names of bundles downloaded.
def download_covers_bundle(url=None, progress_cb=None, cancel_event=None):
"""Download and extract the combined cover bundle into the cover dir.
``url`` defaults to ``config.COVERS_BUNDLE_URL``. Raises on failure; the
caller decides whether to surface the error.
"""
base = (server_url or config.STORE_SERVER_URL or "").rstrip("/")
downloaded = []
for name, path, direct in (
("libretro-covers.zip", _LIBRETRO_ZIP, config.LIBRETRO_COVERS_URL),
("openvgdb-covers.zip", _OVGDB_ZIP, config.OVGDB_COVERS_URL),
):
if path.exists():
continue
url = direct or (f"{base}/api/covers/{name}" if base else "")
if not url:
continue
try:
if _download_file(url, path):
downloaded.append(name)
log.info("downloaded offline cover bundle %s", name)
except Exception as exc: # noqa: BLE001
log.warning("could not download cover bundle %s: %s", name, exc)
return downloaded
url = (url or config.COVERS_BUNDLE_URL or "").strip()
if not url:
raise ValueError("no cover bundle URL configured")
config.COVERS_DIR.mkdir(parents=True, exist_ok=True)
_download_file(url, _BUNDLE_ZIP, timeout=3600,
progress_cb=progress_cb, cancel_event=cancel_event)
inner = ("libretro-covers.zip", "openvgdb-covers.zip")
with zipfile.ZipFile(_BUNDLE_ZIP) as z:
missing = [n for n in inner if n not in z.namelist()]
if missing:
raise ValueError(f"cover bundle is missing: {', '.join(missing)}")
with zipfile.ZipFile(_BUNDLE_ZIP) as z:
for name in inner:
out = config.COVERS_DIR / name
tmp = out.with_name(out.name + ".tmp")
with z.open(name) as src, open(tmp, "wb") as dst:
while True:
chunk = src.read(1 << 20)
if not chunk:
break
dst.write(chunk)
os.replace(tmp, out)
# Invalidate cached namelists so lookups pick up the freshly extracted zips.
global _libretro_names, _ovgdb_names
_libretro_names = None
_ovgdb_names = None
return config.COVERS_DIR
def cover_url(name, kind="Named_Boxarts", system="games"):
@@ -411,6 +449,9 @@ def _fetch_libretro(name, timeout, system="games", indexed_only=False):
data = fetch_libretro_local(name)
if data is not None:
return data
# Theme/DLC share the PSP thumbnails repo, so reuse the games index instead
# of downloading a duplicate copy of the same tree under a new system key.
idx_system = system if system in _PLATFORMS else "games"
if not indexed_only:
# 1. Cheap: exact + progressively stripped names (no index needed).
for kind in _LIBRETRO_KINDS:
@@ -421,7 +462,7 @@ def _fetch_libretro(name, timeout, system="games", indexed_only=False):
# 2. Fuzzy/exact via the precomputed index (covers exact names too, so the
# bulk loader skips the wasteful exact pass entirely).
for kind in _LIBRETRO_KINDS:
best = _index_best(name, kind, system)
best = _index_best(name, kind, idx_system)
if best:
data = _fetch_png(best, kind, timeout, system)
if data is not None:
@@ -479,9 +520,10 @@ def _ovgdb_name_list():
global _ovgdb_names
if _ovgdb_names is None:
_ovgdb_names = set()
if _OVGDB_ZIP.exists():
path = _bundle_zip_path("openvgdb-covers.zip")
if path.exists():
try:
with zipfile.ZipFile(_OVGDB_ZIP) as z:
with zipfile.ZipFile(path) as z:
_ovgdb_names = {n[:-4] for n in z.namelist() if n.endswith(".jpg")}
except Exception: # noqa: BLE001
pass
@@ -532,7 +574,7 @@ def fetch_openvgdb_local(name):
if entry is None:
return None
try:
with zipfile.ZipFile(_OVGDB_ZIP) as z:
with zipfile.ZipFile(_bundle_zip_path("openvgdb-covers.zip")) as z:
data = z.read(entry)
return data if _is_image(data) else None
except Exception: # noqa: BLE001
@@ -556,7 +598,7 @@ def fetch_libretro_local(name):
if entry is None:
return None
try:
with zipfile.ZipFile(_LIBRETRO_ZIP) as z:
with zipfile.ZipFile(_bundle_zip_path("libretro-covers.zip")) as z:
data = z.read(entry)
return data if _is_image(data) else None
except Exception: # noqa: BLE001
@@ -567,9 +609,10 @@ def _libretro_name_list():
global _libretro_names
if _libretro_names is None:
_libretro_names = set()
if _LIBRETRO_ZIP.exists():
path = _bundle_zip_path("libretro-covers.zip")
if path.exists():
try:
with zipfile.ZipFile(_LIBRETRO_ZIP) as z:
with zipfile.ZipFile(path) as z:
_libretro_names = {n[:-4] for n in z.namelist() if n.endswith(".png")}
except Exception: # noqa: BLE001
pass
@@ -607,7 +650,7 @@ def _fetch_screenscraper(name, timeout, system="games"):
params = {
"devid": devid,
"devpassword": devpassword,
"softname": creds.get("screenscraper_softname") or "psp-preserve",
"softname": creds.get("screenscraper_softname") or "psppt",
"output": "json",
"systemeid": _platform(system)["ss_id"],
"romtype": "rom",
@@ -730,11 +773,16 @@ def save_cover(release_id, system, data):
# --------------------------------------------------------------------------- #
# Game cover compositing (IGDB key art -> full PSP cover)
# --------------------------------------------------------------------------- #
_game_template_image = None
_game_template_cache = None # (template QImage or None, window (x, y, w, h))
_game_template_lock = threading.Lock()
def _transparent_rect(img):
"""Bounding box of the fully-transparent pixels in `img`."""
"""Bounding box of the fully-transparent pixels in `img`.
Run once per template (the result is cached) — an O(w*h) scan is far too
slow to repeat for every composed cover.
"""
minx, miny, maxx, maxy = img.width(), img.height(), -1, -1
for y in range(img.height()):
for x in range(img.width()):
@@ -747,24 +795,33 @@ def _transparent_rect(img):
def _game_template():
"""Load the PSP game cover template (top header + transparent window)."""
global _game_template_image
if _game_template_image is None:
try:
from PySide6.QtGui import QImage
except Exception: # noqa: BLE001 - headless API may lack PySide6
return None
img = QImage()
for path in (
config.COVERS_DIR / "UMD_GAME_COVER_TEMPLATE.png",
config.DATA_DIR / "covers" / "UMD_GAME_COVER_TEMPLATE.png",
):
if path.exists():
img.load(str(path))
if not img.isNull():
break
_game_template_image = img.convertToFormat(QImage.Format_ARGB32) if not img.isNull() else None
return _game_template_image
"""Load the PSP game cover template; return ``(template, window)``."""
global _game_template_cache
if _game_template_cache is not None:
return _game_template_cache
try:
from PySide6.QtGui import QImage
except Exception: # noqa: BLE001 - headless API may lack PySide6
_game_template_cache = (None, (0, 0, 0, 0))
return _game_template_cache
img = QImage()
for path in (
config.COVERS_DIR / "UMD_GAME_COVER_TEMPLATE.png",
config.DATA_DIR / "covers" / "UMD_GAME_COVER_TEMPLATE.png",
):
if path.exists():
img.load(str(path))
if not img.isNull():
break
img = img.convertToFormat(QImage.Format_ARGB32) if not img.isNull() else None
entry = (
img,
_transparent_rect(img) if img is not None and not img.isNull() else (0, 0, 0, 0),
)
with _game_template_lock:
if _game_template_cache is None:
_game_template_cache = entry
return _game_template_cache
def compose_game_cover(key_art_bytes):
@@ -779,14 +836,13 @@ def compose_game_cover(key_art_bytes):
except Exception: # noqa: BLE001
return key_art_bytes
template = _game_template()
template, (x, y, w, h) = _game_template()
art = QImage.fromData(key_art_bytes)
if template is None or template.isNull() or art.isNull():
return key_art_bytes
art = art.convertToFormat(QImage.Format_ARGB32)
out = QImage(template.size(), QImage.Format_ARGB32)
out.fill(Qt.transparent)
x, y, w, h = _transparent_rect(template)
painter = QPainter(out)
if w > 0 and h > 0:
@@ -808,7 +864,7 @@ def compose_game_cover(key_art_bytes):
return bytes(ba)
_PS1_TEMPLATE_DIR = config.DATA_DIR / "ps1_templates"
_PS1_TEMPLATE_DIR = config.USER_DATA_DIR / "ps1_templates"
def _ps1_template_bytes(name):
@@ -934,25 +990,28 @@ def _make_pic0(title, description, esrb, QImage, QPainter, QPainterPath, QColor,
QRectF, Qt, _save):
W, H = 310, 180
img = QImage(W, H, QImage.Format_ARGB32)
img.fill(QColor(0, 0, 0, 0))
# Dark gradient backdrop.
# Dark gradient backdrop clipped to a rounded card (transparent corners).
grad = QLinearGradient(0, 0, 0, H)
grad.setColorAt(0.0, QColor(38, 38, 40))
grad.setColorAt(1.0, QColor(12, 12, 14))
card = QPainterPath()
card.addRoundedRect(QRectF(0, 0, W, H), 12, 12)
p = QPainter(img)
p.setRenderHint(QPainter.Antialiasing, True)
p.setRenderHint(QPainter.TextAntialiasing, True)
p.fillRect(0, 0, W, H, grad)
p.fillPath(card, grad)
# Rounded border.
# Rounded card border.
p.setPen(QPen(QColor(110, 110, 115), 1))
p.setBrush(Qt.NoBrush)
p.drawRoundedRect(QRectF(4, 4, W - 8, H - 8), 9, 9)
p.drawPath(card)
# PlayStation button glyphs, top-right.
_draw_ps_glyphs(p, W - 34, 12, 22, QColor(200, 200, 205), QRectF, QPointF,
QPolygonF, QPen, Qt)
_draw_ps_glyphs(p, W - 34, 12, 22, QRectF, QPointF, QPolygonF, QPen, Qt,
QColor)
# Title (bold, upper-left). Shrink the font until the whole title fits, so
# long names are shown in full instead of being cut with an ellipsis.
@@ -974,6 +1033,10 @@ def _make_pic0(title, description, esrb, QImage, QPainter, QPainterPath, QColor,
p.drawText(QRectF(18, 16, title_width, 28), Qt.AlignLeft | Qt.AlignVCenter,
title_text)
# Thin divider under the title (like the original PSN info board).
p.setPen(QPen(QColor(255, 255, 255, 55), 1))
p.drawLine(QPointF(18, 46), QPointF(W - 18, 46))
# Description (wrapped to at most 3 lines, last line elided).
if description:
f_desc = QFont("DejaVu Sans")
@@ -982,7 +1045,7 @@ def _make_pic0(title, description, esrb, QImage, QPainter, QPainterPath, QColor,
p.setPen(QColor(215, 215, 215))
fm = QFontMetrics(f_desc)
lines = _wrap_elided(fm, description.strip(), W - 36, 3, Qt)
y = 46
y = 52
for line in lines:
p.drawText(QRectF(18, y, W - 36, 16), Qt.AlignLeft | Qt.AlignVCenter, line)
y += 16
@@ -1024,23 +1087,31 @@ def _wrap_elided(fm, text, width, max_lines, Qt):
return lines[:max_lines]
def _draw_ps_glyphs(p, x, y, size, color, QRectF, QPointF, QPolygonF, QPen, Qt):
"""Draw the four PlayStation face-button glyphs in a 2x2 grid."""
def _draw_ps_glyphs(p, x, y, size, QRectF, QPointF, QPolygonF, QPen, Qt, QColor):
"""Draw the four PlayStation face-button glyphs in a 2x2 grid, using the
official button colours (green triangle, red circle, blue cross, pink square)."""
cell = size / 2.0
pen = QPen(color, max(1.0, size / 14.0))
pen.setCapStyle(Qt.RoundCap)
pen.setJoinStyle(Qt.RoundJoin)
p.setPen(pen)
p.setBrush(Qt.NoBrush)
# Triangle (top-left).
def _pen(color):
pen = QPen(color, max(1.0, size / 14.0))
pen.setCapStyle(Qt.RoundCap)
pen.setJoinStyle(Qt.RoundJoin)
return pen
# Triangle (top-left, green).
p.setPen(_pen(QColor("#3F9B0F")))
tri = QPolygonF([QPointF(x, y + cell), QPointF(x + cell, y), QPointF(x + cell, y + cell)])
p.drawPolygon(tri)
# Circle (top-right).
# Circle (top-right, red).
p.setPen(_pen(QColor("#DE1C18")))
p.drawEllipse(QRectF(x + cell, y, cell, cell))
# Cross (bottom-left).
# Cross (bottom-left, blue).
p.setPen(_pen(QColor("#287FC1")))
p.drawLine(QPointF(x + 1, y + cell + 1), QPointF(x + cell - 1, y + size - 1))
p.drawLine(QPointF(x + cell - 1, y + cell + 1), QPointF(x + 1, y + size - 1))
# Square (bottom-right).
# Square (bottom-right, pink).
p.setPen(_pen(QColor("#C94E9B")))
p.drawRect(QRectF(x + cell, y + cell, cell, cell))
@@ -1162,6 +1233,21 @@ def fetch_cover(name, release_id, system, timeout=10, quick=False):
except OSError:
pass
# Server is authoritative for game/video covers when configured — no
# libretro/IGDB fallback; if the server is unreachable the cover just
# stays empty. PS1/theme art stays client-side (the server catalog only
# covers games and videos).
if server_client.configured() and system in ("games", "videos"):
try:
data = server_client.fetch_cover(system, name)
except Exception as exc: # noqa: BLE001
log.warning("server cover fetch failed for %r: %s", name, exc)
return None
if data is not None and _is_image(data):
_write_atomic(dest, data)
return dest
return None
key = str(release_id)
# In-memory + persistent negative caches: skip the network for recent misses.
last = _neg_cache.get(key)
+31
View File
@@ -115,6 +115,37 @@ CREATE TABLE IF NOT EXISTS ps1_entries (
error TEXT,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS users (
id INTEGER PRIMARY KEY,
username TEXT NOT NULL UNIQUE,
password_hash TEXT NOT NULL,
role TEXT NOT NULL DEFAULT 'user', -- admin / editor / user
is_active INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL,
last_login_at TEXT
);
CREATE TABLE IF NOT EXISTS api_tokens (
id INTEGER PRIMARY KEY,
user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
token_hash TEXT NOT NULL UNIQUE,
label TEXT,
kind TEXT NOT NULL DEFAULT 'api', -- api / session
created_at TEXT NOT NULL,
last_used_at TEXT,
expires_at TEXT,
revoked_at TEXT
);
CREATE TABLE IF NOT EXISTS download_events (
id INTEGER PRIMARY KEY,
user_id INTEGER REFERENCES users(id) ON DELETE SET NULL,
system TEXT NOT NULL,
name TEXT NOT NULL,
bytes INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL
);
"""
+1224 -314
View File
File diff suppressed because it is too large Load Diff
+45 -16
View File
@@ -12,7 +12,7 @@ import json
import sqlite3
import time
from . import config, covers
from . import config, covers, server_client
from .db import get_connection
_failed = {} # (release_id, system) -> time.time() of last failed fetch
@@ -23,6 +23,9 @@ _miss_db = {}
_miss_loaded = False
_MISS_TTL = 86400.0 # retry failed metadata after 1 day
# Memoised release-date table (invalidated on save/reset).
_dates_cache = None
def _load_miss_db():
global _miss_loaded
@@ -122,6 +125,7 @@ def save(release_id, system, meta, manual=False, title=None):
)
conn.commit()
conn.close()
_invalidate_dates_cache()
return True
except sqlite3.Error:
return False
@@ -140,6 +144,7 @@ def reset(release_id, system):
)
conn.commit()
conn.close()
_invalidate_dates_cache()
except sqlite3.Error:
pass
@@ -161,23 +166,32 @@ def cached_release_dates():
"""Return {(release_id, system): release_date} from cached metadata only.
Pure DB read — no network — so list views can show a release date without
triggering API fetches for every row.
triggering API fetches for every row. Result is memoised and invalidated
whenever metadata is saved.
"""
out = {}
try:
conn = get_connection(config.DB_PATH)
rows = conn.execute("SELECT release_id, system, meta_json FROM item_info").fetchall()
conn.close()
except sqlite3.Error:
return {}
for r in rows:
global _dates_cache
if _dates_cache is None:
out = {}
try:
meta = json.loads(r["meta_json"] or "{}")
except (TypeError, ValueError):
continue
if isinstance(meta, dict) and meta.get("release_date"):
out[(r["release_id"], r["system"])] = meta["release_date"]
return out
conn = get_connection(config.DB_PATH)
rows = conn.execute("SELECT release_id, system, meta_json FROM item_info").fetchall()
conn.close()
except sqlite3.Error:
return out
for r in rows:
try:
meta = json.loads(r["meta_json"] or "{}")
except (TypeError, ValueError):
continue
if isinstance(meta, dict) and meta.get("release_date"):
out[(r["release_id"], r["system"])] = meta["release_date"]
_dates_cache = out
return _dates_cache
def _invalidate_dates_cache():
global _dates_cache
_dates_cache = None
def get_or_fetch(release_id, system, fetch_title):
@@ -185,6 +199,21 @@ def get_or_fetch(release_id, system, fetch_title):
info = load(release_id, system)
if info is not None:
return info
# Server is authoritative for game/video metadata when configured — no
# IGDB/TMDB fallback; an unreachable server just leaves metadata empty.
# PS1 metadata stays client-side (the server catalog only covers games
# and videos).
if server_client.configured() and system in ("games", "videos"):
try:
meta = server_client.fetch_metadata(system, fetch_title)
except Exception as exc: # noqa: BLE001
log.warning("server metadata fetch failed for %r: %s", fetch_title, exc)
return None
if meta:
save(release_id, system, meta, manual=False)
meta["manual"] = False
return meta
return None
key = (str(release_id), system)
last = _failed.get(key)
if last is not None and time.time() - last < _FAIL_TTL:
+36
View File
@@ -88,6 +88,42 @@ def ingest_dat(conn, path):
return dat_name, system, release_count, rom_count
def ensure_catalog(dats_dir=None, conn=None):
"""Ingest any DAT files missing from the DB (idempotent, cheap when current).
Called once at startup so a fresh/reset database still has the catalog and
scanning produces matches instead of everything coming back ``unknown``.
Returns the number of DATs ingested.
"""
dats_dir = Path(dats_dir) if dats_dir else config.DATS_DIR
if not dats_dir.is_dir():
return 0
files = sorted(p for p in dats_dir.glob("*.dat") if p.is_file())
if not files:
return 0
own = conn is None
if own:
conn = get_connection(config.DB_PATH)
try:
known = {r["source_path"] for r in conn.execute("SELECT source_path FROM dats")}
count = 0
for path in files:
if str(path) in known:
continue
try:
ingest_dat(conn, path)
except Exception as exc: # noqa: BLE001 - a bad DAT shouldn't block startup
print(f"warning: could not ingest {path.name}: {exc}", file=sys.stderr)
continue
count += 1
conn.commit()
return count
finally:
if own:
conn.close()
def main(argv=None):
parser = argparse.ArgumentParser(description="Ingest clrmamepro DAT files into SQLite")
parser.add_argument("--dats", help="directory containing .dat files (default: data/dats)")
+4 -2
View File
@@ -48,8 +48,10 @@ def launch_ppsspp(iso, exe=""):
else:
return False, "PPSSPP not found. Install it or set its path in Settings → Saves."
try:
subprocess.Popen(cmd, start_new_session=True, env=_env(),
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
kwargs = {"env": _env(), "stdout": subprocess.DEVNULL, "stderr": subprocess.DEVNULL}
if os.name != "nt":
kwargs["start_new_session"] = True # POSIX-only: detach from our process group
subprocess.Popen(cmd, **kwargs)
return True, ""
except OSError as exc:
return False, str(exc)
+1 -1
View File
@@ -1,4 +1,4 @@
"""PSP Preservation Tool entry point (native GUI).
"""psppt entry point (native GUI).
Run with::
+35 -1
View File
@@ -111,6 +111,36 @@ def normalize_name(name):
return _normalize(name)
_EUROPEAN_REGIONS = {
"europe", "germany", "france", "spain", "italy", "russia", "netherlands",
"portugal", "poland", "united kingdom", "greece", "norway", "sweden",
"denmark", "finland", "austria", "belgium", "switzerland",
}
def region_matches(region, prefer):
"""True if ``region`` satisfies the user's preferred-region hint."""
if not prefer or not region:
return False
tokens = {t.strip().lower() for t in str(region).split(",") if t.strip()}
if not tokens:
return False
prefer = prefer.strip().lower()
if prefer in tokens:
return True
if prefer == "europe":
return bool(tokens & _EUROPEAN_REGIONS)
return False
def apply_region_preference(candidates, prefer):
"""Narrow ``candidates`` to the preferred region (fall back to all)."""
if not prefer:
return candidates
filtered = [c for c in candidates if region_matches(c.get("region"), prefer)]
return filtered or candidates
def serial_candidates(conn, record):
"""Return all catalog releases matching the disc's serial (size-narrowed)."""
serial = record.get("serial")
@@ -131,7 +161,7 @@ def serial_candidates(conn, record):
return _narrow(conn, [dict(r) for r in rows], record.get("size"))
def match(conn, record):
def match(conn, record, prefer_region=None):
size = record.get("size")
# 1. Hash is the most precise key; serials are shared across revisions.
@@ -171,6 +201,10 @@ def match(conn, record):
for cand in candidates:
if cand["id"] == saved["release_id"]:
return "choice", cand
if prefer_region:
filtered = [c for c in candidates if region_matches(c.get("region"), prefer_region)]
if len(filtered) == 1:
return "region", filtered[0]
return "unknown", None
# 4. Fuzzy fallback (filename + PARAM.SFO title), size-disambiguated.
+46 -107
View File
@@ -9,20 +9,23 @@ folder — no Wine or ``pkg2zip`` required.
from __future__ import annotations
import logging
import os
import shutil
import subprocess
import urllib.parse
import urllib.request
from datetime import datetime
from pathlib import Path
from typing import List, Optional
from typing import List
from . import config
log = logging.getLogger("nopaystation")
_TSV_NAMES = {"psp": "PSP_GAMES.tsv", "psx": "PSX_GAMES.tsv"}
_TSV_NAMES = {
"psp": "PSP_GAMES.tsv",
"psx": "PSX_GAMES.tsv",
"theme": "PSP_THEMES.tsv",
"dlc": "PSP_DLCS.tsv",
}
# NoPayStation TSVs are tab-separated with a header row whose exact columns
# differ per platform (PSP has "Type"/"RAP" columns, PSX has "Original Name").
@@ -45,13 +48,13 @@ _HEADER_ALIASES = {
def tsv_url(content_type: str, base: str = "") -> str:
"""TSV URL for a content type (``psp`` / ``psx``)."""
"""TSV URL for a content type (``psp`` / ``psx`` / ``theme`` / ``dlc``)."""
base = (base or config.NPS_TSV_BASE).rstrip("/")
return f"{base}/{_TSV_NAMES.get(content_type, _TSV_NAMES['psp'])}"
def _tsv_dir() -> Path:
d = config.DATA_DIR / "nps"
d = config.USER_DATA_DIR / "nps"
d.mkdir(parents=True, exist_ok=True)
return d
@@ -76,7 +79,7 @@ def parse_tsv(text: str, content_type: str) -> List[dict]:
``release_date``, ``mtime_ns``) plus ``title_id``, ``pkg_url``, ``sha256``,
``content_type`` and the cover ``system`` ("games" for PSP, "ps1" for PSX).
"""
system = "games" if content_type == "psp" else "ps1"
system = {"psp": "games", "psx": "ps1", "theme": "theme", "dlc": "dlc"}.get(content_type, "games")
items: List[dict] = []
header = None
for line in text.splitlines():
@@ -94,7 +97,9 @@ def parse_tsv(text: str, content_type: str) -> List[dict]:
row[canon] = cols[i].strip()
name = row.get("name") or ""
pkg_url = row.get("pkg_url") or ""
if not name or not pkg_url:
# NoPayStation marks delisted packages with the literal "MISSING" in the
# URL column — skip them so they can't be queued for download.
if not name or not pkg_url or not pkg_url.lower().startswith(("http://", "https://")):
continue
title_id = row.get("title_id") or ""
try:
@@ -124,47 +129,40 @@ def parse_tsv(text: str, content_type: str) -> List[dict]:
return items
def fetch_items(content_type: str, url: str = "", timeout: int = 60) -> List[dict]:
"""Download (and cache) a TSV, returning parsed item dicts."""
def fetch_items(content_type: str, url: str = "", timeout: int = 60, force: bool = False) -> List[dict]:
"""Download (and cache) a TSV, returning parsed item dicts.
``force`` re-downloads the TSV (used by the Store's Refresh button); the
local cache is only used as a fallback when the network fails.
"""
url = url or tsv_url(content_type)
name = _TSV_NAMES.get(content_type, "PSP_GAMES.tsv")
cache = _tsv_dir() / name
if cache.is_file():
def read_cache():
try:
return parse_tsv(cache.read_text(encoding="utf-8", errors="replace"), content_type)
except OSError:
pass
req = urllib.request.Request(url, headers={"User-Agent": "psp-preserve/0.1"})
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = resp.read()
text = data.decode("utf-8", errors="replace")
cache.write_text(text, encoding="utf-8")
return parse_tsv(text, content_type)
return None
if not force and cache.is_file():
items = read_cache()
if items is not None:
return items
def find_pkg2zip(configured_path: str = "") -> Optional[Path]:
"""Locate pkg2zip: configured path, then the bundled binary, then PATH."""
if configured_path:
p = Path(configured_path)
return p if p.is_file() else None
bundled = config.DATA_DIR / "bin" / "pkg2zip.exe"
if bundled.is_file():
return bundled
found = shutil.which("pkg2zip")
return Path(found) if found else None
def _pkg2zip_cmd(pkg2zip: Path) -> List[str]:
"""Command prefix for pkg2zip, wrapping the bundled .exe in Wine on Linux."""
if pkg2zip.suffix.lower() == ".exe" and os.name != "nt":
wine = shutil.which("wine") or shutil.which("wine64")
if not wine:
raise RuntimeError(
"The bundled pkg2zip is a Windows binary and Wine is not installed — "
"install Wine (or point Settings → Store at a native pkg2zip)."
)
return [wine, str(pkg2zip)]
return [str(pkg2zip)]
try:
req = urllib.request.Request(url, headers={"User-Agent": "psppt/0.1"})
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = resp.read()
text = data.decode("utf-8", errors="replace")
cache.write_text(text, encoding="utf-8")
return parse_tsv(text, content_type)
except Exception: # noqa: BLE001 - fall back to the cached list on failure
items = read_cache()
if items is not None:
log.warning("could not refresh %s TSV; using cached copy", content_type)
return items
raise
def _safe_folder(value: str) -> str:
@@ -186,12 +184,11 @@ def _ensure_keys_bin(folder, kind):
pass
def extract_pkg(pkg_path, title_id: str, dest_dir, pkg2zip: Path = None,
progress_cb=None, cancel_event=None) -> Path:
def extract_pkg(pkg_path, title_id: str, dest_dir,
kind_hint: str = None, progress_cb=None, cancel_event=None) -> Path:
"""Decrypt/extract a ``.pkg`` into ``dest_dir/<TITLE_ID>/``.
Uses the bundled pure-Python decryptor (no Wine / pkg2zip needed); falls
back to pkg2zip only if that fails and a binary is available. Returns the
Uses the bundled pure-Python decryptor (no Wine or pkg2zip). Returns the
destination game folder path.
"""
from . import pkg as pkgmod
@@ -199,73 +196,15 @@ def extract_pkg(pkg_path, title_id: str, dest_dir, pkg2zip: Path = None,
pkg_path = Path(pkg_path)
dest = Path(dest_dir) / _safe_folder(title_id)
if (dest / "EBOOT.PBP").exists():
if dest.is_dir() and any(dest.iterdir()):
return dest
# 1. Pure-Python decryptor (primary).
try:
_kind, _tid, folder = pkgmod.extract(pkg_path, dest_dir, progress_cb, cancel_event)
_kind, _tid, folder = pkgmod.extract(pkg_path, dest_dir, kind_hint,
progress_cb, cancel_event)
_ensure_keys_bin(folder, _kind)
return folder
except TransferCancelled:
raise
except Exception as exc: # noqa: BLE001
py_exc = exc
log.warning("python pkg decrypt failed (%s); trying pkg2zip fallback", exc)
# 2. pkg2zip fallback (needs a binary; usually Wine + the bundled .exe).
if pkg2zip is None:
pkg2zip = find_pkg2zip()
if pkg2zip is None:
raise RuntimeError(
f"could not decrypt {pkg_path.name} (no pkg2zip available): {py_exc}"
)
folder_name = _safe_folder(title_id)
tmp = Path(dest_dir).with_name(".nps-tmp-" + folder_name)
if tmp.exists():
shutil.rmtree(tmp, ignore_errors=True)
tmp.mkdir(parents=True, exist_ok=True)
if progress_cb is not None:
progress_cb("Extracting", 0.0, "")
try:
cmd = _pkg2zip_cmd(Path(pkg2zip)) + ["-x", str(pkg_path)]
env = dict(os.environ)
env["WINEDEBUG"] = "-all" # silence Wine's console chatter
proc = subprocess.Popen(
cmd,
cwd=str(tmp), stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
text=True, bufsize=1, env=env,
)
assert proc.stdout is not None
for _line in proc.stdout:
if cancel_event is not None and cancel_event.is_set():
proc.kill()
raise TransferCancelled()
proc.wait()
if proc.returncode != 0:
raise RuntimeError(f"pkg2zip failed (exit {proc.returncode}) for {pkg_path.name}")
finally:
if cancel_event is not None and cancel_event.is_set():
shutil.rmtree(tmp, ignore_errors=True)
raise TransferCancelled()
eboots = list(tmp.rglob("EBOOT.PBP"))
if not eboots:
shutil.rmtree(tmp, ignore_errors=True)
raise RuntimeError(f"pkg2zip produced no EBOOT.PBP from {pkg_path.name}")
src_folder = eboots[0].parent
dest.mkdir(parents=True, exist_ok=True)
for p in sorted(src_folder.rglob("*")):
if not p.is_file():
continue
rel = p.relative_to(src_folder)
target = dest / rel
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(p, target)
if progress_cb is not None:
progress_cb("Extracting", 1.0, "")
shutil.rmtree(tmp, ignore_errors=True)
return dest
raise RuntimeError(f"could not decrypt {pkg_path.name}: {exc}") from exc
+1 -1
View File
@@ -155,7 +155,7 @@ def scrape(limit=None, workers=8, rate=3.0):
log.info("downloading OpenVGDB dump (%s)…", OVGDB_ZIP_URL)
tmpdir = Path(tempfile.mkdtemp(prefix="openvgdb-"))
dump_zip = tmpdir / "openvgdb.zip"
req = urllib.request.Request(OVGDB_ZIP_URL, headers={"User-Agent": "psp-preserve/0.1"})
req = urllib.request.Request(OVGDB_ZIP_URL, headers={"User-Agent": "psppt/0.1"})
with urllib.request.urlopen(req, timeout=60) as r, open(dump_zip, "wb") as f:
f.write(r.read())
+2 -2
View File
@@ -2,9 +2,9 @@
from . import iso_reader, matcher, namer
def identify(conn, record):
def identify(conn, record, prefer_region=None):
"""Return ``{method, release, serial, canonical}`` for a scanned disc record."""
method, release = matcher.match(conn, record)
method, release = matcher.match(conn, record, prefer_region=prefer_region)
return identify_with_release(record, release, method)
+151 -30
View File
@@ -1,10 +1,19 @@
"""Pure-Python PSP/PSX PSN ``.pkg`` decryption (no Wine / pkg2zip needed).
"""Pure-Python PSP/PSX/theme/DLC PSN ``.pkg`` decryption (no Wine / pkg2zip).
Ports the folder-mode extraction used by pkgi-psp's depackager and pkg2zip:
parse the ``\\x7FPKG`` header, derive the PSP AES key, AES-CTR-decrypt the file
entries, and write the game's ``EBOOT.PBP`` (+ siblings) into a folder. The
resulting ``EBOOT.PBP`` keeps its DATA.PSP EDAT-encrypted, so it runs on CFW
with the ``npdrm_free`` plugin (ARK ships it) — same as pkgi-psp's install.
parse the ``\\x7FPKG`` header, derive the PSP/PS3 AES key, AES-CTR-decrypt the
file entries, and write the game's ``EBOOT.PBP`` (+ siblings) into a folder.
Supported content:
* PSP games / minis (``EBOOT.PBP``) — ``DATA.PSP`` stays EDAT-encrypted, so it
runs on CFW with ``npdrm_free`` (ARK ships it), same as pkgi-psp's install.
* PS1 Classics (``EBOOT.PBP`` + ``DOCUMENT.DAT``).
* PSP themes (``.ptf``) — the packaged PTF is EDAT-wrapped and is unwrapped here.
* PSP DLC (``.edat``) — extracted as-is into the owning game's folder.
Theme unwrapping needs ``pycryptodome`` (for AES-CMAC + ECB); everything else
works with either pycryptodome or the built-in pure-Python AES fallback.
"""
from __future__ import annotations
@@ -16,6 +25,14 @@ PKG_MAGIC = 0x7F504B47
PSP_KEY = bytes.fromhex("07f2c68290b50d2c33818d709b60e62b")
PS3_KEY = bytes.fromhex("2e7b71d7c9c9a14ea3221f188828b8f8")
# PSP KIRK/AMCTL keys used to unwrap theme PTF EDATs (pkgi-psp / libkirk).
_KIRK38 = bytes.fromhex("12468d7e1c42209bba5426835eb03303")
_KIRK39 = bytes.fromhex("c43bb6d653ee67493ea95fbc0ced6f8a")
_KIRK63 = bytes.fromhex("9c9b1372f8c640cf1c62f5d592ddb582")
_AMCTL3 = bytes.fromhex("e350ed1d910a1fd029bb1c3ef34077fb")
_AMCTL4 = bytes.fromhex("135fa47cab395ba476b8cca98f3a0445")
_AMCTL5 = bytes.fromhex("678d7fa32a9ca0d1508ad8385e4b017e")
try:
from Crypto.Cipher import AES as _AES # pycryptodome (fast)
_HAVE_CRYPTO = True
@@ -136,13 +153,86 @@ def _make_ctr_decryptor(key, iv, block_index):
return decrypt
# --------------------------------------------------------------------------- #
# PSP theme PTF EDAT unwrapping (pkgi-psp's unpack_psp_edat)
# --------------------------------------------------------------------------- #
def _need_crypto():
if not _HAVE_CRYPTO:
raise RuntimeError("theme extraction needs pycryptodome (pip install pycryptodome)")
def _ecb_decrypt(key, block):
_need_crypto()
return _AES.new(key, _AES.MODE_ECB).decrypt(block)
def _cmac(key, data):
_need_crypto()
from Crypto.Hash import CMAC
c = CMAC.new(key, ciphermod=_AES)
c.update(data)
return c.digest()
def _kirk_iv(mac, key_header, offset2):
tmp = _ecb_decrypt(_KIRK38, bytes(key_header[0x70:0x80]))
iv = bytes(mac[i] ^ tmp[i] ^ key_header[offset2 + i] ^ _AMCTL3[i] ^ _AMCTL5[i] for i in range(16))
iv = _ecb_decrypt(_KIRK39, iv)
return bytes(b ^ _AMCTL4[i] for i, b in enumerate(iv))
def _kirk_cbc_decrypt(iv, index, data):
prev = bytearray(16) if index == 0 else bytearray(iv[:12]) + index.to_bytes(4, "little")
block = bytearray(iv[:12]) + index.to_bytes(4, "little")
out = bytearray()
for i in range(0, len(data), 16):
v = int.from_bytes(block[12:16], "little") + 1
block[12:16] = v.to_bytes(4, "little")
o = _ecb_decrypt(_KIRK63, bytes(block))
out += bytes(b ^ prev[k] ^ o[k] for k, b in enumerate(data[i:i + 16]))
prev = bytes(block)
return bytes(out)
def _unwrap_theme_ptf(buf):
"""Unwrap a PSP theme PTF EDAT (``buf`` is the fully CTR-decrypted entry)."""
if len(buf) < 0x90 + 0xA0:
raise RuntimeError("theme EDAT is too short")
kh_off = buf[0xC]
key_header = bytearray(buf[kh_off:kh_off + 0xA0])
if bytes(key_header[:4]) != b"\x00PGD":
raise RuntimeError("theme EDAT: bad PGD header")
if int.from_bytes(key_header[4:8], "little") != 1 or int.from_bytes(key_header[8:12], "little") != 1:
raise RuntimeError("theme EDAT: unsupported key/drm type")
mac = _cmac(_KIRK38, bytes(key_header[:0x70]))
# Step 1: reveal the data size/offset fields (bytes 0x30..0x60).
key_header[0x30:0x60] = _kirk_cbc_decrypt(_kirk_iv(mac, key_header, 0x10), 0, bytes(key_header[0x30:0x60]))
data_size = int.from_bytes(key_header[0x44:0x48], "little")
data_offset = int.from_bytes(key_header[0x4c:0x50], "little")
if data_offset != 0x90:
raise RuntimeError("theme EDAT: unexpected data offset")
# Step 2: decrypt the PTF body.
iv = _kirk_iv(mac, key_header, 0x30)
out = bytearray()
nblocks = (data_size + 15) // 16
for i in range(nblocks):
chunk = buf[kh_off + data_offset + i * 16:kh_off + data_offset + (i + 1) * 16]
out += _kirk_cbc_decrypt(iv, i, chunk)
return bytes(out[:data_size])
# --------------------------------------------------------------------------- #
# PKG extraction
# --------------------------------------------------------------------------- #
def extract(pkg_path, dest_dir, progress_cb=None, cancel_event=None):
"""Decrypt a PSP/PSX PSN pkg into ``dest_dir/<TITLE_ID>/``.
def extract(pkg_path, dest_dir, kind_hint=None, progress_cb=None, cancel_event=None):
"""Decrypt a PSN pkg into ``dest_dir/<TITLE_ID>/``.
Returns ``(kind, title_id, folder)`` where kind is ``"psp"`` or ``"psx"``.
Returns ``(kind, title_id, folder)`` where kind is ``"psp"``, ``"psx"``,
``"theme"`` or ``"dlc"``. ``kind_hint`` ("theme"/"dlc") overrides the
pkg-header content-type detection (NoPayStation DLC ships as two formats).
"""
from .store import TransferCancelled
@@ -179,19 +269,30 @@ def extract(pkg_path, dest_dir, progress_cb=None, cancel_event=None):
items_offset = int.from_bytes(m[8:12], "big")
off += 8 + msize
if content_type == 6:
if kind_hint in ("theme", "dlc"):
kind = kind_hint
elif content_type == 6:
kind = "psx"
elif content_type in (7, 0xE, 0xF, 0x10):
kind = "psp"
elif content_type == 9:
kind = "theme"
elif content_type == 0xB:
kind = "dlc"
else:
raise RuntimeError(f"unsupported NoPayStation content type 0x{content_type:x}")
if key_type != 1:
# NoPayStation DLC is wrapped in a PS3-format pkg (key type 4 in the header
# but actually encrypted with the PS3 key); everything else uses the PSP key.
if content_type == 0xB:
main_key = PS3_KEY
ps3_key = PS3_KEY
elif key_type == 1:
main_key = PSP_KEY
ps3_key = PS3_KEY
else:
raise RuntimeError(f"unsupported PKG key type {key_type}")
main_key = PSP_KEY
ps3_key = PS3_KEY
# Decrypt the whole item table in one shot.
with open(pkg_path, "rb") as fh:
fh.seek(enc_offset + items_offset)
@@ -232,26 +333,46 @@ def extract(pkg_path, dest_dir, progress_cb=None, cancel_event=None):
continue
if kind == "psp" and base not in ("EBOOT.PBP", "PSP-KEY.EDAT", "CONTENT.DAT"):
continue
if kind == "theme" and not base.lower().endswith(".ptf"):
continue
if kind == "dlc" and not base.lower().endswith(".edat"):
continue
if kind == "dlc" and base.upper() == "ISO.BIN.EDAT":
# Small EDAT wrapper shipped inside PSP DLC packages — not DLC
# content itself, so don't install it (matches pkgi-psp).
continue
dest = folder / base
decrypt = _make_ctr_decryptor(item_key, iv, data_offset // 16)
fh.seek(enc_offset + data_offset)
with open(dest, "wb") as out:
remaining = data_size
while remaining > 0:
if cancel_event is not None and cancel_event.is_set():
raise TransferCancelled()
chunk = fh.read(min(1 << 20, remaining))
if not chunk:
break
out.write(decrypt(chunk))
remaining -= len(chunk)
done += len(chunk)
if progress_cb is not None:
progress_cb("Extracting", min(done / total, 1.0), "")
if kind == "theme":
# Read the (small) entry fully, CTR-decrypt, then unwrap the EDAT.
fh.seek(enc_offset + data_offset)
raw = _ctr_decrypt(item_key, iv, data_offset // 16, fh.read(data_size))
dest = folder / base
dest.write_bytes(_unwrap_theme_ptf(raw))
done += data_size
else:
dest = folder / base
decrypt = _make_ctr_decryptor(item_key, iv, data_offset // 16)
fh.seek(enc_offset + data_offset)
with open(dest, "wb") as out:
remaining = data_size
while remaining > 0:
if cancel_event is not None and cancel_event.is_set():
raise TransferCancelled()
chunk = fh.read(min(1 << 20, remaining))
if not chunk:
break
out.write(decrypt(chunk))
remaining -= len(chunk)
done += len(chunk)
if progress_cb is not None:
progress_cb("Extracting", min(done / total, 1.0), "")
written.append(dest)
if not (folder / "EBOOT.PBP").exists():
if kind in ("psp", "psx") and not (folder / "EBOOT.PBP").exists():
raise RuntimeError(f"no EBOOT.PBP found in {pkg_path.name}")
if kind == "theme" and not any(p.suffix.lower() == ".ptf" for p in written):
raise RuntimeError(f"no PTF found in {pkg_path.name}")
if kind == "dlc" and not written:
raise RuntimeError(f"no DLC EDAT found in {pkg_path.name}")
return kind, title_id, folder
+4 -1
View File
@@ -40,6 +40,7 @@ import io
import os
import re
import struct
import sys
import zlib
from dataclasses import dataclass, field
from pathlib import Path
@@ -88,7 +89,9 @@ SYNC_PATTERN = b"\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00"
# (ARK/PRO/ME) loads this ELF directly as the DATA.PSP section, sidestepping
# the base PBP's encrypted ~PSP loader which frequently fails to link on
# 6.60/6.61 (error 80020148 "game could not be started").
_LOADER_DIR = Path(__file__).resolve().parent.parent / "data" / "loader"
_BUNDLE_DIR = getattr(sys, "_MEIPASS", None)
_APP_ROOT = Path(_BUNDLE_DIR).resolve() if _BUNDLE_DIR else Path(__file__).resolve().parent.parent
_LOADER_DIR = _APP_ROOT / "data" / "loader"
BUNDLED_DATA_PSP = _LOADER_DIR / "DATA.PSP"
BUNDLED_KEYS_BIN = _LOADER_DIR / "KEYS.BIN"
+43 -1
View File
@@ -221,8 +221,48 @@ def _pkg_items(root):
return items
def _theme_items(root):
"""Installed themes (``PSP/THEME/*.ptf``), named from the PTF header."""
items = []
theme_dir = root / "PSP" / "THEME"
if theme_dir.is_dir():
try:
entries = sorted(theme_dir.iterdir())
except OSError:
return items
for path in entries:
if not path.is_file() or path.suffix.lower() != ".ptf":
continue
item = _file_item(path, "theme")
try:
from . import theme as theme_mod
name = theme_mod.ptf_name(path)
except Exception: # noqa: BLE001
name = ""
if name:
item["name"] = name
items.append(item)
return items
def _dlc_items(root):
"""Installed DLC (``PSP/GAME/<GAME_ID>/*.edat``)."""
items = []
game_dir = root / "PSP" / "GAME"
if game_dir.is_dir():
for path in sorted(game_dir.rglob("*")):
if not path.is_file() or path.suffix.lower() != ".edat":
continue
if path.name.upper() == "ISO.BIN.EDAT":
continue # package wrapper, not DLC content
item = _file_item(path, "dlc")
item["id"] = path.parent.name
items.append(item)
return items
def list_installed(root):
"""Return ``{games, videos, homebrew, pkg}`` inventories for a PSP root."""
"""Return ``{games, videos, homebrew, pkg, themes, dlc}`` for a PSP root."""
root = Path(root)
games, videos, homebrew_isos = _iso_items(root)
return {
@@ -230,6 +270,8 @@ def list_installed(root):
"videos": videos,
"homebrew": _homebrew_items(root) + homebrew_isos,
"pkg": _pkg_items(root),
"themes": _theme_items(root),
"dlc": _dlc_items(root),
}
+39 -36
View File
@@ -79,13 +79,13 @@ class TransferQueue(QObject):
job_finished = Signal(dict)
job_removed = Signal(int)
def __init__(self, parent=None):
def __init__(self, parent=None, max_concurrent=3):
super().__init__(parent)
self.max_concurrent = max(1, int(max_concurrent))
self.jobs = []
self._pending = []
self._current = None
self._thread = None
self._worker = None
# job_id -> (QThread, JobWorker) for jobs currently running in parallel.
self._running = {}
# Keep strong references to every worker/thread for their full lifetime.
# Dropping a Python ref to a QObject that was moved to another thread
# deletes it directly and crashes (bus error).
@@ -115,8 +115,10 @@ class TransferQueue(QObject):
self.job_updated.emit(job.to_dict())
self.job_finished.emit(job.to_dict())
return
if job.status == "running" and self._worker is not None:
self._worker.cancel()
if job.status == "running":
pair = self._running.get(job_id)
if pair is not None:
pair[1].cancel()
job.status = "cancelling"
self.job_updated.emit(job.to_dict())
@@ -140,8 +142,8 @@ class TransferQueue(QObject):
self.remove(jid)
def shutdown(self):
if self._worker is not None:
self._worker.cancel()
for _t, worker in self._running.values():
worker.cancel()
for t in self._threads:
try:
if t.isRunning():
@@ -157,30 +159,29 @@ class TransferQueue(QObject):
return None
def _start_next(self):
if self._current is not None or not self._pending:
return
job = self._pending.pop(0)
self._current = job
job.status = "running"
log.info("starting job #%d: %s (%s)", job.id, job.title, job.kind)
self.job_updated.emit(job.to_dict())
while len(self._running) < self.max_concurrent and self._pending:
job = self._pending.pop(0)
job.status = "running"
log.info("starting job #%d: %s (%s)", job.id, job.title, job.kind)
self.job_updated.emit(job.to_dict())
self._thread = QThread(self)
self._worker = JobWorker(job.source, job.dest_dir, job)
self._worker.moveToThread(self._thread)
self._thread.started.connect(self._worker.run)
self._worker.progress.connect(self._on_progress)
self._worker.finished.connect(self._on_finished)
self._worker.skipped.connect(self._on_skipped)
self._worker.cancelled.connect(self._on_cancelled)
self._worker.failed.connect(self._on_failed)
for sig in (self._worker.finished, self._worker.skipped,
self._worker.cancelled, self._worker.failed):
sig.connect(self._thread.quit)
self._thread.finished.connect(self._on_thread_finished)
self._threads.append(self._thread)
self._workers.append(self._worker)
self._thread.start()
thread = QThread(self)
worker = JobWorker(job.source, job.dest_dir, job)
worker.moveToThread(thread)
thread.started.connect(worker.run)
worker.progress.connect(self._on_progress)
worker.finished.connect(self._on_finished)
worker.skipped.connect(self._on_skipped)
worker.cancelled.connect(self._on_cancelled)
worker.failed.connect(self._on_failed)
for sig in (worker.finished, worker.skipped,
worker.cancelled, worker.failed):
sig.connect(thread.quit)
thread.finished.connect(self._on_thread_finished)
self._running[job.id] = (thread, worker)
self._threads.append(thread)
self._workers.append(worker)
thread.start()
def _on_progress(self, job_id, stage, fraction, detail):
job = self._find(job_id)
@@ -218,10 +219,12 @@ class TransferQueue(QObject):
self.job_finished.emit(job.to_dict())
def _on_thread_finished(self):
# Queued to the main thread once the worker thread has fully stopped,
# so it is now safe to drop the "current" pointers and advance.
self._current = None
self._thread = None
self._worker = None
# Queued to the main thread once a worker thread has fully stopped, so
# it is now safe to drop its pointers and advance the queue.
thread = self.sender()
for jid, pair in list(self._running.items()):
if pair[0] is thread:
del self._running[jid]
break
self._start_next()
+112
View File
@@ -0,0 +1,112 @@
"""Client-side HTTP access to the psppt server (auth + content fetches).
The server is authoritative for covers and metadata. When a server is
configured (URL + per-user token), the cover/metadata paths use it and show an
error rather than falling back to libretro/IGDB/etc.
"""
import json
import logging
import urllib.error
import urllib.parse
import urllib.request
from . import config
log = logging.getLogger("server_client")
_CFG_FILE = config.STATE_DIR / "server.json"
_cache = {"loaded": False, "cfg": {}}
def _load():
if not _cache["loaded"]:
cfg = {}
try:
cfg = json.loads(_CFG_FILE.read_text(encoding="utf-8"))
except (OSError, ValueError):
cfg = {}
_cache["cfg"] = cfg
_cache["loaded"] = True
return _cache["cfg"]
def _save(cfg):
_CFG_FILE.parent.mkdir(parents=True, exist_ok=True)
_CFG_FILE.write_text(json.dumps(cfg, indent=2), encoding="utf-8")
_cache["cfg"] = cfg
_cache["loaded"] = True
def configured():
cfg = _load()
return bool(cfg.get("url") and cfg.get("token"))
def url():
return (_load().get("url") or config.STORE_SERVER_URL or "").rstrip("/")
def token():
return _load().get("token") or config.STORE_TOKEN or ""
def username():
return _load().get("username") or ""
def logout():
_save({"url": url(), "token": "", "username": ""})
def login(server_url, user, password):
"""Authenticate against the server and store the per-user token."""
base = (server_url or "").rstrip("/")
req = urllib.request.Request(
f"{base}/api/auth/login",
data=json.dumps({"username": user, "password": password}).encode("utf-8"),
headers={"Content-Type": "application/json", "User-Agent": "psppt/0.1"},
method="POST",
)
with urllib.request.urlopen(req, timeout=15) as resp:
data = json.loads(resp.read())
_save({"url": base, "token": data.get("token") or "", "username": data.get("username") or user})
return data
def _get(path, timeout=15):
req = urllib.request.Request(
f"{url()}{path}",
headers={"Authorization": f"Bearer {token()}", "User-Agent": "psppt/0.1"},
)
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.read()
def _lookup_id(system, name):
"""Return the server release id for a name (first match), or None."""
q = urllib.parse.quote(name or "")
data = json.loads(_get(f"/api/releases?system={system}&q={q}&limit=1", timeout=15))
items = data.get("items") or []
return items[0]["id"] if items else None
def fetch_cover(system, name):
"""Return cover bytes for a release, or None (no cover on the server)."""
rid = _lookup_id(system, name)
if rid is None:
return None
try:
return _get(f"/api/cover/{rid}", timeout=30)
except urllib.error.HTTPError as exc:
if exc.code == 404:
return None
raise
def fetch_metadata(system, name):
"""Return metadata dict for a release, or None."""
rid = _lookup_id(system, name)
if rid is None:
return None
detail = json.loads(_get(f"/api/releases/{rid}", timeout=15))
return detail.get("meta") or {}
+1 -1
View File
@@ -11,7 +11,7 @@ from pathlib import Path
from . import config
CREDENTIALS_PATH = Path(
os.environ.get("PSP_CREDENTIALS", str(config.DATA_DIR / "credentials.json"))
os.environ.get("PSP_CREDENTIALS", str(config.USER_DATA_DIR / "credentials.json"))
)
FIELDS = [
+316
View File
@@ -0,0 +1,316 @@
"""Global dark theme for the GUI (Fusion style + palette + stylesheet).
Applied once at startup from ``gui.main``. The per-widget ``setStyleSheet``
colors sprinkled through ``gui.py`` (``#999`` secondary text, ``#e06c6c`` delete
buttons, ``#14151a`` preview panels) were written for a dark background, so a
single cohesive dark theme here makes the whole UI consistent.
"""
from PySide6.QtGui import QColor, QPalette
from . import config
# Design tokens
BG = "#1e1f24" # window background
SURFACE = "#26272e" # cards, inputs, table headers
RAISED = "#2d2e36" # buttons, combos, menus
BORDER = "#3a3b44"
BORDER_SOFT = "#33343d"
TEXT = "#e6e6ea"
TEXT_DIM = "#9a9ba5"
TEXT_FAINT = "#6f7079"
ACCENT = "#4a9eff"
ACCENT_HOVER = "#6bb0ff"
_STYLESHEET = f"""
QMainWindow {{
background: {BG};
}}
QWidget {{
color: {TEXT};
}}
/* --- Tabs --- */
QTabWidget::pane {{
border: 1px solid {BORDER};
border-radius: 6px;
top: -1px;
}}
QTabBar::tab {{
background: transparent;
color: {TEXT_DIM};
padding: 8px 18px;
border: none;
border-bottom: 2px solid transparent;
margin-right: 2px;
}}
QTabBar::tab:hover:!selected {{
color: {TEXT};
}}
QTabBar::tab:selected {{
color: {TEXT};
border-bottom: 2px solid {ACCENT};
}}
/* --- Buttons --- */
QPushButton {{
background: {RAISED};
border: 1px solid {BORDER};
border-radius: 6px;
padding: 6px 12px;
}}
QPushButton:hover {{
background: #343640;
border-color: {ACCENT_HOVER};
}}
QPushButton:pressed {{
background: {SURFACE};
}}
QPushButton:disabled {{
color: {TEXT_FAINT};
background: {SURFACE};
border-color: {BORDER_SOFT};
}}
QPushButton:checked {{
background: {ACCENT};
border-color: {ACCENT};
color: #101116;
}}
/* --- Inputs --- */
QLineEdit, QSpinBox, QPlainTextEdit, QTextEdit {{
background: {SURFACE};
border: 1px solid {BORDER};
border-radius: 6px;
padding: 5px 8px;
selection-background-color: {ACCENT};
selection-color: #101116;
}}
QLineEdit:focus, QSpinBox:focus, QPlainTextEdit:focus, QTextEdit:focus {{
border-color: {ACCENT};
}}
QLineEdit:disabled, QSpinBox:disabled {{
color: {TEXT_FAINT};
}}
/* --- Combo boxes --- */
QComboBox {{
background: {RAISED};
border: 1px solid {BORDER};
border-radius: 6px;
padding: 4px 10px;
}}
QComboBox:hover {{
border-color: {ACCENT_HOVER};
}}
QComboBox::drop-down {{
border: none;
width: 22px;
}}
QComboBox::down-arrow {{
image: url(__CHEVRON__);
width: 12px;
height: 8px;
margin-right: 6px;
}}
QComboBox QAbstractItemView {{
background: {RAISED};
border: 1px solid {BORDER};
border-radius: 6px;
selection-background-color: rgba(74, 158, 255, 0.25);
selection-color: {TEXT};
outline: 0;
}}
/* --- Tables --- */
QTableWidget {{
background: transparent;
border: none;
gridline-color: {BORDER_SOFT};
}}
QHeaderView::section {{
background: {SURFACE};
color: {TEXT_DIM};
border: none;
border-bottom: 1px solid {BORDER};
padding: 6px 8px;
}}
QTableWidget::item {{
padding: 4px 6px;
}}
QTableWidget::item:selected {{
background: rgba(74, 158, 255, 0.22);
color: {TEXT};
}}
QTableCornerButton::section {{
background: {SURFACE};
border: none;
border-bottom: 1px solid {BORDER};
}}
/* --- Cover grid / lists --- */
QListWidget {{
background: transparent;
border: none;
}}
QListWidget::item {{
border-radius: 6px;
}}
QListWidget::item:hover:!selected {{
background: rgba(255, 255, 255, 0.04);
}}
QListWidget::item:selected {{
background: rgba(74, 158, 255, 0.22);
}}
/* --- Cards (details panels use QScrollArea) --- */
QScrollArea {{
background: {SURFACE};
border: none;
}}
/* --- Menus --- */
QMenu {{
background: {RAISED};
border: 1px solid {BORDER};
border-radius: 6px;
padding: 4px;
}}
QMenu::item {{
padding: 6px 26px 6px 12px;
border-radius: 4px;
}}
QMenu::item:selected {{
background: rgba(74, 158, 255, 0.25);
}}
QMenu::separator {{
height: 1px;
background: {BORDER};
margin: 4px 8px;
}}
/* --- Scrollbars --- */
QScrollBar:vertical {{
background: transparent;
width: 10px;
margin: 0;
}}
QScrollBar::handle:vertical {{
background: {BORDER};
border-radius: 5px;
min-height: 30px;
}}
QScrollBar::handle:vertical:hover {{
background: {ACCENT};
}}
QScrollBar:horizontal {{
background: transparent;
height: 10px;
margin: 0;
}}
QScrollBar::handle:horizontal {{
background: {BORDER};
border-radius: 5px;
min-width: 30px;
}}
QScrollBar::handle:horizontal:hover {{
background: {ACCENT};
}}
QScrollBar::add-line, QScrollBar::sub-line {{
width: 0;
height: 0;
}}
QScrollBar::add-page, QScrollBar::sub-page {{
background: transparent;
}}
/* --- Splitter --- */
QSplitter::handle {{
background: transparent;
}}
QSplitter::handle:horizontal {{
width: 1px;
}}
QSplitter::handle:vertical {{
height: 1px;
}}
/* --- Misc --- */
QToolTip {{
background: {RAISED};
color: {TEXT};
border: 1px solid {BORDER};
padding: 4px 6px;
}}
QStatusBar {{
background: {SURFACE};
color: {TEXT_DIM};
border-top: 1px solid {BORDER};
}}
QProgressBar {{
background: {SURFACE};
border: 1px solid {BORDER};
border-radius: 6px;
text-align: center;
}}
QProgressBar::chunk {{
background: {ACCENT};
border-radius: 5px;
}}
QCheckBox {{
spacing: 8px;
}}
QCheckBox::indicator {{
width: 16px;
height: 16px;
border: 1px solid {BORDER};
border-radius: 4px;
background: {SURFACE};
}}
QCheckBox::indicator:checked {{
background: {ACCENT};
border-color: {ACCENT};
}}
QGroupBox {{
border: 1px solid {BORDER};
border-radius: 6px;
margin-top: 10px;
}}
QGroupBox::title {{
subcontrol-origin: margin;
left: 10px;
padding: 0 4px;
color: {TEXT_DIM};
}}
"""
def _dark_palette():
p = QPalette()
p.setColor(QPalette.Window, QColor(BG))
p.setColor(QPalette.WindowText, QColor(TEXT))
p.setColor(QPalette.Base, QColor(SURFACE))
p.setColor(QPalette.AlternateBase, QColor(RAISED))
p.setColor(QPalette.ToolTipBase, QColor(RAISED))
p.setColor(QPalette.ToolTipText, QColor(TEXT))
p.setColor(QPalette.Text, QColor(TEXT))
p.setColor(QPalette.Button, QColor(RAISED))
p.setColor(QPalette.ButtonText, QColor(TEXT))
p.setColor(QPalette.BrightText, QColor("#ff5c5c"))
p.setColor(QPalette.Link, QColor(ACCENT))
p.setColor(QPalette.Highlight, QColor(ACCENT))
p.setColor(QPalette.HighlightedText, QColor("#101116"))
p.setColor(QPalette.PlaceholderText, QColor(TEXT_FAINT))
faint = QColor(TEXT_FAINT)
for role in (QPalette.WindowText, QPalette.Text, QPalette.ButtonText):
p.setColor(QPalette.Disabled, role, faint)
return p
def apply_theme(app):
"""Apply the dark Fusion theme to ``app`` (call once before showing UI)."""
app.setStyle("Fusion")
app.setPalette(_dark_palette())
chevron = str(config.resource_path("img", "chevron.png"))
app.setStyleSheet(_STYLESHEET.replace("__CHEVRON__", chevron))
+511
View File
@@ -0,0 +1,511 @@
"""PSP theme (``.ptf``) introspection: name + preview image.
PTF themes embed a readable name in their header and a preview image (a GIM
bitmap) in object 0. This module decodes both without any external service:
* the name lives at offset 0x08 (128 UTF-8 bytes);
* the preview is object 0's largest image item, stored as a GIM that may be
Sony-LZR compressed.
GIM decoding covers the common RGBA formats (5650/5551/4444/8888) and 8-bit
palettes; anything exotic simply yields no preview.
"""
from __future__ import annotations
import zlib
from pathlib import Path
PTF_MAGIC = b"\x00PTF"
# --------------------------------------------------------------------------- #
# Sony LZR/RLZ decompression (sceLzrDecompress), ported from PTF-Studio.
# --------------------------------------------------------------------------- #
_LZR_PROBABILITY_SLOTS = 2800
_LZR_INITIAL_PROBABILITY = 0x80
_LZR_RENORM_LIMIT = 0x00FFFFFF
_LZR_U32_MOD = 0x100000000
def _lzr_decompress(inp, max_output, offset=0):
inp = bytes(inp)
end = offset + len(inp) - offset
out = bytearray(max_output)
probs = [_LZR_INITIAL_PROBABILITY] * _LZR_PROBABILITY_SLOTS
out_pos = 0
in_pos = offset + 5
code = int.from_bytes(inp[offset + 1:offset + 5], "big")
range_ = 0xFFFFFFFF
spec = 0
corrupt = False
buf_off = 0
last_byte = 0
last_number_flag = 0
type_ = inp[offset]
if type_ >= 0x80:
type_ -= 0x100
def next_byte():
nonlocal in_pos, corrupt
if in_pos >= end:
corrupt = True
return 0
b = inp[in_pos]
in_pos += 1
return b
if type_ > 7:
raise ValueError("unsupported LZR literal-context mode")
if type_ < 0:
stored = code
if stored > len(out) or in_pos + stored > end:
raise ValueError("malformed stored LZR block")
return inp[in_pos:in_pos + stored]
def renormalize():
nonlocal code, range_
if range_ <= _LZR_RENORM_LIMIT:
code = (code * 256 + next_byte()) % _LZR_U32_MOD
range_ = (range_ * 256) % _LZR_U32_MOD
def decode_bit(prob_index):
nonlocal code, range_, corrupt
if prob_index < 0 or prob_index >= _LZR_PROBABILITY_SLOTS:
corrupt = True
return 0
renormalize()
if corrupt:
return 0
p = probs[prob_index]
bound = (range_ // 256) * p
probs[prob_index] = p - (p // 8)
if code < bound:
range_ = bound
probs[prob_index] += 31
return 1
code -= bound
range_ -= bound
return 0
def decode_bit_speculative(prob_index):
nonlocal code, range_, spec, corrupt
if prob_index < 0 or prob_index >= _LZR_PROBABILITY_SLOTS:
corrupt = True
return 0
if spec <= _LZR_RENORM_LIMIT:
code = (code * 256 + next_byte()) % _LZR_U32_MOD
range_ = (spec * 256) % _LZR_U32_MOD
if corrupt:
return 0
p = probs[prob_index]
bound = (range_ // 256) * p
spec = bound
probs[prob_index] = p - (p // 8)
if code < bound:
range_ = bound
probs[prob_index] += 31
return 1
code -= bound
range_ -= bound
return 0
def decode_number(bits, base, step):
nonlocal code, range_, corrupt, last_number_flag
number = 1
if bits >= 3:
number = number * 2 + decode_bit(base + 3 * step)
if bits >= 4:
number = number * 2 + decode_bit(base + 3 * step)
if bits >= 5:
renormalize()
remaining = bits
while remaining >= 5:
if range_ == 0:
corrupt = True
return 0
number *= 2
range_ = range_ // 2
if code < range_:
number += 1
else:
code -= range_
remaining -= 1
last_number_flag = decode_bit(base)
number = number * 2 + last_number_flag
if bits >= 1:
number = number * 2 + decode_bit(base + step)
if bits >= 2:
number = number * 2 + decode_bit(base + 2 * step)
return number
def selector_context():
return buf_off + 2488
def decode_literal(shift):
nonlocal out_pos, last_byte, corrupt, buf_off
if buf_off > 0:
buf_off -= 1
if out_pos == len(out):
return False
context = ((((out_pos & 7) << 8) + last_byte) // (2 ** shift)) & 7
tree_base = context * 0xFF - 1
node = 1
while node <= 0xFF:
node = node * 2 + decode_bit(tree_base + node)
if corrupt:
return False
out[out_pos] = node & 0xFF
last_byte = node & 0xFF
out_pos += 1
return True
def decode_match():
nonlocal out_pos, last_byte, buf_off, spec, code, range_, corrupt, last_number_flag
slot = selector_context()
spec = range_
length_class = -1
bit = 0
while True:
slot += 8
bit = decode_bit_speculative(slot)
if corrupt:
return -1
length_class += bit
if bit == 0 or length_class >= 6:
break
offset_group = length_class + 2033
offset_bias = 64
if bit != 0 or length_class >= 0:
length_context = (length_class << 5) + (((out_pos * (2 ** length_class)) & 3) << 3) + buf_off + 2552
match_length = decode_number(length_class, length_context, 8)
if corrupt:
return -1
if match_length == 0xFF:
return 1
if last_number_flag != 0 or length_class > 0:
offset_group += 56
offset_bias = 352
else:
match_length = 1
index = 1
offset_class = 0
while True:
offset_class = (index << 4) - offset_bias
prob_index = offset_group + (index << 3)
if prob_index >= _LZR_PROBABILITY_SLOTS:
return -1
bit = decode_bit(prob_index)
if corrupt:
return -1
index = (index << 1) | bit
if offset_class >= 0:
break
if bit != 0 or offset_class > 0:
if bit == 0:
offset_class -= 8
base = offset_class + 2344
if base < 0 or base + 3 >= _LZR_PROBABILITY_SLOTS:
return -1
back_offset = decode_number(offset_class // 8, base, 1)
if corrupt:
return -1
else:
back_offset = 1
if back_offset > out_pos:
return -1
copy_end = out_pos + match_length + 1
if copy_end > len(out):
return -1
buf_off = ((copy_end + 1) & 1) + 6
from_ = out_pos - back_offset
while out_pos < copy_end:
out[out_pos] = out[from_]
out_pos += 1
from_ += 1
last_byte = out[out_pos - 1]
return 0
token_budget = max(1024, max_output * 2 + 1024)
while True:
token_budget -= 1
if token_budget < 0:
raise ValueError("malformed LZR stream did not terminate")
token_is_match = decode_bit(selector_context())
if corrupt:
raise ValueError("truncated LZR stream")
if token_is_match == 0:
if not decode_literal(type_):
raise ValueError("LZR stream exceeds declared output size")
else:
result = decode_match()
if result == 1:
return bytes(out[:out_pos])
if result < 0:
raise ValueError("malformed LZR back-reference")
# --------------------------------------------------------------------------- #
# PTF parsing
# --------------------------------------------------------------------------- #
def _decompress_item(blob, comp, ucsize):
if comp == 2 and len(blob) < ucsize:
return zlib.decompress(blob)
if comp == 1 and len(blob) < ucsize:
return _lzr_decompress(blob, ucsize)
return blob
def parse_ptf(data):
"""Return ``(name, items)`` where items are decompressed item blobs."""
if not isinstance(data, (bytes, bytearray)):
data = data.read_bytes() if hasattr(data, "read_bytes") else bytes(data)
data = bytes(data)
if data[:4] != PTF_MAGIC:
raise ValueError("not a PTF file")
raw_name = data[8:8 + 128]
name = raw_name.split(b"\x00", 1)[0].decode("utf-8", "replace").strip()
def u16(o):
return int.from_bytes(data[o:o + 2], "little")
def u32(o):
return int.from_bytes(data[o:o + 4], "little")
items = []
pos = 0x100
while True:
offset = u32(pos)
pos += 4
if offset == 0 or offset >= len(data):
break
count = u16(offset + 2)
data_off = u32(offset + 8)
p = data_off
for _ in range(count):
file_type = u16(p + 4)
comp = u16(p + 6)
size = u32(p + 8)
ucsize = u32(p + 12)
blob = data[p + 0x20:p + 0x20 + size]
items.append((file_type, _decompress_item(blob, comp, ucsize)))
p += 0x20 + size
return name, items
def ptf_name(path):
"""Theme display name embedded in a ``.ptf`` file."""
try:
with open(path, "rb") as fh:
head = fh.read(8 + 128)
except OSError:
return ""
if head[:4] != PTF_MAGIC:
return ""
name = head[8:8 + 128].split(b"\x00", 1)[0].decode("utf-8", "replace").strip()
return name or Path(path).stem
# --------------------------------------------------------------------------- #
# GIM decoding (RGBA formats only)
# --------------------------------------------------------------------------- #
def _overscan(value, tile):
return value if value % tile == 0 else value + (tile - (value % tile))
def _untile(pixels, w, h, tile_w, tile_h):
over_w = _overscan(w, tile_w)
over_h = _overscan(h, tile_h)
out = [0] * (w * h)
tile_pos = 0
origin_x = 0
origin_y = 0
for y in range(over_h):
for x in range(over_w):
dx = origin_x + (tile_pos % tile_w)
dy = origin_y + (tile_pos // tile_w)
if dx < w and dy < h:
out[dy * w + dx] = pixels[y * over_w + x]
tile_pos += 1
if tile_pos == tile_w * tile_h:
tile_pos = 0
origin_x += tile_w
if origin_x >= over_w:
origin_x = 0
origin_y += tile_h
return out
def _gim_to_rgba(data):
"""Decode a GIM to ``(width, height, rgba_bytes)`` or None if unsupported."""
data = bytes(data)
if len(data) < 0x10:
return None
# Detect byte order from the magic. Sony GIMs store the magic as the byte
# string "MIG.00.1PSP" with little-endian data (LibPSPThemes semantics).
if data[:4] == b"MIG.":
byteorder = "little"
elif data[:4] == b".GIM":
byteorder = "big"
else:
return None
def rd(off, size):
return int.from_bytes(data[off:off + size], byteorder)
image = None
palette = None
pos = 0x10
for _ in range(32):
if pos + 16 > len(data):
break
btype = rd(pos, 2)
next_block = rd(pos + 8, 4)
if btype == 4 or btype == 5:
h = pos + 0x10
fmt = rd(h + 4, 2)
pixel_order = rd(h + 6, 2)
width = rd(h + 8, 2)
height = rd(h + 10, 2)
bpp = rd(h + 12, 2)
next_index = rd(h + 24, 4)
frame_data_end = rd(h + 32, 4)
frame_offset = rd(pos + 0x10 + next_index, 4)
block_content = pos + 0x10
start = block_content + frame_offset
end = block_content + frame_data_end
raw = data[start:end]
pixels = _decode_pixels(raw, fmt, bpp, width, height, pixel_order)
if pixels is None:
return None
if btype == 4:
image = (width, height, pixels)
else:
palette = (fmt, width, height, pixels)
if next_block == 0:
break
pos += next_block
if image is None:
return None
width, height, pixels = image
if palette is not None:
_, pw, ph, pvals = palette
if pw * ph > 0:
pixels = [pvals[p] if p < len(pvals) else 0 for p in pixels]
rgba = bytearray(width * height * 4)
for i, v in enumerate(pixels):
# pixels are packed 0xAABBGGRR
o = i * 4
rgba[o] = v & 0xFF
rgba[o + 1] = (v >> 8) & 0xFF
rgba[o + 2] = (v >> 16) & 0xFF
rgba[o + 3] = (v >> 24) & 0xFF
return width, height, bytes(rgba)
def _decode_pixels(raw, fmt, bpp, width, height, pixel_order):
"""Decode a GIM frame into a list of packed 0xAABBGGRR ints."""
if fmt in (0, 1, 2, 3, 5):
pass
else:
return None # indexed-16 / DXT etc. not supported
if fmt == 5: # p8 indexed
bytes_per_pixel = 1
tile_w = 0x80 // max(bpp, 8)
else:
bytes_per_pixel = max(1, (bpp + 7) // 8)
tile_w = 0x80 // max(bpp, 1)
tile_h = 8
if pixel_order == 1:
w = _overscan(width, tile_w)
h = _overscan(height, tile_h)
else:
w, h = width, height
count = w * h
values = []
pos = 0
for _ in range(count):
v = int.from_bytes(raw[pos:pos + bytes_per_pixel], "little")
pos += bytes_per_pixel
values.append(_expand(fmt, bpp, v))
if pos >= len(raw):
# pad missing trailing pixels with transparent
values.extend([0] * (count - len(values)))
break
if pixel_order == 1:
values = _untile(values, width, height, tile_w, tile_h)
return values[:width * height]
def _expand(fmt, bpp, v):
"""Expand a raw GIM pixel value into packed 0xAABBGGRR."""
if fmt == 3: # rgba8888
r = v & 0xFF
g = (v >> 8) & 0xFF
b = (v >> 16) & 0xFF
a = (v >> 24) & 0xFF
return (a << 24) | (b << 16) | (g << 8) | r
if fmt == 0: # rgba5650
r = (v >> 11) & 0x1F
g = (v >> 5) & 0x3F
b = v & 0x1F
r = (r << 3) | (r >> 2)
g = (g << 2) | (g >> 4)
b = (b << 3) | (b >> 2)
return (0xFF << 24) | (b << 16) | (g << 8) | r
if fmt == 1: # rgba5551
r = (v >> 11) & 0x1F
g = (v >> 6) & 0x1F
b = (v >> 1) & 0x1F
a = 0xFF if v & 1 else 0
r = (r << 3) | (r >> 2)
g = (g << 3) | (g >> 2)
b = (b << 3) | (b >> 2)
return (a << 24) | (b << 16) | (g << 8) | r
if fmt == 2: # rgba4444
r = (v >> 12) & 0xF
g = (v >> 8) & 0xF
b = (v >> 4) & 0xF
a = v & 0xF
r = (r << 4) | r
g = (g << 4) | g
b = (b << 4) | b
a = (a << 4) | a
return (a << 24) | (b << 16) | (g << 8) | r
if fmt == 5: # p8 — value is a palette index (expanded later)
return v & 0xFF
return 0
# --------------------------------------------------------------------------- #
# Public API
# --------------------------------------------------------------------------- #
def theme_preview_rgba(ptf_path):
"""Decode the preview image of a theme PTF.
Returns ``(width, height, rgba_bytes)`` or None if it can't be decoded.
"""
try:
data = Path(ptf_path).read_bytes()
_, items = parse_ptf(data)
except (OSError, ValueError):
return None
# The preview image is the largest decodable GIM item.
best = None
for file_type, blob in items:
if file_type != 5:
continue
decoded = _gim_to_rgba(blob)
if decoded is None:
continue
w, h, rgba = decoded
if best is None or w * h > best[0] * best[1]:
best = (w, h, rgba)
return best
+71 -18
View File
@@ -175,6 +175,33 @@ class Source:
raise NotImplementedError
class CoversBundleSource:
"""Download the combined offline cover bundle (runs as a queue job)."""
label = "psppt-covers.zip"
def __init__(self, url=""):
self.url = url
def download(self, item, dest_dir, progress_cb=None, cancel_event=None):
from . import covers
def on_progress(done, total):
if progress_cb is None:
return
frac = (done / total) if total else 0.0
detail = (f"{done / (1 << 20):.0f} / {total / (1 << 20):.0f} MB"
if total else "")
progress_cb("Downloading", frac, detail)
try:
result = covers.download_covers_bundle(self.url,
progress_cb=on_progress,
cancel_event=cancel_event)
except InterruptedError:
raise TransferCancelled() from None
return str(result)
class FileCopySource(Source):
"""Copy a single local file (library -> PSP or PSP -> library).
@@ -279,12 +306,13 @@ class RemoteStoreSource(Source):
label = "Remote store"
def __init__(self, base_url, token=""):
def __init__(self, base_url, token="", system="games"):
self.base_url = (base_url or "").rstrip("/")
self.token = token or ""
self.system = system or "games"
def _headers(self):
headers = {"User-Agent": "psp-preserve/0.1"}
headers = {"User-Agent": "psppt/0.1"}
if self.token:
headers["Authorization"] = f"Bearer {self.token}"
return headers
@@ -292,10 +320,13 @@ class RemoteStoreSource(Source):
def scan(self, root=None):
if not self.base_url:
return []
req = urllib.request.Request(f"{self.base_url}/api/store", headers=self._headers())
req = urllib.request.Request(f"{self.base_url}/api/store?system={self.system}", headers=self._headers())
with urllib.request.urlopen(req, timeout=30) as resp:
data = json.loads(resp.read())
return data.get("items", []) if isinstance(data, dict) else []
items = data.get("items", []) if isinstance(data, dict) else []
for it in items:
it["system"] = it.get("system") or self.system
return items
def download(self, item, dest_dir, progress_cb=None, cancel_event=None):
dest_dir = Path(dest_dir)
@@ -303,10 +334,11 @@ class RemoteStoreSource(Source):
# The listing's "name" has no extension; "filename" carries the real
# archive filename (e.g. "Game (USA).7z") used for the download URL.
name = item.get("filename") or item["name"]
system = item.get("system") or self.system
tmp = Path(tempfile.mkdtemp(prefix="psp-store-"))
archive = tmp / name
try:
url = f"{self.base_url}/api/store/download/{urllib.parse.quote(name, safe='')}"
url = f"{self.base_url}/api/store/download/{system}/{urllib.parse.quote(name, safe='')}"
download_url(url, archive, self._headers(), progress_cb, cancel_event)
iso_name = peek_archive(archive, cancel_event)
if not iso_name:
@@ -347,7 +379,7 @@ class ArchiveOrgSource(Source):
return f"https://archive.org/download/{self.item}"
def _headers(self):
headers = {"User-Agent": "psp-preserve/0.1"}
headers = {"User-Agent": "psppt/0.1"}
cookie = "; ".join(
p for p in (
f"logged-in-sig={self.sig}" if self.sig else "",
@@ -468,32 +500,40 @@ class LocalShareSource(Source):
class NoPayStationSource(Source):
"""Browse NoPayStation .pkg lists and install PSP/PSX PSN content.
``content_type`` is ``"psp"`` (PSP_GAMES.tsv) or ``"psx"`` (PSX_GAMES.tsv).
Downloading decrypts the ``.pkg`` with the bundled pure-Python decryptor
and writes a ``<TITLE_ID>/EBOOT.PBP`` game folder into ``dest_dir``.
``content_type`` is ``"psp"`` (PSP_GAMES.tsv), ``"psx"`` (PSX_GAMES.tsv),
``"theme"`` (PSP_THEMES.tsv) or ``"dlc"`` (PSP_DLCS.tsv). Downloading
decrypts the ``.pkg`` with the bundled pure-Python decryptor and writes a
``<TITLE_ID>/`` game folder into ``dest_dir``.
"""
label = "NoPayStation"
def __init__(self, content_type, tsv_url="", pkg2zip_path=""):
def __init__(self, content_type, tsv_url="", force=False):
from . import nopaystation
self.content_type = content_type
self.tsv_url = tsv_url or nopaystation.tsv_url(content_type)
self.pkg2zip = pkg2zip_path or ""
self.tsv_url = tsv_url or ("" if content_type == "all" else nopaystation.tsv_url(content_type))
self.force = force
def scan(self, root=None):
from . import nopaystation
return nopaystation.fetch_items(self.content_type, self.tsv_url)
if self.content_type == "all":
items = []
for ct in ("psp", "psx", "theme", "dlc"):
try:
items.extend(nopaystation.fetch_items(ct, force=self.force))
except Exception: # noqa: BLE001 - skip a source that fails
log.warning("could not fetch NoPayStation %s list", ct)
return items
return nopaystation.fetch_items(self.content_type, self.tsv_url, force=self.force)
def download(self, item, dest_dir, progress_cb=None, cancel_event=None):
from . import nopaystation
dest_dir = Path(dest_dir)
dest_dir.mkdir(parents=True, exist_ok=True)
# The bundled Python decryptor handles PSP/PSX directly; pkg2zip is
# only an optional fallback if a binary happens to be configured.
pkg2zip = nopaystation.find_pkg2zip(self.pkg2zip)
# The bundled Python decryptor handles PSP/PSX/themes/DLC directly.
title_id = item.get("title_id") or item.get("content_id") or ""
item_ct = item.get("content_type") or self.content_type
pkg_name = re.sub(r"[^A-Za-z0-9_-]", "", title_id) or "pkg"
tmp = Path(tempfile.mkdtemp(prefix="nps-download-"))
pkg = tmp / f"{pkg_name}.pkg"
@@ -501,8 +541,21 @@ class NoPayStationSource(Source):
if progress_cb is not None:
progress_cb("Downloading", 0.0, "")
download_url(item["pkg_url"], pkg, None, progress_cb, cancel_event)
folder = nopaystation.extract_pkg(pkg, title_id, dest_dir, pkg2zip,
progress_cb, cancel_event)
kind_hint = item_ct if item_ct in ("theme", "dlc") else None
folder = nopaystation.extract_pkg(pkg, title_id, dest_dir,
kind_hint, progress_cb, cancel_event)
if item_ct in ("theme", "dlc"):
_write_sidecar(folder, item)
return folder
finally:
shutil.rmtree(tmp, ignore_errors=True)
def _write_sidecar(folder, item):
"""Save the store display name beside a downloaded theme/DLC (for the library)."""
try:
meta = {"name": item.get("name") or "", "title_id": item.get("title_id") or "",
"content_id": item.get("content_id") or ""}
(Path(folder) / "meta.json").write_text(json.dumps(meta, ensure_ascii=False), encoding="utf-8")
except OSError:
pass
-38
View File
@@ -1,38 +0,0 @@
# pkg2zip
The `pkg2zip.exe` binary in this folder is kept as an optional fallback for
decrypting NoPayStation `.pkg` files (PSP / PSX / PS3 / PSVita content). The
app normally uses its own pure-Python decryptor (`app/pkg.py`) and only runs
this binary if that fails.
- Project: <https://github.com/mmozeiko/pkg2zip>
- Author: Marcin Mikołajczak (mmozeiko)
- Version: v1.8 (bundled)
- License: MIT
The Windows binary is run through Wine on Linux/macOS (see
`app/nopaystation.py`).
## License
MIT License
Copyright (c) 2017 Marcin Mikołajczak (mmozeiko)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+11 -10
View File
@@ -1,12 +1,12 @@
services:
psp-preserve:
psppt:
build: .
container_name: psp-preserve
container_name: psppt
# Attach to Unraid's macvlan bridge so the API gets its own LAN IP.
# Plain HTTP for now; see Caddyfile to add Let's Encrypt TLS later.
networks:
br0:
ipv4_address: 192.168.10.100
ipv4_address: ${PSP_LAN_IP}
environment:
PSP_DATA_DIR: /data
PSP_DATS_DIR: /data/dats
@@ -15,15 +15,16 @@ services:
PSP_GAMES_ROOTS: /data/umd_games
PSP_VIDEO_ROOTS: /data/umd_videos
PSP_STORE_DIR: /data/store
# Secret token the desktop app sends as `Authorization: Bearer <token>`.
# Set via a .env file next to this compose file.
PSP_API_TOKEN: ${PSP_API_TOKEN:-}
PSP_STORE_VIDEOS_DIR: /data/store_videos
PSP_ADMIN_USERNAME: ${PSP_ADMIN_USERNAME:-admin}
PSP_ADMIN_PASSWORD: ${PSP_ADMIN_PASSWORD:-admin}
volumes:
- ./data:/data
# Adjust host paths to your Unraid share layout:
- /mnt/user/downloads/Sony - PlayStation Portable:/data/store:ro
- /mnt/user/psp/umd_games:/data/umd_games:ro
- /mnt/user/psp/umd_videos:/data/umd_videos:ro
# Host paths come from .env (see .env.example):
- ${PSP_STORE_HOST}:/data/store
- ${PSP_STORE_VIDEOS_HOST}:/data/store_videos
- ${PSP_UMD_GAMES_HOST}:/data/umd_games:ro
- ${PSP_UMD_VIDEOS_HOST}:/data/umd_videos:ro
restart: unless-stopped
networks:
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 273 B

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 689 KiB

After

Width:  |  Height:  |  Size: 683 KiB

+81
View File
@@ -0,0 +1,81 @@
#!/bin/sh
# Install psppt to a system prefix.
#
# FHS-friendly layout:
# $PREFIX/bin/psppt symlink -> payload binary
# $PREFIX/lib/psppt/ app payload (binary + lib/)
# $PREFIX/share/applications/ desktop entry
# $PREFIX/share/icons/hicolor/256x256/apps/ app icon
#
# The PyInstaller onedir build keeps the executable and its `lib/` contents
# directory side by side, so both live under $PREFIX/lib/psppt/ and a
# symlink exposes the binary at $PREFIX/bin/psppt.
#
# Usage (run from inside the extracted tarball):
# sudo ./install.sh # install to /usr/local (default)
# sudo ./install.sh --prefix=/usr # install to /usr
# sudo ./install.sh --uninstall # remove from the default prefix
set -eu
PREFIX=/usr/local
MODE=install
for arg in "$@"; do
case "$arg" in
--prefix=*) PREFIX=${arg#--prefix=} ;;
--uninstall) MODE=uninstall ;;
*) echo "unknown argument: $arg" >&2; exit 2 ;;
esac
done
SRC_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
LIBDIR="$PREFIX/lib/psppt"
BINDIR="$PREFIX/bin"
APPDIR="$PREFIX/share/applications"
ICONDIR="$PREFIX/share/icons/hicolor/256x256/apps"
APP_BIN="$LIBDIR/psppt"
require_root() {
if [ "$(id -u)" -ne 0 ]; then
echo "run as root (e.g. sudo)" >&2
exit 1
fi
}
uninstall() {
rm -f "$BINDIR/psppt"
rm -f "$APPDIR/psppt.desktop"
rm -f "$ICONDIR/psppt.png"
rm -rf "$LIBDIR"
echo "Uninstalled from $PREFIX"
}
if [ "$MODE" = uninstall ]; then
require_root
uninstall
exit 0
fi
require_root
# The tarball ships these next to this script.
for f in psppt lib psppt.desktop app-icon.png; do
if [ ! -e "$SRC_DIR/$f" ]; then
echo "missing $f (run install.sh from inside the extracted tarball)" >&2
exit 1
fi
done
mkdir -p "$LIBDIR" "$BINDIR" "$APPDIR" "$ICONDIR"
rm -rf "$LIBDIR"
mkdir -p "$LIBDIR"
cp -a "$SRC_DIR/psppt" "$LIBDIR/psppt"
cp -a "$SRC_DIR/lib" "$LIBDIR/lib"
install -m 0644 "$SRC_DIR/psppt.desktop" "$APPDIR/psppt.desktop"
install -m 0644 "$SRC_DIR/app-icon.png" "$ICONDIR/psppt.png"
ln -sfn "$APP_BIN" "$BINDIR/psppt"
echo "Installed to $PREFIX"
echo " binary: $BINDIR/psppt -> $APP_BIN"
echo " payload: $LIBDIR"
echo " desktop: $APPDIR/psppt.desktop"
+9
View File
@@ -0,0 +1,9 @@
[Desktop Entry]
Type=Application
Name=psppt
Comment=Preserve and manage your PSP library
Exec=psppt
Icon=psppt
Terminal=false
Categories=Utility;AudioVideo;
StartupWMClass=psppt
+65
View File
@@ -0,0 +1,65 @@
# -*- mode: python ; coding: utf-8 -*-
"""PyInstaller spec: build psppt (onedir).
Bundles the read-only resources (DATs, PS1 loader, cover templates, index
catalogs, icons) under ``lib/`` next to the executable. User-writable data
(credentials, manual covers, PS1 templates, NPS cache, library DB, cover
cache) lives in the OS state dir at runtime — see ``app/config.py``.
Usage: pyinstaller --clean --noconfirm psppt.spec
Output: dist/psppt/
"""
from PyInstaller.utils.hooks import collect_dynamic_libs, collect_submodules
datas = [
("data/dats", "data/dats"),
("data/loader", "data/loader"),
("data/supplement.json", "data"),
("data/covers/libretro-index.json", "data/covers"),
("data/covers/openvgdb-index.json", "data/covers"),
("data/covers/UMD_COVER_TEMPLATE.png", "data/covers"),
("data/covers/UMD_COVER_TEMPLATE_FEATURE.png", "data/covers"),
("data/covers/UMD_GAME_COVER_TEMPLATE.png", "data/covers"),
("img/app-icon.png", "img"),
("img/chevron.png", "img"),
("packaging/psppt.desktop", "."),
]
a = Analysis(
["run.py"],
pathex=[],
binaries=collect_dynamic_libs("curl_cffi"),
datas=datas,
hiddenimports=collect_submodules("curl_cffi"),
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=["tkinter", "unittest", "pydoc"],
noarchive=False,
)
pyz = PYZ(a.pure)
exe = EXE(
pyz,
a.scripts,
[],
exclude_binaries=True,
name="psppt",
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=False,
console=False,
disable_windowed_traceback=False,
contents_directory="lib",
)
coll = COLLECT(
exe,
a.binaries,
a.datas,
strip=False,
upx=False,
name="psppt",
)
+7
View File
@@ -0,0 +1,7 @@
# Build dependencies (pinned for reproducible CI builds).
# PySide6 is pinned to 6.8.3: newer 6.11.x bundles a Qt6Core.dll that imports
# icuuc.dll on Windows, which is not shipped and crashes at startup.
pyinstaller>=6.11
PySide6==6.8.3
curl_cffi>=0.7
pycryptodome>=3.20
+4
View File
@@ -0,0 +1,4 @@
fastapi>=0.110
uvicorn[standard]>=0.29
jinja2>=3.1
python-multipart>=0.0.9
+10
View File
@@ -0,0 +1,10 @@
"""PyInstaller entry point.
Runs the GUI the same way ``python -m app.main`` does, but as a plain script
so PyInstaller can build it (relative imports inside ``app`` won't resolve when
a module file is executed directly, hence this top-level launcher).
"""
from app.main import main
if __name__ == "__main__":
raise SystemExit(main())
+68
View File
@@ -0,0 +1,68 @@
# Wine-based Gitea Actions runner: builds Windows binaries on a Linux host.
#
# Jobs run in "host" mode inside this container, where a Windows CPython 3.12
# (running under Wine) plus PyInstaller produce the .exe. Node.js and git are
# included so the stock JS actions (actions/checkout, actions/upload-artifact)
# work as-is.
FROM debian:bookworm-slim
ENV DEBIAN_FRONTEND=noninteractive \
WINEPREFIX=/root/.wine \
WINEDEBUG=-all \
WINEDLLOVERRIDES="mscoree,mshtml=" \
PYTHON_VERSION=3.12.8
# WineHQ wine-staging (newer than Debian's 8.0, needed for Python 3.12),
# node + git for JS actions, and build/package tooling.
RUN dpkg --add-architecture i386 \
&& apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl wget gnupg2 \
&& mkdir -p /etc/apt/keyrings \
&& wget -qO- https://dl.winehq.org/wine-builds/winehq.key | gpg --dearmor -o /etc/apt/keyrings/winehq.gpg \
&& echo "deb [signed-by=/etc/apt/keyrings/winehq.gpg] https://dl.winehq.org/wine-builds/debian/ bookworm main" > /etc/apt/sources.list.d/winehq.list \
&& apt-get update \
&& apt-get install -y --install-recommends winehq-staging \
&& apt-get install -y --no-install-recommends unzip zip xvfb xauth cabextract nodejs git \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
# winetricks (standalone script; not packaged in Debian main).
RUN wget -qO /usr/local/bin/winetricks \
"https://raw.githubusercontent.com/Winetricks/winetricks/master/src/winetricks" \
&& chmod +x /usr/local/bin/winetricks
# Gitea Actions runner (linux-amd64 binary).
RUN wget -qO /usr/local/bin/act_runner \
"https://gitea.com/gitea/act_runner/releases/download/v0.6.1/act_runner-0.6.1-linux-amd64" \
&& chmod +x /usr/local/bin/act_runner
# MSVC runtime required by PySide6's Qt DLLs (also initializes the Wine prefix).
RUN xvfb-run -a winetricks --unattended vcrun2022
# Windows CPython 3.12 (embeddable zip) + pip.
RUN wget -qO /tmp/pyembed.zip \
"https://www.python.org/ftp/python/${PYTHON_VERSION}/python-${PYTHON_VERSION}-embed-amd64.zip" \
&& mkdir -p /root/.wine/drive_c/Python312 \
&& cd /root/.wine/drive_c/Python312 \
&& unzip -q /tmp/pyembed.zip \
&& rm -f /tmp/pyembed.zip \
&& printf 'python312.zip\n.\nimport site\nLib\\site-packages\n' > python312._pth \
&& wine "C:\\Python312\\python.exe" --version
# Convenience wrapper: winpython -> wine C:\Python312\python.exe
RUN printf '#!/bin/sh\nexec wine "C:\\Python312\\python.exe" "$@"\n' > /usr/local/bin/winpython \
&& chmod +x /usr/local/bin/winpython
# Bootstrap pip, then pre-install the Windows build deps so CI jobs start fast.
RUN wget -qO /tmp/get-pip.py https://bootstrap.pypa.io/get-pip.py \
&& wine "C:\\Python312\\python.exe" /tmp/get-pip.py \
&& rm -f /tmp/get-pip.py
COPY requirements-build.txt /tmp/requirements-build.txt
RUN winpython -m pip install --upgrade pip \
&& winpython -m pip install -r /tmp/requirements-build.txt \
&& rm -f /tmp/requirements-build.txt
COPY runner/entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
WORKDIR /data
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
set -e
# Register on first run, then run the daemon (host mode).
if [ ! -f /data/.runner ]; then
echo "Registering Wine runner with Gitea…"
act_runner register --no-interactive \
--instance "${GITEA_INSTANCE_URL:?GITEA_INSTANCE_URL is required}" \
--token "${GITEA_RUNNER_REGISTRATION_TOKEN:?GITEA_RUNNER_REGISTRATION_TOKEN is required}" \
--name "${RUNNER_NAME:-windows-builder}" \
--labels "${RUNNER_LABELS:-windows:host}"
fi
exec act_runner daemon
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env python3
"""Package the offline cover zips into a single distributable bundle.
Outputs ``psppt-covers.zip`` containing ``libretro-covers.zip`` and
``openvgdb-covers.zip``. Attach it to the Gitea ``covers`` release; the app
downloads it on first run when the user opts in.
Usage: python3 scripts/make-covers-bundle.py [out-path]
"""
import sys
import zipfile
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
COVERS = ROOT / "data" / "covers"
INNER = ("libretro-covers.zip", "openvgdb-covers.zip")
def main(argv):
out = Path(argv[1]) if len(argv) > 1 else ROOT / "psppt-covers.zip"
missing = [n for n in INNER if not (COVERS / n).exists()]
if missing:
print(f"missing: {', '.join(missing)} in {COVERS}", file=sys.stderr)
return 2
# Inner zips are already compressed; store them to avoid recompressing.
with zipfile.ZipFile(out, "w", zipfile.ZIP_STORED) as z:
for n in INNER:
print(f"adding {n} …")
z.write(COVERS / n, arcname=n)
size = out.stat().st_size / (1 << 20)
print(f"wrote {out} ({size:.0f} MB)")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv))
+1
View File
@@ -0,0 +1 @@
"""psppt server package (FastAPI + admin WebUI)."""
+491
View File
@@ -0,0 +1,491 @@
"""REST API for the psppt server (JSON, token-authenticated).
Endpoints:
POST /api/auth/login -> {token, username, role}
GET /api/auth/me -> current user
POST /api/auth/logout -> revoke current token
GET /api/users -> list (admin)
POST /api/users -> create (admin)
PATCH /api/users/{id} -> update role/password/active (admin)
DELETE /api/users/{id} -> delete (admin)
GET /api/users/{id}/tokens -> list API tokens (admin or self)
DELETE /api/users/{id}/tokens/{tid} -> revoke a token (admin or self)
GET /api/releases?system=&q=&limit= -> catalog list
GET /api/releases/{id} -> detail + metadata + roms
PATCH /api/releases/{id} -> edit metadata (editor+)
GET /api/cover/{id} -> cover image
POST /api/cover/{id} -> upload/replace cover (editor+)
DELETE /api/cover/{id} -> delete cover (editor+)
GET /api/store?system=games|videos -> archive list
GET /api/store/download/{system}/{name} -> stream archive
POST /api/store/upload?system= -> upload archive (editor+)
DELETE /api/store/{system}/{name} -> delete archive (editor+)
GET /api/stats -> dashboard numbers (admin)
"""
import json
import re
from pathlib import Path
from fastapi import APIRouter, Depends, File, Header, HTTPException, Query, Request, UploadFile
from fastapi.responses import FileResponse, JSONResponse
from app import covers, store
from app.db import get_connection
from . import config
from .security import now_iso, require_role, revoke_token, get_current_user, hash_password, verify_password, issue_token
router = APIRouter(prefix="/api")
_META_FIELDS = ("release_date", "developer", "publisher", "genre", "synopsis", "title")
_SAFE_NAME_RE = re.compile(r"[^A-Za-z0-9._()\- ]")
# --------------------------------------------------------------------------- #
# helpers
# --------------------------------------------------------------------------- #
def _conn():
return get_connection()
def _release_system(conn, release_id):
row = conn.execute(
"SELECT d.system FROM releases r JOIN dats d ON d.id = r.dat_id WHERE r.id = ?",
(release_id,),
).fetchone()
return row["system"] if row else None
def _metadata(conn, release_id, system):
row = conn.execute(
"SELECT title, meta_json, manual FROM item_info WHERE release_id = ? AND system = ?",
(str(release_id), system),
).fetchone()
meta = {}
if row and row["meta_json"]:
try:
meta = json.loads(row["meta_json"])
except (TypeError, ValueError):
meta = {}
if row:
meta["title"] = row["title"]
meta["manual"] = bool(row["manual"])
return meta
def _save_metadata(conn, release_id, system, fields):
key = str(release_id)
row = conn.execute(
"SELECT meta_json FROM item_info WHERE release_id = ? AND system = ?", (key, system)
).fetchone()
meta = {}
if row and row["meta_json"]:
try:
meta = json.loads(row["meta_json"])
except (TypeError, ValueError):
meta = {}
meta.update(fields)
title = fields.get("title") or meta.get("title") or ""
conn.execute(
"INSERT INTO item_info (release_id, system, title, meta_json, manual, updated_at) "
"VALUES (?, ?, ?, ?, 1, ?) "
"ON CONFLICT(release_id, system) DO UPDATE SET "
"title = excluded.title, meta_json = excluded.meta_json, manual = 1, updated_at = excluded.updated_at",
(key, system, title, json.dumps(meta, ensure_ascii=False), now_iso()),
)
def _cover_paths(release_id, system):
base = config.COVERS_DIR / system / str(release_id)
return base, [base.with_suffix(ext) for ext in (".png", ".jpg", ".jpeg", ".webp")]
def _find_cover(release_id, system):
base, candidates = _cover_paths(release_id, system)
for p in candidates:
if p.exists():
return p
return base.with_suffix(".png")
def _save_cover(release_id, system, data):
base, candidates = _cover_paths(release_id, system)
base.parent.mkdir(parents=True, exist_ok=True)
ext = ".png"
if data[:3] == b"\xff\xd8\xff":
ext = ".jpg"
elif data[:4] == b"RIFF" and data[8:12] == b"WEBP":
ext = ".webp"
dest = base.with_suffix(ext)
for p in candidates:
if p.exists() and p != dest:
p.unlink()
tmp = dest.with_name(dest.name + ".tmp")
tmp.write_bytes(data)
tmp.replace(dest)
return dest
def _store_root(system):
return config.STORE_VIDEOS_ROOT if system == "videos" else config.STORE_ROOT
def _log_download(user, system, name, size):
conn = _conn()
try:
conn.execute(
"INSERT INTO download_events (user_id, system, name, bytes, created_at) VALUES (?, ?, ?, ?, ?)",
(user["id"] if user else None, system, name, size or 0, now_iso()),
)
conn.commit()
finally:
conn.close()
# --------------------------------------------------------------------------- #
# auth
# --------------------------------------------------------------------------- #
@router.post("/auth/login")
def auth_login(payload: dict):
username = (payload.get("username") or "").strip()
password = payload.get("password") or ""
conn = _conn()
try:
row = conn.execute(
"SELECT id, username, password_hash, role, is_active FROM users WHERE username = ?",
(username,),
).fetchone()
finally:
conn.close()
if row is None or not row["is_active"] or not verify_password(password, row["password_hash"]):
raise HTTPException(status_code=401, detail="Invalid username or password")
token = issue_token(row["id"], kind="api", label="desktop")
return {"token": token, "username": row["username"], "role": row["role"], "user_id": row["id"]}
@router.get("/auth/me")
def auth_me(user=Depends(get_current_user)):
return user
@router.post("/auth/logout")
def auth_logout(request: Request, authorization: str = Header(default="")):
token = ""
if authorization.startswith("Bearer "):
token = authorization[7:].strip()
else:
token = request.cookies.get("psppt_session", "")
if token:
revoke_token(token)
return {"ok": True}
# --------------------------------------------------------------------------- #
# users (admin)
# --------------------------------------------------------------------------- #
@router.get("/users")
def users_list(user=Depends(require_role("admin"))):
conn = _conn()
try:
rows = conn.execute(
"SELECT id, username, role, is_active, created_at, last_login_at FROM users ORDER BY username"
).fetchall()
return {"items": [dict(r) for r in rows]}
finally:
conn.close()
@router.post("/users")
def users_create(payload: dict, user=Depends(require_role("admin"))):
username = (payload.get("username") or "").strip()
password = payload.get("password") or ""
role = payload.get("role") or "user"
if not username or not password:
raise HTTPException(status_code=400, detail="username and password are required")
if role not in ("admin", "editor", "user"):
raise HTTPException(status_code=400, detail="invalid role")
conn = _conn()
try:
try:
conn.execute(
"INSERT INTO users (username, password_hash, role, is_active, created_at) VALUES (?, ?, ?, 1, ?)",
(username, hash_password(password), role, now_iso()),
)
conn.commit()
except Exception:
raise HTTPException(status_code=409, detail="username already exists")
row = conn.execute("SELECT id, username, role, is_active FROM users WHERE username = ?", (username,)).fetchone()
return dict(row)
finally:
conn.close()
@router.patch("/users/{user_id}")
def users_update(user_id: int, payload: dict, user=Depends(require_role("admin"))):
conn = _conn()
try:
row = conn.execute("SELECT id FROM users WHERE id = ?", (user_id,)).fetchone()
if row is None:
raise HTTPException(status_code=404, detail="not found")
if "role" in payload:
if payload["role"] not in ("admin", "editor", "user"):
raise HTTPException(status_code=400, detail="invalid role")
conn.execute("UPDATE users SET role = ? WHERE id = ?", (payload["role"], user_id))
if "is_active" in payload:
conn.execute("UPDATE users SET is_active = ? WHERE id = ?", (1 if payload["is_active"] else 0, user_id))
if payload.get("password"):
conn.execute("UPDATE users SET password_hash = ? WHERE id = ?", (hash_password(payload["password"]), user_id))
conn.commit()
row = conn.execute("SELECT id, username, role, is_active FROM users WHERE id = ?", (user_id,)).fetchone()
return dict(row)
finally:
conn.close()
@router.delete("/users/{user_id}")
def users_delete(user_id: int, user=Depends(require_role("admin"))):
if user_id == user["id"]:
raise HTTPException(status_code=400, detail="cannot delete yourself")
conn = _conn()
try:
conn.execute("DELETE FROM users WHERE id = ?", (user_id,))
conn.commit()
finally:
conn.close()
return {"ok": True}
@router.get("/users/{user_id}/tokens")
def users_tokens(user_id: int, user=Depends(require_role("admin"))):
conn = _conn()
try:
rows = conn.execute(
"SELECT id, label, kind, created_at, last_used_at, expires_at, revoked_at FROM api_tokens "
"WHERE user_id = ? ORDER BY id DESC",
(user_id,),
).fetchall()
return {"items": [dict(r) for r in rows]}
finally:
conn.close()
@router.delete("/users/{user_id}/tokens/{token_id}")
def users_revoke_token(user_id: int, token_id: int, user=Depends(require_role("admin"))):
conn = _conn()
try:
conn.execute(
"UPDATE api_tokens SET revoked_at = ? WHERE id = ? AND user_id = ?",
(now_iso(), token_id, user_id),
)
conn.commit()
finally:
conn.close()
return {"ok": True}
# --------------------------------------------------------------------------- #
# releases / catalog
# --------------------------------------------------------------------------- #
@router.get("/releases")
def releases_list(system: str = Query(default=""), q: str = Query(default=""), limit: int = 50, offset: int = 0):
limit = max(1, min(limit, 500))
offset = max(0, offset)
sql = "SELECT r.id, d.system, r.name, r.region, r.serial FROM releases r JOIN dats d ON d.id = r.dat_id"
where, params = [], []
if system:
where.append("d.system = ?")
params.append(system)
if q:
where.append("r.name LIKE ?")
params.append(f"%{q}%")
if where:
sql += " WHERE " + " AND ".join(where)
sql += " ORDER BY r.name LIMIT ? OFFSET ?"
params += [limit, offset]
conn = _conn()
try:
rows = conn.execute(sql, params).fetchall()
items = []
for r in rows:
d = dict(r)
d["cover"] = f"/api/cover/{d['id']}"
items.append(d)
return {"items": items, "limit": limit, "offset": offset}
finally:
conn.close()
@router.get("/releases/{release_id}")
def releases_detail(release_id: int):
conn = _conn()
try:
rel = conn.execute(
"SELECT r.id, d.system, r.name, r.description, r.region, r.serial "
"FROM releases r JOIN dats d ON d.id = r.dat_id WHERE r.id = ?",
(release_id,),
).fetchone()
if rel is None:
raise HTTPException(status_code=404, detail="not found")
roms = conn.execute(
"SELECT name, size, crc32, md5, sha1, serial FROM roms WHERE release_id = ?", (release_id,)
).fetchall()
result = dict(rel)
result["cover"] = f"/api/cover/{release_id}"
result["roms"] = [dict(r) for r in roms]
result["meta"] = _metadata(conn, release_id, rel["system"])
return result
finally:
conn.close()
@router.patch("/releases/{release_id}")
def releases_update(release_id: int, payload: dict, user=Depends(require_role("admin", "editor"))):
conn = _conn()
try:
rel = conn.execute(
"SELECT r.id, d.system FROM releases r JOIN dats d ON d.id = r.dat_id WHERE r.id = ?",
(release_id,),
).fetchone()
if rel is None:
raise HTTPException(status_code=404, detail="not found")
# release-table columns
for col in ("name", "region", "serial", "description"):
if col in payload:
conn.execute(f"UPDATE releases SET {col} = ? WHERE id = ?", (payload[col], release_id))
# metadata (item_info)
meta_fields = {k: payload[k] for k in _META_FIELDS if k in payload}
if meta_fields:
_save_metadata(conn, release_id, rel["system"], meta_fields)
conn.commit()
return {"ok": True}
finally:
conn.close()
# --------------------------------------------------------------------------- #
# covers
# --------------------------------------------------------------------------- #
@router.get("/cover/{release_id}")
def cover_get(release_id: int):
conn = _conn()
try:
system = _release_system(conn, release_id)
finally:
conn.close()
if system is None:
raise HTTPException(status_code=404, detail="not found")
path = _find_cover(release_id, system)
if not path.exists():
raise HTTPException(status_code=404, detail="no cover available")
media = "image/jpeg" if path.suffix.lower() in (".jpg", ".jpeg") else "image/png"
if path.suffix.lower() == ".webp":
media = "image/webp"
return FileResponse(path, media_type=media)
@router.post("/cover/{release_id}")
async def cover_upload(release_id: int, file: UploadFile = File(...), user=Depends(require_role("admin", "editor"))):
conn = _conn()
try:
system = _release_system(conn, release_id)
finally:
conn.close()
if system is None:
raise HTTPException(status_code=404, detail="not found")
data = await file.read()
if len(data) > 20 * (1 << 20):
raise HTTPException(status_code=400, detail="image too large")
_save_cover(release_id, system, data)
return {"ok": True, "url": f"/api/cover/{release_id}"}
@router.delete("/cover/{release_id}")
def cover_delete(release_id: int, user=Depends(require_role("admin", "editor"))):
conn = _conn()
try:
system = _release_system(conn, release_id)
finally:
conn.close()
if system is None:
raise HTTPException(status_code=404, detail="not found")
path = _find_cover(release_id, system)
if path.exists():
path.unlink()
return {"ok": True}
# --------------------------------------------------------------------------- #
# store
# --------------------------------------------------------------------------- #
@router.get("/store")
def store_list(system: str = Query(default="games")):
root = _store_root(system)
if not root.is_dir():
return {"items": [], "count": 0, "system": system}
items = [
{"name": it["name"], "filename": Path(it["path"]).name, "size": it["size"], "mtime_ns": it["mtime_ns"], "system": system}
for it in store.scan(root)
]
return {"items": items, "count": len(items), "system": system}
@router.get("/store/download/{system}/{name}")
def store_download(system: str, name: str, user=Depends(get_current_user)):
root = _store_root(system).resolve()
target = (root / name).resolve()
if not target.is_file() or root not in target.parents:
raise HTTPException(status_code=404, detail="not found")
_log_download(user, system, name, target.stat().st_size)
return FileResponse(target, filename=name, media_type="application/octet-stream")
@router.post("/store/upload")
async def store_upload(system: str = Query(default="games"), file: UploadFile = File(...), user=Depends(require_role("admin", "editor"))):
root = _store_root(system)
root.mkdir(parents=True, exist_ok=True)
safe = _SAFE_NAME_RE.sub("", file.filename or "")
if not safe:
raise HTTPException(status_code=400, detail="invalid filename")
dest = root / safe
data = await file.read()
dest.write_bytes(data)
return {"ok": True, "filename": safe, "size": len(data)}
@router.delete("/store/{system}/{name}")
def store_delete(system: str, name: str, user=Depends(require_role("admin", "editor"))):
root = _store_root(system).resolve()
target = (root / name).resolve()
if not target.is_file() or root not in target.parents:
raise HTTPException(status_code=404, detail="not found")
target.unlink()
return {"ok": True}
# --------------------------------------------------------------------------- #
# stats (admin)
# --------------------------------------------------------------------------- #
@router.get("/stats")
def stats(user=Depends(require_role("admin"))):
conn = _conn()
try:
systems = conn.execute(
"SELECT d.system AS system, COUNT(*) AS releases FROM releases r "
"JOIN dats d ON d.id = r.dat_id GROUP BY d.system ORDER BY d.system"
).fetchall()
users = conn.execute("SELECT COUNT(*) AS n FROM users").fetchone()["n"]
covers_count = sum(1 for _ in config.COVERS_DIR.rglob("*") if _.is_file()) if config.COVERS_DIR.is_dir() else 0
downloads = conn.execute("SELECT COUNT(*) AS n FROM download_events").fetchone()["n"]
recent = conn.execute(
"SELECT e.system, e.name, e.bytes, e.created_at, u.username FROM download_events e "
"LEFT JOIN users u ON u.id = e.user_id ORDER BY e.id DESC LIMIT 20"
).fetchall()
return {
"systems": [dict(r) for r in systems],
"users": users,
"covers": covers_count,
"downloads": downloads,
"recent": [dict(r) for r in recent],
}
finally:
conn.close()
+9
View File
@@ -0,0 +1,9 @@
"""Server-specific settings (env-driven)."""
import os
from pathlib import Path
from app.config import COVERS_DIR, DB_PATH, STORE_ROOT
STORE_VIDEOS_ROOT = Path(os.environ.get("PSP_STORE_VIDEOS_DIR", "/data/store_videos"))
ADMIN_USERNAME = os.environ.get("PSP_ADMIN_USERNAME", "admin")
ADMIN_PASSWORD = os.environ.get("PSP_ADMIN_PASSWORD", "admin")
+39
View File
@@ -0,0 +1,39 @@
"""psppt server: FastAPI app assembly."""
import logging
from contextlib import asynccontextmanager
from pathlib import Path
from fastapi import FastAPI
from fastapi.staticfiles import StaticFiles
from app.db import init_db
from . import api, config, web
from .security import ensure_admin
log = logging.getLogger("psppt.server")
@asynccontextmanager
async def lifespan(_app: FastAPI):
init_db()
created = ensure_admin(config.ADMIN_USERNAME, config.ADMIN_PASSWORD)
if created:
log.warning(
"Created initial admin user %r with the configured password — change it now.",
config.ADMIN_USERNAME,
)
yield
app = FastAPI(title="psppt server", lifespan=lifespan)
app.include_router(api.router)
app.include_router(web.router)
app.mount("/static", StaticFiles(directory=str(Path(__file__).parent / "static")), name="static")
@app.get("/health")
def health():
return {"ok": True}
+138
View File
@@ -0,0 +1,138 @@
"""Password hashing, API/session tokens, and FastAPI auth dependencies.
Stdlib-only crypto: PBKDF2-SHA256 for passwords, SHA-256 hashed bearer tokens
for both the desktop client (kind="api") and the WebUI session (kind="session").
"""
import hashlib
import secrets
from datetime import datetime, timezone
from fastapi import Depends, Header, HTTPException, Request, status
from app.db import get_connection
_PBKDF2_ITERATIONS = 600_000
ADMIN_ROLE = "admin"
EDITOR_ROLE = "editor"
USER_ROLE = "user"
ROLES = (ADMIN_ROLE, EDITOR_ROLE, USER_ROLE)
def now_iso():
return datetime.now(timezone.utc).isoformat(timespec="seconds")
def hash_password(password: str) -> str:
salt = secrets.token_bytes(16)
digest = hashlib.pbkdf2_hmac("sha256", password.encode("utf-8"), salt, _PBKDF2_ITERATIONS)
return f"pbkdf2_sha256${_PBKDF2_ITERATIONS}${salt.hex()}${digest.hex()}"
def verify_password(password: str, stored: str) -> bool:
try:
scheme, iterations, salt_hex, digest_hex = stored.split("$")
if scheme != "pbkdf2_sha256":
return False
salt = bytes.fromhex(salt_hex)
expected = bytes.fromhex(digest_hex)
actual = hashlib.pbkdf2_hmac("sha256", password.encode("utf-8"), salt, int(iterations))
return secrets.compare_digest(actual, expected)
except (ValueError, TypeError):
return False
def _hash_token(raw: str) -> str:
return hashlib.sha256(raw.encode("utf-8")).hexdigest()
def issue_token(user_id: int, kind: str = "api", label: str = "", expires_at: str = "") -> str:
"""Create a token row and return the raw token (only shown once)."""
raw = secrets.token_urlsafe(32)
conn = get_connection()
try:
conn.execute(
"INSERT INTO api_tokens (user_id, token_hash, label, kind, created_at, last_used_at, expires_at, revoked_at) "
"VALUES (?, ?, ?, ?, ?, ?, ?, NULL)",
(user_id, _hash_token(raw), label, kind, now_iso(), now_iso(), expires_at or None),
)
conn.commit()
finally:
conn.close()
return raw
def revoke_token(raw: str) -> None:
conn = get_connection()
try:
conn.execute(
"UPDATE api_tokens SET revoked_at = ? WHERE token_hash = ?",
(now_iso(), _hash_token(raw)),
)
conn.commit()
finally:
conn.close()
def get_user_by_token(raw: str):
if not raw:
return None
conn = get_connection()
try:
row = conn.execute(
"SELECT u.id, u.username, u.role, u.is_active FROM api_tokens t "
"JOIN users u ON u.id = t.user_id "
"WHERE t.token_hash = ? AND t.revoked_at IS NULL "
"AND (t.expires_at IS NULL OR t.expires_at = '' OR t.expires_at > ?)",
(_hash_token(raw), now_iso()),
).fetchone()
if row is None or not row["is_active"]:
return None
conn.execute(
"UPDATE api_tokens SET last_used_at = ? WHERE token_hash = ?",
(now_iso(), _hash_token(raw)),
)
conn.commit()
return dict(row)
finally:
conn.close()
def get_current_user(request: Request, authorization: str = Header(default="")):
"""Authenticate via ``Authorization: Bearer`` (client) or session cookie (WebUI)."""
token = ""
if authorization.startswith("Bearer "):
token = authorization[7:].strip()
else:
token = request.cookies.get("psppt_session", "")
user = get_user_by_token(token)
if user is None:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Not authenticated")
return user
def require_role(*roles):
"""Dependency factory: only users with one of ``roles`` may pass."""
def dependency(user=Depends(get_current_user)):
if user["role"] not in roles:
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Insufficient permissions")
return user
return dependency
def ensure_admin(username: str, password: str) -> bool:
"""Create the initial admin account when the users table is empty."""
conn = get_connection()
try:
count = conn.execute("SELECT COUNT(*) AS n FROM users").fetchone()["n"]
if count == 0:
conn.execute(
"INSERT INTO users (username, password_hash, role, is_active, created_at) "
"VALUES (?, ?, 'admin', 1, ?)",
(username, hash_password(password), now_iso()),
)
conn.commit()
return True
finally:
conn.close()
return False
+52
View File
@@ -0,0 +1,52 @@
:root { color-scheme: dark; }
* { box-sizing: border-box; }
body {
margin: 0; font-family: system-ui, -apple-system, Segoe UI, Roboto, sans-serif;
background: #12131a; color: #e8e9ed; line-height: 1.45;
}
nav {
display: flex; align-items: center; gap: 16px; padding: 10px 20px;
background: #1a1c26; border-bottom: 1px solid #2a2d3a;
}
nav a { color: #9fb4ff; text-decoration: none; font-weight: 600; }
nav a:hover { color: #c7d3ff; }
nav .spacer { flex: 1; }
nav .who { color: #8a8da0; font-size: 0.9em; }
main { max-width: 1100px; margin: 0 auto; padding: 24px 20px 60px; }
h1 { font-size: 1.6em; margin-top: 0; }
h2 { font-size: 1.15em; margin-top: 28px; color: #b9bdd0; }
.card { background: #1a1c26; border: 1px solid #2a2d3a; border-radius: 10px; padding: 24px; }
.card.narrow { max-width: 380px; margin: 60px auto; }
form label { display: block; margin: 10px 0 2px; font-size: 0.85em; color: #9aa0b5; }
input, select, textarea {
width: 100%; padding: 8px 10px; border-radius: 6px; border: 1px solid #34384a;
background: #0f1016; color: #e8e9ed; font: inherit;
}
form.inline { display: flex; gap: 8px; align-items: center; flex-wrap: wrap; }
form.inline input, form.inline select { width: auto; }
form.filters input { width: 220px; }
form.grid { display: grid; grid-template-columns: 1fr 1fr; gap: 4px 18px; max-width: 760px; }
form.grid .wide { grid-column: 1 / -1; }
button {
padding: 8px 16px; border: 0; border-radius: 6px; cursor: pointer;
background: #3b5bdb; color: #fff; font: inherit; font-weight: 600;
}
button:hover { background: #4c6ef5; }
button.ghost { background: transparent; border: 1px solid #34384a; color: #c7cadb; }
button.danger { background: #b03a3a; }
button.danger:hover { background: #d14a4a; }
table { width: 100%; border-collapse: collapse; margin-top: 12px; font-size: 0.95em; }
th, td { text-align: left; padding: 7px 10px; border-bottom: 1px solid #262936; }
th { color: #9aa0b5; font-weight: 600; }
tr:hover td { background: #171923; }
.cards { display: flex; gap: 14px; flex-wrap: wrap; }
.stat { flex: 1; min-width: 120px; background: #1a1c26; border: 1px solid #2a2d3a;
border-radius: 10px; padding: 16px; text-align: center; }
.stat b { display: block; font-size: 1.7em; }
.stat span { color: #8a8da0; font-size: 0.85em; }
.error { color: #ff8787; }
.muted { color: #8a8da0; }
.cover { max-width: 200px; border-radius: 8px; border: 1px solid #2a2d3a; margin-bottom: 10px; }
code { background: #0f1016; padding: 2px 6px; border-radius: 4px; }
label.check { display: flex; align-items: center; gap: 6px; margin: 0; }
label.check input { width: auto; }
+26
View File
@@ -0,0 +1,26 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{% block title %}psppt server{% endblock %}</title>
<link rel="stylesheet" href="/static/app.css">
</head>
<body>
{% if user %}
<nav>
<a href="/dashboard">Dashboard</a>
<a href="/releases">Games &amp; Videos</a>
{% if user.role in ('admin', 'editor') %}<a href="/store">Store</a>{% endif %}
{% if user.role == 'admin' %}<a href="/users">Users</a>{% endif %}
{% if user.role == 'admin' %}<a href="/settings">Settings</a>{% endif %}
<span class="spacer"></span>
<span class="who">{{ user.username }} · {{ user.role }}</span>
<form method="post" action="/logout" class="inline"><button class="ghost">Logout</button></form>
</nav>
{% endif %}
<main>
{% block content %}{% endblock %}
</main>
</body>
</html>
+37
View File
@@ -0,0 +1,37 @@
{% extends "base.html" %}
{% block title %}Dashboard · psppt{% endblock %}
{% block content %}
<h1>Dashboard</h1>
<div class="cards">
<div class="stat"><b>{{ users }}</b><span>users</span></div>
<div class="stat"><b>{{ downloads }}</b><span>downloads</span></div>
<div class="stat"><b>{{ covers }}</b><span>covers</span></div>
<div class="stat"><b>{{ store_games }}</b><span>game archives</span></div>
<div class="stat"><b>{{ store_videos }}</b><span>video archives</span></div>
</div>
<h2>Catalog</h2>
<ul>
{% for s in systems %}
<li>{{ s.system }} — {{ s.releases }} release(s)</li>
{% else %}
<li>No catalog data ingested yet.</li>
{% endfor %}
</ul>
<h2>Recent downloads</h2>
<table>
<tr><th>When</th><th>User</th><th>System</th><th>Name</th><th>Size</th></tr>
{% for r in recent %}
<tr>
<td>{{ r.created_at }}</td>
<td>{{ r.username or '—' }}</td>
<td>{{ r.system }}</td>
<td>{{ r.name }}</td>
<td>{{ (r.bytes / 1048576) | round(1) }} MB</td>
</tr>
{% else %}
<tr><td colspan="5">No downloads yet.</td></tr>
{% endfor %}
</table>
{% endblock %}
+13
View File
@@ -0,0 +1,13 @@
{% extends "base.html" %}
{% block title %}Login · psppt{% endblock %}
{% block content %}
<div class="card narrow">
<h1>psppt server</h1>
{% if error %}<p class="error">{{ error }}</p>{% endif %}
<form method="post" action="/login">
<label>Username <input name="username" required autofocus></label>
<label>Password <input name="password" type="password" required></label>
<button>Log in</button>
</form>
</div>
{% endblock %}
+31
View File
@@ -0,0 +1,31 @@
{% extends "base.html" %}
{% block title %}{{ rel.name }} · psppt{% endblock %}
{% block content %}
<p><a href="/releases">← back</a></p>
<h1>{{ rel.name }}</h1>
<form method="post" action="/releases/{{ rel.id }}/edit" class="grid">
<label>Name <input name="name" value="{{ rel.name }}"></label>
<label>Region <input name="region" value="{{ rel.region or '' }}"></label>
<label>Serial <input name="serial" value="{{ rel.serial or '' }}"></label>
<label>Description <textarea name="description">{{ rel.description or '' }}</textarea></label>
<label>Title <input name="title" value="{{ meta.title or '' }}"></label>
<label>Release date <input name="release_date" value="{{ meta.release_date or '' }}"></label>
<label>Developer <input name="developer" value="{{ meta.developer or '' }}"></label>
<label>Publisher <input name="publisher" value="{{ meta.publisher or '' }}"></label>
<label>Genre <input name="genre" value="{{ meta.genre or '' }}"></label>
<label class="wide">Synopsis <textarea name="synopsis">{{ meta.synopsis or '' }}</textarea></label>
<button>Save</button>
</form>
<h2>Cover</h2>
{% if has_cover %}<img src="/api/cover/{{ rel.id }}" class="cover" alt="cover">{% else %}<p class="muted">No cover.</p>{% endif %}
<form method="post" action="/releases/{{ rel.id }}/cover" enctype="multipart/form-data" class="inline">
<input type="file" name="cover" accept="image/*">
<button>Upload cover</button>
</form>
{% if has_cover %}
<form method="post" action="/releases/{{ rel.id }}/cover/delete" class="inline">
<button class="danger">Delete cover</button>
</form>
{% endif %}
{% endblock %}
+29
View File
@@ -0,0 +1,29 @@
{% extends "base.html" %}
{% block title %}Games &amp; Videos · psppt{% endblock %}
{% block content %}
<h1>Games &amp; Videos</h1>
<form method="get" action="/releases" class="filters">
<select name="system">
<option value="">All systems</option>
<option value="games" {% if system == 'games' %}selected{% endif %}>Games</option>
<option value="videos" {% if system == 'videos' %}selected{% endif %}>Videos</option>
</select>
<input name="q" value="{{ q }}" placeholder="Search…">
<button>Filter</button>
</form>
<table>
<tr><th>Name</th><th>System</th><th>Region</th><th>Serial</th><th>Cover</th><th></th></tr>
{% for it in items %}
<tr>
<td>{{ it.name }}</td>
<td>{{ it.system }}</td>
<td>{{ it.region or '' }}</td>
<td>{{ it.serial or '' }}</td>
<td>{% if it.has_cover %}✓{% else %}—{% endif %}</td>
<td><a href="/releases/{{ it.id }}/edit">Edit</a></td>
</tr>
{% else %}
<tr><td colspan="6">Nothing found.</td></tr>
{% endfor %}
</table>
{% endblock %}
+11
View File
@@ -0,0 +1,11 @@
{% extends "base.html" %}
{% block title %}Settings · psppt{% endblock %}
{% block content %}
<h1>Settings</h1>
<ul>
<li>Store root (games): <code>{{ store_root }}</code></li>
<li>Store root (videos): <code>{{ store_videos_root }}</code></li>
<li>Covers directory: <code>{{ covers_dir }}</code></li>
</ul>
<p class="muted">These paths are configured via environment variables — restart the server to change them.</p>
{% endblock %}
+50
View File
@@ -0,0 +1,50 @@
{% extends "base.html" %}
{% block title %}Store · psppt{% endblock %}
{% block content %}
<h1>Store</h1>
<h2>Upload archive</h2>
<form method="post" action="/store/upload" enctype="multipart/form-data" class="inline">
<select name="system">
<option value="games">Games</option>
<option value="videos">Videos</option>
</select>
<input type="file" name="file">
<button>Upload</button>
</form>
<h2>Games ({{ games | length }})</h2>
<table>
<tr><th>Name</th><th>Size</th><th></th></tr>
{% for g in games %}
<tr>
<td>{{ g.name }}</td>
<td>{{ (g.size / 1048576) | round(1) }} MB</td>
<td>
<form method="post" action="/store/games/{{ g.filename }}/delete" class="inline">
<button class="danger">Delete</button>
</form>
</td>
</tr>
{% else %}
<tr><td colspan="3">No game archives.</td></tr>
{% endfor %}
</table>
<h2>Videos ({{ videos | length }})</h2>
<table>
<tr><th>Name</th><th>Size</th><th></th></tr>
{% for v in videos %}
<tr>
<td>{{ v.name }}</td>
<td>{{ (v.size / 1048576) | round(1) }} MB</td>
<td>
<form method="post" action="/store/videos/{{ v.filename }}/delete" class="inline">
<button class="danger">Delete</button>
</form>
</td>
</tr>
{% else %}
<tr><td colspan="3">No video archives.</td></tr>
{% endfor %}
</table>
{% endblock %}
+41
View File
@@ -0,0 +1,41 @@
{% extends "base.html" %}
{% block title %}Users · psppt{% endblock %}
{% block content %}
<h1>Users</h1>
<h2>Add user</h2>
<form method="post" action="/users" class="inline">
<input name="username" placeholder="username" required>
<input name="password" type="password" placeholder="password" required>
<select name="role">
<option value="user">user</option>
<option value="editor">editor</option>
<option value="admin">admin</option>
</select>
<button>Add</button>
</form>
<h2>Existing</h2>
<table>
<tr><th>Username</th><th>Role</th><th>Active</th><th>Actions</th></tr>
{% for u in users %}
<tr>
<td>{{ u.username }}</td>
<td>{{ u.role }}</td>
<td>{{ 'yes' if u.is_active else 'no' }}</td>
<td>
<form method="post" action="/users/{{ u.id }}" class="inline">
<select name="role">
<option value="user" {% if u.role == 'user' %}selected{% endif %}>user</option>
<option value="editor" {% if u.role == 'editor' %}selected{% endif %}>editor</option>
<option value="admin" {% if u.role == 'admin' %}selected{% endif %}>admin</option>
</select>
<input name="password" type="password" placeholder="new password">
<label class="check"><input type="checkbox" name="is_active" {% if u.is_active %}checked{% endif %}> active</label>
<button name="action" value="save">Save</button>
<button name="action" value="delete" class="danger">Delete</button>
</form>
</td>
</tr>
{% endfor %}
</table>
{% endblock %}
+339
View File
@@ -0,0 +1,339 @@
"""Server-rendered admin WebUI (Jinja2, plain HTML forms — no JS build)."""
from pathlib import Path
from fastapi import APIRouter, Depends, HTTPException, Request, status
from fastapi.responses import RedirectResponse
from fastapi.templating import Jinja2Templates
from app import store as store_mod
from app.db import get_connection
from . import config
from .api import _find_cover, _metadata, _release_system, _store_root, _SAFE_NAME_RE
from .security import get_user_by_token, hash_password, issue_token, now_iso, require_role, revoke_token, verify_password
router = APIRouter(include_in_schema=False)
templates = Jinja2Templates(directory=str(Path(__file__).parent / "templates"))
def _render(request, name, **ctx):
ctx.setdefault("user", getattr(request.state, "user", None))
return templates.TemplateResponse(request, name, ctx)
def web_user(request: Request) -> dict:
user = get_user_by_token(request.cookies.get("psppt_session", ""))
if user is None:
raise HTTPException(status_code=status.HTTP_303_SEE_OTHER, headers={"Location": "/login"})
request.state.user = user
return user
def web_admin(request: Request, user=Depends(web_user)) -> dict:
if user["role"] != "admin":
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Admin only")
return user
def web_editor(request: Request, user=Depends(web_user)) -> dict:
if user["role"] not in ("admin", "editor"):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Editor or admin required")
return user
# --------------------------------------------------------------------------- #
# auth pages
# --------------------------------------------------------------------------- #
@router.get("/login")
def login_page(request: Request):
return _render(request, "login.html")
@router.post("/login")
async def login_submit(request: Request):
form = await request.form()
username = (form.get("username") or "").strip()
password = form.get("password") or ""
conn = get_connection()
try:
row = conn.execute(
"SELECT id, username, password_hash, role, is_active FROM users WHERE username = ?", (username,)
).fetchone()
finally:
conn.close()
if row is None or not row["is_active"] or not verify_password(password, row["password_hash"]):
return _render(request, "login.html", error="Invalid username or password")
token = issue_token(row["id"], kind="session", label="web session")
conn = get_connection()
try:
conn.execute("UPDATE users SET last_login_at = ? WHERE id = ?", (now_iso(), row["id"]))
conn.commit()
finally:
conn.close()
resp = RedirectResponse("/dashboard", status_code=status.HTTP_303_SEE_OTHER)
resp.set_cookie("psppt_session", token, httponly=True, samesite="lax", path="/")
return resp
@router.post("/logout")
def logout(request: Request):
token = request.cookies.get("psppt_session", "")
if token:
revoke_token(token)
resp = RedirectResponse("/login", status_code=status.HTTP_303_SEE_OTHER)
resp.delete_cookie("psppt_session", path="/")
return resp
# --------------------------------------------------------------------------- #
# dashboard
# --------------------------------------------------------------------------- #
@router.get("/")
def root():
return RedirectResponse("/dashboard", status_code=status.HTTP_303_SEE_OTHER)
@router.get("/dashboard")
def dashboard(request: Request, user=Depends(web_user)):
conn = get_connection()
try:
systems = conn.execute(
"SELECT d.system AS system, COUNT(*) AS releases FROM releases r "
"JOIN dats d ON d.id = r.dat_id GROUP BY d.system ORDER BY d.system"
).fetchall()
users = conn.execute("SELECT COUNT(*) AS n FROM users").fetchone()["n"]
downloads = conn.execute("SELECT COUNT(*) AS n FROM download_events").fetchone()["n"]
covers = sum(1 for _ in config.COVERS_DIR.rglob("*") if _.is_file()) if config.COVERS_DIR.is_dir() else 0
recent = conn.execute(
"SELECT e.system, e.name, e.bytes, e.created_at, u.username FROM download_events e "
"LEFT JOIN users u ON u.id = e.user_id ORDER BY e.id DESC LIMIT 15"
).fetchall()
finally:
conn.close()
store_games = len(store_mod.scan(config.STORE_ROOT)) if config.STORE_ROOT.is_dir() else 0
store_videos = len(store_mod.scan(config.STORE_VIDEOS_ROOT)) if config.STORE_VIDEOS_ROOT.is_dir() else 0
return _render(request, "dashboard.html", systems=[dict(r) for r in systems], users=users,
downloads=downloads, covers=covers, recent=[dict(r) for r in recent],
store_games=store_games, store_videos=store_videos)
# --------------------------------------------------------------------------- #
# releases (games / videos browser)
# --------------------------------------------------------------------------- #
@router.get("/releases")
def releases_page(request: Request, system: str = "", q: str = "", user=Depends(web_user)):
sql = "SELECT r.id, d.system, r.name, r.region, r.serial FROM releases r JOIN dats d ON d.id = r.dat_id"
where, params = [], []
if system:
where.append("d.system = ?")
params.append(system)
if q:
where.append("r.name LIKE ?")
params.append(f"%{q}%")
if where:
sql += " WHERE " + " AND ".join(where)
sql += " ORDER BY r.name LIMIT 200"
conn = get_connection()
try:
rows = conn.execute(sql, params).fetchall()
items = []
for r in rows:
d = dict(r)
d["has_cover"] = _find_cover(r["id"], r["system"]).exists()
items.append(d)
finally:
conn.close()
return _render(request, "releases.html", items=items, system=system, q=q)
@router.get("/releases/{release_id}/edit")
def release_edit_page(release_id: int, request: Request, user=Depends(web_editor)):
conn = get_connection()
try:
rel = conn.execute(
"SELECT r.id, d.system, r.name, r.description, r.region, r.serial "
"FROM releases r JOIN dats d ON d.id = r.dat_id WHERE r.id = ?",
(release_id,),
).fetchone()
if rel is None:
raise HTTPException(status_code=404, detail="not found")
meta = _metadata(conn, release_id, rel["system"])
finally:
conn.close()
has_cover = _find_cover(release_id, rel["system"]).exists()
return _render(request, "release_edit.html", rel=dict(rel), meta=meta, has_cover=has_cover)
@router.post("/releases/{release_id}/edit")
async def release_edit_save(release_id: int, request: Request, user=Depends(web_editor)):
form = await request.form()
conn = get_connection()
try:
rel = conn.execute(
"SELECT d.system FROM releases r JOIN dats d ON d.id = r.dat_id WHERE r.id = ?", (release_id,)
).fetchone()
if rel is None:
raise HTTPException(status_code=404, detail="not found")
conn.execute(
"UPDATE releases SET name = ?, region = ?, serial = ?, description = ? WHERE id = ?",
(form.get("name") or "", form.get("region") or "", form.get("serial") or "",
form.get("description") or "", release_id),
)
meta = {}
for k in ("title", "release_date", "developer", "publisher", "genre", "synopsis"):
v = form.get(k)
if v is not None and str(v).strip():
meta[k] = str(v).strip()
if meta:
from .api import _save_metadata
_save_metadata(conn, release_id, rel["system"], meta)
conn.commit()
finally:
conn.close()
return RedirectResponse(f"/releases/{release_id}/edit", status_code=status.HTTP_303_SEE_OTHER)
@router.post("/releases/{release_id}/cover")
async def release_cover_upload(release_id: int, request: Request, user=Depends(web_editor)):
from .api import _save_cover
form = await request.form()
file = form.get("cover")
if file is None or not getattr(file, "filename", ""):
raise HTTPException(status_code=400, detail="no file")
data = await file.read()
conn = get_connection()
try:
system = _release_system(conn, release_id)
finally:
conn.close()
if system is None:
raise HTTPException(status_code=404, detail="not found")
_save_cover(release_id, system, data)
return RedirectResponse(f"/releases/{release_id}/edit", status_code=status.HTTP_303_SEE_OTHER)
@router.post("/releases/{release_id}/cover/delete")
def release_cover_delete(release_id: int, user=Depends(web_editor)):
from .api import _find_cover
conn = get_connection()
try:
system = _release_system(conn, release_id)
finally:
conn.close()
if system:
p = _find_cover(release_id, system)
if p.exists():
p.unlink()
return RedirectResponse(f"/releases/{release_id}/edit", status_code=status.HTTP_303_SEE_OTHER)
# --------------------------------------------------------------------------- #
# users (admin)
# --------------------------------------------------------------------------- #
@router.get("/users")
def users_page(request: Request, user=Depends(web_admin)):
conn = get_connection()
try:
rows = conn.execute(
"SELECT id, username, role, is_active, created_at, last_login_at FROM users ORDER BY username"
).fetchall()
finally:
conn.close()
return _render(request, "users.html", users=[dict(r) for r in rows])
@router.post("/users")
async def users_create(request: Request, user=Depends(web_admin)):
form = await request.form()
username = (form.get("username") or "").strip()
password = form.get("password") or ""
role = form.get("role") or "user"
if not username or not password:
raise HTTPException(status_code=400, detail="username and password required")
conn = get_connection()
try:
try:
conn.execute(
"INSERT INTO users (username, password_hash, role, is_active, created_at) VALUES (?, ?, ?, 1, ?)",
(username, hash_password(password), role, now_iso()),
)
conn.commit()
except Exception:
raise HTTPException(status_code=409, detail="username already exists")
finally:
conn.close()
return RedirectResponse("/users", status_code=status.HTTP_303_SEE_OTHER)
@router.post("/users/{user_id}")
async def users_update(user_id: int, request: Request, user=Depends(web_admin)):
form = await request.form()
action = form.get("action") or ""
conn = get_connection()
try:
if action == "delete":
if user_id != user["id"]:
conn.execute("DELETE FROM users WHERE id = ?", (user_id,))
else:
role = form.get("role")
if role in ("admin", "editor", "user"):
conn.execute("UPDATE users SET role = ? WHERE id = ?", (role, user_id))
conn.execute("UPDATE users SET is_active = ? WHERE id = ?", (1 if form.get("is_active") else 0, user_id))
pw = form.get("password") or ""
if pw:
conn.execute("UPDATE users SET password_hash = ? WHERE id = ?", (hash_password(pw), user_id))
conn.commit()
finally:
conn.close()
return RedirectResponse("/users", status_code=status.HTTP_303_SEE_OTHER)
# --------------------------------------------------------------------------- #
# store management (editor+)
# --------------------------------------------------------------------------- #
@router.get("/store")
def store_page(request: Request, user=Depends(web_editor)):
def _rows(root):
out = []
for it in store_mod.scan(root):
out.append({"name": it["name"], "filename": Path(it["path"]).name, "size": it["size"]})
return out
games = _rows(config.STORE_ROOT) if config.STORE_ROOT.is_dir() else []
videos = _rows(config.STORE_VIDEOS_ROOT) if config.STORE_VIDEOS_ROOT.is_dir() else []
return _render(request, "store.html", games=games, videos=videos)
@router.post("/store/upload")
async def store_upload(request: Request, user=Depends(web_editor)):
form = await request.form()
system = form.get("system") or "games"
file = form.get("file")
if file is None or not getattr(file, "filename", ""):
raise HTTPException(status_code=400, detail="no file")
data = await file.read()
root = _store_root(system)
root.mkdir(parents=True, exist_ok=True)
safe = _SAFE_NAME_RE.sub("", file.filename or "")
if not safe:
raise HTTPException(status_code=400, detail="invalid filename")
(root / safe).write_bytes(data)
return RedirectResponse("/store", status_code=status.HTTP_303_SEE_OTHER)
@router.post("/store/{system}/{name}/delete")
def store_delete(system: str, name: str, user=Depends(web_editor)):
root = _store_root(system).resolve()
target = (root / name).resolve()
if target.is_file() and root in target.parents:
target.unlink()
return RedirectResponse("/store", status_code=status.HTTP_303_SEE_OTHER)
# --------------------------------------------------------------------------- #
# settings (admin)
# --------------------------------------------------------------------------- #
@router.get("/settings")
def settings_page(request: Request, user=Depends(web_admin)):
return _render(request, "settings.html",
store_root=config.STORE_ROOT, store_videos_root=config.STORE_VIDEOS_ROOT,
covers_dir=config.COVERS_DIR)