Compare commits

...
22 Commits
Author SHA1 Message Date
quoije b378214e9b Run transfer queue jobs concurrently (max 3) so downloads don't block each other
build / Linux x86_64 (push) Successful in 3m15s
build / Windows x64 (Wine) (push) Successful in 3m41s
2026-09-14 12:50:05 -04:00
quoije 0110b50a07 Download cover bundle through the transfer queue (fix QProgressDialog NameError) 2026-09-14 12:47:13 -04:00
quoije 76eb6a6ef4 Smooth store source transitions: cancel stale batches and batch grid updates 2026-09-14 00:49:50 -04:00
quoije 58d348859a Fix store FD exhaustion: run cover/date loaders one at a time 2026-09-14 00:38:37 -04:00
quoije 3359ecf131 Fix store lag: decode cached covers off the GUI thread
build / Linux x86_64 (push) Successful in 2m48s
build / Windows x64 (Wine) (push) Successful in 3m17s
2026-09-14 00:24:48 -04:00
quoije 38a3859a21 Unbundle covers: fetch psppt-covers.zip from repo on demand with user prompt
build / Linux x86_64 (push) Successful in 2m28s
build / Windows x64 (Wine) (push) Successful in 3m8s
2026-09-14 00:03:09 -04:00
quoije 96e876b8ec Harden shutdown: no segfault closing mid-load (thread cleanup + graceful Ctrl+C) 2026-09-13 23:49:17 -04:00
quoije 733e3c9089 Fix UI freeze on first cover load (decode+scale off the GUI thread)
build / Linux x86_64 (push) Failing after 8m22s
build / Windows x64 (Wine) (push) Canceled after 17m46s
2026-09-13 23:20:08 -04:00
quoije dc7d6c9720 Rebrand to psppt (psp-preserve -> psppt) 2026-09-13 23:08:22 -04:00
quoije 5c5bb6e5d6 fix line break 2026-09-13 22:57:08 -04:00
quoije 35db22e3c5 Add grrr meme to README (resized to 156x148) 2026-09-13 22:55:46 -04:00
quoije b0cdac5483 Put contents_directory on EXE (COLLECT reads it from the exe) 2026-09-13 22:33:04 -04:00
quoije f367bbe298 Clean /usr/bin install: contents dir 'lib' + install.sh
build / Linux x86_64 (push) Successful in 13m11s
build / Windows x64 (Wine) (push) Successful in 20m38s
2026-09-13 21:47:32 -04:00
quoije c61d8e9d3f Revert contents_directory (collides with exe name)
build / Linux x86_64 (push) Failing after 3m8s
build / Windows x64 (Wine) (push) Canceled after 0s
2026-09-13 21:43:28 -04:00
quoije 78f6d44c73 Fix frozen-build SSL certs, taskbar icon, and rename _internal to psp-preserve
build / Linux x86_64 (push) Failing after 7m17s
build / Windows x64 (Wine) (push) Canceled after 0s
- config: point SSL at certifi bundle when frozen (fixes store/NPS/TMDB SSL errors)
- gui: set app name + window icon for the taskbar
- spec: contents_directory=psp-preserve; bundle a .desktop file for Wayland launchers
2026-09-13 21:19:56 -04:00
quoije a7f90cd5a7 Runner image: use WineHQ wine-staging + embeddable Python (Wine 8.0 too old for Py3.12) 2026-09-13 21:13:30 -04:00
quoije c9575817fe Runner image: use full Python installer (bundles pip + OpenSSL) 2026-09-13 21:11:45 -04:00
quoije b576ff5747 Settings: add Storage section showing state dir, DB, cover cache, and user data paths 2026-09-13 21:07:23 -04:00
quoije 6510229b07 Add cover-grid sorting to Library and Store
Library: Title / Release date / Size / Region / Type / Serial
Store: Newest first / Title / Region / Size / In library first
2026-09-13 21:03:49 -04:00
quoije e61d91418e Runner image: add wine32 (winetricks regedit needs 32-bit Wine) 2026-09-13 20:52:42 -04:00
quoije a1c461f6aa Runner image: use embeddable Python (avoid 32-bit installer bootstrap) 2026-09-13 20:50:57 -04:00
quoije 985c8df2cc Runner image: add xauth (xvfb-run requirement) 2026-09-13 20:38:48 -04:00
23 changed files with 684 additions and 179 deletions
+12 -8
View File
@@ -22,15 +22,19 @@ jobs:
pip install -r requirements-build.txt pip install -r requirements-build.txt
- name: Build - name: Build
run: pyinstaller --clean --noconfirm psp-preserve.spec run: pyinstaller --clean --noconfirm psppt.spec
- name: Package tarball - name: Package tarball
run: tar -czf psp-preserve-linux-x86_64.tar.gz -C dist psp-preserve run: |
cp packaging/install.sh dist/psppt/
cp packaging/psppt.desktop dist/psppt/
cp img/app-icon.png dist/psppt/
tar -czf psppt-linux-x86_64.tar.gz -C dist psppt
- uses: actions/upload-artifact@v3 - uses: actions/upload-artifact@v3
with: with:
name: psp-preserve-linux-x86_64 name: psppt-linux-x86_64
path: psp-preserve-linux-x86_64.tar.gz path: psppt-linux-x86_64.tar.gz
windows: windows:
name: Windows x64 (Wine) name: Windows x64 (Wine)
@@ -44,13 +48,13 @@ jobs:
winpython -m pip install -r requirements-build.txt winpython -m pip install -r requirements-build.txt
- name: Build - name: Build
run: winpython -m PyInstaller --clean --noconfirm psp-preserve.spec run: winpython -m PyInstaller --clean --noconfirm psppt.spec
- name: Package zip - name: Package zip
run: | run: |
cd dist && zip -rq ../psp-preserve-windows-x64.zip psp-preserve cd dist && zip -rq ../psppt-windows-x64.zip psppt
- uses: actions/upload-artifact@v3 - uses: actions/upload-artifact@v3
with: with:
name: psp-preserve-windows-x64 name: psppt-windows-x64
path: psp-preserve-windows-x64.zip path: psppt-windows-x64.zip
+4
View File
@@ -24,3 +24,7 @@ data/nps/*.tsv
# PyInstaller output # PyInstaller output
build/ build/
dist/ dist/
# Offline cover bundles (large; fetched from the project release on first run)
data/covers/*.zip
/psppt-covers.zip
+1 -1
View File
@@ -3,5 +3,5 @@
# Port-forward 80 and 443 on your router to this host. # Port-forward 80 and 443 on your router to this host.
hostname.com { hostname.com {
reverse_proxy psp-preserve:8000 reverse_proxy psppt:8000
} }
+37 -18
View File
@@ -1,6 +1,10 @@
# PSP Preservation Tool # psppt
the name is terrible, i know. the name is terrible, i know.
![grrr](img/grrr.png)
![AI Generated](https://raw.githubusercontent.com/dotnetdave/ai-usage-badges/refs/heads/main/badges/png/1x/ai-generated.png) ![AI Generated](https://raw.githubusercontent.com/dotnetdave/ai-usage-badges/refs/heads/main/badges/png/1x/ai-generated.png)
A cross-platform (Linux + Windows) desktop GUI — PySide6 — for preserving PSP A cross-platform (Linux + Windows) desktop GUI — PySide6 — for preserving PSP
@@ -11,7 +15,7 @@ save-data with a real PSP, can pull missing discs from a Redump 7z archive
collection, downloads PSN content from NoPayStation, and converts PlayStation 1 collection, downloads PSN content from NoPayStation, and converts PlayStation 1
ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew. ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew.
![PSP Preservation Tool](img/qt-gui.png) ![psppt](img/qt-gui.png)
* Dark theme with a blue accent — the Library tab in cover view. * * Dark theme with a blue accent — the Library tab in cover view. *
## Features ## Features
@@ -69,14 +73,16 @@ see the note inside).
python3 -m venv .venv-build python3 -m venv .venv-build
. .venv-build/bin/activate . .venv-build/bin/activate
pip install -r requirements-build.txt pip install -r requirements-build.txt
pyinstaller --clean --noconfirm psp-preserve.spec pyinstaller --clean --noconfirm psppt.spec
# output: dist/psp-preserve/ → run dist/psp-preserve/psp-preserve # output: dist/psppt/ → run dist/psppt/psppt
``` ```
Notes: Notes:
- The build bundles the DATs, PS1 loader, and the offline cover bundles. The - The build bundles the DATs, PS1 loader, cover templates, and index catalogs
cover zips are large (~850 MB), so builds are slow and the output is big. (~80 MB total). The large offline cover zips (~850 MB) are **not** bundled —
the app fetches them on first run (see
[Offline bundles](#offline-bundles--auto-download)).
- `7z` (p7zip-full on Linux, 7-Zip on Windows) is a **runtime** dependency for - `7z` (p7zip-full on Linux, 7-Zip on Windows) is a **runtime** dependency for
reading store archives — it is not bundled. reading store archives — it is not bundled.
- User data (credentials, manual covers, PS1 templates, NPS cache, library DB, - User data (credentials, manual covers, PS1 templates, NPS cache, library DB,
@@ -166,9 +172,10 @@ Discs are matched by hash first (most precise), then serial, then filename.
- `data/credentials.json` — cover/metadata API keys (IGDB, TheMovieDB, - `data/credentials.json` — cover/metadata API keys (IGDB, TheMovieDB,
ScreenScraper, Google Custom Search). Environment variables take precedence. ScreenScraper, Google Custom Search). Environment variables take precedence.
- `data/manual_covers/games|videos/` — your own cover images. - `data/manual_covers/games|videos/` — your own cover images.
- `data/covers/` — checked-in assets: the UMD video/game cover templates, - `data/covers/` — checked-in assets: the UMD video/game cover templates and
`libretro-index.json` + `openvgdb-index.json` catalogs, `libretro-covers.zip` + the `libretro-index.json` + `openvgdb-index.json` catalogs. The large
`openvgdb-covers.zip` offline cover bundles, and the legacy cover cache. `libretro-covers.zip` + `openvgdb-covers.zip` bundles are git-ignored and
distributed as `psppt-covers.zip` (see above).
- `data/base.PBP` — a PSN PS1 classic EBOOT (any game) supplying `STARTDAT` - `data/base.PBP` — a PSN PS1 classic EBOOT (any game) supplying `STARTDAT`
+ a base `PARAM.SFO`/icon for PS1 conversion (git-ignored, copyrighted). + a base `PARAM.SFO`/icon for PS1 conversion (git-ignored, copyrighted).
- `data/loader/DATA.PSP` + `data/loader/KEYS.BIN` — bundled plain-ELF POPS - `data/loader/DATA.PSP` + `data/loader/KEYS.BIN` — bundled plain-ELF POPS
@@ -178,7 +185,7 @@ Discs are matched by hash first (most precise), then serial, then filename.
to drawn art when they're absent. to drawn art when they're absent.
- `data/nps/` — NoPayStation TSV lists (cached here on first store scan in dev; - `data/nps/` — NoPayStation TSV lists (cached here on first store scan in dev;
a packaged build stores them under the state dir instead). a packaged build stores them under the state dir instead).
- State dir (`~/.local/state/psp-preserve/` on Linux, `%LOCALAPPDATA%\PSPPreserve` - State dir (`~/.local/state/psppt/` on Linux, `%LOCALAPPDATA%\psppt`
on Windows, or `$PSP_STATE_DIR`): on Windows, or `$PSP_STATE_DIR`):
- `library.sqlite` — the catalog + library database. - `library.sqlite` — the catalog + library database.
- `covers/` — downloaded cover cache. - `covers/` — downloaded cover cache.
@@ -192,13 +199,13 @@ bundled with the app; user-writable data lives in the state dir.
## Configuration ## Configuration
Everything is driven by environment variables, with GUI settings (stored in Everything is driven by environment variables, with GUI settings (stored in
QSettings under `PSPPreserve`) taking precedence where noted. QSettings under `psppt`) taking precedence where noted.
| Variable | Default | Purpose | | Variable | Default | Purpose |
|---|---|---| |---|---|---|
| `PSP_DATA_DIR` | `./data` | data root (DATs, credentials, manual covers) | | `PSP_DATA_DIR` | `./data` | data root (DATs, credentials, manual covers) |
| `PSP_DATS_DIR` | `$PSP_DATA_DIR/dats` | DAT folder | | `PSP_DATS_DIR` | `$PSP_DATA_DIR/dats` | DAT folder |
| `PSP_STATE_DIR` | `~/.local/state/psp-preserve` | local state root | | `PSP_STATE_DIR` | `~/.local/state/psppt` | local state root |
| `PSP_DB_PATH` | `$PSP_STATE_DIR/library.sqlite` | SQLite library | | `PSP_DB_PATH` | `$PSP_STATE_DIR/library.sqlite` | SQLite library |
| `PSP_COVERS_DIR` | `$PSP_STATE_DIR/covers` | cover cache | | `PSP_COVERS_DIR` | `$PSP_STATE_DIR/covers` | cover cache |
| `PSP_GAMES_ROOTS` | `/data/umd_games` | comma-separated game disc roots | | `PSP_GAMES_ROOTS` | `/data/umd_games` | comma-separated game disc roots |
@@ -219,6 +226,7 @@ QSettings under `PSPPreserve`) taking precedence where noted.
| `PSP_CREDENTIALS` | `$PSP_USER_DATA_DIR/credentials.json` | credentials file | | `PSP_CREDENTIALS` | `$PSP_USER_DATA_DIR/credentials.json` | credentials file |
| `PSP_LIBRETRO_COVERS_URL` | *(empty)* | direct URL for `libretro-covers.zip` (else derived from store server) | | `PSP_LIBRETRO_COVERS_URL` | *(empty)* | direct URL for `libretro-covers.zip` (else derived from store server) |
| `PSP_OVGDB_COVERS_URL` | *(empty)* | direct URL for `openvgdb-covers.zip` (else derived from store server) | | `PSP_OVGDB_COVERS_URL` | *(empty)* | direct URL for `openvgdb-covers.zip` (else derived from store server) |
| `PSP_COVERS_BUNDLE_URL` | Gitea `covers` release asset | combined `psppt-covers.zip` download URL |
Cover/metadata keys can also be set as env vars: `IGDB_CLIENT_ID`, Cover/metadata keys can also be set as env vars: `IGDB_CLIENT_ID`,
`IGDB_CLIENT_SECRET`, `TMDB_API_KEY`, `GOOGLE_API_KEY`, `GOOGLE_CX`, `IGDB_CLIENT_SECRET`, `TMDB_API_KEY`, `GOOGLE_API_KEY`, `GOOGLE_CX`,
@@ -251,12 +259,23 @@ Manual covers in `data/manual_covers/videos/` always win.
### Offline bundles & auto-download ### Offline bundles & auto-download
The two bundles (`libretro-covers.zip`, `openvgdb-covers.zip`) are shipped in The two bundles (`libretro-covers.zip`, `openvgdb-covers.zip`) are packaged
`data/covers/`, so a fresh install works with **no internet and no API keys**. into one `psppt-covers.zip` (~850 MB) and attached to the repo's `covers`
If either is missing at startup, the app downloads it from the store server's release — **not** bundled into the app binary.
`/api/covers/<name>.zip` endpoint (see [Server setup](#server-setup)).
Rebuild them yourself: On first launch, if the bundle isn't installed, the app asks whether to
download it (with a progress bar). Covers keep working online via libretro /
OpenVGDB / ScreenScraper / IGDB without it — the bundle just makes them instant
and offline. "Don't ask again" is remembered, and you can re-download from
**Settings → Cover sources → Download offline cover bundle…**.
Build the bundle yourself:
```bash
python3 scripts/make-covers-bundle.py # -> psppt-covers.zip
```
Rebuild the inner zips:
```bash ```bash
python3 -m app.libretro # libretro-thumbnails -> libretro-covers.zip python3 -m app.libretro # libretro-thumbnails -> libretro-covers.zip
@@ -423,7 +442,7 @@ Token protection: set `PSP_API_TOKEN`; the client sends
macvlan bridge so it gets its own LAN IP (e.g. `192.168.1.100`, plain HTTP on 8000): macvlan bridge so it gets its own LAN IP (e.g. `192.168.1.100`, plain HTTP on 8000):
```bash ```bash
# in /path/to/psp-preserve/ # in /path/to/psppt/
cp .env.example .env # then edit: LAN IP, share paths, API token cp .env.example .env # then edit: LAN IP, share paths, API token
docker compose up -d --build docker compose up -d --build
``` ```
+1 -1
View File
@@ -1 +1 @@
"""PSP Preservation Tool application package.""" """psppt application package."""
+2 -2
View File
@@ -251,7 +251,7 @@ def _library(conn, system=None, refresh=False):
class Handler(BaseHTTPRequestHandler): class Handler(BaseHTTPRequestHandler):
server_version = "PSPPreserve/0.1" server_version = "psppt/0.1"
def do_GET(self): def do_GET(self):
parsed = urlparse(self.path) parsed = urlparse(self.path)
@@ -262,7 +262,7 @@ class Handler(BaseHTTPRequestHandler):
try: try:
if path == "/": if path == "/":
_json(self, { _json(self, {
"name": "PSP Preservation Tool API", "name": "psppt API",
"endpoints": [ "endpoints": [
"/api/summary", "/api/summary",
"/api/releases?system=games|videos&q=&limit=&offset=", "/api/releases?system=games|videos&q=&limit=&offset=",
+22 -2
View File
@@ -11,6 +11,19 @@ _BUNDLE_DIR = getattr(sys, "_MEIPASS", None)
PROJECT_ROOT = Path(_BUNDLE_DIR).resolve() if _BUNDLE_DIR else Path(__file__).resolve().parent.parent PROJECT_ROOT = Path(_BUNDLE_DIR).resolve() if _BUNDLE_DIR else Path(__file__).resolve().parent.parent
# Frozen bundles don't ship the system CA store, so Python's urllib would fail
# TLS verification against the store server / NoPayStation / TMDB / archive.org.
# Point OpenSSL at the certifi bundle (a curl_cffi dependency) when frozen only;
# source runs keep using the OS CA store.
if IS_FROZEN:
try:
import certifi
_CA_BUNDLE = certifi.where()
os.environ.setdefault("SSL_CERT_FILE", _CA_BUNDLE)
os.environ.setdefault("REQUESTS_CA_BUNDLE", _CA_BUNDLE)
except Exception: # noqa: BLE001
pass
def resource_path(*parts): def resource_path(*parts):
"""Absolute path to a read-only bundled resource (works when frozen).""" """Absolute path to a read-only bundled resource (works when frozen)."""
@@ -29,10 +42,10 @@ def _default_state_dir():
""" """
if os.name == "nt": if os.name == "nt":
base = os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or str(Path.home()) base = os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or str(Path.home())
return Path(base) / "PSPPreserve" return Path(base) / "psppt"
xdg = os.environ.get("XDG_STATE_HOME") xdg = os.environ.get("XDG_STATE_HOME")
base = Path(xdg) if xdg else Path.home() / ".local" / "state" base = Path(xdg) if xdg else Path.home() / ".local" / "state"
return base / "psp-preserve" return base / "psppt"
STATE_DIR = Path(os.environ.get("PSP_STATE_DIR", str(_default_state_dir()))) STATE_DIR = Path(os.environ.get("PSP_STATE_DIR", str(_default_state_dir())))
@@ -63,6 +76,13 @@ STORE_TOKEN = os.environ.get("PSP_STORE_TOKEN", "")
LIBRETRO_COVERS_URL = os.environ.get("PSP_LIBRETRO_COVERS_URL", "") LIBRETRO_COVERS_URL = os.environ.get("PSP_LIBRETRO_COVERS_URL", "")
OVGDB_COVERS_URL = os.environ.get("PSP_OVGDB_COVERS_URL", "") OVGDB_COVERS_URL = os.environ.get("PSP_OVGDB_COVERS_URL", "")
# Combined offline cover bundle (both inner zips) fetched on first run when the
# user opts in. Defaults to the project's Gitea release asset.
COVERS_BUNDLE_URL = os.environ.get(
"PSP_COVERS_BUNDLE_URL",
"https://git.ceaf.dev/quoije/psp-preservation-tool/releases/download/covers/psppt-covers.zip",
)
# archive.org item hosting the same Redump 7z set (identifier or full URL). # archive.org item hosting the same Redump 7z set (identifier or full URL).
ARCHIVE_ITEM = os.environ.get("PSP_ARCHIVE_ITEM", "psp-redump-roms321.com") ARCHIVE_ITEM = os.environ.get("PSP_ARCHIVE_ITEM", "psp-redump-roms321.com")
+74 -34
View File
@@ -53,8 +53,20 @@ _TMDB_SEARCH = "https://api.themoviedb.org/3/search/movie"
_TMDB_IMAGE = "https://image.tmdb.org/t/p/" _TMDB_IMAGE = "https://image.tmdb.org/t/p/"
_GOOGLE_CSE_API = "https://www.googleapis.com/customsearch/v1" _GOOGLE_CSE_API = "https://www.googleapis.com/customsearch/v1"
_MANUAL_DIR = config.USER_DATA_DIR / "manual_covers" _MANUAL_DIR = config.USER_DATA_DIR / "manual_covers"
_OVGDB_ZIP = config.DATA_DIR / "covers" / "openvgdb-covers.zip" # Offline cover bundles live in the writable cover dir (downloaded on demand),
_LIBRETRO_ZIP = config.DATA_DIR / "covers" / "libretro-covers.zip" # not the read-only app bundle. A source checkout keeps them in data/covers/.
_OVGDB_ZIP = config.COVERS_DIR / "openvgdb-covers.zip"
_LIBRETRO_ZIP = config.COVERS_DIR / "libretro-covers.zip"
_BUNDLE_ZIP = config.COVERS_DIR / "psppt-covers.zip"
def _bundle_zip_path(name):
"""Resolve a cover bundle zip: writable dir first, legacy checkout second."""
for base in (config.COVERS_DIR, config.LEGACY_COVERS_DIR):
p = base / name
if p.exists():
return p
return config.COVERS_DIR / name
_WIKI_SEARCH = "https://en.wikipedia.org/w/api.php" _WIKI_SEARCH = "https://en.wikipedia.org/w/api.php"
_WIKI_SUMMARY = "https://en.wikipedia.org/api/rest_v1/page/summary/" _WIKI_SUMMARY = "https://en.wikipedia.org/api/rest_v1/page/summary/"
@@ -62,7 +74,7 @@ _WIKI_SUMMARY = "https://en.wikipedia.org/api/rest_v1/page/summary/"
_PNG_MAGIC = b"\x89PNG\r\n\x1a\n" _PNG_MAGIC = b"\x89PNG\r\n\x1a\n"
_JPEG_MAGIC = b"\xff\xd8\xff" _JPEG_MAGIC = b"\xff\xd8\xff"
_MAX_HOPS = 3 # follow libretro symlink text bodies at most this many times _MAX_HOPS = 3 # follow libretro symlink text bodies at most this many times
_UA = {"User-Agent": "psp-preserve/0.1 (https://github.com; local tool)"} _UA = {"User-Agent": "psppt/0.1 (https://github.com; local tool)"}
_igdb_token_state = {"token": "", "expires": 0.0} # IGDB OAuth2 token cache _igdb_token_state = {"token": "", "expires": 0.0} # IGDB OAuth2 token cache
_igdb_lock = threading.Lock() _igdb_lock = threading.Lock()
_igdb_last = 0.0 # monotonic clock of the last IGDB API call _igdb_last = 0.0 # monotonic clock of the last IGDB API call
@@ -158,48 +170,74 @@ def clear_cover_cache():
pass pass
def _download_file(url, dest, timeout=600): def _download_file(url, dest, timeout=600, progress_cb=None, cancel_event=None):
"""Stream `url` to `dest` (atomic); returns True on success.""" """Stream `url` to `dest` (atomic); returns True on success.
``progress_cb(done, total)`` is called with bytes so far (total is 0 when
the server doesn't report a content length). ``cancel_event`` aborts the
download when set.
"""
req = urllib.request.Request(url, headers=_UA) req = urllib.request.Request(url, headers=_UA)
with urllib.request.urlopen(req, timeout=timeout) as resp: with urllib.request.urlopen(req, timeout=timeout) as resp:
if resp.status != 200: if resp.status != 200:
return False return False
total = int(resp.headers.get("Content-Length") or 0)
tmp = dest.with_name(dest.name + ".tmp") tmp = dest.with_name(dest.name + ".tmp")
done = 0
with open(tmp, "wb") as f: with open(tmp, "wb") as f:
while True: while True:
if cancel_event is not None and cancel_event.is_set():
raise InterruptedError("download cancelled")
chunk = resp.read(1 << 20) chunk = resp.read(1 << 20)
if not chunk: if not chunk:
break break
f.write(chunk) f.write(chunk)
done += len(chunk)
if progress_cb is not None:
progress_cb(done, total)
os.replace(tmp, dest) os.replace(tmp, dest)
return True return True
def ensure_cover_bundles(server_url=""): def covers_bundle_present():
"""Download missing offline cover bundle zips from the store server. """True when the offline cover bundle is installed (both inner zips)."""
return (_bundle_zip_path("libretro-covers.zip").exists()
and _bundle_zip_path("openvgdb-covers.zip").exists())
``server_url`` is the base URL (e.g. ``http://192.168.10.100:8000``); the
zips are fetched from ``<base>/api/covers/<name>.zip`` unless a direct URL def download_covers_bundle(url=None, progress_cb=None, cancel_event=None):
is set in config. Returns the names of bundles downloaded. """Download and extract the combined cover bundle into the cover dir.
``url`` defaults to ``config.COVERS_BUNDLE_URL``. Raises on failure; the
caller decides whether to surface the error.
""" """
base = (server_url or config.STORE_SERVER_URL or "").rstrip("/") url = (url or config.COVERS_BUNDLE_URL or "").strip()
downloaded = []
for name, path, direct in (
("libretro-covers.zip", _LIBRETRO_ZIP, config.LIBRETRO_COVERS_URL),
("openvgdb-covers.zip", _OVGDB_ZIP, config.OVGDB_COVERS_URL),
):
if path.exists():
continue
url = direct or (f"{base}/api/covers/{name}" if base else "")
if not url: if not url:
continue raise ValueError("no cover bundle URL configured")
try: config.COVERS_DIR.mkdir(parents=True, exist_ok=True)
if _download_file(url, path): _download_file(url, _BUNDLE_ZIP, timeout=3600,
downloaded.append(name) progress_cb=progress_cb, cancel_event=cancel_event)
log.info("downloaded offline cover bundle %s", name) inner = ("libretro-covers.zip", "openvgdb-covers.zip")
except Exception as exc: # noqa: BLE001 with zipfile.ZipFile(_BUNDLE_ZIP) as z:
log.warning("could not download cover bundle %s: %s", name, exc) missing = [n for n in inner if n not in z.namelist()]
return downloaded if missing:
raise ValueError(f"cover bundle is missing: {', '.join(missing)}")
with zipfile.ZipFile(_BUNDLE_ZIP) as z:
for name in inner:
out = config.COVERS_DIR / name
tmp = out.with_name(out.name + ".tmp")
with z.open(name) as src, open(tmp, "wb") as dst:
while True:
chunk = src.read(1 << 20)
if not chunk:
break
dst.write(chunk)
os.replace(tmp, out)
# Invalidate cached namelists so lookups pick up the freshly extracted zips.
global _libretro_names, _ovgdb_names
_libretro_names = None
_ovgdb_names = None
return config.COVERS_DIR
def cover_url(name, kind="Named_Boxarts", system="games"): def cover_url(name, kind="Named_Boxarts", system="games"):
@@ -482,9 +520,10 @@ def _ovgdb_name_list():
global _ovgdb_names global _ovgdb_names
if _ovgdb_names is None: if _ovgdb_names is None:
_ovgdb_names = set() _ovgdb_names = set()
if _OVGDB_ZIP.exists(): path = _bundle_zip_path("openvgdb-covers.zip")
if path.exists():
try: try:
with zipfile.ZipFile(_OVGDB_ZIP) as z: with zipfile.ZipFile(path) as z:
_ovgdb_names = {n[:-4] for n in z.namelist() if n.endswith(".jpg")} _ovgdb_names = {n[:-4] for n in z.namelist() if n.endswith(".jpg")}
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
pass pass
@@ -535,7 +574,7 @@ def fetch_openvgdb_local(name):
if entry is None: if entry is None:
return None return None
try: try:
with zipfile.ZipFile(_OVGDB_ZIP) as z: with zipfile.ZipFile(_bundle_zip_path("openvgdb-covers.zip")) as z:
data = z.read(entry) data = z.read(entry)
return data if _is_image(data) else None return data if _is_image(data) else None
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
@@ -559,7 +598,7 @@ def fetch_libretro_local(name):
if entry is None: if entry is None:
return None return None
try: try:
with zipfile.ZipFile(_LIBRETRO_ZIP) as z: with zipfile.ZipFile(_bundle_zip_path("libretro-covers.zip")) as z:
data = z.read(entry) data = z.read(entry)
return data if _is_image(data) else None return data if _is_image(data) else None
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
@@ -570,9 +609,10 @@ def _libretro_name_list():
global _libretro_names global _libretro_names
if _libretro_names is None: if _libretro_names is None:
_libretro_names = set() _libretro_names = set()
if _LIBRETRO_ZIP.exists(): path = _bundle_zip_path("libretro-covers.zip")
if path.exists():
try: try:
with zipfile.ZipFile(_LIBRETRO_ZIP) as z: with zipfile.ZipFile(path) as z:
_libretro_names = {n[:-4] for n in z.namelist() if n.endswith(".png")} _libretro_names = {n[:-4] for n in z.namelist() if n.endswith(".png")}
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
pass pass
@@ -610,7 +650,7 @@ def _fetch_screenscraper(name, timeout, system="games"):
params = { params = {
"devid": devid, "devid": devid,
"devpassword": devpassword, "devpassword": devpassword,
"softname": creds.get("screenscraper_softname") or "psp-preserve", "softname": creds.get("screenscraper_softname") or "psppt",
"output": "json", "output": "json",
"systemeid": _platform(system)["ss_id"], "systemeid": _platform(system)["ss_id"],
"romtype": "rom", "romtype": "rom",
+289 -43
View File
@@ -1,4 +1,4 @@
"""Native Linux GUI for the PSP Preservation Tool (PySide6 / Qt). """Native Linux GUI for psppt (PySide6 / Qt).
Run with:: Run with::
@@ -7,6 +7,7 @@ Run with::
import datetime import datetime
import json import json
import logging import logging
import signal
import sys import sys
import threading import threading
from pathlib import Path from pathlib import Path
@@ -117,14 +118,16 @@ def _thread_running(thread):
class CoverLoader(QObject): class CoverLoader(QObject):
loaded = Signal(int, int, str) # gen, item index, cached image path loaded = Signal(int, int, QImage) # gen, item index, fitted cover image
failed = Signal(int, int, str) # gen, item index, name (no cover found) failed = Signal(int, int, str) # gen, item index, name (no cover found)
finished = Signal() finished = Signal()
def __init__(self, items, gen): def __init__(self, items, gen, icon_w=150, icon_h=220):
super().__init__() super().__init__()
self.items = items self.items = items
self.gen = gen self.gen = gen
self.icon_w = icon_w
self.icon_h = icon_h
self._cancel = threading.Event() self._cancel = threading.Event()
def cancel(self): def cancel(self):
@@ -148,11 +151,17 @@ class CoverLoader(QObject):
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
path = None path = None
if path is not None: if path is not None:
self.loaded.emit(self.gen, i, str(path)) img = QImage(str(path))
if img.isNull():
self.failed.emit(self.gen, i, self.items[i]["name"])
else:
self.loaded.emit(self.gen, i, _fit_image(img, self.icon_w, self.icon_h))
else: else:
self.failed.emit(self.gen, i, self.items[i]["name"]) self.failed.emit(self.gen, i, self.items[i]["name"])
finally: finally:
pool.shutdown(wait=True, cancel_futures=True) # Return promptly on cancel/exit instead of blocking the GUI thread
# on in-flight network timeouts during shutdown.
pool.shutdown(wait=not self._cancel.is_set(), cancel_futures=True)
self.finished.emit() self.finished.emit()
@@ -276,6 +285,25 @@ def _fit_pixmap(pm, w, h):
return canvas return canvas
def _fit_image(img, w, h):
"""Scale a QImage into a letterboxed ``w x h`` ARGB image (thread-safe).
Decoding + scaling happen off the GUI thread in the cover loaders; the
main thread only converts the result to a QPixmap (cheap).
"""
canvas = QImage(w, h, QImage.Format_ARGB32)
canvas.fill(Qt.transparent)
if img.isNull():
return canvas
scaled = img.scaled(w, h, Qt.KeepAspectRatio, Qt.SmoothTransformation)
x = (w - scaled.width()) // 2
y = (h - scaled.height()) // 2
painter = QPainter(canvas)
painter.drawImage(x, y, scaled)
painter.end()
return canvas
def _cover_pixmap(rid, system, max_w, max_h): def _cover_pixmap(rid, system, max_w, max_h):
"""Return a scaled cover pixmap for an item, or None if unavailable.""" """Return a scaled cover pixmap for an item, or None if unavailable."""
if rid is None: if rid is None:
@@ -429,7 +457,7 @@ class CoverShowcase(QWidget):
for loader in self._loaders: for loader in self._loaders:
loader.cancel() loader.cancel()
self.thread = QThread(self) self.thread = QThread(self)
self.loader = CoverLoader(self.items, gen) self.loader = CoverLoader(self.items, gen, self.ICON_W, self.ICON_H)
self.loader.moveToThread(self.thread) self.loader.moveToThread(self.thread)
self.thread.started.connect(self.loader.run) self.thread.started.connect(self.loader.run)
self.loader.loaded.connect(self._on_loaded) self.loader.loaded.connect(self._on_loaded)
@@ -454,12 +482,12 @@ class CoverShowcase(QWidget):
painter.end() painter.end()
return QIcon(pm) return QIcon(pm)
def _on_loaded(self, gen, index, path): def _on_loaded(self, gen, index, image):
if gen != self._generation: if self._closing or gen != self._generation:
return return
item = self.list.item(index) item = self.list.item(index)
if item is not None: if item is not None:
item.setIcon(self._fit_icon(QPixmap(path))) item.setIcon(QIcon(QPixmap.fromImage(image)))
if 0 <= index < len(self.items): if 0 <= index < len(self.items):
self.cover_loaded.emit(self.items[index]) self.cover_loaded.emit(self.items[index])
@@ -477,7 +505,7 @@ class CoverShowcase(QWidget):
self.play_requested.emit(data) self.play_requested.emit(data)
def _on_failed(self, gen, index, name): def _on_failed(self, gen, index, name):
if gen != self._generation: if self._closing or gen != self._generation:
return return
if index not in self.missing: if index not in self.missing:
self.missing.append(index) self.missing.append(index)
@@ -485,6 +513,8 @@ class CoverShowcase(QWidget):
self._update_status() self._update_status()
def _on_loading_done(self): def _on_loading_done(self):
if self._closing:
return
sender = self.sender() sender = self.sender()
if sender is not None and getattr(sender, "gen", None) != self._generation: if sender is not None and getattr(sender, "gen", None) != self._generation:
return return
@@ -492,6 +522,8 @@ class CoverShowcase(QWidget):
self._update_status() self._update_status()
def _update_status(self): def _update_status(self):
if self._closing:
return
if self._loading: if self._loading:
self.status.setText("Loading covers…") self.status.setText("Loading covers…")
self.missing_changed.emit(0) self.missing_changed.emit(0)
@@ -1111,13 +1143,14 @@ class ScrapeDialog(QDialog):
class SettingsPage(QWidget): class SettingsPage(QWidget):
saved = Signal() saved = Signal()
download_covers = Signal()
def __init__(self, games_dir, videos_dir, store_dir, ppsspp_dir, cloud_dir, ppsspp_exe, def __init__(self, games_dir, videos_dir, store_dir, ppsspp_dir, cloud_dir, ppsspp_exe,
store_server_url, store_token, store_archive_id, store_archive_sig, store_archive_user, store_server_url, store_token, store_archive_id, store_archive_sig, store_archive_user,
iso_category, ps1_category, psp_category, ps1_dir, ps1_output_dir, ps1_base_pbp, iso_category, ps1_category, psp_category, ps1_dir, ps1_output_dir, ps1_base_pbp,
nps_dir="", parent=None): nps_dir="", parent=None):
super().__init__(parent) super().__init__(parent)
s = QSettings("PSPPreserve", "PSPPreserve") s = QSettings("psppt", "psppt")
outer = QVBoxLayout(self) outer = QVBoxLayout(self)
outer.setContentsMargins(16, 16, 16, 16) outer.setContentsMargins(16, 16, 16, 16)
@@ -1278,6 +1311,25 @@ class SettingsPage(QWidget):
self.clear_covers_btn = QPushButton("Clear cover cache (re-fetch all covers)") self.clear_covers_btn = QPushButton("Clear cover cache (re-fetch all covers)")
self.clear_covers_btn.clicked.connect(self._clear_cover_cache) self.clear_covers_btn.clicked.connect(self._clear_cover_cache)
form.addRow(self.clear_covers_btn) form.addRow(self.clear_covers_btn)
self.download_bundle_btn = QPushButton("Download offline cover bundle…")
self.download_bundle_btn.clicked.connect(self.download_covers.emit)
form.addRow(self.download_bundle_btn)
col.addWidget(box)
# -- Storage ---------------------------------------------------------- #
box, form = self._group("Storage")
self._path_row(form, "State directory", str(config.STATE_DIR))
self._path_row(form, "Library database", str(config.DB_PATH))
self._path_row(form, "Cover cache", str(config.COVERS_DIR))
self._path_row(form, "User data (credentials, NPS cache…)", str(config.USER_DATA_DIR))
note = QLabel(
"Your library, covers, and credentials live here — outside the app "
"bundle, so they survive upgrades. On Linux this is the "
"~/.local/state/psppt folder."
)
note.setWordWrap(True)
note.setStyleSheet("color: #9a9ba5;")
form.addRow(note)
col.addWidget(box) col.addWidget(box)
col.addStretch(1) col.addStretch(1)
@@ -1342,6 +1394,29 @@ class SettingsPage(QWidget):
form.addRow(label, edit) form.addRow(label, edit)
return edit return edit
def _path_row(self, form, label, path):
edit = QLineEdit(path)
edit.setReadOnly(True)
open_btn = QPushButton("Open")
open_btn.clicked.connect(lambda _=False, p=Path(path): self._open_path(p))
row = QWidget()
h = QHBoxLayout(row)
h.setContentsMargins(0, 0, 0, 0)
h.addWidget(edit, 1)
h.addWidget(open_btn)
form.addRow(label, row)
return edit
@staticmethod
def _open_path(path):
path = Path(path)
target = path if path.is_dir() else path.parent
if target.is_dir():
QDesktopServices.openUrl(QUrl.fromLocalFile(str(target)))
else:
QMessageBox.information(None, "Not found",
f"Folder not created yet:\n{target}\n\nIt is created on first use.")
def games_dir(self): def games_dir(self):
return self.games_edit.text().strip() return self.games_edit.text().strip()
@@ -1405,7 +1480,7 @@ class SettingsPage(QWidget):
def _save(self): def _save(self):
sources.save({key: edit.text() for key, edit in self.fields.items()}) sources.save({key: edit.text() for key, edit in self.fields.items()})
covers.clear_miss_cache() covers.clear_miss_cache()
s = QSettings("PSPPreserve", "PSPPreserve") s = QSettings("psppt", "psppt")
s.setValue("auto_scan", self.auto_scan.isChecked()) s.setValue("auto_scan", self.auto_scan.isChecked())
s.setValue("scan_workers", self.workers_spin.value()) s.setValue("scan_workers", self.workers_spin.value())
s.setValue("auto_sync_saves", self.auto_sync.isChecked()) s.setValue("auto_sync_saves", self.auto_sync.isChecked())
@@ -1440,14 +1515,16 @@ class StoreScanWorker(QObject):
class StoreCoverLoader(QObject): class StoreCoverLoader(QObject):
loaded = Signal(int, int, str) # gen, grid row, cached image path loaded = Signal(int, int, QImage) # gen, grid row, fitted cover image
finished = Signal() finished = Signal()
def __init__(self, tasks, gen, quick=True): def __init__(self, tasks, gen, quick=True, icon_w=150, icon_h=220):
super().__init__() super().__init__()
self.tasks = tasks # list of (grid_row, item_dict) self.tasks = tasks # list of (grid_row, item_dict)
self.gen = gen self.gen = gen
self.quick = quick self.quick = quick
self.icon_w = icon_w
self.icon_h = icon_h
self._cancel = threading.Event() self._cancel = threading.Event()
def cancel(self): def cancel(self):
@@ -1471,9 +1548,13 @@ class StoreCoverLoader(QObject):
except Exception: # noqa: BLE001 except Exception: # noqa: BLE001
path = None path = None
if path is not None: if path is not None:
self.loaded.emit(self.gen, row, str(path)) img = QImage(str(path))
if not img.isNull():
self.loaded.emit(self.gen, row, _fit_image(img, self.icon_w, self.icon_h))
finally: finally:
pool.shutdown(wait=True, cancel_futures=True) # Return promptly on cancel/exit instead of blocking the GUI thread
# on in-flight network timeouts during shutdown.
pool.shutdown(wait=not self._cancel.is_set(), cancel_futures=True)
self.finished.emit() self.finished.emit()
@@ -1750,6 +1831,8 @@ class StorePage(QWidget):
self._lib_cover = {} # norm -> (release_id, system) self._lib_cover = {} # norm -> (release_id, system)
self._psp_norm = set() # norm of library items installed on the PSP self._psp_norm = set() # norm of library items installed on the PSP
self._cover_attempted = set() self._cover_attempted = set()
self._cover_busy = False
self._date_busy = False
self._generation = 0 self._generation = 0
self._scan_gen = 0 self._scan_gen = 0
self._meta_gen = 0 self._meta_gen = 0
@@ -1789,6 +1872,11 @@ class StorePage(QWidget):
self.psp_filter.setToolTip("Filter by whether the archive matches a disc installed on the connected PSP") self.psp_filter.setToolTip("Filter by whether the archive matches a disc installed on the connected PSP")
self.psp_filter.currentIndexChanged.connect(self._apply_filter) self.psp_filter.currentIndexChanged.connect(self._apply_filter)
bar.addWidget(self.psp_filter) bar.addWidget(self.psp_filter)
self.store_sort = QComboBox()
self.store_sort.addItems(["Newest first", "Title (A–Z)", "Region (A–Z)", "Size (largest)", "In library first"])
self.store_sort.setToolTip("Sort the cover grid")
self.store_sort.currentTextChanged.connect(self._apply_filter)
bar.addWidget(self.store_sort)
refresh_btn = QPushButton("Refresh") refresh_btn = QPushButton("Refresh")
refresh_btn.clicked.connect(lambda: self.refresh(force=True)) refresh_btn.clicked.connect(lambda: self.refresh(force=True))
bar.addWidget(refresh_btn) bar.addWidget(refresh_btn)
@@ -1931,9 +2019,27 @@ class StorePage(QWidget):
if self._items: if self._items:
self._apply_filter() self._apply_filter()
def _begin_scan(self):
"""Clear the stale view and invalidate in-flight cover/date batches."""
self._generation += 1
self._cover_busy = False
self._date_busy = False
self._cover_queue = []
self._cover_next = 0
self._date_queue = []
self._date_next = 0
self._cover_attempted.clear()
for loader in self._loaders:
loader.cancel()
self._loaders = []
self.grid.clear()
self.table.setRowCount(0)
self.status.setText("Loading…")
def refresh(self, force=False): def refresh(self, force=False):
if self._closing: if self._closing:
return return
self._begin_scan()
self._scan_gen += 1 self._scan_gen += 1
gen = self._scan_gen gen = self._scan_gen
if self.source_combo.currentIndex() == 0: # Server if self.source_combo.currentIndex() == 0: # Server
@@ -1994,6 +2100,8 @@ class StorePage(QWidget):
self._items = [] self._items = []
self._filtered = [] self._filtered = []
self._dirty = False self._dirty = False
self.grid.clear()
self.table.setRowCount(0)
log.warning("store unavailable: %s", message) log.warning("store unavailable: %s", message)
self.status.setText(f"Store unavailable: {message}") self.status.setText(f"Store unavailable: {message}")
@@ -2046,7 +2154,7 @@ class StorePage(QWidget):
if not it.get("release_date"): if not it.get("release_date"):
it["release_date"] = dates.get((str(rid), system)) or "" it["release_date"] = dates.get((str(rid), system)) or ""
filtered.append(it) filtered.append(it)
filtered.sort(key=self._store_sort_key) self._sort_store_items(filtered)
self._filtered = filtered self._filtered = filtered
if self.all_check.isChecked(): if self.all_check.isChecked():
show = len(filtered) show = len(filtered)
@@ -2054,6 +2162,19 @@ class StorePage(QWidget):
show = min(self.RECENT_LIMIT, len(filtered)) show = min(self.RECENT_LIMIT, len(filtered))
self._populate(filtered, show) self._populate(filtered, show)
def _sort_store_items(self, items):
mode = self.store_sort.currentText()
if mode == "Title (A–Z)":
items.sort(key=lambda it: store.display_name(it["name"]).lower())
elif mode == "Region (A–Z)":
items.sort(key=lambda it: (store.region(it["name"]) or "").lower())
elif mode == "Size (largest)":
items.sort(key=lambda it: it.get("size") or 0, reverse=True)
elif mode == "In library first":
items.sort(key=lambda it: (not it.get("in_library"), store.display_name(it["name"]).lower()))
else: # Newest first
items.sort(key=self._store_sort_key)
@staticmethod @staticmethod
def _store_sort_key(it): def _store_sort_key(it):
d = it.get("release_date") or "" d = it.get("release_date") or ""
@@ -2081,6 +2202,8 @@ class StorePage(QWidget):
self._cover_next = 0 self._cover_next = 0
self._date_queue = [] self._date_queue = []
self._date_next = 0 self._date_next = 0
self._cover_busy = False
self._date_busy = False
self._cover_attempted.clear() self._cover_attempted.clear()
self._placeholder = QIcon(_placeholder_pixmap(self.ICON_W, self.ICON_H)) self._placeholder = QIcon(_placeholder_pixmap(self.ICON_W, self.ICON_H))
@@ -2090,6 +2213,7 @@ class StorePage(QWidget):
self._loaders = [] self._loaders = []
self.grid.clear() self.grid.clear()
self.grid.setUpdatesEnabled(False)
self.table.setUpdatesEnabled(False) self.table.setUpdatesEnabled(False)
self.table.setSortingEnabled(False) self.table.setSortingEnabled(False)
self.table.setRowCount(show_count) self.table.setRowCount(show_count)
@@ -2132,10 +2256,10 @@ class StorePage(QWidget):
rid, system = lc rid, system = lc
else: else:
rid, system = covers.stable_key(it["name"]), it.get("system") or "games" rid, system = covers.stable_key(it["name"]), it.get("system") or "games"
p = covers.cover_path(rid, system) # Covers are decoded + scaled off the GUI thread (StoreCoverLoader).
if p.exists() and covers.file_is_image(p): # Never load a QPixmap here — doing so for cached covers made the store
item.setIcon(QIcon(_fit_pixmap(QPixmap(str(p)), self.ICON_W, self.ICON_H))) # unresponsive while filling/scrolling.
elif system not in ("theme", "dlc") and rid not in self._cover_attempted: if system not in ("theme", "dlc") and rid not in self._cover_attempted:
# Themes and DLC have no meaningful box art in libretro/IGDB, and # Themes and DLC have no meaningful box art in libretro/IGDB, and
# bulk-scraping hundreds of them on every switch to the DLC list # bulk-scraping hundreds of them on every switch to the DLC list
# saturates the network/CPU and makes the store unresponsive. # saturates the network/CPU and makes the store unresponsive.
@@ -2194,6 +2318,7 @@ class StorePage(QWidget):
def _finish_fill(self): def _finish_fill(self):
if self._fill_timer is not None: if self._fill_timer is not None:
self._fill_timer.stop() self._fill_timer.stop()
self.grid.setUpdatesEnabled(True)
self.table.setUpdatesEnabled(True) self.table.setUpdatesEnabled(True)
self.table.setSortingEnabled(True) self.table.setSortingEnabled(True)
total = len(self._filtered) total = len(self._filtered)
@@ -2206,13 +2331,14 @@ class StorePage(QWidget):
self._load_date_batch(self.DATE_BATCH) self._load_date_batch(self.DATE_BATCH)
def _load_cover_batch(self, n): def _load_cover_batch(self, n):
if self._closing or not self._cover_queue: if self._closing or not self._cover_queue or self._cover_busy:
return return
end = min(self._cover_next + n, len(self._cover_queue)) end = min(self._cover_next + n, len(self._cover_queue))
if self._cover_next >= end: if self._cover_next >= end:
return return
tasks = self._cover_queue[self._cover_next:end] tasks = self._cover_queue[self._cover_next:end]
self._cover_next = end self._cover_next = end
self._cover_busy = True
self._start_cover_loader(self._generation, tasks) self._start_cover_loader(self._generation, tasks)
def _active_scrollbar(self): def _active_scrollbar(self):
@@ -2243,13 +2369,14 @@ class StorePage(QWidget):
self._fill_timer.start() self._fill_timer.start()
def _load_date_batch(self, n): def _load_date_batch(self, n):
if self._closing or not self._date_queue: if self._closing or not self._date_queue or self._date_busy:
return return
end = min(self._date_next + n, len(self._date_queue)) end = min(self._date_next + n, len(self._date_queue))
if self._date_next >= end: if self._date_next >= end:
return return
tasks = self._date_queue[self._date_next:end] tasks = self._date_queue[self._date_next:end]
self._date_next = end self._date_next = end
self._date_busy = True
self._start_date_loader(self._generation, tasks) self._start_date_loader(self._generation, tasks)
def _start_date_loader(self, gen, tasks): def _start_date_loader(self, gen, tasks):
@@ -2260,14 +2387,24 @@ class StorePage(QWidget):
loader.moveToThread(thread) loader.moveToThread(thread)
thread.started.connect(loader.run) thread.started.connect(loader.run)
loader.loaded.connect(self._on_date_loaded) loader.loaded.connect(self._on_date_loaded)
loader.finished.connect(self._on_date_batch_done)
loader.finished.connect(thread.quit) loader.finished.connect(thread.quit)
thread.finished.connect(self._on_thread_finished) thread.finished.connect(self._on_thread_finished)
self._threads.append(thread) self._threads.append(thread)
self._loaders.append(loader) self._loaders.append(loader)
thread.start() thread.start()
def _on_date_batch_done(self):
if self._closing:
return
loader = self.sender()
if getattr(loader, "gen", None) != self._generation:
return
self._date_busy = False
self._load_date_batch(self.DATE_BATCH)
def _on_date_loaded(self, gen, row, date): def _on_date_loaded(self, gen, row, date):
if gen != self._generation or not date: if self._closing or gen != self._generation or not date:
return return
if 0 <= row < len(self._filtered): if 0 <= row < len(self._filtered):
self._filtered[row]["release_date"] = date self._filtered[row]["release_date"] = date
@@ -2279,22 +2416,32 @@ class StorePage(QWidget):
if self._closing or not tasks: if self._closing or not tasks:
return return
thread = QThread(self) thread = QThread(self)
loader = StoreCoverLoader(tasks, gen, quick=True) loader = StoreCoverLoader(tasks, gen, quick=True, icon_w=self.ICON_W, icon_h=self.ICON_H)
loader.moveToThread(thread) loader.moveToThread(thread)
thread.started.connect(loader.run) thread.started.connect(loader.run)
loader.loaded.connect(self._on_cover_loaded) loader.loaded.connect(self._on_cover_loaded)
loader.finished.connect(self._on_cover_batch_done)
loader.finished.connect(thread.quit) loader.finished.connect(thread.quit)
thread.finished.connect(self._on_thread_finished) thread.finished.connect(self._on_thread_finished)
self._threads.append(thread) self._threads.append(thread)
self._loaders.append(loader) self._loaders.append(loader)
thread.start() thread.start()
def _on_cover_loaded(self, gen, row, path): def _on_cover_batch_done(self):
if gen != self._generation: if self._closing:
return
loader = self.sender()
if getattr(loader, "gen", None) != self._generation:
return
self._cover_busy = False
self._load_cover_batch(self.COVER_SCROLL_BATCH)
def _on_cover_loaded(self, gen, row, image):
if self._closing or gen != self._generation:
return return
item = self.grid.item(row) item = self.grid.item(row)
if item is not None: if item is not None:
item.setIcon(QIcon(_fit_pixmap(QPixmap(path), self.ICON_W, self.ICON_H))) item.setIcon(QIcon(QPixmap.fromImage(image)))
it = item.data(Qt.UserRole) it = item.data(Qt.UserRole)
shown = getattr(self.details, "_item", None) shown = getattr(self.details, "_item", None)
if it is not None and shown is not None and it.get("path") == shown.get("path"): if it is not None and shown is not None and it.get("path") == shown.get("path"):
@@ -2523,6 +2670,7 @@ class QueuePage(QWidget):
"umd-dump": "UMD dump", "umd-dump": "UMD dump",
"save-sync": "Save sync", "save-sync": "Save sync",
"nps-download": "NoPayStation", "nps-download": "NoPayStation",
"covers": "Covers bundle",
} }
def __init__(self, parent=None): def __init__(self, parent=None):
@@ -4044,6 +4192,7 @@ class Ps1Page(QWidget):
self._preview_gen = 0 self._preview_gen = 0
self._preview_threads = [] self._preview_threads = []
self._preview_workers = [] self._preview_workers = []
self._closing = False
layout = QVBoxLayout(self) layout = QVBoxLayout(self)
bar = QHBoxLayout() bar = QHBoxLayout()
@@ -4142,7 +4291,7 @@ class Ps1Page(QWidget):
self._preview_workers.append(worker) self._preview_workers.append(worker)
def _on_preview_result(self, gen, game_id, icon0, pic0, pic1): def _on_preview_result(self, gen, game_id, icon0, pic0, pic1):
if gen != self._preview_gen: if self._closing or gen != self._preview_gen:
return return
self.preview_icon0.setText("") self.preview_icon0.setText("")
self.preview_pic0.setText("") self.preview_pic0.setText("")
@@ -4190,6 +4339,8 @@ class Ps1Page(QWidget):
self._thread.start() self._thread.start()
def _on_results(self, entries): def _on_results(self, entries):
if self._closing:
return
# Only show discs that still need converting; converted games already # Only show discs that still need converting; converted games already
# appear in the Library tab. # appear in the Library tab.
entries = [e for e in entries if e["kind"] == "disc" and e.get("status") != "converted"] entries = [e for e in entries if e["kind"] == "disc" and e.get("status") != "converted"]
@@ -4205,8 +4356,23 @@ class Ps1Page(QWidget):
self.table.selectRow(0) self.table.selectRow(0)
def _on_failed(self, message): def _on_failed(self, message):
if not self._closing:
self.status.setText(f"PS1 scan failed: {message}") self.status.setText(f"PS1 scan failed: {message}")
def shutdown(self):
self._closing = True
if self._thread is not None and _thread_running(self._thread):
self._thread.quit()
self._thread.wait()
for t in self._preview_threads:
if _thread_running(t):
t.quit()
t.wait()
self._preview_threads = []
self._preview_workers = []
self._thread = None
self._worker = None
def _convert_selected(self): def _convert_selected(self):
row = self.table.currentRow() row = self.table.currentRow()
if row < 0 or row >= len(self._entries): if row < 0 or row >= len(self._entries):
@@ -4230,7 +4396,7 @@ class Ps1Page(QWidget):
class MainWindow(QMainWindow): class MainWindow(QMainWindow):
def __init__(self): def __init__(self):
super().__init__() super().__init__()
self.setWindowTitle("psp preservation tool ~ quoije") self.setWindowTitle("psppt ~ quoije")
self.resize(1240, 780) self.resize(1240, 780)
self._rows = [] self._rows = []
self._renames = [] self._renames = []
@@ -4253,7 +4419,7 @@ class MainWindow(QMainWindow):
self._library_workers = [] self._library_workers = []
self._library_loading = False self._library_loading = False
self.settings = QSettings("PSPPreserve", "PSPPreserve") self.settings = QSettings("psppt", "psppt")
self._games_dir = self._first_existing(config.GAMES_ROOTS) or self.settings.value("games_dir", "") or "" self._games_dir = self._first_existing(config.GAMES_ROOTS) or self.settings.value("games_dir", "") or ""
self._videos_dir = self._first_existing(config.VIDEO_ROOTS) or self.settings.value("videos_dir", "") or "" self._videos_dir = self._first_existing(config.VIDEO_ROOTS) or self.settings.value("videos_dir", "") or ""
self._store_dir = str(self.settings.value("store_dir", str(config.STORE_ROOT)) or "") self._store_dir = str(self.settings.value("store_dir", str(config.STORE_ROOT)) or "")
@@ -4296,7 +4462,7 @@ class MainWindow(QMainWindow):
pm = QPixmap(str(icon_path)) pm = QPixmap(str(icon_path))
pm = pm.scaled(28, 28, Qt.KeepAspectRatio, Qt.SmoothTransformation) pm = pm.scaled(28, 28, Qt.KeepAspectRatio, Qt.SmoothTransformation)
logo.setPixmap(pm) logo.setPixmap(pm)
logo.setToolTip("PSP Preservation Tool") logo.setToolTip("psppt")
self.tabs.setCornerWidget(logo, Qt.TopRightCorner) self.tabs.setCornerWidget(logo, Qt.TopRightCorner)
self.queue = queue.TransferQueue(self) self.queue = queue.TransferQueue(self)
@@ -4366,6 +4532,11 @@ class MainWindow(QMainWindow):
self.library_view = QComboBox() self.library_view = QComboBox()
self.library_view.addItems(["Cover view", "List view"]) self.library_view.addItems(["Cover view", "List view"])
bar.addWidget(self.library_view) bar.addWidget(self.library_view)
self.library_sort = QComboBox()
self.library_sort.addItems(["Title", "Release date", "Size", "Region", "Type", "Serial"])
self.library_sort.setToolTip("Sort the cover grid")
self.library_sort.currentTextChanged.connect(self._apply_library_filter)
bar.addWidget(self.library_sort)
reload_btn = QPushButton("Reload") reload_btn = QPushButton("Reload")
reload_btn.clicked.connect(self.load_library) reload_btn.clicked.connect(self.load_library)
bar.addWidget(reload_btn) bar.addWidget(reload_btn)
@@ -4497,6 +4668,7 @@ class MainWindow(QMainWindow):
self._ps1_dir, self._ps1_output_dir, self._ps1_base_pbp, self._ps1_dir, self._ps1_output_dir, self._ps1_base_pbp,
self._nps_dir) self._nps_dir)
self.settings_page.saved.connect(self._on_settings_saved) self.settings_page.saved.connect(self._on_settings_saved)
self.settings_page.download_covers.connect(self._download_covers_bundle)
return self.settings_page return self.settings_page
def _on_settings_saved(self): def _on_settings_saved(self):
@@ -4633,12 +4805,30 @@ class MainWindow(QMainWindow):
if psp_f == "Not on PSP" and it.get("on_psp"): if psp_f == "Not on PSP" and it.get("on_psp"):
continue continue
items.append(it) items.append(it)
self._sort_library_items(items)
self._library_filtered = items self._library_filtered = items
self._populate_library_table(items) self._populate_library_table(items)
self.cover_grid.set_items(items) self.cover_grid.set_items(items)
if hasattr(self, "psp_page"): if hasattr(self, "psp_page"):
self.psp_page.set_library_items(self._library_cache) self.psp_page.set_library_items(self._library_cache)
def _sort_library_items(self, items):
mode = self.library_sort.currentText()
title = lambda it: (it.get("title_override") or it.get("canonical") or it["name"] or "").lower()
if mode == "Release date":
items.sort(key=lambda it: it.get("release_date") or "", reverse=True)
elif mode == "Size":
items.sort(key=lambda it: it.get("size") or 0, reverse=True)
elif mode == "Region":
items.sort(key=lambda it: (it.get("region") or "").lower())
elif mode == "Type":
order = {"games": 0, "videos": 1, "ps1": 2, "theme": 3, "dlc": 4}
items.sort(key=lambda it: (order.get(it.get("system"), 9), title(it)))
elif mode == "Serial":
items.sort(key=lambda it: (it.get("serial") or "").lower())
else: # Title
items.sort(key=title)
def _populate_library_table(self, items): def _populate_library_table(self, items):
self.library_table.setUpdatesEnabled(False) self.library_table.setUpdatesEnabled(False)
self.library_table.setSortingEnabled(False) self.library_table.setSortingEnabled(False)
@@ -5216,6 +5406,9 @@ class MainWindow(QMainWindow):
QTimer.singleShot(0, self._refresh_psp) QTimer.singleShot(0, self._refresh_psp)
if kind == "save-sync": if kind == "save-sync":
QTimer.singleShot(0, self._refresh_saves) QTimer.singleShot(0, self._refresh_saves)
if kind == "covers":
covers.clear_miss_cache()
QTimer.singleShot(0, self.load_library)
if kind == "ps1-convert": if kind == "ps1-convert":
QTimer.singleShot(0, self.ps1_page.refresh) QTimer.singleShot(0, self.ps1_page.refresh)
QTimer.singleShot(0, self.load_library) QTimer.singleShot(0, self.load_library)
@@ -5551,6 +5744,48 @@ class MainWindow(QMainWindow):
QMessageBox.information(self, "Done", f"Renamed {renamed} file(s).") QMessageBox.information(self, "Done", f"Renamed {renamed} file(s).")
self.load_library() self.load_library()
def maybe_offer_covers_bundle(self):
"""Offer to download the offline cover bundle once, if it's missing."""
if self._closing or covers.covers_bundle_present():
return
s = QSettings("psppt", "psppt")
if str(s.value("covers_bundle_skip", "false")).lower() == "true":
return
box = QMessageBox(self)
box.setWindowTitle("Offline covers")
box.setIcon(QMessageBox.Question)
box.setText(
"The offline cover bundle (~843 MB) isn't installed.\n\n"
"Download it now from the project repo? Covers keep working online "
"without it — the bundle just makes them instant and offline."
)
download = box.addButton("Download", QMessageBox.AcceptRole)
box.addButton("Not now", QMessageBox.RejectRole)
never = box.addButton("Don't ask again", QMessageBox.DestructiveRole)
box.setDefaultButton(download)
box.exec()
clicked = box.clickedButton()
if clicked is never:
s.setValue("covers_bundle_skip", "true")
elif clicked is download:
self._download_covers_bundle()
def _download_covers_bundle(self):
"""Queue the offline cover bundle download in the transfer queue."""
if self._closing:
return
for j in self.queue.jobs:
if j.kind == "covers" and j.status in ("queued", "running", "cancelling"):
return # already queued or running
self.queue.enqueue(
transfers.CoversBundleSource(),
str(config.COVERS_DIR),
None,
"Offline cover bundle",
kind="covers",
)
self.tabs.setCurrentWidget(self.queue_page)
def closeEvent(self, event): def closeEvent(self, event):
self._closing = True self._closing = True
self.settings.setValue("window_geometry", self.saveGeometry()) self.settings.setValue("window_geometry", self.saveGeometry())
@@ -5576,18 +5811,11 @@ class MainWindow(QMainWindow):
self.psp_page.shutdown() self.psp_page.shutdown()
if hasattr(self, "saves_page"): if hasattr(self, "saves_page"):
self.saves_page.shutdown() self.saves_page.shutdown()
if hasattr(self, "ps1_page"):
self.ps1_page.shutdown()
event.accept() event.accept()
def _start_bundle_check(window):
"""Download any missing offline cover bundles in the background."""
server = getattr(window, "_store_server_url", "") or ""
try:
covers.ensure_cover_bundles(server)
except Exception: # noqa: BLE001 - background best-effort
pass
def main(): def main():
logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)-7s %(name)-14s %(message)s") logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)-7s %(name)-14s %(message)s")
init_db() # migrate legacy on-share DB/covers to local storage, then schema init_db() # migrate legacy on-share DB/covers to local storage, then schema
@@ -5596,16 +5824,34 @@ def main():
n = ingest.ensure_catalog() n = ingest.ensure_catalog()
if n: if n:
log.info("ingested %d DAT(s) into the catalog", n) log.info("ingested %d DAT(s) into the catalog", n)
log.info("starting psp-preserve (db: %s)", config.DB_PATH) log.info("starting psppt (db: %s)", config.DB_PATH)
app = QApplication(sys.argv) app = QApplication(sys.argv)
style.apply_theme(app) style.apply_theme(app)
app.setApplicationName("psppt")
app.setDesktopFileName("psppt")
icon_path = config.PROJECT_ROOT / "img" / "app-icon.png" icon_path = config.PROJECT_ROOT / "img" / "app-icon.png"
if icon_path.is_file(): if icon_path.is_file():
app.setWindowIcon(QIcon(str(icon_path))) app.setWindowIcon(QIcon(str(icon_path)))
window = MainWindow() window = MainWindow()
window.setWindowIcon(app.windowIcon())
window.show() window.show()
log.info("main window ready") log.info("main window ready")
threading.Thread(target=_start_bundle_check, args=(window,), daemon=True).start() QTimer.singleShot(1200, window.maybe_offer_covers_bundle)
# Ctrl+C should close the window gracefully (running closeEvent's thread
# cleanup) rather than tearing Qt down mid-thread, which segfaults.
interrupt = {"pending": False}
def _sigint_handler(signum, frame):
interrupt["pending"] = True
signal.signal(signal.SIGINT, _sigint_handler)
sigint_timer = QTimer()
sigint_timer.setInterval(100)
sigint_timer.timeout.connect(
lambda: window.close() if interrupt["pending"] else None)
sigint_timer.start()
return app.exec() return app.exec()
+1 -1
View File
@@ -1,4 +1,4 @@
"""PSP Preservation Tool entry point (native GUI). """psppt entry point (native GUI).
Run with:: Run with::
+1 -1
View File
@@ -151,7 +151,7 @@ def fetch_items(content_type: str, url: str = "", timeout: int = 60, force: bool
return items return items
try: try:
req = urllib.request.Request(url, headers={"User-Agent": "psp-preserve/0.1"}) req = urllib.request.Request(url, headers={"User-Agent": "psppt/0.1"})
with urllib.request.urlopen(req, timeout=timeout) as resp: with urllib.request.urlopen(req, timeout=timeout) as resp:
data = resp.read() data = resp.read()
text = data.decode("utf-8", errors="replace") text = data.decode("utf-8", errors="replace")
+1 -1
View File
@@ -155,7 +155,7 @@ def scrape(limit=None, workers=8, rate=3.0):
log.info("downloading OpenVGDB dump (%s)…", OVGDB_ZIP_URL) log.info("downloading OpenVGDB dump (%s)…", OVGDB_ZIP_URL)
tmpdir = Path(tempfile.mkdtemp(prefix="openvgdb-")) tmpdir = Path(tempfile.mkdtemp(prefix="openvgdb-"))
dump_zip = tmpdir / "openvgdb.zip" dump_zip = tmpdir / "openvgdb.zip"
req = urllib.request.Request(OVGDB_ZIP_URL, headers={"User-Agent": "psp-preserve/0.1"}) req = urllib.request.Request(OVGDB_ZIP_URL, headers={"User-Agent": "psppt/0.1"})
with urllib.request.urlopen(req, timeout=60) as r, open(dump_zip, "wb") as f: with urllib.request.urlopen(req, timeout=60) as r, open(dump_zip, "wb") as f:
f.write(r.read()) f.write(r.read())
+35 -32
View File
@@ -79,13 +79,13 @@ class TransferQueue(QObject):
job_finished = Signal(dict) job_finished = Signal(dict)
job_removed = Signal(int) job_removed = Signal(int)
def __init__(self, parent=None): def __init__(self, parent=None, max_concurrent=3):
super().__init__(parent) super().__init__(parent)
self.max_concurrent = max(1, int(max_concurrent))
self.jobs = [] self.jobs = []
self._pending = [] self._pending = []
self._current = None # job_id -> (QThread, JobWorker) for jobs currently running in parallel.
self._thread = None self._running = {}
self._worker = None
# Keep strong references to every worker/thread for their full lifetime. # Keep strong references to every worker/thread for their full lifetime.
# Dropping a Python ref to a QObject that was moved to another thread # Dropping a Python ref to a QObject that was moved to another thread
# deletes it directly and crashes (bus error). # deletes it directly and crashes (bus error).
@@ -115,8 +115,10 @@ class TransferQueue(QObject):
self.job_updated.emit(job.to_dict()) self.job_updated.emit(job.to_dict())
self.job_finished.emit(job.to_dict()) self.job_finished.emit(job.to_dict())
return return
if job.status == "running" and self._worker is not None: if job.status == "running":
self._worker.cancel() pair = self._running.get(job_id)
if pair is not None:
pair[1].cancel()
job.status = "cancelling" job.status = "cancelling"
self.job_updated.emit(job.to_dict()) self.job_updated.emit(job.to_dict())
@@ -140,8 +142,8 @@ class TransferQueue(QObject):
self.remove(jid) self.remove(jid)
def shutdown(self): def shutdown(self):
if self._worker is not None: for _t, worker in self._running.values():
self._worker.cancel() worker.cancel()
for t in self._threads: for t in self._threads:
try: try:
if t.isRunning(): if t.isRunning():
@@ -157,30 +159,29 @@ class TransferQueue(QObject):
return None return None
def _start_next(self): def _start_next(self):
if self._current is not None or not self._pending: while len(self._running) < self.max_concurrent and self._pending:
return
job = self._pending.pop(0) job = self._pending.pop(0)
self._current = job
job.status = "running" job.status = "running"
log.info("starting job #%d: %s (%s)", job.id, job.title, job.kind) log.info("starting job #%d: %s (%s)", job.id, job.title, job.kind)
self.job_updated.emit(job.to_dict()) self.job_updated.emit(job.to_dict())
self._thread = QThread(self) thread = QThread(self)
self._worker = JobWorker(job.source, job.dest_dir, job) worker = JobWorker(job.source, job.dest_dir, job)
self._worker.moveToThread(self._thread) worker.moveToThread(thread)
self._thread.started.connect(self._worker.run) thread.started.connect(worker.run)
self._worker.progress.connect(self._on_progress) worker.progress.connect(self._on_progress)
self._worker.finished.connect(self._on_finished) worker.finished.connect(self._on_finished)
self._worker.skipped.connect(self._on_skipped) worker.skipped.connect(self._on_skipped)
self._worker.cancelled.connect(self._on_cancelled) worker.cancelled.connect(self._on_cancelled)
self._worker.failed.connect(self._on_failed) worker.failed.connect(self._on_failed)
for sig in (self._worker.finished, self._worker.skipped, for sig in (worker.finished, worker.skipped,
self._worker.cancelled, self._worker.failed): worker.cancelled, worker.failed):
sig.connect(self._thread.quit) sig.connect(thread.quit)
self._thread.finished.connect(self._on_thread_finished) thread.finished.connect(self._on_thread_finished)
self._threads.append(self._thread) self._running[job.id] = (thread, worker)
self._workers.append(self._worker) self._threads.append(thread)
self._thread.start() self._workers.append(worker)
thread.start()
def _on_progress(self, job_id, stage, fraction, detail): def _on_progress(self, job_id, stage, fraction, detail):
job = self._find(job_id) job = self._find(job_id)
@@ -218,10 +219,12 @@ class TransferQueue(QObject):
self.job_finished.emit(job.to_dict()) self.job_finished.emit(job.to_dict())
def _on_thread_finished(self): def _on_thread_finished(self):
# Queued to the main thread once the worker thread has fully stopped, # Queued to the main thread once a worker thread has fully stopped, so
# so it is now safe to drop the "current" pointers and advance. # it is now safe to drop its pointers and advance the queue.
self._current = None thread = self.sender()
self._thread = None for jid, pair in list(self._running.items()):
self._worker = None if pair[0] is thread:
del self._running[jid]
break
self._start_next() self._start_next()
+29 -2
View File
@@ -175,6 +175,33 @@ class Source:
raise NotImplementedError raise NotImplementedError
class CoversBundleSource:
"""Download the combined offline cover bundle (runs as a queue job)."""
label = "psppt-covers.zip"
def __init__(self, url=""):
self.url = url
def download(self, item, dest_dir, progress_cb=None, cancel_event=None):
from . import covers
def on_progress(done, total):
if progress_cb is None:
return
frac = (done / total) if total else 0.0
detail = (f"{done / (1 << 20):.0f} / {total / (1 << 20):.0f} MB"
if total else "")
progress_cb("Downloading", frac, detail)
try:
result = covers.download_covers_bundle(self.url,
progress_cb=on_progress,
cancel_event=cancel_event)
except InterruptedError:
raise TransferCancelled() from None
return str(result)
class FileCopySource(Source): class FileCopySource(Source):
"""Copy a single local file (library -> PSP or PSP -> library). """Copy a single local file (library -> PSP or PSP -> library).
@@ -284,7 +311,7 @@ class RemoteStoreSource(Source):
self.token = token or "" self.token = token or ""
def _headers(self): def _headers(self):
headers = {"User-Agent": "psp-preserve/0.1"} headers = {"User-Agent": "psppt/0.1"}
if self.token: if self.token:
headers["Authorization"] = f"Bearer {self.token}" headers["Authorization"] = f"Bearer {self.token}"
return headers return headers
@@ -347,7 +374,7 @@ class ArchiveOrgSource(Source):
return f"https://archive.org/download/{self.item}" return f"https://archive.org/download/{self.item}"
def _headers(self): def _headers(self):
headers = {"User-Agent": "psp-preserve/0.1"} headers = {"User-Agent": "psppt/0.1"}
cookie = "; ".join( cookie = "; ".join(
p for p in ( p for p in (
f"logged-in-sig={self.sig}" if self.sig else "", f"logged-in-sig={self.sig}" if self.sig else "",
Binary file not shown.
Binary file not shown.
+2 -2
View File
@@ -1,7 +1,7 @@
services: services:
psp-preserve: psppt:
build: . build: .
container_name: psp-preserve container_name: psppt
# Attach to Unraid's macvlan bridge so the API gets its own LAN IP. # Attach to Unraid's macvlan bridge so the API gets its own LAN IP.
# Plain HTTP for now; see Caddyfile to add Let's Encrypt TLS later. # Plain HTTP for now; see Caddyfile to add Let's Encrypt TLS later.
networks: networks:
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

+81
View File
@@ -0,0 +1,81 @@
#!/bin/sh
# Install psppt to a system prefix.
#
# FHS-friendly layout:
# $PREFIX/bin/psppt symlink -> payload binary
# $PREFIX/lib/psppt/ app payload (binary + lib/)
# $PREFIX/share/applications/ desktop entry
# $PREFIX/share/icons/hicolor/256x256/apps/ app icon
#
# The PyInstaller onedir build keeps the executable and its `lib/` contents
# directory side by side, so both live under $PREFIX/lib/psppt/ and a
# symlink exposes the binary at $PREFIX/bin/psppt.
#
# Usage (run from inside the extracted tarball):
# sudo ./install.sh # install to /usr/local (default)
# sudo ./install.sh --prefix=/usr # install to /usr
# sudo ./install.sh --uninstall # remove from the default prefix
set -eu
PREFIX=/usr/local
MODE=install
for arg in "$@"; do
case "$arg" in
--prefix=*) PREFIX=${arg#--prefix=} ;;
--uninstall) MODE=uninstall ;;
*) echo "unknown argument: $arg" >&2; exit 2 ;;
esac
done
SRC_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
LIBDIR="$PREFIX/lib/psppt"
BINDIR="$PREFIX/bin"
APPDIR="$PREFIX/share/applications"
ICONDIR="$PREFIX/share/icons/hicolor/256x256/apps"
APP_BIN="$LIBDIR/psppt"
require_root() {
if [ "$(id -u)" -ne 0 ]; then
echo "run as root (e.g. sudo)" >&2
exit 1
fi
}
uninstall() {
rm -f "$BINDIR/psppt"
rm -f "$APPDIR/psppt.desktop"
rm -f "$ICONDIR/psppt.png"
rm -rf "$LIBDIR"
echo "Uninstalled from $PREFIX"
}
if [ "$MODE" = uninstall ]; then
require_root
uninstall
exit 0
fi
require_root
# The tarball ships these next to this script.
for f in psppt lib psppt.desktop app-icon.png; do
if [ ! -e "$SRC_DIR/$f" ]; then
echo "missing $f (run install.sh from inside the extracted tarball)" >&2
exit 1
fi
done
mkdir -p "$LIBDIR" "$BINDIR" "$APPDIR" "$ICONDIR"
rm -rf "$LIBDIR"
mkdir -p "$LIBDIR"
cp -a "$SRC_DIR/psppt" "$LIBDIR/psppt"
cp -a "$SRC_DIR/lib" "$LIBDIR/lib"
install -m 0644 "$SRC_DIR/psppt.desktop" "$APPDIR/psppt.desktop"
install -m 0644 "$SRC_DIR/app-icon.png" "$ICONDIR/psppt.png"
ln -sfn "$APP_BIN" "$BINDIR/psppt"
echo "Installed to $PREFIX"
echo " binary: $BINDIR/psppt -> $APP_BIN"
echo " payload: $LIBDIR"
echo " desktop: $APPDIR/psppt.desktop"
+9
View File
@@ -0,0 +1,9 @@
[Desktop Entry]
Type=Application
Name=psppt
Comment=Preserve and manage your PSP library
Exec=psppt
Icon=psppt
Terminal=false
Categories=Utility;AudioVideo;
StartupWMClass=psppt
+9 -9
View File
@@ -1,13 +1,13 @@
# -*- mode: python ; coding: utf-8 -*- # -*- mode: python ; coding: utf-8 -*-
"""PyInstaller spec: build the PSP Preservation Tool (onedir). """PyInstaller spec: build psppt (onedir).
Bundles the read-only resources (DATs, PS1 loader, offline cover bundles, Bundles the read-only resources (DATs, PS1 loader, cover templates, index
icons) under ``_internal/`` next to the executable. User-writable data catalogs, icons) under ``lib/`` next to the executable. User-writable data
(credentials, manual covers, PS1 templates, NPS cache, library DB, cover (credentials, manual covers, PS1 templates, NPS cache, library DB, cover
cache) lives in the OS state dir at runtime — see ``app/config.py``. cache) lives in the OS state dir at runtime — see ``app/config.py``.
Usage: pyinstaller --clean --noconfirm psp-preserve.spec Usage: pyinstaller --clean --noconfirm psppt.spec
Output: dist/psp-preserve/ Output: dist/psppt/
""" """
from PyInstaller.utils.hooks import collect_dynamic_libs, collect_submodules from PyInstaller.utils.hooks import collect_dynamic_libs, collect_submodules
@@ -15,8 +15,6 @@ datas = [
("data/dats", "data/dats"), ("data/dats", "data/dats"),
("data/loader", "data/loader"), ("data/loader", "data/loader"),
("data/supplement.json", "data"), ("data/supplement.json", "data"),
("data/covers/libretro-covers.zip", "data/covers"),
("data/covers/openvgdb-covers.zip", "data/covers"),
("data/covers/libretro-index.json", "data/covers"), ("data/covers/libretro-index.json", "data/covers"),
("data/covers/openvgdb-index.json", "data/covers"), ("data/covers/openvgdb-index.json", "data/covers"),
("data/covers/UMD_COVER_TEMPLATE.png", "data/covers"), ("data/covers/UMD_COVER_TEMPLATE.png", "data/covers"),
@@ -24,6 +22,7 @@ datas = [
("data/covers/UMD_GAME_COVER_TEMPLATE.png", "data/covers"), ("data/covers/UMD_GAME_COVER_TEMPLATE.png", "data/covers"),
("img/app-icon.png", "img"), ("img/app-icon.png", "img"),
("img/chevron.png", "img"), ("img/chevron.png", "img"),
("packaging/psppt.desktop", "."),
] ]
a = Analysis( a = Analysis(
@@ -46,13 +45,14 @@ exe = EXE(
a.scripts, a.scripts,
[], [],
exclude_binaries=True, exclude_binaries=True,
name="psp-preserve", name="psppt",
debug=False, debug=False,
bootloader_ignore_signals=False, bootloader_ignore_signals=False,
strip=False, strip=False,
upx=False, upx=False,
console=False, console=False,
disable_windowed_traceback=False, disable_windowed_traceback=False,
contents_directory="lib",
) )
coll = COLLECT( coll = COLLECT(
@@ -61,5 +61,5 @@ coll = COLLECT(
a.datas, a.datas,
strip=False, strip=False,
upx=False, upx=False,
name="psp-preserve", name="psppt",
) )
+31 -15
View File
@@ -12,34 +12,50 @@ ENV DEBIAN_FRONTEND=noninteractive \
WINEDLLOVERRIDES="mscoree,mshtml=" \ WINEDLLOVERRIDES="mscoree,mshtml=" \
PYTHON_VERSION=3.12.8 PYTHON_VERSION=3.12.8
# Wine (64-bit), node + git for JS actions, and build/package tooling. # WineHQ wine-staging (newer than Debian's 8.0, needed for Python 3.12),
RUN apt-get update && apt-get install -y --no-install-recommends \ # node + git for JS actions, and build/package tooling.
ca-certificates curl wget unzip zip \ RUN dpkg --add-architecture i386 \
wine wine64 xvfb winetricks cabextract \ && apt-get update \
nodejs git \ && apt-get install -y --no-install-recommends ca-certificates curl wget gnupg2 \
&& mkdir -p /etc/apt/keyrings \
&& wget -qO- https://dl.winehq.org/wine-builds/winehq.key | gpg --dearmor -o /etc/apt/keyrings/winehq.gpg \
&& echo "deb [signed-by=/etc/apt/keyrings/winehq.gpg] https://dl.winehq.org/wine-builds/debian/ bookworm main" > /etc/apt/sources.list.d/winehq.list \
&& apt-get update \
&& apt-get install -y --install-recommends winehq-staging \
&& apt-get install -y --no-install-recommends unzip zip xvfb xauth cabextract nodejs git \
&& apt-get clean && rm -rf /var/lib/apt/lists/* && apt-get clean && rm -rf /var/lib/apt/lists/*
# winetricks (standalone script; not packaged in Debian main).
RUN wget -qO /usr/local/bin/winetricks \
"https://raw.githubusercontent.com/Winetricks/winetricks/master/src/winetricks" \
&& chmod +x /usr/local/bin/winetricks
# Gitea Actions runner (linux-amd64 binary). # Gitea Actions runner (linux-amd64 binary).
RUN wget -qO /usr/local/bin/act_runner \ RUN wget -qO /usr/local/bin/act_runner \
"https://gitea.com/gitea/act_runner/releases/download/v0.6.1/act_runner-0.6.1-linux-amd64" \ "https://gitea.com/gitea/act_runner/releases/download/v0.6.1/act_runner-0.6.1-linux-amd64" \
&& chmod +x /usr/local/bin/act_runner && chmod +x /usr/local/bin/act_runner
# Windows CPython 3.12, silent install into C:\Python312. # MSVC runtime required by PySide6's Qt DLLs (also initializes the Wine prefix).
RUN wget -qO /tmp/python-installer.exe \
"https://www.python.org/ftp/python/${PYTHON_VERSION}/python-${PYTHON_VERSION}-amd64.exe" \
&& xvfb-run -a wine /tmp/python-installer.exe \
/quiet InstallAllUsers=1 PrependPath=1 Include_test=0 Include_launcher=0 Shortcuts=0 \
&& rm -f /tmp/python-installer.exe \
&& wine "C:\\Python312\\python.exe" --version
# MSVC runtime required by PySide6's Qt DLLs.
RUN xvfb-run -a winetricks --unattended vcrun2022 RUN xvfb-run -a winetricks --unattended vcrun2022
# Windows CPython 3.12 (embeddable zip) + pip.
RUN wget -qO /tmp/pyembed.zip \
"https://www.python.org/ftp/python/${PYTHON_VERSION}/python-${PYTHON_VERSION}-embed-amd64.zip" \
&& mkdir -p /root/.wine/drive_c/Python312 \
&& cd /root/.wine/drive_c/Python312 \
&& unzip -q /tmp/pyembed.zip \
&& rm -f /tmp/pyembed.zip \
&& printf 'python312.zip\n.\nimport site\nLib\\site-packages\n' > python312._pth \
&& wine "C:\\Python312\\python.exe" --version
# Convenience wrapper: winpython -> wine C:\Python312\python.exe # Convenience wrapper: winpython -> wine C:\Python312\python.exe
RUN printf '#!/bin/sh\nexec wine "C:\\Python312\\python.exe" "$@"\n' > /usr/local/bin/winpython \ RUN printf '#!/bin/sh\nexec wine "C:\\Python312\\python.exe" "$@"\n' > /usr/local/bin/winpython \
&& chmod +x /usr/local/bin/winpython && chmod +x /usr/local/bin/winpython
# Pre-install the Windows build deps so CI jobs start fast. # Bootstrap pip, then pre-install the Windows build deps so CI jobs start fast.
RUN wget -qO /tmp/get-pip.py https://bootstrap.pypa.io/get-pip.py \
&& wine "C:\\Python312\\python.exe" /tmp/get-pip.py \
&& rm -f /tmp/get-pip.py
COPY requirements-build.txt /tmp/requirements-build.txt COPY requirements-build.txt /tmp/requirements-build.txt
RUN winpython -m pip install --upgrade pip \ RUN winpython -m pip install --upgrade pip \
&& winpython -m pip install -r /tmp/requirements-build.txt \ && winpython -m pip install -r /tmp/requirements-build.txt \
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env python3
"""Package the offline cover zips into a single distributable bundle.
Outputs ``psppt-covers.zip`` containing ``libretro-covers.zip`` and
``openvgdb-covers.zip``. Attach it to the Gitea ``covers`` release; the app
downloads it on first run when the user opts in.
Usage: python3 scripts/make-covers-bundle.py [out-path]
"""
import sys
import zipfile
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
COVERS = ROOT / "data" / "covers"
INNER = ("libretro-covers.zip", "openvgdb-covers.zip")
def main(argv):
out = Path(argv[1]) if len(argv) > 1 else ROOT / "psppt-covers.zip"
missing = [n for n in INNER if not (COVERS / n).exists()]
if missing:
print(f"missing: {', '.join(missing)} in {COVERS}", file=sys.stderr)
return 2
# Inner zips are already compressed; store them to avoid recompressing.
with zipfile.ZipFile(out, "w", zipfile.ZIP_STORED) as z:
for n in INNER:
print(f"adding {n} …")
z.write(COVERS / n, arcname=n)
size = out.stat().st_size / (1 << 20)
print(f"wrote {out} ({size:.0f} MB)")
return 0
if __name__ == "__main__":
raise SystemExit(main(sys.argv))