Compare commits

...
24 Commits
Author SHA1 Message Date
quoije 6eab2eca9c Use upload-artifact v3 (Gitea lacks v4 artifact protocol)
build / Linux x86_64 (push) Successful in 13m34s
build / Windows x64 (Wine) (push) Canceled after 0s
2026-09-13 20:28:57 -04:00
quoije eac29c428d Add Wine-based runner image for Windows builds
- runner/Dockerfile: Debian + Wine + Windows CPython 3.12 + PyInstaller + act_runner
- runner/entrypoint.sh: self-register with the windows:host label
- workflow windows job now builds via winpython under Wine
2026-09-13 20:18:06 -04:00
quoije ce00e3f518 Add PyInstaller spec and Gitea Actions CI for Linux/Windows binaries
build / Linux x86_64 (push) Failing after 13m46s
build / Windows x64 (push) Canceled after 0s
- run.py: top-level entry point for PyInstaller (relative imports need it)
- psp-preserve.spec: onedir build bundling DATs, loader, cover bundles, icons
- requirements-build.txt: pinned build deps (PySide6 6.8.3 for Windows ICU fix)
- .gitea/workflows/build.yml: builds linux + windows artifacts on tags/manual
- gitignore build/ dist/
2026-09-13 20:09:55 -04:00
quoije c19717402c Polish settings, add matching defaults, drop pkg2zip, externalize infra config
- Rework Settings tab into grouped sections
- Add preferred-region matching and full-hash-by-default setting
- Replace combo-box square arrow with a proper chevron
- Fix UMD Refresh button (clicked bool was clearing the view)
- Remove pkg2zip.exe fallback (pure-Python decryptor is the primary path)
- Move LAN IP/share paths to .env; use example defaults in config
2026-09-13 20:03:44 -04:00
quoije 1ec2e81e15 Scan tab: remove duplicate Scan button, add redo-match-choice context menu 2026-09-13 19:31:28 -04:00
quoije bd9cf47ca9 Run saves scan in background so app startup doesn't freeze 2026-09-09 23:15:18 -04:00
quoije 6e06d1389c Background scrape/eject/refresh, PS1 PIC0 art polish, top-right logo, and updated screenshot 2026-09-09 21:25:34 -04:00
quoije 8c5f9b6ea0 Dark theme, scan auto-ingest, packaging prep, and README refresh 2026-09-09 18:17:23 -04:00
quoije 74164b33ec Update screenshot (5-column grid) 2026-09-09 12:28:18 -04:00
quoije 75db7a3faa Fill cover grid width in library and store (no dead gap) 2026-09-09 12:27:46 -04:00
quoije ed3cc9bc8a Update README screenshot 2026-09-08 23:59:34 -04:00
quoije e01013ce6a Fix DLC switch freeze: skip bulk cover scraping for themes/DLC 2026-09-08 23:55:51 -04:00
quoije 318a920873 Fix store scan signals running on worker thread (bound methods, not lambdas) 2026-09-08 23:27:29 -04:00
quoije ec9f5e6d8e Store: fix NPS refresh, scan race, thread cleanup, and matching perf 2026-09-08 23:18:23 -04:00
quoije be54ff00aa Re-enable cover/metadata scraping for themes and DLC 2026-09-08 23:04:24 -04:00
quoije 115cdd84b0 Store: show cached theme previews in grid and details panel 2026-09-08 22:56:58 -04:00
quoije 62ce4b1913 PSP Sync: show friendly theme/DLC names instead of raw filenames 2026-09-08 22:51:36 -04:00
quoije 46c675ce79 PSP Sync: list themes and DLC in the device inventory 2026-09-08 22:45:36 -04:00
quoije 42fac24cf1 Skip ISO.BIN.EDAT wrapper so DLC shows as a single library item 2026-09-08 22:15:36 -04:00
quoije 7cc5debfa6 Skip NoPayStation items with MISSING (delisted) pkg URLs 2026-09-08 22:00:56 -04:00
quoije c03f4619e6 Show theme/DLC names and embedded theme previews in the library 2026-09-08 17:32:11 -04:00
quoije 1be33be4ab Fix NPS store content-type dropdown fetching wrong list 2026-09-07 23:58:54 -04:00
quoije 2fed95da05 Skip cover/metadata scraping for NoPayStation themes and DLC 2026-09-07 23:53:55 -04:00
quoije c94bcdc993 Add NoPayStation themes, DLC and minis support 2026-09-07 23:34:56 -04:00
30 changed files with 2386 additions and 520 deletions
+12
View File
@@ -0,0 +1,12 @@
# Copy this file to `.env` and fill in your values. `.env` is git-ignored.
# Static LAN IP for the API container on Unraid's macvlan bridge (br0).
PSP_LAN_IP=192.168.10.100
# Host paths mounted into the container (Unraid shares).
PSP_STORE_HOST=/mnt/user/downloads/Sony - PlayStation Portable
PSP_UMD_GAMES_HOST=/mnt/user/psp/umd_games
PSP_UMD_VIDEOS_HOST=/mnt/user/psp/umd_videos
# Secret token the desktop app sends as `Authorization: Bearer <token>`.
PSP_API_TOKEN=
+56
View File
@@ -0,0 +1,56 @@
name: build
on:
push:
tags: ["v*"]
workflow_dispatch:
jobs:
linux:
name: Linux x86_64
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install build dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-build.txt
- name: Build
run: pyinstaller --clean --noconfirm psp-preserve.spec
- name: Package tarball
run: tar -czf psp-preserve-linux-x86_64.tar.gz -C dist psp-preserve
- uses: actions/upload-artifact@v3
with:
name: psp-preserve-linux-x86_64
path: psp-preserve-linux-x86_64.tar.gz
windows:
name: Windows x64 (Wine)
runs-on: windows
steps:
- uses: actions/checkout@v4
- name: Install build dependencies
run: |
winpython -m pip install --upgrade pip
winpython -m pip install -r requirements-build.txt
- name: Build
run: winpython -m PyInstaller --clean --noconfirm psp-preserve.spec
- name: Package zip
run: |
cd dist && zip -rq ../psp-preserve-windows-x64.zip psp-preserve
- uses: actions/upload-artifact@v3
with:
name: psp-preserve-windows-x64
path: psp-preserve-windows-x64.zip
+8 -1
View File
@@ -16,4 +16,11 @@ data/*.pbp
data/ps1_templates/
# nps tsv files
data/nps/*.tsv
data/nps/*.tsv
# Local environment overrides (LAN paths, secrets). See .env.example
.env
# PyInstaller output
build/
dist/
+128 -43
View File
@@ -3,15 +3,16 @@ the name is terrible, i know.
![AI Generated](https://raw.githubusercontent.com/dotnetdave/ai-usage-badges/refs/heads/main/badges/png/1x/ai-generated.png)
Native Linux GUI (PySide6) for preserving PSP UMD **games** and **videos** as
`.iso` files. It scans your discs, verifies them against the full Redump catalog
(Redump DATs), reports what's missing, and retags/renames discs to a canonical
scheme. It also fetches cover art and metadata, syncs discs and save-data with a
real PSP, can pull missing discs from your own Redump 7z archive collection, and
converts PlayStation 1 ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew.
A cross-platform (Linux + Windows) desktop GUI — PySide6 — for preserving PSP
UMD **games** and **videos** as `.iso` files. It scans your discs, verifies them
against the Redump/No-Intro catalogs, reports what's missing, and retags/renames
discs to a canonical scheme. It fetches cover art and metadata, syncs discs and
save-data with a real PSP, can pull missing discs from a Redump 7z archive
collection, downloads PSN content from NoPayStation, and converts PlayStation 1
ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew.
![PSP Preservation Tool](img/qt-gui.png)
* Use default app theme for Qt.
* Dark theme with a blue accent — the Library tab in cover view. *
## Features
@@ -35,9 +36,9 @@ converts PlayStation 1 ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew.
## Requirements
- Linux + Python 3
- `PySide6` + `pycryptodome` (`pip install -r requirements.txt`)
- `7z` (p7zip) — used to peek/extract disc images from store archives
- Python 3.10+ on Linux or Windows
- `PySide6`, `curl_cffi`, `pycryptodome` (`pip install -r requirements.txt`)
- `7z` (p7zip / 7-Zip) — used to peek/extract disc images from store archives
- PPSSPP (optional) — to launch discs from the details panel
NoPayStation `.pkg` files are decrypted by the bundled **pure-Python
@@ -58,6 +59,75 @@ pip install -r requirements.txt
python3 -m app.main
```
## Building binaries
The GUI is packaged with PyInstaller into a self-contained folder. Build
requirements are pinned in `requirements-build.txt` (PySide6 6.8.3 on purpose —
see the note inside).
```bash
python3 -m venv .venv-build
. .venv-build/bin/activate
pip install -r requirements-build.txt
pyinstaller --clean --noconfirm psp-preserve.spec
# output: dist/psp-preserve/ → run dist/psp-preserve/psp-preserve
```
Notes:
- The build bundles the DATs, PS1 loader, and the offline cover bundles. The
cover zips are large (~850 MB), so builds are slow and the output is big.
- `7z` (p7zip-full on Linux, 7-Zip on Windows) is a **runtime** dependency for
reading store archives — it is not bundled.
- User data (credentials, manual covers, PS1 templates, NPS cache, library DB,
cover cache) lives in the OS state dir, never next to the binary.
### CI (Gitea Actions)
`.gitea/workflows/build.yml` builds Linux and Windows artifacts on version-tag
pushes (`v*`) or on manual trigger (**Actions → build → Run workflow**).
Artifacts appear on the Actions run page and can be attached to a release.
Each job needs a [Gitea Actions runner](https://docs.gitea.com/usage/actions/act-runner)
registered with the matching label:
```bash
# Linux runner (Docker mode, run on your server)
./act_runner register \
--instance https://git.ceaf.dev \
--token <registration-token> \
--name linux-builder \
--labels ubuntu-latest:docker://node:20-bookworm
# Windows runner (host mode, run on a Windows machine)
act_runner.exe register \
--instance https://git.ceaf.dev \
--token <registration-token> \
--name win-builder \
--labels windows:host
```
The registration token is generated in Gitea under
**Site Administration → Actions → Runners → Create new runner**.
#### Windows builds without a Windows box (Wine runner)
The `windows` job targets a runner labelled `windows` that builds under Wine on
Linux. Build it from `runner/`:
```bash
docker build -f runner/Dockerfile -t psp-wine-runner .
docker run -d --name psp-wine-runner --restart unless-stopped \
-e GITEA_INSTANCE_URL=https://git.ceaf.dev \
-e GITEA_RUNNER_REGISTRATION_TOKEN=<registration-token> \
-v /mnt/user/appdata/psp-wine-runner:/data \
psp-wine-runner
```
The image ships Wine + Windows CPython 3.12 + PyInstaller, and registers itself
with the `windows:host` label. (If you have a real Windows machine instead,
register a native `act_runner.exe` with the same `windows` label.)
## First-time setup
1. Put your DAT files in `data/dats/`:
@@ -66,14 +136,15 @@ python3 -m app.main
Redump lacks — keeps serial-based matching working).
- videos: No-Intro `Sony - PlayStation Portable (UMD Video)` (Redump has
no UMD Video DAT).
2. Ingest them into the local SQLite library:
2. Start the app — any DATs missing from the library are ingested automatically
at startup (no manual step). You can also re-ingest on demand:
```bash
python3 -m app.ingest
```
3. Open the GUI and point **Settings → General → Games/Videos directory** at
your disc folders (or set `PSP_GAMES_ROOTS` / `PSP_VIDEO_ROOTS`).
3. Point **Settings → General → Games/Videos directory** at your disc folders
(or set `PSP_GAMES_ROOTS` / `PSP_VIDEO_ROOTS`).
4. (Optional) PS1 conversion — copy any PSN PS1 classic `EBOOT.PBP` to
`data/base.PBP` (it supplies Sony's encrypted loader). Then set the **PS1
directory**, **PS1 output directory** and **PS1 base.pbp** in
@@ -105,19 +176,18 @@ Discs are matched by hash first (most precise), then serial, then filename.
- `data/ps1_templates/` — optional PS1 Classics art templates
(`ICON0.PNG` case frame, `PIC1.PNG` launch background); the app falls back
to drawn art when they're absent.
- `data/nps/` — NoPayStation TSV lists (cached here on first store scan; drop
`PSP_GAMES.tsv` / `PSX_GAMES.tsv` manually to work offline).
- `data/bin/pkg2zip.exe` — bundled `pkg2zip` binary (MIT, by mmozeiko) kept
only as an optional fallback decryptor; the app normally uses its own
pure-Python decryptor (`app/pkg.py`) and never needs Wine.
See `data/bin/ATTRIBUTION.md`.
- State dir (`~/.local/state/psp-preserve/` by default, or `$XDG_STATE_HOME/psp-preserve`):
- `data/nps/` — NoPayStation TSV lists (cached here on first store scan in dev;
a packaged build stores them under the state dir instead).
- State dir (`~/.local/state/psp-preserve/` on Linux, `%LOCALAPPDATA%\PSPPreserve`
on Windows, or `$PSP_STATE_DIR`):
- `library.sqlite` — the catalog + library database.
- `covers/` — downloaded cover cache.
- `data/` — user-writable data in packaged builds (credentials, manual
covers, PS1 templates, NPS TSV cache).
SQLite is kept on a local disk because SQLite locking is unreliable over
network filesystems; the DATs, manual covers and credentials stay in `data/`
(which may be a network share).
network filesystems. Read-only resources (DATs, the POPS loader, icons) are
bundled with the app; user-writable data lives in the state dir.
## Configuration
@@ -136,7 +206,7 @@ QSettings under `PSPPreserve`) taking precedence where noted.
| `PSP_PS1_ROOTS` | `/data/ps1` | comma-separated PS1 disc roots |
| `PSP_PS1_OUTPUT_DIR` | `/data/ps1_converted` | converted EBOOT.PBP output folder |
| `PSP_SCAN_WORKERS` | `1` | parallel hashing workers |
| `PSP_STORE_DIR` | `/media/downloads/Sony - PlayStation Portable` | local 7z store folder |
| `PSP_STORE_DIR` | `/data/store` | local 7z store folder |
| `PSP_STORE_SERVER` | `https://psp.example.com` | store server base URL |
| `PSP_STORE_TOKEN` | *(empty)* | store server bearer token |
| `PSP_ARCHIVE_ITEM` | `psp-redump-roms321.com` | archive.org item identifier |
@@ -144,9 +214,9 @@ QSettings under `PSPPreserve`) taking precedence where noted.
| `PSP_ARCHIVE_LOGGED_IN_USER` | *(empty)* | archive.org `logged-in-user` cookie |
| `PSP_NPS_DIR` | `<state>/nps` | NoPayStation output folder |
| `PSP_NPS_TSV_BASE` | `https://nopaystation.com/tsv/` | NoPayStation TSV base URL |
| `PSP_PKG2ZIP_PATH` | *(empty)* | pkg2zip binary path (else auto-detected) |
| `PSP_API_TOKEN` | *(empty)* | server API bearer token (unset = open) |
| `PSP_CREDENTIALS` | `$PSP_DATA_DIR/credentials.json` | credentials file |
| `PSP_USER_DATA_DIR` | `<state>/data` (packaged) / `./data` (dev) | user-writable data (credentials, manual covers, PS1 templates, NPS cache) |
| `PSP_CREDENTIALS` | `$PSP_USER_DATA_DIR/credentials.json` | credentials file |
| `PSP_LIBRETRO_COVERS_URL` | *(empty)* | direct URL for `libretro-covers.zip` (else derived from store server) |
| `PSP_OVGDB_COVERS_URL` | *(empty)* | direct URL for `openvgdb-covers.zip` (else derived from store server) |
@@ -235,7 +305,7 @@ Configure the folders in **Settings → General** (or `PSP_PS1_ROOTS` /
`PSP_PS1_OUTPUT_DIR`). Current limits: single-disc games only (no multi-disc
PBP); disc data is DEFLATE-compressed (level 1 by default, `--complevel 0-9`).
## Store — archive.org and local folder
## Store — four sources
The **Store** tab downloads missing discs from a Redump 7z archive collection
(`Name (Region) (tags).7z`, containing the `.iso`/`.cso` inside). It has three
@@ -274,30 +344,44 @@ skipped — and you'll get a clear error if *everything* in the item is private.
### 4. NoPayStation
The **NoPayStation** source lists PSN content from
[NoPayStation](https://nopaystation.com/) — **PSP games** (`PSP_GAMES.tsv`) and
**PS1 Classics** (`PSX_GAMES.tsv`), selectable from the content-type dropdown
next to the source selector.
[NoPayStation](https://nopaystation.com/), selectable from the content-type
dropdown next to the source selector:
- **PSP Games** (`PSP_GAMES.tsv`) — includes **minis** (also listed separately
in the `Type` column), PC Engine and NeoGeo releases.
- **PS1 Classics** (`PSX_GAMES.tsv`)
- **PSP Themes** (`PSP_THEMES.tsv`)
- **PSP DLC** (`PSP_DLCS.tsv`)
Requirements & behaviour:
1. `.pkg` files are decrypted by the built-in **pure-Python decryptor**
(`app/pkg.py`, powered by `pycryptodome` for speed) — no Wine and no
`pkg2zip` install needed. A bundled `pkg2zip.exe` is kept as a fallback
only, and only runs if Python decryption fails and Wine is present.
`pkg2zip` needed.
2. Set **NoPayStation output dir** in **Settings → Store** (default
`$PSP_STATE_DIR/nps`).
3. Downloading a PSP game writes `PSP/<TITLE_ID>/EBOOT.PBP` (plus
`PSP-KEY.EDAT` / `CONTENT.DAT` when the package ships them); a PS1 Classic
writes `PSX/<TITLE_ID>/EBOOT.PBP` (+ `DOCUMENT.DAT` and `KEYS.BIN`). Both
then show up in the **Library** (PS1 Classics under **PS1**, PSP games
under **Games**) and can be copied to the PSP like any other item.
4. The TSV lists are cached in `data/nps/` and refreshed on each store scan
3. Downloads land in the NPS folder and appear in the **Library**:
- PSP game → `PSP/<TITLE_ID>/EBOOT.PBP` (`psp_kind="eboot"`).
- PS1 Classic → `PSX/<TITLE_ID>/EBOOT.PBP` (+ `DOCUMENT.DAT`, `KEYS.BIN`).
- Theme → `THEME/<TITLE_ID>/<content_id>.ptf` (the packaged PTF is
EDAT-unwrapped to a raw theme file).
- DLC → `DLC/<GAME_ID>/<content_id>.edat` (installed as-is into the owning
game's folder).
4. Copying to the PSP puts games/PS1 under `/PSP/GAME/` (optionally nested in
a Category Lite folder), themes under `/PSP/THEME/`, and DLC under
`/PSP/GAME/<GAME_ID>/`.
5. The TSV lists are cached in `data/nps/` and refreshed on each store scan
(drop the `.tsv` files there manually to work offline).
> **DRM note:** NPS content is PSN-DRM'd. To *run* it on the PSP you need the
> [`npdrm_free`](https://github.com/qwikrazor87/npdrm_free) plugin (ARK and most
> CFW already ship it). The desktop app only downloads and decrypts; it doesn't
> strip DRM.
>
> **Covers/metadata:** themes and DLC have no cover art or game metadata of
> their own — the app scrapes them like games (IGDB/libretro/etc.), which can
> return unrelated results. Downloaded themes keep their embedded preview as
> the primary cover (used instead of a scraped image when available).
## Server setup
@@ -339,13 +423,14 @@ Token protection: set `PSP_API_TOKEN`; the client sends
macvlan bridge so it gets its own LAN IP (e.g. `192.168.1.100`, plain HTTP on 8000):
```bash
# in /path/to/psp-preserve/ — adjust the volume paths for your shares
printf 'PSP_API_TOKEN=<long-random-secret>\n' > .env
# in /path/to/psp-preserve/
cp .env.example .env # then edit: LAN IP, share paths, API token
docker compose up -d --build
```
The compose file mounts your Unraid shares into `/data/store`, `/data/umd_games`
and `/data/umd_videos` (the store is mounted read-only).
The compose file reads the LAN IP and Unraid share paths from `.env` and
mounts them into `/data/store`, `/data/umd_games` and `/data/umd_videos`
(the store is mounted read-only).
### TLS (optional)
@@ -365,8 +450,8 @@ token**, then pick **Server** in the Store tab's source selector. Use
Connect a PSP over USB (or mount a UMD) and the **PSP Sync** tab detects it:
- **Games / UMD Videos / Homebrew-PSN / PKG** — tables of what's installed, with
delete-from-PSP (checkboxes and right-click).
- **Games / UMD Videos / Homebrew-PSN / Themes / DLC / PKG** — tables of what's
installed, with delete-from-PSP (checkboxes and right-click).
- **UMD** — reads `PARAM.SFO` from a mounted UMD, shows the title/serial, and
can dump the disc image into your library.
- **Sync from Library** — copy discs from your library onto the PSP, optionally
+30 -8
View File
@@ -1,8 +1,20 @@
"""Central configuration, driven by environment variables."""
import os
import sys
from pathlib import Path
PROJECT_ROOT = Path(__file__).resolve().parent.parent
# PyInstaller sets ``sys.frozen`` and unpacks bundled files to ``sys._MEIPASS``.
# Read-only resources (DATs, loader, icons) then live in the bundle; otherwise
# they live in the project checkout.
IS_FROZEN = getattr(sys, "frozen", False)
_BUNDLE_DIR = getattr(sys, "_MEIPASS", None)
PROJECT_ROOT = Path(_BUNDLE_DIR).resolve() if _BUNDLE_DIR else Path(__file__).resolve().parent.parent
def resource_path(*parts):
"""Absolute path to a read-only bundled resource (works when frozen)."""
return PROJECT_ROOT.joinpath(*parts)
DATA_DIR = Path(os.environ.get("PSP_DATA_DIR", str(PROJECT_ROOT / "data")))
DATS_DIR = Path(os.environ.get("PSP_DATS_DIR", str(DATA_DIR / "dats")))
@@ -12,9 +24,12 @@ def _default_state_dir():
"""Local state directory (fast, reliable local disk).
SQLite locking is unreliable over network filesystems (SMB/FUSE), so the
library DB and the cover cache live on local storage while DATs, manual
covers and credentials stay in ``DATA_DIR`` (which may be a network share).
library DB and the cover cache live on local storage while read-only DATs
stay bundled with the app.
"""
if os.name == "nt":
base = os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA") or str(Path.home())
return Path(base) / "PSPPreserve"
xdg = os.environ.get("XDG_STATE_HOME")
base = Path(xdg) if xdg else Path.home() / ".local" / "state"
return base / "psp-preserve"
@@ -23,14 +38,24 @@ def _default_state_dir():
STATE_DIR = Path(os.environ.get("PSP_STATE_DIR", str(_default_state_dir())))
DB_PATH = Path(os.environ.get("PSP_DB_PATH", str(STATE_DIR / "library.sqlite")))
COVERS_DIR = Path(os.environ.get("PSP_COVERS_DIR", str(STATE_DIR / "covers")))
# User-writable data (credentials, manual covers, PS1 templates, NPS TSV cache).
# In dev this is the checkout's ``data/``; when frozen the bundle is read-only,
# so it points at local state that survives app upgrades.
USER_DATA_DIR = Path(os.environ.get(
"PSP_USER_DATA_DIR", str(DATA_DIR if not IS_FROZEN else STATE_DIR / "data")))
LEGACY_DB_PATH = DATA_DIR / "library.sqlite"
LEGACY_COVERS_DIR = DATA_DIR / "covers"
def user_data_path(*parts):
"""Absolute path to a user-writable data file/folder (works when frozen)."""
return USER_DATA_DIR.joinpath(*parts)
# Redump 7z store (browsable in the Store tab). Env override; GUI setting wins.
STORE_ROOT = Path(os.environ.get("PSP_STORE_DIR", "/media/downloads/Sony - PlayStation Portable"))
STORE_ROOT = Path(os.environ.get("PSP_STORE_DIR", "/data/store"))
# Remote store server (HTTPS). Used by the Store tab as the primary source.
STORE_SERVER_URL = os.environ.get("PSP_STORE_SERVER", "https://psp.ceaf.dev")
STORE_SERVER_URL = os.environ.get("PSP_STORE_SERVER", "https://psp.example.com")
STORE_TOKEN = os.environ.get("PSP_STORE_TOKEN", "")
# Optional direct URLs for the offline cover bundle zips. When unset, the GUI
@@ -51,9 +76,6 @@ ARCHIVE_LOGGED_IN_USER = os.environ.get("PSP_ARCHIVE_LOGGED_IN_USER", "")
NPS_OUTPUT_DIR = Path(os.environ.get("PSP_NPS_DIR", str(STATE_DIR / "nps")))
NPS_TSV_BASE = os.environ.get("PSP_NPS_TSV_BASE", "https://nopaystation.com/tsv/")
# pkg2zip binary used to decrypt/extract NPS .pkg files (blank = auto-detect).
PKG2ZIP_PATH = os.environ.get("PSP_PKG2ZIP_PATH", "")
# Scan roots (used by the scanner phase). Comma-separated in env.
GAMES_ROOTS = [Path(p) for p in os.environ.get("PSP_GAMES_ROOTS", "/data/umd_games").split(",") if p.strip()]
VIDEO_ROOTS = [Path(p) for p in os.environ.get("PSP_VIDEO_ROOTS", "/data/umd_videos").split(",") if p.strip()]
+38 -20
View File
@@ -52,7 +52,7 @@ _IGDB_IMAGE = "https://images.igdb.com/igdb/image/upload/"
_TMDB_SEARCH = "https://api.themoviedb.org/3/search/movie"
_TMDB_IMAGE = "https://image.tmdb.org/t/p/"
_GOOGLE_CSE_API = "https://www.googleapis.com/customsearch/v1"
_MANUAL_DIR = config.DATA_DIR / "manual_covers"
_MANUAL_DIR = config.USER_DATA_DIR / "manual_covers"
_OVGDB_ZIP = config.DATA_DIR / "covers" / "openvgdb-covers.zip"
_LIBRETRO_ZIP = config.DATA_DIR / "covers" / "libretro-covers.zip"
@@ -411,6 +411,9 @@ def _fetch_libretro(name, timeout, system="games", indexed_only=False):
data = fetch_libretro_local(name)
if data is not None:
return data
# Theme/DLC share the PSP thumbnails repo, so reuse the games index instead
# of downloading a duplicate copy of the same tree under a new system key.
idx_system = system if system in _PLATFORMS else "games"
if not indexed_only:
# 1. Cheap: exact + progressively stripped names (no index needed).
for kind in _LIBRETRO_KINDS:
@@ -421,7 +424,7 @@ def _fetch_libretro(name, timeout, system="games", indexed_only=False):
# 2. Fuzzy/exact via the precomputed index (covers exact names too, so the
# bulk loader skips the wasteful exact pass entirely).
for kind in _LIBRETRO_KINDS:
best = _index_best(name, kind, system)
best = _index_best(name, kind, idx_system)
if best:
data = _fetch_png(best, kind, timeout, system)
if data is not None:
@@ -808,7 +811,7 @@ def compose_game_cover(key_art_bytes):
return bytes(ba)
_PS1_TEMPLATE_DIR = config.DATA_DIR / "ps1_templates"
_PS1_TEMPLATE_DIR = config.USER_DATA_DIR / "ps1_templates"
def _ps1_template_bytes(name):
@@ -934,25 +937,28 @@ def _make_pic0(title, description, esrb, QImage, QPainter, QPainterPath, QColor,
QRectF, Qt, _save):
W, H = 310, 180
img = QImage(W, H, QImage.Format_ARGB32)
img.fill(QColor(0, 0, 0, 0))
# Dark gradient backdrop.
# Dark gradient backdrop clipped to a rounded card (transparent corners).
grad = QLinearGradient(0, 0, 0, H)
grad.setColorAt(0.0, QColor(38, 38, 40))
grad.setColorAt(1.0, QColor(12, 12, 14))
card = QPainterPath()
card.addRoundedRect(QRectF(0, 0, W, H), 12, 12)
p = QPainter(img)
p.setRenderHint(QPainter.Antialiasing, True)
p.setRenderHint(QPainter.TextAntialiasing, True)
p.fillRect(0, 0, W, H, grad)
p.fillPath(card, grad)
# Rounded border.
# Rounded card border.
p.setPen(QPen(QColor(110, 110, 115), 1))
p.setBrush(Qt.NoBrush)
p.drawRoundedRect(QRectF(4, 4, W - 8, H - 8), 9, 9)
p.drawPath(card)
# PlayStation button glyphs, top-right.
_draw_ps_glyphs(p, W - 34, 12, 22, QColor(200, 200, 205), QRectF, QPointF,
QPolygonF, QPen, Qt)
_draw_ps_glyphs(p, W - 34, 12, 22, QRectF, QPointF, QPolygonF, QPen, Qt,
QColor)
# Title (bold, upper-left). Shrink the font until the whole title fits, so
# long names are shown in full instead of being cut with an ellipsis.
@@ -974,6 +980,10 @@ def _make_pic0(title, description, esrb, QImage, QPainter, QPainterPath, QColor,
p.drawText(QRectF(18, 16, title_width, 28), Qt.AlignLeft | Qt.AlignVCenter,
title_text)
# Thin divider under the title (like the original PSN info board).
p.setPen(QPen(QColor(255, 255, 255, 55), 1))
p.drawLine(QPointF(18, 46), QPointF(W - 18, 46))
# Description (wrapped to at most 3 lines, last line elided).
if description:
f_desc = QFont("DejaVu Sans")
@@ -982,7 +992,7 @@ def _make_pic0(title, description, esrb, QImage, QPainter, QPainterPath, QColor,
p.setPen(QColor(215, 215, 215))
fm = QFontMetrics(f_desc)
lines = _wrap_elided(fm, description.strip(), W - 36, 3, Qt)
y = 46
y = 52
for line in lines:
p.drawText(QRectF(18, y, W - 36, 16), Qt.AlignLeft | Qt.AlignVCenter, line)
y += 16
@@ -1024,23 +1034,31 @@ def _wrap_elided(fm, text, width, max_lines, Qt):
return lines[:max_lines]
def _draw_ps_glyphs(p, x, y, size, color, QRectF, QPointF, QPolygonF, QPen, Qt):
"""Draw the four PlayStation face-button glyphs in a 2x2 grid."""
def _draw_ps_glyphs(p, x, y, size, QRectF, QPointF, QPolygonF, QPen, Qt, QColor):
"""Draw the four PlayStation face-button glyphs in a 2x2 grid, using the
official button colours (green triangle, red circle, blue cross, pink square)."""
cell = size / 2.0
pen = QPen(color, max(1.0, size / 14.0))
pen.setCapStyle(Qt.RoundCap)
pen.setJoinStyle(Qt.RoundJoin)
p.setPen(pen)
p.setBrush(Qt.NoBrush)
# Triangle (top-left).
def _pen(color):
pen = QPen(color, max(1.0, size / 14.0))
pen.setCapStyle(Qt.RoundCap)
pen.setJoinStyle(Qt.RoundJoin)
return pen
# Triangle (top-left, green).
p.setPen(_pen(QColor("#3F9B0F")))
tri = QPolygonF([QPointF(x, y + cell), QPointF(x + cell, y), QPointF(x + cell, y + cell)])
p.drawPolygon(tri)
# Circle (top-right).
# Circle (top-right, red).
p.setPen(_pen(QColor("#DE1C18")))
p.drawEllipse(QRectF(x + cell, y, cell, cell))
# Cross (bottom-left).
# Cross (bottom-left, blue).
p.setPen(_pen(QColor("#287FC1")))
p.drawLine(QPointF(x + 1, y + cell + 1), QPointF(x + cell - 1, y + size - 1))
p.drawLine(QPointF(x + cell - 1, y + cell + 1), QPointF(x + 1, y + size - 1))
# Square (bottom-right).
# Square (bottom-right, pink).
p.setPen(_pen(QColor("#C94E9B")))
p.drawRect(QRectF(x + cell, y + cell, cell, cell))
+759 -233
View File
File diff suppressed because it is too large Load Diff
+29 -15
View File
@@ -23,6 +23,9 @@ _miss_db = {}
_miss_loaded = False
_MISS_TTL = 86400.0 # retry failed metadata after 1 day
# Memoised release-date table (invalidated on save/reset).
_dates_cache = None
def _load_miss_db():
global _miss_loaded
@@ -122,6 +125,7 @@ def save(release_id, system, meta, manual=False, title=None):
)
conn.commit()
conn.close()
_invalidate_dates_cache()
return True
except sqlite3.Error:
return False
@@ -140,6 +144,7 @@ def reset(release_id, system):
)
conn.commit()
conn.close()
_invalidate_dates_cache()
except sqlite3.Error:
pass
@@ -161,23 +166,32 @@ def cached_release_dates():
"""Return {(release_id, system): release_date} from cached metadata only.
Pure DB read — no network — so list views can show a release date without
triggering API fetches for every row.
triggering API fetches for every row. Result is memoised and invalidated
whenever metadata is saved.
"""
out = {}
try:
conn = get_connection(config.DB_PATH)
rows = conn.execute("SELECT release_id, system, meta_json FROM item_info").fetchall()
conn.close()
except sqlite3.Error:
return {}
for r in rows:
global _dates_cache
if _dates_cache is None:
out = {}
try:
meta = json.loads(r["meta_json"] or "{}")
except (TypeError, ValueError):
continue
if isinstance(meta, dict) and meta.get("release_date"):
out[(r["release_id"], r["system"])] = meta["release_date"]
return out
conn = get_connection(config.DB_PATH)
rows = conn.execute("SELECT release_id, system, meta_json FROM item_info").fetchall()
conn.close()
except sqlite3.Error:
return out
for r in rows:
try:
meta = json.loads(r["meta_json"] or "{}")
except (TypeError, ValueError):
continue
if isinstance(meta, dict) and meta.get("release_date"):
out[(r["release_id"], r["system"])] = meta["release_date"]
_dates_cache = out
return _dates_cache
def _invalidate_dates_cache():
global _dates_cache
_dates_cache = None
def get_or_fetch(release_id, system, fetch_title):
+36
View File
@@ -88,6 +88,42 @@ def ingest_dat(conn, path):
return dat_name, system, release_count, rom_count
def ensure_catalog(dats_dir=None, conn=None):
"""Ingest any DAT files missing from the DB (idempotent, cheap when current).
Called once at startup so a fresh/reset database still has the catalog and
scanning produces matches instead of everything coming back ``unknown``.
Returns the number of DATs ingested.
"""
dats_dir = Path(dats_dir) if dats_dir else config.DATS_DIR
if not dats_dir.is_dir():
return 0
files = sorted(p for p in dats_dir.glob("*.dat") if p.is_file())
if not files:
return 0
own = conn is None
if own:
conn = get_connection(config.DB_PATH)
try:
known = {r["source_path"] for r in conn.execute("SELECT source_path FROM dats")}
count = 0
for path in files:
if str(path) in known:
continue
try:
ingest_dat(conn, path)
except Exception as exc: # noqa: BLE001 - a bad DAT shouldn't block startup
print(f"warning: could not ingest {path.name}: {exc}", file=sys.stderr)
continue
count += 1
conn.commit()
return count
finally:
if own:
conn.close()
def main(argv=None):
parser = argparse.ArgumentParser(description="Ingest clrmamepro DAT files into SQLite")
parser.add_argument("--dats", help="directory containing .dat files (default: data/dats)")
+4 -2
View File
@@ -48,8 +48,10 @@ def launch_ppsspp(iso, exe=""):
else:
return False, "PPSSPP not found. Install it or set its path in Settings → Saves."
try:
subprocess.Popen(cmd, start_new_session=True, env=_env(),
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
kwargs = {"env": _env(), "stdout": subprocess.DEVNULL, "stderr": subprocess.DEVNULL}
if os.name != "nt":
kwargs["start_new_session"] = True # POSIX-only: detach from our process group
subprocess.Popen(cmd, **kwargs)
return True, ""
except OSError as exc:
return False, str(exc)
+35 -1
View File
@@ -111,6 +111,36 @@ def normalize_name(name):
return _normalize(name)
_EUROPEAN_REGIONS = {
"europe", "germany", "france", "spain", "italy", "russia", "netherlands",
"portugal", "poland", "united kingdom", "greece", "norway", "sweden",
"denmark", "finland", "austria", "belgium", "switzerland",
}
def region_matches(region, prefer):
"""True if ``region`` satisfies the user's preferred-region hint."""
if not prefer or not region:
return False
tokens = {t.strip().lower() for t in str(region).split(",") if t.strip()}
if not tokens:
return False
prefer = prefer.strip().lower()
if prefer in tokens:
return True
if prefer == "europe":
return bool(tokens & _EUROPEAN_REGIONS)
return False
def apply_region_preference(candidates, prefer):
"""Narrow ``candidates`` to the preferred region (fall back to all)."""
if not prefer:
return candidates
filtered = [c for c in candidates if region_matches(c.get("region"), prefer)]
return filtered or candidates
def serial_candidates(conn, record):
"""Return all catalog releases matching the disc's serial (size-narrowed)."""
serial = record.get("serial")
@@ -131,7 +161,7 @@ def serial_candidates(conn, record):
return _narrow(conn, [dict(r) for r in rows], record.get("size"))
def match(conn, record):
def match(conn, record, prefer_region=None):
size = record.get("size")
# 1. Hash is the most precise key; serials are shared across revisions.
@@ -171,6 +201,10 @@ def match(conn, record):
for cand in candidates:
if cand["id"] == saved["release_id"]:
return "choice", cand
if prefer_region:
filtered = [c for c in candidates if region_matches(c.get("region"), prefer_region)]
if len(filtered) == 1:
return "region", filtered[0]
return "unknown", None
# 4. Fuzzy fallback (filename + PARAM.SFO title), size-disambiguated.
+46 -107
View File
@@ -9,20 +9,23 @@ folder — no Wine or ``pkg2zip`` required.
from __future__ import annotations
import logging
import os
import shutil
import subprocess
import urllib.parse
import urllib.request
from datetime import datetime
from pathlib import Path
from typing import List, Optional
from typing import List
from . import config
log = logging.getLogger("nopaystation")
_TSV_NAMES = {"psp": "PSP_GAMES.tsv", "psx": "PSX_GAMES.tsv"}
_TSV_NAMES = {
"psp": "PSP_GAMES.tsv",
"psx": "PSX_GAMES.tsv",
"theme": "PSP_THEMES.tsv",
"dlc": "PSP_DLCS.tsv",
}
# NoPayStation TSVs are tab-separated with a header row whose exact columns
# differ per platform (PSP has "Type"/"RAP" columns, PSX has "Original Name").
@@ -45,13 +48,13 @@ _HEADER_ALIASES = {
def tsv_url(content_type: str, base: str = "") -> str:
"""TSV URL for a content type (``psp`` / ``psx``)."""
"""TSV URL for a content type (``psp`` / ``psx`` / ``theme`` / ``dlc``)."""
base = (base or config.NPS_TSV_BASE).rstrip("/")
return f"{base}/{_TSV_NAMES.get(content_type, _TSV_NAMES['psp'])}"
def _tsv_dir() -> Path:
d = config.DATA_DIR / "nps"
d = config.USER_DATA_DIR / "nps"
d.mkdir(parents=True, exist_ok=True)
return d
@@ -76,7 +79,7 @@ def parse_tsv(text: str, content_type: str) -> List[dict]:
``release_date``, ``mtime_ns``) plus ``title_id``, ``pkg_url``, ``sha256``,
``content_type`` and the cover ``system`` ("games" for PSP, "ps1" for PSX).
"""
system = "games" if content_type == "psp" else "ps1"
system = {"psp": "games", "psx": "ps1", "theme": "theme", "dlc": "dlc"}.get(content_type, "games")
items: List[dict] = []
header = None
for line in text.splitlines():
@@ -94,7 +97,9 @@ def parse_tsv(text: str, content_type: str) -> List[dict]:
row[canon] = cols[i].strip()
name = row.get("name") or ""
pkg_url = row.get("pkg_url") or ""
if not name or not pkg_url:
# NoPayStation marks delisted packages with the literal "MISSING" in the
# URL column — skip them so they can't be queued for download.
if not name or not pkg_url or not pkg_url.lower().startswith(("http://", "https://")):
continue
title_id = row.get("title_id") or ""
try:
@@ -124,47 +129,40 @@ def parse_tsv(text: str, content_type: str) -> List[dict]:
return items
def fetch_items(content_type: str, url: str = "", timeout: int = 60) -> List[dict]:
"""Download (and cache) a TSV, returning parsed item dicts."""
def fetch_items(content_type: str, url: str = "", timeout: int = 60, force: bool = False) -> List[dict]:
"""Download (and cache) a TSV, returning parsed item dicts.
``force`` re-downloads the TSV (used by the Store's Refresh button); the
local cache is only used as a fallback when the network fails.
"""
url = url or tsv_url(content_type)
name = _TSV_NAMES.get(content_type, "PSP_GAMES.tsv")
cache = _tsv_dir() / name
if cache.is_file():
def read_cache():
try:
return parse_tsv(cache.read_text(encoding="utf-8", errors="replace"), content_type)
except OSError:
pass
req = urllib.request.Request(url, headers={"User-Agent": "psp-preserve/0.1"})
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = resp.read()
text = data.decode("utf-8", errors="replace")
cache.write_text(text, encoding="utf-8")
return parse_tsv(text, content_type)
return None
if not force and cache.is_file():
items = read_cache()
if items is not None:
return items
def find_pkg2zip(configured_path: str = "") -> Optional[Path]:
"""Locate pkg2zip: configured path, then the bundled binary, then PATH."""
if configured_path:
p = Path(configured_path)
return p if p.is_file() else None
bundled = config.DATA_DIR / "bin" / "pkg2zip.exe"
if bundled.is_file():
return bundled
found = shutil.which("pkg2zip")
return Path(found) if found else None
def _pkg2zip_cmd(pkg2zip: Path) -> List[str]:
"""Command prefix for pkg2zip, wrapping the bundled .exe in Wine on Linux."""
if pkg2zip.suffix.lower() == ".exe" and os.name != "nt":
wine = shutil.which("wine") or shutil.which("wine64")
if not wine:
raise RuntimeError(
"The bundled pkg2zip is a Windows binary and Wine is not installed — "
"install Wine (or point Settings → Store at a native pkg2zip)."
)
return [wine, str(pkg2zip)]
return [str(pkg2zip)]
try:
req = urllib.request.Request(url, headers={"User-Agent": "psp-preserve/0.1"})
with urllib.request.urlopen(req, timeout=timeout) as resp:
data = resp.read()
text = data.decode("utf-8", errors="replace")
cache.write_text(text, encoding="utf-8")
return parse_tsv(text, content_type)
except Exception: # noqa: BLE001 - fall back to the cached list on failure
items = read_cache()
if items is not None:
log.warning("could not refresh %s TSV; using cached copy", content_type)
return items
raise
def _safe_folder(value: str) -> str:
@@ -186,12 +184,11 @@ def _ensure_keys_bin(folder, kind):
pass
def extract_pkg(pkg_path, title_id: str, dest_dir, pkg2zip: Path = None,
progress_cb=None, cancel_event=None) -> Path:
def extract_pkg(pkg_path, title_id: str, dest_dir,
kind_hint: str = None, progress_cb=None, cancel_event=None) -> Path:
"""Decrypt/extract a ``.pkg`` into ``dest_dir/<TITLE_ID>/``.
Uses the bundled pure-Python decryptor (no Wine / pkg2zip needed); falls
back to pkg2zip only if that fails and a binary is available. Returns the
Uses the bundled pure-Python decryptor (no Wine or pkg2zip). Returns the
destination game folder path.
"""
from . import pkg as pkgmod
@@ -199,73 +196,15 @@ def extract_pkg(pkg_path, title_id: str, dest_dir, pkg2zip: Path = None,
pkg_path = Path(pkg_path)
dest = Path(dest_dir) / _safe_folder(title_id)
if (dest / "EBOOT.PBP").exists():
if dest.is_dir() and any(dest.iterdir()):
return dest
# 1. Pure-Python decryptor (primary).
try:
_kind, _tid, folder = pkgmod.extract(pkg_path, dest_dir, progress_cb, cancel_event)
_kind, _tid, folder = pkgmod.extract(pkg_path, dest_dir, kind_hint,
progress_cb, cancel_event)
_ensure_keys_bin(folder, _kind)
return folder
except TransferCancelled:
raise
except Exception as exc: # noqa: BLE001
py_exc = exc
log.warning("python pkg decrypt failed (%s); trying pkg2zip fallback", exc)
# 2. pkg2zip fallback (needs a binary; usually Wine + the bundled .exe).
if pkg2zip is None:
pkg2zip = find_pkg2zip()
if pkg2zip is None:
raise RuntimeError(
f"could not decrypt {pkg_path.name} (no pkg2zip available): {py_exc}"
)
folder_name = _safe_folder(title_id)
tmp = Path(dest_dir).with_name(".nps-tmp-" + folder_name)
if tmp.exists():
shutil.rmtree(tmp, ignore_errors=True)
tmp.mkdir(parents=True, exist_ok=True)
if progress_cb is not None:
progress_cb("Extracting", 0.0, "")
try:
cmd = _pkg2zip_cmd(Path(pkg2zip)) + ["-x", str(pkg_path)]
env = dict(os.environ)
env["WINEDEBUG"] = "-all" # silence Wine's console chatter
proc = subprocess.Popen(
cmd,
cwd=str(tmp), stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
text=True, bufsize=1, env=env,
)
assert proc.stdout is not None
for _line in proc.stdout:
if cancel_event is not None and cancel_event.is_set():
proc.kill()
raise TransferCancelled()
proc.wait()
if proc.returncode != 0:
raise RuntimeError(f"pkg2zip failed (exit {proc.returncode}) for {pkg_path.name}")
finally:
if cancel_event is not None and cancel_event.is_set():
shutil.rmtree(tmp, ignore_errors=True)
raise TransferCancelled()
eboots = list(tmp.rglob("EBOOT.PBP"))
if not eboots:
shutil.rmtree(tmp, ignore_errors=True)
raise RuntimeError(f"pkg2zip produced no EBOOT.PBP from {pkg_path.name}")
src_folder = eboots[0].parent
dest.mkdir(parents=True, exist_ok=True)
for p in sorted(src_folder.rglob("*")):
if not p.is_file():
continue
rel = p.relative_to(src_folder)
target = dest / rel
target.parent.mkdir(parents=True, exist_ok=True)
shutil.copy2(p, target)
if progress_cb is not None:
progress_cb("Extracting", 1.0, "")
shutil.rmtree(tmp, ignore_errors=True)
return dest
raise RuntimeError(f"could not decrypt {pkg_path.name}: {exc}") from exc
+2 -2
View File
@@ -2,9 +2,9 @@
from . import iso_reader, matcher, namer
def identify(conn, record):
def identify(conn, record, prefer_region=None):
"""Return ``{method, release, serial, canonical}`` for a scanned disc record."""
method, release = matcher.match(conn, record)
method, release = matcher.match(conn, record, prefer_region=prefer_region)
return identify_with_release(record, release, method)
+151 -30
View File
@@ -1,10 +1,19 @@
"""Pure-Python PSP/PSX PSN ``.pkg`` decryption (no Wine / pkg2zip needed).
"""Pure-Python PSP/PSX/theme/DLC PSN ``.pkg`` decryption (no Wine / pkg2zip).
Ports the folder-mode extraction used by pkgi-psp's depackager and pkg2zip:
parse the ``\\x7FPKG`` header, derive the PSP AES key, AES-CTR-decrypt the file
entries, and write the game's ``EBOOT.PBP`` (+ siblings) into a folder. The
resulting ``EBOOT.PBP`` keeps its DATA.PSP EDAT-encrypted, so it runs on CFW
with the ``npdrm_free`` plugin (ARK ships it) — same as pkgi-psp's install.
parse the ``\\x7FPKG`` header, derive the PSP/PS3 AES key, AES-CTR-decrypt the
file entries, and write the game's ``EBOOT.PBP`` (+ siblings) into a folder.
Supported content:
* PSP games / minis (``EBOOT.PBP``) — ``DATA.PSP`` stays EDAT-encrypted, so it
runs on CFW with ``npdrm_free`` (ARK ships it), same as pkgi-psp's install.
* PS1 Classics (``EBOOT.PBP`` + ``DOCUMENT.DAT``).
* PSP themes (``.ptf``) — the packaged PTF is EDAT-wrapped and is unwrapped here.
* PSP DLC (``.edat``) — extracted as-is into the owning game's folder.
Theme unwrapping needs ``pycryptodome`` (for AES-CMAC + ECB); everything else
works with either pycryptodome or the built-in pure-Python AES fallback.
"""
from __future__ import annotations
@@ -16,6 +25,14 @@ PKG_MAGIC = 0x7F504B47
PSP_KEY = bytes.fromhex("07f2c68290b50d2c33818d709b60e62b")
PS3_KEY = bytes.fromhex("2e7b71d7c9c9a14ea3221f188828b8f8")
# PSP KIRK/AMCTL keys used to unwrap theme PTF EDATs (pkgi-psp / libkirk).
_KIRK38 = bytes.fromhex("12468d7e1c42209bba5426835eb03303")
_KIRK39 = bytes.fromhex("c43bb6d653ee67493ea95fbc0ced6f8a")
_KIRK63 = bytes.fromhex("9c9b1372f8c640cf1c62f5d592ddb582")
_AMCTL3 = bytes.fromhex("e350ed1d910a1fd029bb1c3ef34077fb")
_AMCTL4 = bytes.fromhex("135fa47cab395ba476b8cca98f3a0445")
_AMCTL5 = bytes.fromhex("678d7fa32a9ca0d1508ad8385e4b017e")
try:
from Crypto.Cipher import AES as _AES # pycryptodome (fast)
_HAVE_CRYPTO = True
@@ -136,13 +153,86 @@ def _make_ctr_decryptor(key, iv, block_index):
return decrypt
# --------------------------------------------------------------------------- #
# PSP theme PTF EDAT unwrapping (pkgi-psp's unpack_psp_edat)
# --------------------------------------------------------------------------- #
def _need_crypto():
if not _HAVE_CRYPTO:
raise RuntimeError("theme extraction needs pycryptodome (pip install pycryptodome)")
def _ecb_decrypt(key, block):
_need_crypto()
return _AES.new(key, _AES.MODE_ECB).decrypt(block)
def _cmac(key, data):
_need_crypto()
from Crypto.Hash import CMAC
c = CMAC.new(key, ciphermod=_AES)
c.update(data)
return c.digest()
def _kirk_iv(mac, key_header, offset2):
tmp = _ecb_decrypt(_KIRK38, bytes(key_header[0x70:0x80]))
iv = bytes(mac[i] ^ tmp[i] ^ key_header[offset2 + i] ^ _AMCTL3[i] ^ _AMCTL5[i] for i in range(16))
iv = _ecb_decrypt(_KIRK39, iv)
return bytes(b ^ _AMCTL4[i] for i, b in enumerate(iv))
def _kirk_cbc_decrypt(iv, index, data):
prev = bytearray(16) if index == 0 else bytearray(iv[:12]) + index.to_bytes(4, "little")
block = bytearray(iv[:12]) + index.to_bytes(4, "little")
out = bytearray()
for i in range(0, len(data), 16):
v = int.from_bytes(block[12:16], "little") + 1
block[12:16] = v.to_bytes(4, "little")
o = _ecb_decrypt(_KIRK63, bytes(block))
out += bytes(b ^ prev[k] ^ o[k] for k, b in enumerate(data[i:i + 16]))
prev = bytes(block)
return bytes(out)
def _unwrap_theme_ptf(buf):
"""Unwrap a PSP theme PTF EDAT (``buf`` is the fully CTR-decrypted entry)."""
if len(buf) < 0x90 + 0xA0:
raise RuntimeError("theme EDAT is too short")
kh_off = buf[0xC]
key_header = bytearray(buf[kh_off:kh_off + 0xA0])
if bytes(key_header[:4]) != b"\x00PGD":
raise RuntimeError("theme EDAT: bad PGD header")
if int.from_bytes(key_header[4:8], "little") != 1 or int.from_bytes(key_header[8:12], "little") != 1:
raise RuntimeError("theme EDAT: unsupported key/drm type")
mac = _cmac(_KIRK38, bytes(key_header[:0x70]))
# Step 1: reveal the data size/offset fields (bytes 0x30..0x60).
key_header[0x30:0x60] = _kirk_cbc_decrypt(_kirk_iv(mac, key_header, 0x10), 0, bytes(key_header[0x30:0x60]))
data_size = int.from_bytes(key_header[0x44:0x48], "little")
data_offset = int.from_bytes(key_header[0x4c:0x50], "little")
if data_offset != 0x90:
raise RuntimeError("theme EDAT: unexpected data offset")
# Step 2: decrypt the PTF body.
iv = _kirk_iv(mac, key_header, 0x30)
out = bytearray()
nblocks = (data_size + 15) // 16
for i in range(nblocks):
chunk = buf[kh_off + data_offset + i * 16:kh_off + data_offset + (i + 1) * 16]
out += _kirk_cbc_decrypt(iv, i, chunk)
return bytes(out[:data_size])
# --------------------------------------------------------------------------- #
# PKG extraction
# --------------------------------------------------------------------------- #
def extract(pkg_path, dest_dir, progress_cb=None, cancel_event=None):
"""Decrypt a PSP/PSX PSN pkg into ``dest_dir/<TITLE_ID>/``.
def extract(pkg_path, dest_dir, kind_hint=None, progress_cb=None, cancel_event=None):
"""Decrypt a PSN pkg into ``dest_dir/<TITLE_ID>/``.
Returns ``(kind, title_id, folder)`` where kind is ``"psp"`` or ``"psx"``.
Returns ``(kind, title_id, folder)`` where kind is ``"psp"``, ``"psx"``,
``"theme"`` or ``"dlc"``. ``kind_hint`` ("theme"/"dlc") overrides the
pkg-header content-type detection (NoPayStation DLC ships as two formats).
"""
from .store import TransferCancelled
@@ -179,19 +269,30 @@ def extract(pkg_path, dest_dir, progress_cb=None, cancel_event=None):
items_offset = int.from_bytes(m[8:12], "big")
off += 8 + msize
if content_type == 6:
if kind_hint in ("theme", "dlc"):
kind = kind_hint
elif content_type == 6:
kind = "psx"
elif content_type in (7, 0xE, 0xF, 0x10):
kind = "psp"
elif content_type == 9:
kind = "theme"
elif content_type == 0xB:
kind = "dlc"
else:
raise RuntimeError(f"unsupported NoPayStation content type 0x{content_type:x}")
if key_type != 1:
# NoPayStation DLC is wrapped in a PS3-format pkg (key type 4 in the header
# but actually encrypted with the PS3 key); everything else uses the PSP key.
if content_type == 0xB:
main_key = PS3_KEY
ps3_key = PS3_KEY
elif key_type == 1:
main_key = PSP_KEY
ps3_key = PS3_KEY
else:
raise RuntimeError(f"unsupported PKG key type {key_type}")
main_key = PSP_KEY
ps3_key = PS3_KEY
# Decrypt the whole item table in one shot.
with open(pkg_path, "rb") as fh:
fh.seek(enc_offset + items_offset)
@@ -232,26 +333,46 @@ def extract(pkg_path, dest_dir, progress_cb=None, cancel_event=None):
continue
if kind == "psp" and base not in ("EBOOT.PBP", "PSP-KEY.EDAT", "CONTENT.DAT"):
continue
if kind == "theme" and not base.lower().endswith(".ptf"):
continue
if kind == "dlc" and not base.lower().endswith(".edat"):
continue
if kind == "dlc" and base.upper() == "ISO.BIN.EDAT":
# Small EDAT wrapper shipped inside PSP DLC packages — not DLC
# content itself, so don't install it (matches pkgi-psp).
continue
dest = folder / base
decrypt = _make_ctr_decryptor(item_key, iv, data_offset // 16)
fh.seek(enc_offset + data_offset)
with open(dest, "wb") as out:
remaining = data_size
while remaining > 0:
if cancel_event is not None and cancel_event.is_set():
raise TransferCancelled()
chunk = fh.read(min(1 << 20, remaining))
if not chunk:
break
out.write(decrypt(chunk))
remaining -= len(chunk)
done += len(chunk)
if progress_cb is not None:
progress_cb("Extracting", min(done / total, 1.0), "")
if kind == "theme":
# Read the (small) entry fully, CTR-decrypt, then unwrap the EDAT.
fh.seek(enc_offset + data_offset)
raw = _ctr_decrypt(item_key, iv, data_offset // 16, fh.read(data_size))
dest = folder / base
dest.write_bytes(_unwrap_theme_ptf(raw))
done += data_size
else:
dest = folder / base
decrypt = _make_ctr_decryptor(item_key, iv, data_offset // 16)
fh.seek(enc_offset + data_offset)
with open(dest, "wb") as out:
remaining = data_size
while remaining > 0:
if cancel_event is not None and cancel_event.is_set():
raise TransferCancelled()
chunk = fh.read(min(1 << 20, remaining))
if not chunk:
break
out.write(decrypt(chunk))
remaining -= len(chunk)
done += len(chunk)
if progress_cb is not None:
progress_cb("Extracting", min(done / total, 1.0), "")
written.append(dest)
if not (folder / "EBOOT.PBP").exists():
if kind in ("psp", "psx") and not (folder / "EBOOT.PBP").exists():
raise RuntimeError(f"no EBOOT.PBP found in {pkg_path.name}")
if kind == "theme" and not any(p.suffix.lower() == ".ptf" for p in written):
raise RuntimeError(f"no PTF found in {pkg_path.name}")
if kind == "dlc" and not written:
raise RuntimeError(f"no DLC EDAT found in {pkg_path.name}")
return kind, title_id, folder
+4 -1
View File
@@ -40,6 +40,7 @@ import io
import os
import re
import struct
import sys
import zlib
from dataclasses import dataclass, field
from pathlib import Path
@@ -88,7 +89,9 @@ SYNC_PATTERN = b"\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00"
# (ARK/PRO/ME) loads this ELF directly as the DATA.PSP section, sidestepping
# the base PBP's encrypted ~PSP loader which frequently fails to link on
# 6.60/6.61 (error 80020148 "game could not be started").
_LOADER_DIR = Path(__file__).resolve().parent.parent / "data" / "loader"
_BUNDLE_DIR = getattr(sys, "_MEIPASS", None)
_APP_ROOT = Path(_BUNDLE_DIR).resolve() if _BUNDLE_DIR else Path(__file__).resolve().parent.parent
_LOADER_DIR = _APP_ROOT / "data" / "loader"
BUNDLED_DATA_PSP = _LOADER_DIR / "DATA.PSP"
BUNDLED_KEYS_BIN = _LOADER_DIR / "KEYS.BIN"
+43 -1
View File
@@ -221,8 +221,48 @@ def _pkg_items(root):
return items
def _theme_items(root):
"""Installed themes (``PSP/THEME/*.ptf``), named from the PTF header."""
items = []
theme_dir = root / "PSP" / "THEME"
if theme_dir.is_dir():
try:
entries = sorted(theme_dir.iterdir())
except OSError:
return items
for path in entries:
if not path.is_file() or path.suffix.lower() != ".ptf":
continue
item = _file_item(path, "theme")
try:
from . import theme as theme_mod
name = theme_mod.ptf_name(path)
except Exception: # noqa: BLE001
name = ""
if name:
item["name"] = name
items.append(item)
return items
def _dlc_items(root):
"""Installed DLC (``PSP/GAME/<GAME_ID>/*.edat``)."""
items = []
game_dir = root / "PSP" / "GAME"
if game_dir.is_dir():
for path in sorted(game_dir.rglob("*")):
if not path.is_file() or path.suffix.lower() != ".edat":
continue
if path.name.upper() == "ISO.BIN.EDAT":
continue # package wrapper, not DLC content
item = _file_item(path, "dlc")
item["id"] = path.parent.name
items.append(item)
return items
def list_installed(root):
"""Return ``{games, videos, homebrew, pkg}`` inventories for a PSP root."""
"""Return ``{games, videos, homebrew, pkg, themes, dlc}`` for a PSP root."""
root = Path(root)
games, videos, homebrew_isos = _iso_items(root)
return {
@@ -230,6 +270,8 @@ def list_installed(root):
"videos": videos,
"homebrew": _homebrew_items(root) + homebrew_isos,
"pkg": _pkg_items(root),
"themes": _theme_items(root),
"dlc": _dlc_items(root),
}
+1 -1
View File
@@ -11,7 +11,7 @@ from pathlib import Path
from . import config
CREDENTIALS_PATH = Path(
os.environ.get("PSP_CREDENTIALS", str(config.DATA_DIR / "credentials.json"))
os.environ.get("PSP_CREDENTIALS", str(config.USER_DATA_DIR / "credentials.json"))
)
FIELDS = [
+316
View File
@@ -0,0 +1,316 @@
"""Global dark theme for the GUI (Fusion style + palette + stylesheet).
Applied once at startup from ``gui.main``. The per-widget ``setStyleSheet``
colors sprinkled through ``gui.py`` (``#999`` secondary text, ``#e06c6c`` delete
buttons, ``#14151a`` preview panels) were written for a dark background, so a
single cohesive dark theme here makes the whole UI consistent.
"""
from PySide6.QtGui import QColor, QPalette
from . import config
# Design tokens
BG = "#1e1f24" # window background
SURFACE = "#26272e" # cards, inputs, table headers
RAISED = "#2d2e36" # buttons, combos, menus
BORDER = "#3a3b44"
BORDER_SOFT = "#33343d"
TEXT = "#e6e6ea"
TEXT_DIM = "#9a9ba5"
TEXT_FAINT = "#6f7079"
ACCENT = "#4a9eff"
ACCENT_HOVER = "#6bb0ff"
_STYLESHEET = f"""
QMainWindow {{
background: {BG};
}}
QWidget {{
color: {TEXT};
}}
/* --- Tabs --- */
QTabWidget::pane {{
border: 1px solid {BORDER};
border-radius: 6px;
top: -1px;
}}
QTabBar::tab {{
background: transparent;
color: {TEXT_DIM};
padding: 8px 18px;
border: none;
border-bottom: 2px solid transparent;
margin-right: 2px;
}}
QTabBar::tab:hover:!selected {{
color: {TEXT};
}}
QTabBar::tab:selected {{
color: {TEXT};
border-bottom: 2px solid {ACCENT};
}}
/* --- Buttons --- */
QPushButton {{
background: {RAISED};
border: 1px solid {BORDER};
border-radius: 6px;
padding: 6px 12px;
}}
QPushButton:hover {{
background: #343640;
border-color: {ACCENT_HOVER};
}}
QPushButton:pressed {{
background: {SURFACE};
}}
QPushButton:disabled {{
color: {TEXT_FAINT};
background: {SURFACE};
border-color: {BORDER_SOFT};
}}
QPushButton:checked {{
background: {ACCENT};
border-color: {ACCENT};
color: #101116;
}}
/* --- Inputs --- */
QLineEdit, QSpinBox, QPlainTextEdit, QTextEdit {{
background: {SURFACE};
border: 1px solid {BORDER};
border-radius: 6px;
padding: 5px 8px;
selection-background-color: {ACCENT};
selection-color: #101116;
}}
QLineEdit:focus, QSpinBox:focus, QPlainTextEdit:focus, QTextEdit:focus {{
border-color: {ACCENT};
}}
QLineEdit:disabled, QSpinBox:disabled {{
color: {TEXT_FAINT};
}}
/* --- Combo boxes --- */
QComboBox {{
background: {RAISED};
border: 1px solid {BORDER};
border-radius: 6px;
padding: 4px 10px;
}}
QComboBox:hover {{
border-color: {ACCENT_HOVER};
}}
QComboBox::drop-down {{
border: none;
width: 22px;
}}
QComboBox::down-arrow {{
image: url(__CHEVRON__);
width: 12px;
height: 8px;
margin-right: 6px;
}}
QComboBox QAbstractItemView {{
background: {RAISED};
border: 1px solid {BORDER};
border-radius: 6px;
selection-background-color: rgba(74, 158, 255, 0.25);
selection-color: {TEXT};
outline: 0;
}}
/* --- Tables --- */
QTableWidget {{
background: transparent;
border: none;
gridline-color: {BORDER_SOFT};
}}
QHeaderView::section {{
background: {SURFACE};
color: {TEXT_DIM};
border: none;
border-bottom: 1px solid {BORDER};
padding: 6px 8px;
}}
QTableWidget::item {{
padding: 4px 6px;
}}
QTableWidget::item:selected {{
background: rgba(74, 158, 255, 0.22);
color: {TEXT};
}}
QTableCornerButton::section {{
background: {SURFACE};
border: none;
border-bottom: 1px solid {BORDER};
}}
/* --- Cover grid / lists --- */
QListWidget {{
background: transparent;
border: none;
}}
QListWidget::item {{
border-radius: 6px;
}}
QListWidget::item:hover:!selected {{
background: rgba(255, 255, 255, 0.04);
}}
QListWidget::item:selected {{
background: rgba(74, 158, 255, 0.22);
}}
/* --- Cards (details panels use QScrollArea) --- */
QScrollArea {{
background: {SURFACE};
border: none;
}}
/* --- Menus --- */
QMenu {{
background: {RAISED};
border: 1px solid {BORDER};
border-radius: 6px;
padding: 4px;
}}
QMenu::item {{
padding: 6px 26px 6px 12px;
border-radius: 4px;
}}
QMenu::item:selected {{
background: rgba(74, 158, 255, 0.25);
}}
QMenu::separator {{
height: 1px;
background: {BORDER};
margin: 4px 8px;
}}
/* --- Scrollbars --- */
QScrollBar:vertical {{
background: transparent;
width: 10px;
margin: 0;
}}
QScrollBar::handle:vertical {{
background: {BORDER};
border-radius: 5px;
min-height: 30px;
}}
QScrollBar::handle:vertical:hover {{
background: {ACCENT};
}}
QScrollBar:horizontal {{
background: transparent;
height: 10px;
margin: 0;
}}
QScrollBar::handle:horizontal {{
background: {BORDER};
border-radius: 5px;
min-width: 30px;
}}
QScrollBar::handle:horizontal:hover {{
background: {ACCENT};
}}
QScrollBar::add-line, QScrollBar::sub-line {{
width: 0;
height: 0;
}}
QScrollBar::add-page, QScrollBar::sub-page {{
background: transparent;
}}
/* --- Splitter --- */
QSplitter::handle {{
background: transparent;
}}
QSplitter::handle:horizontal {{
width: 1px;
}}
QSplitter::handle:vertical {{
height: 1px;
}}
/* --- Misc --- */
QToolTip {{
background: {RAISED};
color: {TEXT};
border: 1px solid {BORDER};
padding: 4px 6px;
}}
QStatusBar {{
background: {SURFACE};
color: {TEXT_DIM};
border-top: 1px solid {BORDER};
}}
QProgressBar {{
background: {SURFACE};
border: 1px solid {BORDER};
border-radius: 6px;
text-align: center;
}}
QProgressBar::chunk {{
background: {ACCENT};
border-radius: 5px;
}}
QCheckBox {{
spacing: 8px;
}}
QCheckBox::indicator {{
width: 16px;
height: 16px;
border: 1px solid {BORDER};
border-radius: 4px;
background: {SURFACE};
}}
QCheckBox::indicator:checked {{
background: {ACCENT};
border-color: {ACCENT};
}}
QGroupBox {{
border: 1px solid {BORDER};
border-radius: 6px;
margin-top: 10px;
}}
QGroupBox::title {{
subcontrol-origin: margin;
left: 10px;
padding: 0 4px;
color: {TEXT_DIM};
}}
"""
def _dark_palette():
p = QPalette()
p.setColor(QPalette.Window, QColor(BG))
p.setColor(QPalette.WindowText, QColor(TEXT))
p.setColor(QPalette.Base, QColor(SURFACE))
p.setColor(QPalette.AlternateBase, QColor(RAISED))
p.setColor(QPalette.ToolTipBase, QColor(RAISED))
p.setColor(QPalette.ToolTipText, QColor(TEXT))
p.setColor(QPalette.Text, QColor(TEXT))
p.setColor(QPalette.Button, QColor(RAISED))
p.setColor(QPalette.ButtonText, QColor(TEXT))
p.setColor(QPalette.BrightText, QColor("#ff5c5c"))
p.setColor(QPalette.Link, QColor(ACCENT))
p.setColor(QPalette.Highlight, QColor(ACCENT))
p.setColor(QPalette.HighlightedText, QColor("#101116"))
p.setColor(QPalette.PlaceholderText, QColor(TEXT_FAINT))
faint = QColor(TEXT_FAINT)
for role in (QPalette.WindowText, QPalette.Text, QPalette.ButtonText):
p.setColor(QPalette.Disabled, role, faint)
return p
def apply_theme(app):
"""Apply the dark Fusion theme to ``app`` (call once before showing UI)."""
app.setStyle("Fusion")
app.setPalette(_dark_palette())
chevron = str(config.resource_path("img", "chevron.png"))
app.setStyleSheet(_STYLESHEET.replace("__CHEVRON__", chevron))
+511
View File
@@ -0,0 +1,511 @@
"""PSP theme (``.ptf``) introspection: name + preview image.
PTF themes embed a readable name in their header and a preview image (a GIM
bitmap) in object 0. This module decodes both without any external service:
* the name lives at offset 0x08 (128 UTF-8 bytes);
* the preview is object 0's largest image item, stored as a GIM that may be
Sony-LZR compressed.
GIM decoding covers the common RGBA formats (5650/5551/4444/8888) and 8-bit
palettes; anything exotic simply yields no preview.
"""
from __future__ import annotations
import zlib
from pathlib import Path
PTF_MAGIC = b"\x00PTF"
# --------------------------------------------------------------------------- #
# Sony LZR/RLZ decompression (sceLzrDecompress), ported from PTF-Studio.
# --------------------------------------------------------------------------- #
_LZR_PROBABILITY_SLOTS = 2800
_LZR_INITIAL_PROBABILITY = 0x80
_LZR_RENORM_LIMIT = 0x00FFFFFF
_LZR_U32_MOD = 0x100000000
def _lzr_decompress(inp, max_output, offset=0):
inp = bytes(inp)
end = offset + len(inp) - offset
out = bytearray(max_output)
probs = [_LZR_INITIAL_PROBABILITY] * _LZR_PROBABILITY_SLOTS
out_pos = 0
in_pos = offset + 5
code = int.from_bytes(inp[offset + 1:offset + 5], "big")
range_ = 0xFFFFFFFF
spec = 0
corrupt = False
buf_off = 0
last_byte = 0
last_number_flag = 0
type_ = inp[offset]
if type_ >= 0x80:
type_ -= 0x100
def next_byte():
nonlocal in_pos, corrupt
if in_pos >= end:
corrupt = True
return 0
b = inp[in_pos]
in_pos += 1
return b
if type_ > 7:
raise ValueError("unsupported LZR literal-context mode")
if type_ < 0:
stored = code
if stored > len(out) or in_pos + stored > end:
raise ValueError("malformed stored LZR block")
return inp[in_pos:in_pos + stored]
def renormalize():
nonlocal code, range_
if range_ <= _LZR_RENORM_LIMIT:
code = (code * 256 + next_byte()) % _LZR_U32_MOD
range_ = (range_ * 256) % _LZR_U32_MOD
def decode_bit(prob_index):
nonlocal code, range_, corrupt
if prob_index < 0 or prob_index >= _LZR_PROBABILITY_SLOTS:
corrupt = True
return 0
renormalize()
if corrupt:
return 0
p = probs[prob_index]
bound = (range_ // 256) * p
probs[prob_index] = p - (p // 8)
if code < bound:
range_ = bound
probs[prob_index] += 31
return 1
code -= bound
range_ -= bound
return 0
def decode_bit_speculative(prob_index):
nonlocal code, range_, spec, corrupt
if prob_index < 0 or prob_index >= _LZR_PROBABILITY_SLOTS:
corrupt = True
return 0
if spec <= _LZR_RENORM_LIMIT:
code = (code * 256 + next_byte()) % _LZR_U32_MOD
range_ = (spec * 256) % _LZR_U32_MOD
if corrupt:
return 0
p = probs[prob_index]
bound = (range_ // 256) * p
spec = bound
probs[prob_index] = p - (p // 8)
if code < bound:
range_ = bound
probs[prob_index] += 31
return 1
code -= bound
range_ -= bound
return 0
def decode_number(bits, base, step):
nonlocal code, range_, corrupt, last_number_flag
number = 1
if bits >= 3:
number = number * 2 + decode_bit(base + 3 * step)
if bits >= 4:
number = number * 2 + decode_bit(base + 3 * step)
if bits >= 5:
renormalize()
remaining = bits
while remaining >= 5:
if range_ == 0:
corrupt = True
return 0
number *= 2
range_ = range_ // 2
if code < range_:
number += 1
else:
code -= range_
remaining -= 1
last_number_flag = decode_bit(base)
number = number * 2 + last_number_flag
if bits >= 1:
number = number * 2 + decode_bit(base + step)
if bits >= 2:
number = number * 2 + decode_bit(base + 2 * step)
return number
def selector_context():
return buf_off + 2488
def decode_literal(shift):
nonlocal out_pos, last_byte, corrupt, buf_off
if buf_off > 0:
buf_off -= 1
if out_pos == len(out):
return False
context = ((((out_pos & 7) << 8) + last_byte) // (2 ** shift)) & 7
tree_base = context * 0xFF - 1
node = 1
while node <= 0xFF:
node = node * 2 + decode_bit(tree_base + node)
if corrupt:
return False
out[out_pos] = node & 0xFF
last_byte = node & 0xFF
out_pos += 1
return True
def decode_match():
nonlocal out_pos, last_byte, buf_off, spec, code, range_, corrupt, last_number_flag
slot = selector_context()
spec = range_
length_class = -1
bit = 0
while True:
slot += 8
bit = decode_bit_speculative(slot)
if corrupt:
return -1
length_class += bit
if bit == 0 or length_class >= 6:
break
offset_group = length_class + 2033
offset_bias = 64
if bit != 0 or length_class >= 0:
length_context = (length_class << 5) + (((out_pos * (2 ** length_class)) & 3) << 3) + buf_off + 2552
match_length = decode_number(length_class, length_context, 8)
if corrupt:
return -1
if match_length == 0xFF:
return 1
if last_number_flag != 0 or length_class > 0:
offset_group += 56
offset_bias = 352
else:
match_length = 1
index = 1
offset_class = 0
while True:
offset_class = (index << 4) - offset_bias
prob_index = offset_group + (index << 3)
if prob_index >= _LZR_PROBABILITY_SLOTS:
return -1
bit = decode_bit(prob_index)
if corrupt:
return -1
index = (index << 1) | bit
if offset_class >= 0:
break
if bit != 0 or offset_class > 0:
if bit == 0:
offset_class -= 8
base = offset_class + 2344
if base < 0 or base + 3 >= _LZR_PROBABILITY_SLOTS:
return -1
back_offset = decode_number(offset_class // 8, base, 1)
if corrupt:
return -1
else:
back_offset = 1
if back_offset > out_pos:
return -1
copy_end = out_pos + match_length + 1
if copy_end > len(out):
return -1
buf_off = ((copy_end + 1) & 1) + 6
from_ = out_pos - back_offset
while out_pos < copy_end:
out[out_pos] = out[from_]
out_pos += 1
from_ += 1
last_byte = out[out_pos - 1]
return 0
token_budget = max(1024, max_output * 2 + 1024)
while True:
token_budget -= 1
if token_budget < 0:
raise ValueError("malformed LZR stream did not terminate")
token_is_match = decode_bit(selector_context())
if corrupt:
raise ValueError("truncated LZR stream")
if token_is_match == 0:
if not decode_literal(type_):
raise ValueError("LZR stream exceeds declared output size")
else:
result = decode_match()
if result == 1:
return bytes(out[:out_pos])
if result < 0:
raise ValueError("malformed LZR back-reference")
# --------------------------------------------------------------------------- #
# PTF parsing
# --------------------------------------------------------------------------- #
def _decompress_item(blob, comp, ucsize):
if comp == 2 and len(blob) < ucsize:
return zlib.decompress(blob)
if comp == 1 and len(blob) < ucsize:
return _lzr_decompress(blob, ucsize)
return blob
def parse_ptf(data):
"""Return ``(name, items)`` where items are decompressed item blobs."""
if not isinstance(data, (bytes, bytearray)):
data = data.read_bytes() if hasattr(data, "read_bytes") else bytes(data)
data = bytes(data)
if data[:4] != PTF_MAGIC:
raise ValueError("not a PTF file")
raw_name = data[8:8 + 128]
name = raw_name.split(b"\x00", 1)[0].decode("utf-8", "replace").strip()
def u16(o):
return int.from_bytes(data[o:o + 2], "little")
def u32(o):
return int.from_bytes(data[o:o + 4], "little")
items = []
pos = 0x100
while True:
offset = u32(pos)
pos += 4
if offset == 0 or offset >= len(data):
break
count = u16(offset + 2)
data_off = u32(offset + 8)
p = data_off
for _ in range(count):
file_type = u16(p + 4)
comp = u16(p + 6)
size = u32(p + 8)
ucsize = u32(p + 12)
blob = data[p + 0x20:p + 0x20 + size]
items.append((file_type, _decompress_item(blob, comp, ucsize)))
p += 0x20 + size
return name, items
def ptf_name(path):
"""Theme display name embedded in a ``.ptf`` file."""
try:
with open(path, "rb") as fh:
head = fh.read(8 + 128)
except OSError:
return ""
if head[:4] != PTF_MAGIC:
return ""
name = head[8:8 + 128].split(b"\x00", 1)[0].decode("utf-8", "replace").strip()
return name or Path(path).stem
# --------------------------------------------------------------------------- #
# GIM decoding (RGBA formats only)
# --------------------------------------------------------------------------- #
def _overscan(value, tile):
return value if value % tile == 0 else value + (tile - (value % tile))
def _untile(pixels, w, h, tile_w, tile_h):
over_w = _overscan(w, tile_w)
over_h = _overscan(h, tile_h)
out = [0] * (w * h)
tile_pos = 0
origin_x = 0
origin_y = 0
for y in range(over_h):
for x in range(over_w):
dx = origin_x + (tile_pos % tile_w)
dy = origin_y + (tile_pos // tile_w)
if dx < w and dy < h:
out[dy * w + dx] = pixels[y * over_w + x]
tile_pos += 1
if tile_pos == tile_w * tile_h:
tile_pos = 0
origin_x += tile_w
if origin_x >= over_w:
origin_x = 0
origin_y += tile_h
return out
def _gim_to_rgba(data):
"""Decode a GIM to ``(width, height, rgba_bytes)`` or None if unsupported."""
data = bytes(data)
if len(data) < 0x10:
return None
# Detect byte order from the magic. Sony GIMs store the magic as the byte
# string "MIG.00.1PSP" with little-endian data (LibPSPThemes semantics).
if data[:4] == b"MIG.":
byteorder = "little"
elif data[:4] == b".GIM":
byteorder = "big"
else:
return None
def rd(off, size):
return int.from_bytes(data[off:off + size], byteorder)
image = None
palette = None
pos = 0x10
for _ in range(32):
if pos + 16 > len(data):
break
btype = rd(pos, 2)
next_block = rd(pos + 8, 4)
if btype == 4 or btype == 5:
h = pos + 0x10
fmt = rd(h + 4, 2)
pixel_order = rd(h + 6, 2)
width = rd(h + 8, 2)
height = rd(h + 10, 2)
bpp = rd(h + 12, 2)
next_index = rd(h + 24, 4)
frame_data_end = rd(h + 32, 4)
frame_offset = rd(pos + 0x10 + next_index, 4)
block_content = pos + 0x10
start = block_content + frame_offset
end = block_content + frame_data_end
raw = data[start:end]
pixels = _decode_pixels(raw, fmt, bpp, width, height, pixel_order)
if pixels is None:
return None
if btype == 4:
image = (width, height, pixels)
else:
palette = (fmt, width, height, pixels)
if next_block == 0:
break
pos += next_block
if image is None:
return None
width, height, pixels = image
if palette is not None:
_, pw, ph, pvals = palette
if pw * ph > 0:
pixels = [pvals[p] if p < len(pvals) else 0 for p in pixels]
rgba = bytearray(width * height * 4)
for i, v in enumerate(pixels):
# pixels are packed 0xAABBGGRR
o = i * 4
rgba[o] = v & 0xFF
rgba[o + 1] = (v >> 8) & 0xFF
rgba[o + 2] = (v >> 16) & 0xFF
rgba[o + 3] = (v >> 24) & 0xFF
return width, height, bytes(rgba)
def _decode_pixels(raw, fmt, bpp, width, height, pixel_order):
"""Decode a GIM frame into a list of packed 0xAABBGGRR ints."""
if fmt in (0, 1, 2, 3, 5):
pass
else:
return None # indexed-16 / DXT etc. not supported
if fmt == 5: # p8 indexed
bytes_per_pixel = 1
tile_w = 0x80 // max(bpp, 8)
else:
bytes_per_pixel = max(1, (bpp + 7) // 8)
tile_w = 0x80 // max(bpp, 1)
tile_h = 8
if pixel_order == 1:
w = _overscan(width, tile_w)
h = _overscan(height, tile_h)
else:
w, h = width, height
count = w * h
values = []
pos = 0
for _ in range(count):
v = int.from_bytes(raw[pos:pos + bytes_per_pixel], "little")
pos += bytes_per_pixel
values.append(_expand(fmt, bpp, v))
if pos >= len(raw):
# pad missing trailing pixels with transparent
values.extend([0] * (count - len(values)))
break
if pixel_order == 1:
values = _untile(values, width, height, tile_w, tile_h)
return values[:width * height]
def _expand(fmt, bpp, v):
"""Expand a raw GIM pixel value into packed 0xAABBGGRR."""
if fmt == 3: # rgba8888
r = v & 0xFF
g = (v >> 8) & 0xFF
b = (v >> 16) & 0xFF
a = (v >> 24) & 0xFF
return (a << 24) | (b << 16) | (g << 8) | r
if fmt == 0: # rgba5650
r = (v >> 11) & 0x1F
g = (v >> 5) & 0x3F
b = v & 0x1F
r = (r << 3) | (r >> 2)
g = (g << 2) | (g >> 4)
b = (b << 3) | (b >> 2)
return (0xFF << 24) | (b << 16) | (g << 8) | r
if fmt == 1: # rgba5551
r = (v >> 11) & 0x1F
g = (v >> 6) & 0x1F
b = (v >> 1) & 0x1F
a = 0xFF if v & 1 else 0
r = (r << 3) | (r >> 2)
g = (g << 3) | (g >> 2)
b = (b << 3) | (b >> 2)
return (a << 24) | (b << 16) | (g << 8) | r
if fmt == 2: # rgba4444
r = (v >> 12) & 0xF
g = (v >> 8) & 0xF
b = (v >> 4) & 0xF
a = v & 0xF
r = (r << 4) | r
g = (g << 4) | g
b = (b << 4) | b
a = (a << 4) | a
return (a << 24) | (b << 16) | (g << 8) | r
if fmt == 5: # p8 — value is a palette index (expanded later)
return v & 0xFF
return 0
# --------------------------------------------------------------------------- #
# Public API
# --------------------------------------------------------------------------- #
def theme_preview_rgba(ptf_path):
"""Decode the preview image of a theme PTF.
Returns ``(width, height, rgba_bytes)`` or None if it can't be decoded.
"""
try:
data = Path(ptf_path).read_bytes()
_, items = parse_ptf(data)
except (OSError, ValueError):
return None
# The preview image is the largest decodable GIM item.
best = None
for file_type, blob in items:
if file_type != 5:
continue
decoded = _gim_to_rgba(blob)
if decoded is None:
continue
w, h, rgba = decoded
if best is None or w * h > best[0] * best[1]:
best = (w, h, rgba)
return best
+23 -11
View File
@@ -468,31 +468,30 @@ class LocalShareSource(Source):
class NoPayStationSource(Source):
"""Browse NoPayStation .pkg lists and install PSP/PSX PSN content.
``content_type`` is ``"psp"`` (PSP_GAMES.tsv) or ``"psx"`` (PSX_GAMES.tsv).
Downloading decrypts the ``.pkg`` with the bundled pure-Python decryptor
and writes a ``<TITLE_ID>/EBOOT.PBP`` game folder into ``dest_dir``.
``content_type`` is ``"psp"`` (PSP_GAMES.tsv), ``"psx"`` (PSX_GAMES.tsv),
``"theme"`` (PSP_THEMES.tsv) or ``"dlc"`` (PSP_DLCS.tsv). Downloading
decrypts the ``.pkg`` with the bundled pure-Python decryptor and writes a
``<TITLE_ID>/`` game folder into ``dest_dir``.
"""
label = "NoPayStation"
def __init__(self, content_type, tsv_url="", pkg2zip_path=""):
def __init__(self, content_type, tsv_url="", force=False):
from . import nopaystation
self.content_type = content_type
self.tsv_url = tsv_url or nopaystation.tsv_url(content_type)
self.pkg2zip = pkg2zip_path or ""
self.force = force
def scan(self, root=None):
from . import nopaystation
return nopaystation.fetch_items(self.content_type, self.tsv_url)
return nopaystation.fetch_items(self.content_type, self.tsv_url, force=self.force)
def download(self, item, dest_dir, progress_cb=None, cancel_event=None):
from . import nopaystation
dest_dir = Path(dest_dir)
dest_dir.mkdir(parents=True, exist_ok=True)
# The bundled Python decryptor handles PSP/PSX directly; pkg2zip is
# only an optional fallback if a binary happens to be configured.
pkg2zip = nopaystation.find_pkg2zip(self.pkg2zip)
# The bundled Python decryptor handles PSP/PSX/themes/DLC directly.
title_id = item.get("title_id") or item.get("content_id") or ""
pkg_name = re.sub(r"[^A-Za-z0-9_-]", "", title_id) or "pkg"
tmp = Path(tempfile.mkdtemp(prefix="nps-download-"))
@@ -501,8 +500,21 @@ class NoPayStationSource(Source):
if progress_cb is not None:
progress_cb("Downloading", 0.0, "")
download_url(item["pkg_url"], pkg, None, progress_cb, cancel_event)
folder = nopaystation.extract_pkg(pkg, title_id, dest_dir, pkg2zip,
progress_cb, cancel_event)
kind_hint = self.content_type if self.content_type in ("theme", "dlc") else None
folder = nopaystation.extract_pkg(pkg, title_id, dest_dir,
kind_hint, progress_cb, cancel_event)
if self.content_type in ("theme", "dlc"):
_write_sidecar(folder, item)
return folder
finally:
shutil.rmtree(tmp, ignore_errors=True)
def _write_sidecar(folder, item):
"""Save the store display name beside a downloaded theme/DLC (for the library)."""
try:
meta = {"name": item.get("name") or "", "title_id": item.get("title_id") or "",
"content_id": item.get("content_id") or ""}
(Path(folder) / "meta.json").write_text(json.dumps(meta, ensure_ascii=False), encoding="utf-8")
except OSError:
pass
-38
View File
@@ -1,38 +0,0 @@
# pkg2zip
The `pkg2zip.exe` binary in this folder is kept as an optional fallback for
decrypting NoPayStation `.pkg` files (PSP / PSX / PS3 / PSVita content). The
app normally uses its own pure-Python decryptor (`app/pkg.py`) and only runs
this binary if that fails.
- Project: <https://github.com/mmozeiko/pkg2zip>
- Author: Marcin Mikołajczak (mmozeiko)
- Version: v1.8 (bundled)
- License: MIT
The Windows binary is run through Wine on Linux/macOS (see
`app/nopaystation.py`).
## License
MIT License
Copyright (c) 2017 Marcin Mikołajczak (mmozeiko)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Binary file not shown.
+6 -6
View File
@@ -6,7 +6,7 @@ services:
# Plain HTTP for now; see Caddyfile to add Let's Encrypt TLS later.
networks:
br0:
ipv4_address: 192.168.10.100
ipv4_address: ${PSP_LAN_IP}
environment:
PSP_DATA_DIR: /data
PSP_DATS_DIR: /data/dats
@@ -16,14 +16,14 @@ services:
PSP_VIDEO_ROOTS: /data/umd_videos
PSP_STORE_DIR: /data/store
# Secret token the desktop app sends as `Authorization: Bearer <token>`.
# Set via a .env file next to this compose file.
# Set in the .env file next to this compose file.
PSP_API_TOKEN: ${PSP_API_TOKEN:-}
volumes:
- ./data:/data
# Adjust host paths to your Unraid share layout:
- /mnt/user/downloads/Sony - PlayStation Portable:/data/store:ro
- /mnt/user/psp/umd_games:/data/umd_games:ro
- /mnt/user/psp/umd_videos:/data/umd_videos:ro
# Host paths come from .env (see .env.example):
- ${PSP_STORE_HOST}:/data/store:ro
- ${PSP_UMD_GAMES_HOST}:/data/umd_games:ro
- ${PSP_UMD_VIDEOS_HOST}:/data/umd_videos:ro
restart: unless-stopped
networks:
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 273 B

BIN
View File
Binary file not shown.

Before

Width:  |  Height:  |  Size: 689 KiB

After

Width:  |  Height:  |  Size: 683 KiB

+65
View File
@@ -0,0 +1,65 @@
# -*- mode: python ; coding: utf-8 -*-
"""PyInstaller spec: build the PSP Preservation Tool (onedir).
Bundles the read-only resources (DATs, PS1 loader, offline cover bundles,
icons) under ``_internal/`` next to the executable. User-writable data
(credentials, manual covers, PS1 templates, NPS cache, library DB, cover
cache) lives in the OS state dir at runtime — see ``app/config.py``.
Usage: pyinstaller --clean --noconfirm psp-preserve.spec
Output: dist/psp-preserve/
"""
from PyInstaller.utils.hooks import collect_dynamic_libs, collect_submodules
datas = [
("data/dats", "data/dats"),
("data/loader", "data/loader"),
("data/supplement.json", "data"),
("data/covers/libretro-covers.zip", "data/covers"),
("data/covers/openvgdb-covers.zip", "data/covers"),
("data/covers/libretro-index.json", "data/covers"),
("data/covers/openvgdb-index.json", "data/covers"),
("data/covers/UMD_COVER_TEMPLATE.png", "data/covers"),
("data/covers/UMD_COVER_TEMPLATE_FEATURE.png", "data/covers"),
("data/covers/UMD_GAME_COVER_TEMPLATE.png", "data/covers"),
("img/app-icon.png", "img"),
("img/chevron.png", "img"),
]
a = Analysis(
["run.py"],
pathex=[],
binaries=collect_dynamic_libs("curl_cffi"),
datas=datas,
hiddenimports=collect_submodules("curl_cffi"),
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=["tkinter", "unittest", "pydoc"],
noarchive=False,
)
pyz = PYZ(a.pure)
exe = EXE(
pyz,
a.scripts,
[],
exclude_binaries=True,
name="psp-preserve",
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=False,
console=False,
disable_windowed_traceback=False,
)
coll = COLLECT(
exe,
a.binaries,
a.datas,
strip=False,
upx=False,
name="psp-preserve",
)
+7
View File
@@ -0,0 +1,7 @@
# Build dependencies (pinned for reproducible CI builds).
# PySide6 is pinned to 6.8.3: newer 6.11.x bundles a Qt6Core.dll that imports
# icuuc.dll on Windows, which is not shipped and crashes at startup.
pyinstaller>=6.11
PySide6==6.8.3
curl_cffi>=0.7
pycryptodome>=3.20
+10
View File
@@ -0,0 +1,10 @@
"""PyInstaller entry point.
Runs the GUI the same way ``python -m app.main`` does, but as a plain script
so PyInstaller can build it (relative imports inside ``app`` won't resolve when
a module file is executed directly, hence this top-level launcher).
"""
from app.main import main
if __name__ == "__main__":
raise SystemExit(main())
+52
View File
@@ -0,0 +1,52 @@
# Wine-based Gitea Actions runner: builds Windows binaries on a Linux host.
#
# Jobs run in "host" mode inside this container, where a Windows CPython 3.12
# (running under Wine) plus PyInstaller produce the .exe. Node.js and git are
# included so the stock JS actions (actions/checkout, actions/upload-artifact)
# work as-is.
FROM debian:bookworm-slim
ENV DEBIAN_FRONTEND=noninteractive \
WINEPREFIX=/root/.wine \
WINEDEBUG=-all \
WINEDLLOVERRIDES="mscoree,mshtml=" \
PYTHON_VERSION=3.12.8
# Wine (64-bit), node + git for JS actions, and build/package tooling.
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates curl wget unzip zip \
wine wine64 xvfb winetricks cabextract \
nodejs git \
&& apt-get clean && rm -rf /var/lib/apt/lists/*
# Gitea Actions runner (linux-amd64 binary).
RUN wget -qO /usr/local/bin/act_runner \
"https://gitea.com/gitea/act_runner/releases/download/v0.6.1/act_runner-0.6.1-linux-amd64" \
&& chmod +x /usr/local/bin/act_runner
# Windows CPython 3.12, silent install into C:\Python312.
RUN wget -qO /tmp/python-installer.exe \
"https://www.python.org/ftp/python/${PYTHON_VERSION}/python-${PYTHON_VERSION}-amd64.exe" \
&& xvfb-run -a wine /tmp/python-installer.exe \
/quiet InstallAllUsers=1 PrependPath=1 Include_test=0 Include_launcher=0 Shortcuts=0 \
&& rm -f /tmp/python-installer.exe \
&& wine "C:\\Python312\\python.exe" --version
# MSVC runtime required by PySide6's Qt DLLs.
RUN xvfb-run -a winetricks --unattended vcrun2022
# Convenience wrapper: winpython -> wine C:\Python312\python.exe
RUN printf '#!/bin/sh\nexec wine "C:\\Python312\\python.exe" "$@"\n' > /usr/local/bin/winpython \
&& chmod +x /usr/local/bin/winpython
# Pre-install the Windows build deps so CI jobs start fast.
COPY requirements-build.txt /tmp/requirements-build.txt
RUN winpython -m pip install --upgrade pip \
&& winpython -m pip install -r /tmp/requirements-build.txt \
&& rm -f /tmp/requirements-build.txt
COPY runner/entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
WORKDIR /data
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
+14
View File
@@ -0,0 +1,14 @@
#!/bin/sh
set -e
# Register on first run, then run the daemon (host mode).
if [ ! -f /data/.runner ]; then
echo "Registering Wine runner with Gitea…"
act_runner register --no-interactive \
--instance "${GITEA_INSTANCE_URL:?GITEA_INSTANCE_URL is required}" \
--token "${GITEA_RUNNER_REGISTRATION_TOKEN:?GITEA_RUNNER_REGISTRATION_TOKEN is required}" \
--name "${RUNNER_NAME:-windows-builder}" \
--labels "${RUNNER_LABELS:-windows:host}"
fi
exec act_runner daemon