299 lines
12 KiB
Markdown
299 lines
12 KiB
Markdown
# PSP Preservation Tool
|
|
|
|
|
|
|
|
Native Linux GUI (PySide6) for preserving PSP UMD **games** and **videos** as
|
|
`.iso` files. It scans your discs, verifies them against the full Redump catalog
|
|
(Redump DATs), reports what's missing, and retags/renames discs to a canonical
|
|
scheme. It also fetches cover art and metadata, syncs discs and save-data with a
|
|
real PSP, and can pull missing discs from your own Redump 7z archive collection.
|
|
|
|
|
|
* Use default app theme for Qt.
|
|
|
|
## Features
|
|
|
|
- **Library** — scan folders of `.iso` discs, verify against the DAT catalog,
|
|
browse with cover art, search/filter, see per-game details (developer,
|
|
publisher, genre, release date, synopsis, trailer), and launch discs in PPSSPP.
|
|
- **Store** — browse and download discs from three sources: a local 7z folder,
|
|
your own HTTP store server, or an archive.org item (see
|
|
[Store](#store--archiveorg-and-local-folder)).
|
|
- **Queue** — background transfers with progress, cancel, per-row delete, etc.
|
|
- **PSP Sync** — detect a connected PSP (USB) or a mounted UMD, inventory what's
|
|
installed (games / UMD videos / homebrew / PKG), copy discs to the PSP, delete
|
|
from the PSP, and dump a UMD to the library.
|
|
- **Saves** — two-way sync of save-data between the PSP, PPSSPP, and a cloud
|
|
folder.
|
|
- **Settings** — directories, cover-source API keys, store/server config, and
|
|
misc options.
|
|
|
|
## Requirements
|
|
|
|
- Linux + Python 3
|
|
- `PySide6` — the only Python dependency (`pip install -r requirements.txt`)
|
|
- `7z` (p7zip) — used to peek/extract disc images from store archives
|
|
- PPSSPP (optional) — to launch discs from the details panel
|
|
|
|
## Install
|
|
|
|
```bash
|
|
python3 -m venv .venv # keep the venv on a local disk, not an SMB mount
|
|
. .venv/bin/activate
|
|
pip install -r requirements.txt
|
|
```
|
|
|
|
## Run the GUI
|
|
|
|
```bash
|
|
python3 -m app.main
|
|
```
|
|
|
|
## First-time setup
|
|
|
|
1. Put your DAT files in `data/dats/`:
|
|
- games: Redump `Sony - PlayStation Portable` (authoritative hashes)
|
|
**and** No-Intro `Sony - PlayStation Portable` (supplies the serials
|
|
Redump lacks — keeps serial-based matching working).
|
|
- videos: No-Intro `Sony - PlayStation Portable (UMD Video)` (Redump has
|
|
no UMD Video DAT).
|
|
2. Ingest them into the local SQLite library:
|
|
|
|
```bash
|
|
python3 -m app.ingest
|
|
```
|
|
|
|
3. Open the GUI and point **Settings → General → Games/Videos directory** at
|
|
your disc folders (or set `PSP_GAMES_ROOTS` / `PSP_VIDEO_ROOTS`).
|
|
|
|
Discs that aren't in the Redump DATs can be added to `data/supplement.json`
|
|
(`name`, `region`, `crc32`, `sha1`, `md5`) so the tool still recognizes them.
|
|
|
|
## Nomenclature
|
|
|
|
`Title (Region) (tags) (Serial).iso` — leading articles stay in front
|
|
(`The Sims 2`), and revision/disc tags are preserved (`(v1.03)`, `(Disc 1)`).
|
|
Discs are matched by hash first (most precise), then serial, then filename.
|
|
|
|
## Directory layout
|
|
|
|
- `data/dats/` — clrmamepro DAT files (games + videos).
|
|
- `data/supplement.json` — extra discs not in Redump.
|
|
- `data/credentials.json` — cover/metadata API keys (IGDB, TheMovieDB,
|
|
ScreenScraper). Environment variables take precedence.
|
|
- `data/manual_covers/games|videos/` — your own cover images.
|
|
- `data/covers/` — checked-in assets: the UMD video/game cover templates,
|
|
`libretro-index.json` catalog, `libretro-covers.zip` + `openvgdb-covers.zip`
|
|
offline cover bundles, and the legacy cover cache.
|
|
- State dir (`~/.local/state/psp-preserve/` by default, or `$XDG_STATE_HOME/psp-preserve`):
|
|
- `library.sqlite` — the catalog + library database.
|
|
- `covers/` — downloaded cover cache.
|
|
|
|
SQLite is kept on a local disk because SQLite locking is unreliable over
|
|
network filesystems; the DATs, manual covers and credentials stay in `data/`
|
|
(which may be a network share).
|
|
|
|
## Configuration
|
|
|
|
Everything is driven by environment variables, with GUI settings (stored in
|
|
QSettings under `PSPPreserve`) taking precedence where noted.
|
|
|
|
| Variable | Default | Purpose |
|
|
|---|---|---|
|
|
| `PSP_DATA_DIR` | `./data` | data root (DATs, credentials, manual covers) |
|
|
| `PSP_DATS_DIR` | `$PSP_DATA_DIR/dats` | DAT folder |
|
|
| `PSP_STATE_DIR` | `~/.local/state/psp-preserve` | local state root |
|
|
| `PSP_DB_PATH` | `$PSP_STATE_DIR/library.sqlite` | SQLite library |
|
|
| `PSP_COVERS_DIR` | `$PSP_STATE_DIR/covers` | cover cache |
|
|
| `PSP_GAMES_ROOTS` | `/data/umd_games` | comma-separated game disc roots |
|
|
| `PSP_VIDEO_ROOTS` | `/data/umd_videos` | comma-separated video disc roots |
|
|
| `PSP_SCAN_WORKERS` | `1` | parallel hashing workers |
|
|
| `PSP_STORE_DIR` | `/media/downloads/Sony - PlayStation Portable` | local 7z store folder |
|
|
| `PSP_STORE_SERVER` | `https://psp.example.com` | store server base URL |
|
|
| `PSP_STORE_TOKEN` | *(empty)* | store server bearer token |
|
|
| `PSP_ARCHIVE_ITEM` | `psp-redump-roms321.com` | archive.org item identifier |
|
|
| `PSP_ARCHIVE_LOGGED_IN_SIG` | *(empty)* | archive.org `logged-in-sig` cookie |
|
|
| `PSP_ARCHIVE_LOGGED_IN_USER` | *(empty)* | archive.org `logged-in-user` cookie |
|
|
| `PSP_API_TOKEN` | *(empty)* | server API bearer token (unset = open) |
|
|
| `PSP_CREDENTIALS` | `$PSP_DATA_DIR/credentials.json` | credentials file |
|
|
| `PSP_LIBRETRO_COVERS_URL` | *(empty)* | direct URL for `libretro-covers.zip` (else derived from store server) |
|
|
| `PSP_OVGDB_COVERS_URL` | *(empty)* | direct URL for `openvgdb-covers.zip` (else derived from store server) |
|
|
|
|
Cover/metadata keys can also be set as env vars: `IGDB_CLIENT_ID`,
|
|
`IGDB_CLIENT_SECRET`, `TMDB_API_KEY`, `SCREENSCRAPER_DEVID`,
|
|
`SCREENSCRAPER_DEVPASSWORD`, `SCREENSCRAPER_SSID`, `SCREENSCRAPER_SSPASSWORD`,
|
|
`SCREENSCRAPER_SOFTNAME`.
|
|
|
|
## Cover art
|
|
|
|
Games are resolved in this order (offline-first, then network):
|
|
|
|
1. **Manual covers** — images you drop in `data/manual_covers/games/`
|
|
(fuzzy-matched; strips `(Region)`, handles `Title, The`).
|
|
2. **libretro-thumbnails (offline bundle)** — full PSP box art from
|
|
`data/covers/libretro-covers.zip`. If that zip is absent, it downloads from
|
|
GitHub and caches instead.
|
|
3. **OpenVGDB (offline bundle)** — full covers from
|
|
`data/covers/openvgdb-covers.zip`.
|
|
4. **ScreenScraper** — full box art + rich metadata; needs a free dev
|
|
ID/password.
|
|
5. **Wikipedia** — keyless fallback.
|
|
6. **IGDB** — needs a Twitch/IGDB client ID + secret. IGDB only stores cropped
|
|
3:4 key art, so the app composites it into the PSP game cover template
|
|
(`data/covers/UMD_GAME_COVER_TEMPLATE.png`) to restore the full cover.
|
|
|
|
UMD **videos** have no public box-art API: they use your manual covers, or a
|
|
TheMovieDB poster composited into `data/covers/UMD_COVER_TEMPLATE.png`.
|
|
|
|
### Offline bundles & auto-download
|
|
|
|
The two bundles (`libretro-covers.zip`, `openvgdb-covers.zip`) are shipped in
|
|
`data/covers/`, so a fresh install works with **no internet and no API keys**.
|
|
If either is missing at startup, the app downloads it from the store server's
|
|
`/api/covers/<name>.zip` endpoint (see [Server setup](#server-setup)).
|
|
|
|
Rebuild them yourself:
|
|
|
|
```bash
|
|
python3 -m app.libretro # libretro-thumbnails -> libretro-covers.zip
|
|
python3 -m app.openvgdb # OpenVGDB/GameFAQs -> openvgdb-covers.zip
|
|
```
|
|
|
|
(`app.openvgdb` needs `curl_cffi` to pass GameFAQs' Cloudflare challenge.)
|
|
|
|
## Store — archive.org and local folder
|
|
|
|
The **Store** tab downloads missing discs from a Redump 7z archive collection
|
|
(`Name (Region) (tags).7z`, containing the `.iso`/`.cso` inside). It has three
|
|
sources, selectable in the tab:
|
|
|
|
### 1. Local folder
|
|
|
|
Point **Settings → Store → Local store dir** (or `PSP_STORE_DIR`) at a folder of
|
|
`.7z`/`.zip` archives. The app scans the folder, lists the archives, and on
|
|
download:
|
|
|
|
1. reads the archive listing with `7z` to find the disc image inside,
|
|
2. copies the archive to a temp dir and extracts the image with `7z`,
|
|
3. moves the image into your games/videos directory.
|
|
|
|
### 2. Server (HTTP API)
|
|
|
|
See [Server setup](#server-setup). The app talks to `GET /api/store` (list) and
|
|
`GET /api/store/download/<name>` (stream), using your `PSP_API_TOKEN` as
|
|
`Authorization: Bearer <token>`.
|
|
|
|
### 3. Archive.org
|
|
|
|
Point **Settings → Store → Archive.org item** (or `PSP_ARCHIVE_ITEM`) at an
|
|
archive.org item identifier (or a full `details`/`download` URL). The app:
|
|
|
|
1. lists the item via `https://archive.org/metadata/<item>`,
|
|
2. downloads archives from `https://archive.org/download/<item>/<file>`,
|
|
3. extracts the disc image and moves it into your library.
|
|
|
|
If the item's files are marked **private** (login required), set the
|
|
`logged-in-sig` and `logged-in-user` cookies in **Settings → Store** (grab them
|
|
from a logged-in archive.org browser session). Without them, private files are
|
|
skipped — and you'll get a clear error if *everything* in the item is private.
|
|
|
|
## Server setup
|
|
|
|
The catalog + store API is a single-file, stdlib-only HTTP server
|
|
(`app/api.py`) — no pip install needed on the server.
|
|
|
|
```bash
|
|
PSP_DATA_DIR=/data \
|
|
PSP_DATS_DIR=/data/dats \
|
|
PSP_DB_PATH=/data/library.sqlite \
|
|
PSP_COVERS_DIR=/data/covers \
|
|
PSP_GAMES_ROOTS=/data/umd_games \
|
|
PSP_VIDEO_ROOTS=/data/umd_videos \
|
|
PSP_STORE_DIR=/data/store \
|
|
PSP_API_TOKEN=<long-random-secret> \
|
|
python3 -m app.api --host 0.0.0.0 --port 8000
|
|
```
|
|
|
|
Endpoints:
|
|
|
|
| Endpoint | Description |
|
|
|---|---|
|
|
| `GET /api/summary` | release counts per system |
|
|
| `GET /api/releases?system=&q=&limit=&offset=` | search the catalog |
|
|
| `GET /api/releases/<id>` | one release + its rom hashes |
|
|
| `GET /api/library?system=&refresh=` | your scanned library |
|
|
| `GET /api/cover/<id>` | cover image (fetched on demand) |
|
|
| `GET /api/covers/<name>.zip` | offline cover bundle (public) |
|
|
| `GET /api/store` | list store archives *(token)* |
|
|
| `GET /api/store/download/<name>` | stream one archive *(token)* |
|
|
|
|
Token protection: set `PSP_API_TOKEN`; the client sends
|
|
`Authorization: Bearer <token>`. Leave it unset to allow anonymous access
|
|
(not recommended).
|
|
|
|
### Docker (Unraid)
|
|
|
|
`docker-compose.yml` builds the API image and attaches it to Unraid's `br0`
|
|
macvlan bridge so it gets its own LAN IP (e.g. `192.168.1.100`, plain HTTP on 8000):
|
|
|
|
```bash
|
|
# in /path/to/psp-preserve/ — adjust the volume paths for your shares
|
|
printf 'PSP_API_TOKEN=<long-random-secret>\n' > .env
|
|
docker compose up -d --build
|
|
```
|
|
|
|
The compose file mounts your Unraid shares into `/data/store`, `/data/umd_games`
|
|
and `/data/umd_videos` (the store is mounted read-only).
|
|
|
|
### TLS (optional)
|
|
|
|
`Caddyfile` is a ready-made reverse proxy for `https://psp.example.com` with
|
|
automatic Let's Encrypt certificates. To enable it, add the `caddy` service to
|
|
the compose file, give it its own `br0` IP, point it at the API, and forward
|
|
ports 80/443 on your router to it.
|
|
|
|
### Client
|
|
|
|
In the GUI: **Settings → Store** → enter the **Server URL** (e.g.
|
|
`http://<server-ip>:8000` or `https://psp.example.com`) and the same **API
|
|
token**, then pick **Server** in the Store tab's source selector. Use
|
|
**Archive.org** or **Local folder** the same way.
|
|
|
|
## PSP Sync
|
|
|
|
Connect a PSP over USB (or mount a UMD) and the **PSP Sync** tab detects it:
|
|
|
|
- **Games / UMD Videos / Homebrew-PSN / PKG** — tables of what's installed, with
|
|
delete-from-PSP (checkboxes and right-click).
|
|
- **UMD** — reads `PARAM.SFO` from a mounted UMD, shows the title/serial, and
|
|
can dump the disc image into your library.
|
|
- **Sync from Library** — copy discs from your library onto the PSP, optionally
|
|
into a Category Lite category (set in **Settings → PSP → ISO category**).
|
|
|
|
## Saves sync
|
|
|
|
The **Saves** tab shows save-data across three locations — **PSP** (USB),
|
|
**PPSSPP**, and a **Cloud folder** — and syncs both ways. Save signatures are
|
|
cached (keyed by size + mtime) so rescanning is fast, and an **Auto-sync**
|
|
option runs the sync automatically whenever a PSP is connected.
|
|
|
|
## CLI tools
|
|
|
|
```bash
|
|
# scan + report (dry-run)
|
|
python3 -m app.scan --roots /path/to/umd_games,/path/to/umd_videos
|
|
|
|
# scan + apply renames
|
|
python3 -m app.scan --roots /path/to/umd_games,/path/to/umd_videos --apply
|
|
|
|
# ingest DAT files into the library DB
|
|
python3 -m app.ingest --dats data/dats --db data/library.sqlite
|
|
|
|
# (re)build the offline cover bundles (see Cover art)
|
|
python3 -m app.libretro
|
|
python3 -m app.openvgdb
|
|
```
|
|
|
|
`python3 -m app.scan --json` prints machine-readable results, and `--quick`
|
|
skips full-file hashing (serial/title match only).
|