quoije/psp-preservation-tool
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
a9c371c2210d0733e086098213773e17773b7f57
psp-preservation-tool/README.md
T

15 KiB
Raw Blame History

PSP Preservation Tool

AI Generated

Native Linux GUI (PySide6) for preserving PSP UMD games and videos as .iso files. It scans your discs, verifies them against the full Redump catalog (Redump DATs), reports what's missing, and retags/renames discs to a canonical scheme. It also fetches cover art and metadata, syncs discs and save-data with a real PSP, can pull missing discs from your own Redump 7z archive collection, and converts PlayStation 1 ISO/CUE/BIN rips into PSP EBOOT.PBP homebrew.

PSP Preservation Tool

  • Use default app theme for Qt.

Features

  • Library — scan folders of .iso discs, verify against the DAT catalog, browse with cover art, search/filter, see per-game details (developer, publisher, genre, release date, synopsis, trailer), and launch discs in PPSSPP.
  • Store — browse and download discs from three sources: a local 7z folder, your own HTTP store server, or an archive.org item (see Store).
  • Queue — background transfers with progress, cancel, per-row delete, etc.
  • PSP Sync — detect a connected PSP (USB) or a mounted UMD, inventory what's installed (games / UMD videos / homebrew / PKG), copy discs to the PSP, delete from the PSP, and dump a UMD to the library.
  • Saves — two-way sync of save-data between the PSP, PPSSPP, and a cloud folder.
  • PS1 — scan a folder of PlayStation 1 discs (ISO/BIN/CUE), read each disc's serial, convert to PSP EBOOT.PBP with a pure-Python popstation converter, and track already-converted games.
  • Settings — directories, cover-source API keys, store/server config, and misc options.

Requirements

  • Linux + Python 3
  • PySide6 — the only Python dependency (pip install -r requirements.txt)
  • 7z (p7zip) — used to peek/extract disc images from store archives
  • PPSSPP (optional) — to launch discs from the details panel

Install

python3 -m venv .venv          # keep the venv on a local disk, not an SMB mount
. .venv/bin/activate
pip install -r requirements.txt

Run the GUI

python3 -m app.main

First-time setup

  1. Put your DAT files in data/dats/:

    • games: Redump Sony - PlayStation Portable (authoritative hashes) and No-Intro Sony - PlayStation Portable (supplies the serials Redump lacks — keeps serial-based matching working).
    • videos: No-Intro Sony - PlayStation Portable (UMD Video) (Redump has no UMD Video DAT).

  2. Ingest them into the local SQLite library:

    python3 -m app.ingest

  3. Open the GUI and point Settings → General → Games/Videos directory at your disc folders (or set PSP_GAMES_ROOTS / PSP_VIDEO_ROOTS).

  4. (Optional) PS1 conversion — copy any PSN PS1 classic EBOOT.PBP to data/base.PBP (it supplies Sony's encrypted loader). Then set the PS1 directory, PS1 output directory and PS1 base.pbp in Settings → General and use the PS1 tab (see PS1 support).

Discs that aren't in the Redump DATs can be added to data/supplement.json (name, region, crc32, sha1, md5) so the tool still recognizes them.

Nomenclature

Title (Region) (tags) (Serial).iso — leading articles stay in front (The Sims 2), and revision/disc tags are preserved ((v1.03), (Disc 1)). Discs are matched by hash first (most precise), then serial, then filename.

Directory layout

  • data/dats/ — clrmamepro DAT files (games + videos).
  • data/supplement.json — extra discs not in Redump.
  • data/credentials.json — cover/metadata API keys (IGDB, TheMovieDB, ScreenScraper, Google Custom Search). Environment variables take precedence.
  • data/manual_covers/games|videos/ — your own cover images.
  • data/covers/ — checked-in assets: the UMD video/game cover templates, libretro-index.json + openvgdb-index.json catalogs, libretro-covers.zip + openvgdb-covers.zip offline cover bundles, and the legacy cover cache.
  • data/base.PBP — a PSN PS1 classic EBOOT (any game) used as the encrypted loader when converting PS1 discs to EBOOT.PBP (git-ignored, copyrighted).
  • State dir (~/.local/state/psp-preserve/ by default, or $XDG_STATE_HOME/psp-preserve):
    • library.sqlite — the catalog + library database.
    • covers/ — downloaded cover cache.

SQLite is kept on a local disk because SQLite locking is unreliable over network filesystems; the DATs, manual covers and credentials stay in data/ (which may be a network share).

Configuration

Everything is driven by environment variables, with GUI settings (stored in QSettings under PSPPreserve) taking precedence where noted.

Variable Default Purpose
PSP_DATA_DIR ./data data root (DATs, credentials, manual covers)
PSP_DATS_DIR $PSP_DATA_DIR/dats DAT folder
PSP_STATE_DIR ~/.local/state/psp-preserve local state root
PSP_DB_PATH $PSP_STATE_DIR/library.sqlite SQLite library
PSP_COVERS_DIR $PSP_STATE_DIR/covers cover cache
PSP_GAMES_ROOTS /data/umd_games comma-separated game disc roots
PSP_VIDEO_ROOTS /data/umd_videos comma-separated video disc roots
PSP_PS1_ROOTS /data/ps1 comma-separated PS1 disc roots
PSP_PS1_OUTPUT_DIR /data/ps1_converted converted EBOOT.PBP output folder
PSP_SCAN_WORKERS 1 parallel hashing workers
PSP_STORE_DIR /media/downloads/Sony - PlayStation Portable local 7z store folder
PSP_STORE_SERVER https://psp.example.com store server base URL
PSP_STORE_TOKEN (empty) store server bearer token
PSP_ARCHIVE_ITEM psp-redump-roms321.com archive.org item identifier
PSP_ARCHIVE_LOGGED_IN_SIG (empty) archive.org logged-in-sig cookie
PSP_ARCHIVE_LOGGED_IN_USER (empty) archive.org logged-in-user cookie
PSP_API_TOKEN (empty) server API bearer token (unset = open)
PSP_CREDENTIALS $PSP_DATA_DIR/credentials.json credentials file
PSP_LIBRETRO_COVERS_URL (empty) direct URL for libretro-covers.zip (else derived from store server)
PSP_OVGDB_COVERS_URL (empty) direct URL for openvgdb-covers.zip (else derived from store server)

Cover/metadata keys can also be set as env vars: IGDB_CLIENT_ID, IGDB_CLIENT_SECRET, TMDB_API_KEY, GOOGLE_API_KEY, GOOGLE_CX, SCREENSCRAPER_DEVID, SCREENSCRAPER_DEVPASSWORD, SCREENSCRAPER_SSID, SCREENSCRAPER_SSPASSWORD, SCREENSCRAPER_SOFTNAME.

Cover art

Games are resolved in this order (offline-first, then network):

  1. Manual covers — images you drop in data/manual_covers/games/ (fuzzy-matched; strips (Region), handles Title, The).
  2. libretro-thumbnails (offline bundle) — full PSP box art from data/covers/libretro-covers.zip. If that zip is absent, it downloads from GitHub and caches instead.
  3. OpenVGDB (offline bundle) — full covers from data/covers/openvgdb-covers.zip.
  4. ScreenScraper — full box art + rich metadata; needs a free dev ID/password.
  5. Wikipedia — keyless fallback.
  6. IGDB — needs a Twitch/IGDB client ID + secret. IGDB only stores cropped 3:4 key art, so the app composites it into the PSP game cover template (data/covers/UMD_GAME_COVER_TEMPLATE.png) to restore the full cover.

UMD videos have no public box-art API. The manual Scrape… dialog offers three sources: TheMovieDB posters, Wikipedia image search (keyless), and Google Images (needs a Google API key + Search Engine ID in Settings → Cover sources). The pick is composited into data/covers/UMD_COVER_TEMPLATE.png. Manual covers in data/manual_covers/videos/ always win.

Offline bundles & auto-download

The two bundles (libretro-covers.zip, openvgdb-covers.zip) are shipped in data/covers/, so a fresh install works with no internet and no API keys. If either is missing at startup, the app downloads it from the store server's /api/covers/<name>.zip endpoint (see Server setup).

Rebuild them yourself:

python3 -m app.libretro          # libretro-thumbnails -> libretro-covers.zip
python3 -m app.openvgdb          # OpenVGDB/GameFAQs  -> openvgdb-covers.zip

(app.openvgdb needs curl_cffi to pass GameFAQs' Cloudflare challenge.)

PS1 support

The PS1 tab scans a folder of PlayStation 1 rips and converts them into PSP EBOOT.PBP files with a pure-Python popstation converter (no external tools). PS1 discs (pending) and converted EBOOTs also show up in the Library tab (filter by PS1), where you can convert a disc or copy an EBOOT to the PSP.

  • Source formats — .iso, .img, .bin (with a matching .cue for multi-track discs).
  • Serial detection — reads SYSTEM.CNF to derive the game ID (e.g. SLUS-00001) and writes it into PARAM.SFO + the PSAR header.
  • base.pbp — conversion needs a PSN PS1 classic EBOOT.PBP (any game) to supply Sony's encrypted DATA.PSP loader. Copy one to data/base.PBP and point Settings → General → PS1 base.pbp at it.
  • Output — a PSN-style folder <out>/<Game ID>/ containing EBOOT.PBP, KEYS.BIN and DOCUMENT.DAT, ready to copy to <PSP>/PSP/GAME/<Game ID>/.

Configure the folders in Settings → General (or PSP_PS1_ROOTS / PSP_PS1_OUTPUT_DIR). Current limits: single-disc games only (no multi-disc PBP), and images are stored uncompressed (LZRC compression is on the roadmap).

Store — archive.org and local folder

The Store tab downloads missing discs from a Redump 7z archive collection (Name (Region) (tags).7z, containing the .iso/.cso inside). It has three sources, selectable in the tab:

1. Local folder

Point Settings → Store → Local store dir (or PSP_STORE_DIR) at a folder of .7z/.zip archives. The app scans the folder, lists the archives, and on download:

  1. reads the archive listing with 7z to find the disc image inside,
  2. copies the archive to a temp dir and extracts the image with 7z,
  3. moves the image into your games/videos directory.

2. Server (HTTP API)

See Server setup. The app talks to GET /api/store (list) and GET /api/store/download/<name> (stream), using your PSP_API_TOKEN as Authorization: Bearer <token>.

3. Archive.org

Point Settings → Store → Archive.org item (or PSP_ARCHIVE_ITEM) at an archive.org item identifier (or a full details/download URL). The app:

  1. lists the item via https://archive.org/metadata/<item>,
  2. downloads archives from https://archive.org/download/<item>/<file>,
  3. extracts the disc image and moves it into your library.

If the item's files are marked private (login required), set the logged-in-sig and logged-in-user cookies in Settings → Store (grab them from a logged-in archive.org browser session). Without them, private files are skipped — and you'll get a clear error if everything in the item is private.

Server setup

The catalog + store API is a single-file, stdlib-only HTTP server (app/api.py) — no pip install needed on the server.

PSP_DATA_DIR=/data \
PSP_DATS_DIR=/data/dats \
PSP_DB_PATH=/data/library.sqlite \
PSP_COVERS_DIR=/data/covers \
PSP_GAMES_ROOTS=/data/umd_games \
PSP_VIDEO_ROOTS=/data/umd_videos \
PSP_STORE_DIR=/data/store \
PSP_API_TOKEN=<long-random-secret> \
python3 -m app.api --host 0.0.0.0 --port 8000

Endpoints:

Endpoint Description
GET /api/summary release counts per system
GET /api/releases?system=&q=&limit=&offset= search the catalog
GET /api/releases/<id> one release + its rom hashes
GET /api/library?system=&refresh= your scanned library
GET /api/cover/<id> cover image (fetched on demand)
GET /api/covers/<name>.zip offline cover bundle (public)
GET /api/store list store archives (token)
GET /api/store/download/<name> stream one archive (token)

Token protection: set PSP_API_TOKEN; the client sends Authorization: Bearer <token>. Leave it unset to allow anonymous access (not recommended).

Docker (Unraid)

docker-compose.yml builds the API image and attaches it to Unraid's br0 macvlan bridge so it gets its own LAN IP (e.g. 192.168.1.100, plain HTTP on 8000):

# in /path/to/psp-preserve/ — adjust the volume paths for your shares
printf 'PSP_API_TOKEN=<long-random-secret>\n' > .env
docker compose up -d --build

The compose file mounts your Unraid shares into /data/store, /data/umd_games and /data/umd_videos (the store is mounted read-only).

TLS (optional)

Caddyfile is a ready-made reverse proxy for https://psp.example.com with automatic Let's Encrypt certificates. To enable it, add the caddy service to the compose file, give it its own br0 IP, point it at the API, and forward ports 80/443 on your router to it.

Client

In the GUI: Settings → Store → enter the Server URL (e.g. http://<server-ip>:8000 or https://psp.example.com) and the same API token, then pick Server in the Store tab's source selector. Use Archive.org or Local folder the same way.

PSP Sync

Connect a PSP over USB (or mount a UMD) and the PSP Sync tab detects it:

  • Games / UMD Videos / Homebrew-PSN / PKG — tables of what's installed, with delete-from-PSP (checkboxes and right-click).
  • UMD — reads PARAM.SFO from a mounted UMD, shows the title/serial, and can dump the disc image into your library.
  • Sync from Library — copy discs from your library onto the PSP, optionally into a Category Lite category (set in Settings → PSP → ISO category).

Saves sync

The Saves tab shows save-data across three locations — PSP (USB), PPSSPP, and a Cloud folder — and syncs both ways. Save signatures are cached (keyed by size + mtime) so rescanning is fast, and an Auto-sync option runs the sync automatically whenever a PSP is connected.

CLI tools

# scan + report (dry-run)
python3 -m app.scan --roots /path/to/umd_games,/path/to/umd_videos

# scan + apply renames
python3 -m app.scan --roots /path/to/umd_games,/path/to/umd_videos --apply

# ingest DAT files into the library DB
python3 -m app.ingest --dats data/dats --db data/library.sqlite

# (re)build the offline cover bundles (see Cover art)
python3 -m app.libretro
python3 -m app.openvgdb

# convert one PS1 disc to EBOOT.PBP (CLI)
python3 -m app.ps1_convert --source /path/to/game.cue --title "Game" \
  --game-id SLUS00001 --base-pbp data/base.PBP --out /out/Game.pbp

python3 -m app.scan --json prints machine-readable results, and --quick skips full-file hashing (serial/title match only).

Reference in New Issue View Git Blame Copy Permalink