Compare commits
14
Commits
00f9e24c79
...
02b4adb1c1
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
02b4adb1c1 | ||
|
|
a9c371c221 | ||
|
|
4d5c48cb94 | ||
|
|
777d316c0f | ||
|
|
625e2f8718 | ||
|
|
7d8b449846 | ||
|
|
27e482d344 | ||
|
|
5d619e5f59 | ||
|
|
743d5e8e90 | ||
|
|
bb1b8dea66 | ||
|
|
2d599141f9 | ||
|
|
922c01d17c | ||
|
|
128b62e5b8 | ||
|
|
da3429d918 |
@@ -6,3 +6,7 @@ data/*.sqlite
|
|||||||
data/*.sqlite-*
|
data/*.sqlite-*
|
||||||
data/credentials.json
|
data/credentials.json
|
||||||
session-transcript-2026-09-01.md
|
session-transcript-2026-09-01.md
|
||||||
|
|
||||||
|
# Copyrighted PSN EBOOTs (base.pbp) used only for local PS1 conversion testing
|
||||||
|
data/*.PBP
|
||||||
|
data/*.pbp
|
||||||
|
|||||||
@@ -3,10 +3,11 @@
|
|||||||

|

|
||||||
|
|
||||||
Native Linux GUI (PySide6) for preserving PSP UMD **games** and **videos** as
|
Native Linux GUI (PySide6) for preserving PSP UMD **games** and **videos** as
|
||||||
`.iso` files. It scans your discs, verifies them against a full catalog
|
`.iso` files. It scans your discs, verifies them against the full Redump catalog
|
||||||
(No-Intro DATs), reports what's missing, and retags/renames discs to a canonical
|
(Redump DATs), reports what's missing, and retags/renames discs to a canonical
|
||||||
scheme. It also fetches cover art and metadata, syncs discs and save-data with a
|
scheme. It also fetches cover art and metadata, syncs discs and save-data with a
|
||||||
real PSP, and can pull missing discs from your own Redump 7z archive collection.
|
real PSP, can pull missing discs from your own Redump 7z archive collection, and
|
||||||
|
converts PlayStation 1 ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew.
|
||||||
|
|
||||||

|

|
||||||
* Use default app theme for Qt.
|
* Use default app theme for Qt.
|
||||||
@@ -25,6 +26,9 @@ real PSP, and can pull missing discs from your own Redump 7z archive collection.
|
|||||||
from the PSP, and dump a UMD to the library.
|
from the PSP, and dump a UMD to the library.
|
||||||
- **Saves** — two-way sync of save-data between the PSP, PPSSPP, and a cloud
|
- **Saves** — two-way sync of save-data between the PSP, PPSSPP, and a cloud
|
||||||
folder.
|
folder.
|
||||||
|
- **PS1** — scan a folder of PlayStation 1 discs (ISO/BIN/CUE), read each disc's
|
||||||
|
serial, convert to PSP `EBOOT.PBP` with a pure-Python popstation converter,
|
||||||
|
and track already-converted games.
|
||||||
- **Settings** — directories, cover-source API keys, store/server config, and
|
- **Settings** — directories, cover-source API keys, store/server config, and
|
||||||
misc options.
|
misc options.
|
||||||
|
|
||||||
@@ -52,8 +56,11 @@ python3 -m app.main
|
|||||||
## First-time setup
|
## First-time setup
|
||||||
|
|
||||||
1. Put your DAT files in `data/dats/`:
|
1. Put your DAT files in `data/dats/`:
|
||||||
- games: No-Intro `Sony - PlayStation Portable`
|
- games: Redump `Sony - PlayStation Portable` (authoritative hashes)
|
||||||
- videos: No-Intro `Sony - PlayStation Portable (UMD Video)`
|
**and** No-Intro `Sony - PlayStation Portable` (supplies the serials
|
||||||
|
Redump lacks — keeps serial-based matching working).
|
||||||
|
- videos: No-Intro `Sony - PlayStation Portable (UMD Video)` (Redump has
|
||||||
|
no UMD Video DAT).
|
||||||
2. Ingest them into the local SQLite library:
|
2. Ingest them into the local SQLite library:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -62,8 +69,12 @@ python3 -m app.main
|
|||||||
|
|
||||||
3. Open the GUI and point **Settings → General → Games/Videos directory** at
|
3. Open the GUI and point **Settings → General → Games/Videos directory** at
|
||||||
your disc folders (or set `PSP_GAMES_ROOTS` / `PSP_VIDEO_ROOTS`).
|
your disc folders (or set `PSP_GAMES_ROOTS` / `PSP_VIDEO_ROOTS`).
|
||||||
|
4. (Optional) PS1 conversion — copy any PSN PS1 classic `EBOOT.PBP` to
|
||||||
|
`data/base.PBP` (it supplies Sony's encrypted loader). Then set the **PS1
|
||||||
|
directory**, **PS1 output directory** and **PS1 base.pbp** in
|
||||||
|
**Settings → General** and use the **PS1** tab (see [PS1 support](#ps1-support)).
|
||||||
|
|
||||||
Discs that aren't in the No-Intro DATs can be added to `data/supplement.json`
|
Discs that aren't in the Redump DATs can be added to `data/supplement.json`
|
||||||
(`name`, `region`, `crc32`, `sha1`, `md5`) so the tool still recognizes them.
|
(`name`, `region`, `crc32`, `sha1`, `md5`) so the tool still recognizes them.
|
||||||
|
|
||||||
## Nomenclature
|
## Nomenclature
|
||||||
@@ -75,13 +86,15 @@ Discs are matched by hash first (most precise), then serial, then filename.
|
|||||||
## Directory layout
|
## Directory layout
|
||||||
|
|
||||||
- `data/dats/` — clrmamepro DAT files (games + videos).
|
- `data/dats/` — clrmamepro DAT files (games + videos).
|
||||||
- `data/supplement.json` — extra discs not in No-Intro.
|
- `data/supplement.json` — extra discs not in Redump.
|
||||||
- `data/credentials.json` — cover/metadata API keys (IGDB, TheMovieDB,
|
- `data/credentials.json` — cover/metadata API keys (IGDB, TheMovieDB,
|
||||||
ScreenScraper). Environment variables take precedence.
|
ScreenScraper, Google Custom Search). Environment variables take precedence.
|
||||||
- `data/manual_covers/games|videos/` — your own cover images.
|
- `data/manual_covers/games|videos/` — your own cover images.
|
||||||
- `data/covers/` — checked-in assets: the UMD video/game cover templates,
|
- `data/covers/` — checked-in assets: the UMD video/game cover templates,
|
||||||
`libretro-index.json` catalog, `libretro-covers.zip` + `openvgdb-covers.zip`
|
`libretro-index.json` + `openvgdb-index.json` catalogs, `libretro-covers.zip` +
|
||||||
offline cover bundles, and the legacy cover cache.
|
`openvgdb-covers.zip` offline cover bundles, and the legacy cover cache.
|
||||||
|
- `data/base.PBP` — a PSN PS1 classic EBOOT (any game) used as the encrypted
|
||||||
|
loader when converting PS1 discs to `EBOOT.PBP` (git-ignored, copyrighted).
|
||||||
- State dir (`~/.local/state/psp-preserve/` by default, or `$XDG_STATE_HOME/psp-preserve`):
|
- State dir (`~/.local/state/psp-preserve/` by default, or `$XDG_STATE_HOME/psp-preserve`):
|
||||||
- `library.sqlite` — the catalog + library database.
|
- `library.sqlite` — the catalog + library database.
|
||||||
- `covers/` — downloaded cover cache.
|
- `covers/` — downloaded cover cache.
|
||||||
@@ -104,6 +117,8 @@ QSettings under `PSPPreserve`) taking precedence where noted.
|
|||||||
| `PSP_COVERS_DIR` | `$PSP_STATE_DIR/covers` | cover cache |
|
| `PSP_COVERS_DIR` | `$PSP_STATE_DIR/covers` | cover cache |
|
||||||
| `PSP_GAMES_ROOTS` | `/data/umd_games` | comma-separated game disc roots |
|
| `PSP_GAMES_ROOTS` | `/data/umd_games` | comma-separated game disc roots |
|
||||||
| `PSP_VIDEO_ROOTS` | `/data/umd_videos` | comma-separated video disc roots |
|
| `PSP_VIDEO_ROOTS` | `/data/umd_videos` | comma-separated video disc roots |
|
||||||
|
| `PSP_PS1_ROOTS` | `/data/ps1` | comma-separated PS1 disc roots |
|
||||||
|
| `PSP_PS1_OUTPUT_DIR` | `/data/ps1_converted` | converted EBOOT.PBP output folder |
|
||||||
| `PSP_SCAN_WORKERS` | `1` | parallel hashing workers |
|
| `PSP_SCAN_WORKERS` | `1` | parallel hashing workers |
|
||||||
| `PSP_STORE_DIR` | `/media/downloads/Sony - PlayStation Portable` | local 7z store folder |
|
| `PSP_STORE_DIR` | `/media/downloads/Sony - PlayStation Portable` | local 7z store folder |
|
||||||
| `PSP_STORE_SERVER` | `https://psp.example.com` | store server base URL |
|
| `PSP_STORE_SERVER` | `https://psp.example.com` | store server base URL |
|
||||||
@@ -117,9 +132,9 @@ QSettings under `PSPPreserve`) taking precedence where noted.
|
|||||||
| `PSP_OVGDB_COVERS_URL` | *(empty)* | direct URL for `openvgdb-covers.zip` (else derived from store server) |
|
| `PSP_OVGDB_COVERS_URL` | *(empty)* | direct URL for `openvgdb-covers.zip` (else derived from store server) |
|
||||||
|
|
||||||
Cover/metadata keys can also be set as env vars: `IGDB_CLIENT_ID`,
|
Cover/metadata keys can also be set as env vars: `IGDB_CLIENT_ID`,
|
||||||
`IGDB_CLIENT_SECRET`, `TMDB_API_KEY`, `SCREENSCRAPER_DEVID`,
|
`IGDB_CLIENT_SECRET`, `TMDB_API_KEY`, `GOOGLE_API_KEY`, `GOOGLE_CX`,
|
||||||
`SCREENSCRAPER_DEVPASSWORD`, `SCREENSCRAPER_SSID`, `SCREENSCRAPER_SSPASSWORD`,
|
`SCREENSCRAPER_DEVID`, `SCREENSCRAPER_DEVPASSWORD`, `SCREENSCRAPER_SSID`,
|
||||||
`SCREENSCRAPER_SOFTNAME`.
|
`SCREENSCRAPER_SSPASSWORD`, `SCREENSCRAPER_SOFTNAME`.
|
||||||
|
|
||||||
## Cover art
|
## Cover art
|
||||||
|
|
||||||
@@ -139,8 +154,11 @@ Games are resolved in this order (offline-first, then network):
|
|||||||
3:4 key art, so the app composites it into the PSP game cover template
|
3:4 key art, so the app composites it into the PSP game cover template
|
||||||
(`data/covers/UMD_GAME_COVER_TEMPLATE.png`) to restore the full cover.
|
(`data/covers/UMD_GAME_COVER_TEMPLATE.png`) to restore the full cover.
|
||||||
|
|
||||||
UMD **videos** have no public box-art API: they use your manual covers, or a
|
UMD **videos** have no public box-art API. The manual **Scrape…** dialog offers
|
||||||
TheMovieDB poster composited into `data/covers/UMD_COVER_TEMPLATE.png`.
|
three sources: TheMovieDB posters, Wikipedia image search (keyless), and Google
|
||||||
|
Images (needs a Google API key + Search Engine ID in **Settings → Cover
|
||||||
|
sources**). The pick is composited into `data/covers/UMD_COVER_TEMPLATE.png`.
|
||||||
|
Manual covers in `data/manual_covers/videos/` always win.
|
||||||
|
|
||||||
### Offline bundles & auto-download
|
### Offline bundles & auto-download
|
||||||
|
|
||||||
@@ -158,6 +176,29 @@ python3 -m app.openvgdb # OpenVGDB/GameFAQs -> openvgdb-covers.zip
|
|||||||
|
|
||||||
(`app.openvgdb` needs `curl_cffi` to pass GameFAQs' Cloudflare challenge.)
|
(`app.openvgdb` needs `curl_cffi` to pass GameFAQs' Cloudflare challenge.)
|
||||||
|
|
||||||
|
## PS1 support
|
||||||
|
|
||||||
|
The **PS1** tab scans a folder of PlayStation 1 rips and converts them into PSP
|
||||||
|
`EBOOT.PBP` files with a pure-Python popstation converter (no external tools).
|
||||||
|
PS1 discs (pending) and converted EBOOTs also show up in the **Library** tab
|
||||||
|
(filter by **PS1**), where you can convert a disc or copy an EBOOT to the PSP.
|
||||||
|
|
||||||
|
- **Source formats** — `.iso`, `.img`, `.bin` (with a matching `.cue` for
|
||||||
|
multi-track discs).
|
||||||
|
- **Serial detection** — reads `SYSTEM.CNF` to derive the game ID (e.g.
|
||||||
|
`SLUS-00001`) and writes it into `PARAM.SFO` + the PSAR header.
|
||||||
|
- **base.pbp** — conversion needs a PSN PS1 classic `EBOOT.PBP` (any game) to
|
||||||
|
supply Sony's encrypted `DATA.PSP` loader. Copy one to `data/base.PBP` and
|
||||||
|
point **Settings → General → PS1 base.pbp** at it.
|
||||||
|
- **Output** — a folder `<out>/<Game ID>/` containing a single `EBOOT.PBP`
|
||||||
|
(the popstation way: uncompressed level-0 PSAR, no `KEYS.BIN`/`DOCUMENT.DAT`),
|
||||||
|
ready to copy to `<PSP>/PSP/GAME/<Game ID>/`. Compatible with CFW POPS
|
||||||
|
(ARK-4).
|
||||||
|
|
||||||
|
Configure the folders in **Settings → General** (or `PSP_PS1_ROOTS` /
|
||||||
|
`PSP_PS1_OUTPUT_DIR`). Current limits: single-disc games only (no multi-disc
|
||||||
|
PBP), and images are stored uncompressed (LZRC compression is on the roadmap).
|
||||||
|
|
||||||
## Store — archive.org and local folder
|
## Store — archive.org and local folder
|
||||||
|
|
||||||
The **Store** tab downloads missing discs from a Redump 7z archive collection
|
The **Store** tab downloads missing discs from a Redump 7z archive collection
|
||||||
@@ -289,6 +330,10 @@ python3 -m app.ingest --dats data/dats --db data/library.sqlite
|
|||||||
# (re)build the offline cover bundles (see Cover art)
|
# (re)build the offline cover bundles (see Cover art)
|
||||||
python3 -m app.libretro
|
python3 -m app.libretro
|
||||||
python3 -m app.openvgdb
|
python3 -m app.openvgdb
|
||||||
|
|
||||||
|
# convert one PS1 disc to EBOOT.PBP (CLI)
|
||||||
|
python3 -m app.ps1_convert --source /path/to/game.cue --title "Game" \
|
||||||
|
--game-id SLUS00001 --base-pbp data/base.PBP --out /out/Game.pbp
|
||||||
```
|
```
|
||||||
|
|
||||||
`python3 -m app.scan --json` prints machine-readable results, and `--quick`
|
`python3 -m app.scan --json` prints machine-readable results, and `--quick`
|
||||||
|
|||||||
@@ -50,10 +50,17 @@ ARCHIVE_LOGGED_IN_USER = os.environ.get("PSP_ARCHIVE_LOGGED_IN_USER", "")
|
|||||||
GAMES_ROOTS = [Path(p) for p in os.environ.get("PSP_GAMES_ROOTS", "/data/umd_games").split(",") if p.strip()]
|
GAMES_ROOTS = [Path(p) for p in os.environ.get("PSP_GAMES_ROOTS", "/data/umd_games").split(",") if p.strip()]
|
||||||
VIDEO_ROOTS = [Path(p) for p in os.environ.get("PSP_VIDEO_ROOTS", "/data/umd_videos").split(",") if p.strip()]
|
VIDEO_ROOTS = [Path(p) for p in os.environ.get("PSP_VIDEO_ROOTS", "/data/umd_videos").split(",") if p.strip()]
|
||||||
|
|
||||||
|
# PS1: source folder (raw ISO/BIN/CUE) and converted EBOOT.PBP output folder.
|
||||||
|
PS1_ROOTS = [Path(p) for p in os.environ.get("PSP_PS1_ROOTS", "/data/ps1").split(",") if p.strip()]
|
||||||
|
PS1_OUTPUT_DIR = Path(os.environ.get("PSP_PS1_OUTPUT_DIR", "/data/ps1_converted"))
|
||||||
|
|
||||||
# Only .iso discs are tracked; everything below is explicitly ignored.
|
# Only .iso discs are tracked; everything below is explicitly ignored.
|
||||||
DISC_EXTENSIONS = {".iso"}
|
DISC_EXTENSIONS = {".iso"}
|
||||||
IGNORED_EXTENSIONS = {".pkg", ".pbp", ".cso"}
|
IGNORED_EXTENSIONS = {".pkg", ".pbp", ".cso"}
|
||||||
|
|
||||||
|
# PS1 source images (scanned separately from PSP .iso discs).
|
||||||
|
PS1_EXTENSIONS = {".cue", ".iso", ".img", ".bin"}
|
||||||
|
|
||||||
# Parallel hashing workers for the scanner. Keep at 1 on spinning disks
|
# Parallel hashing workers for the scanner. Keep at 1 on spinning disks
|
||||||
# (concurrent reads cause seek thrash); raise it for SSD / fast arrays.
|
# (concurrent reads cause seek thrash); raise it for SSD / fast arrays.
|
||||||
SCAN_WORKERS = int(os.environ.get("PSP_SCAN_WORKERS", "1"))
|
SCAN_WORKERS = int(os.environ.get("PSP_SCAN_WORKERS", "1"))
|
||||||
|
|||||||
+281
-93
@@ -24,18 +24,34 @@ from .db import get_connection
|
|||||||
|
|
||||||
log = logging.getLogger("covers")
|
log = logging.getLogger("covers")
|
||||||
|
|
||||||
_LIBRETRO_BASE = "https://raw.githubusercontent.com/libretro-thumbnails/Sony_-_PlayStation_Portable/master"
|
|
||||||
_LIBRETRO_KINDS = ("Named_Boxarts",)
|
_LIBRETRO_KINDS = ("Named_Boxarts",)
|
||||||
_TREE_URL = "https://api.github.com/repos/libretro-thumbnails/Sony_-_PlayStation_Portable/git/trees/master?recursive=1"
|
_PLATFORMS = {
|
||||||
|
"games": {"libretro_repo": "Sony_-_PlayStation_Portable", "igdb_id": 38, "ss_id": "61"},
|
||||||
|
"ps1": {"libretro_repo": "Sony_-_PlayStation", "igdb_id": 7, "ss_id": "57"},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _platform(system):
|
||||||
|
"""Platform parameters (libretro repo, IGDB id, ScreenScraper id)."""
|
||||||
|
return _PLATFORMS.get(system) or _PLATFORMS["games"]
|
||||||
|
|
||||||
|
|
||||||
|
def _libretro_base(system="games"):
|
||||||
|
return ("https://raw.githubusercontent.com/libretro-thumbnails/"
|
||||||
|
f"{_platform(system)['libretro_repo']}/master")
|
||||||
|
|
||||||
|
|
||||||
|
def _libretro_tree_url(system="games"):
|
||||||
|
return ("https://api.github.com/repos/libretro-thumbnails/"
|
||||||
|
f"{_platform(system)['libretro_repo']}/git/trees/master?recursive=1")
|
||||||
|
|
||||||
_SS_API = "https://api.screenscraper.fr/api2/jeuInfos.php"
|
_SS_API = "https://api.screenscraper.fr/api2/jeuInfos.php"
|
||||||
_SS_PSP_ID = "61" # Sony PSP
|
|
||||||
_IGDB_TOKEN_URL = "https://id.twitch.tv/oauth2/token"
|
_IGDB_TOKEN_URL = "https://id.twitch.tv/oauth2/token"
|
||||||
_IGDB_API = "https://api.igdb.com/v4/games"
|
_IGDB_API = "https://api.igdb.com/v4/games"
|
||||||
_IGDB_IMAGE = "https://images.igdb.com/igdb/image/upload/"
|
_IGDB_IMAGE = "https://images.igdb.com/igdb/image/upload/"
|
||||||
_IGDB_PSP_ID = 38 # Sony PSP
|
|
||||||
_TMDB_SEARCH = "https://api.themoviedb.org/3/search/movie"
|
_TMDB_SEARCH = "https://api.themoviedb.org/3/search/movie"
|
||||||
_TMDB_IMAGE = "https://image.tmdb.org/t/p/"
|
_TMDB_IMAGE = "https://image.tmdb.org/t/p/"
|
||||||
|
_GOOGLE_CSE_API = "https://www.googleapis.com/customsearch/v1"
|
||||||
_MANUAL_DIR = config.DATA_DIR / "manual_covers"
|
_MANUAL_DIR = config.DATA_DIR / "manual_covers"
|
||||||
_OVGDB_ZIP = config.DATA_DIR / "covers" / "openvgdb-covers.zip"
|
_OVGDB_ZIP = config.DATA_DIR / "covers" / "openvgdb-covers.zip"
|
||||||
_LIBRETRO_ZIP = config.DATA_DIR / "covers" / "libretro-covers.zip"
|
_LIBRETRO_ZIP = config.DATA_DIR / "covers" / "libretro-covers.zip"
|
||||||
@@ -51,7 +67,7 @@ _igdb_token_state = {"token": "", "expires": 0.0} # IGDB OAuth2 token cache
|
|||||||
_igdb_lock = threading.Lock()
|
_igdb_lock = threading.Lock()
|
||||||
_igdb_last = 0.0 # monotonic clock of the last IGDB API call
|
_igdb_last = 0.0 # monotonic clock of the last IGDB API call
|
||||||
|
|
||||||
_index_state = {"ts": 0.0, "names": {}, "norm": {}}
|
_index_state = {} # system -> {"ts", "names", "norm"}
|
||||||
_INDEX_TTL = 86400.0 # refresh the libretro file index once a day
|
_INDEX_TTL = 86400.0 # refresh the libretro file index once a day
|
||||||
|
|
||||||
_neg_cache = {} # cache key -> time.time() of the last failed fetch
|
_neg_cache = {} # cache key -> time.time() of the last failed fetch
|
||||||
@@ -186,9 +202,9 @@ def ensure_cover_bundles(server_url=""):
|
|||||||
return downloaded
|
return downloaded
|
||||||
|
|
||||||
|
|
||||||
def cover_url(name, kind="Named_Boxarts"):
|
def cover_url(name, kind="Named_Boxarts", system="games"):
|
||||||
"""Public libretro URL of an image for a catalog name."""
|
"""Public libretro URL of an image for a catalog name."""
|
||||||
return f"{_LIBRETRO_BASE}/{kind}/{urllib.parse.quote(name)}.png"
|
return f"{_libretro_base(system)}/{kind}/{urllib.parse.quote(name)}.png"
|
||||||
|
|
||||||
|
|
||||||
def cover_path(release_id, system):
|
def cover_path(release_id, system):
|
||||||
@@ -260,7 +276,7 @@ def _candidate_names(name):
|
|||||||
yield s
|
yield s
|
||||||
|
|
||||||
|
|
||||||
def _fetch_png(name, kind, timeout):
|
def _fetch_png(name, kind, timeout, system="games"):
|
||||||
"""Fetch a real image from libretro, following symlink text bodies.
|
"""Fetch a real image from libretro, following symlink text bodies.
|
||||||
|
|
||||||
GitHub serves a repo symlink as a tiny text body containing the target
|
GitHub serves a repo symlink as a tiny text body containing the target
|
||||||
@@ -268,7 +284,7 @@ def _fetch_png(name, kind, timeout):
|
|||||||
"""
|
"""
|
||||||
target = name
|
target = name
|
||||||
for _ in range(_MAX_HOPS + 1):
|
for _ in range(_MAX_HOPS + 1):
|
||||||
data = _http_get(cover_url(target, kind), timeout)
|
data = _http_get(cover_url(target, kind, system), timeout)
|
||||||
if data is None:
|
if data is None:
|
||||||
return None
|
return None
|
||||||
if _is_image(data):
|
if _is_image(data):
|
||||||
@@ -283,26 +299,29 @@ def _fetch_png(name, kind, timeout):
|
|||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
# libretro filename index + fuzzy matching
|
# libretro filename index + fuzzy matching
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
def _index_path():
|
def _index_path(system="games"):
|
||||||
return config.COVERS_DIR / "libretro-index.json"
|
if system == "games":
|
||||||
|
return config.COVERS_DIR / "libretro-index.json"
|
||||||
|
return config.COVERS_DIR / f"libretro-index-{system}.json"
|
||||||
|
|
||||||
|
|
||||||
def _load_index(force=False):
|
def _load_index(force=False, system="games"):
|
||||||
st = _index_state
|
st = _index_state.get(system) or {"ts": 0.0, "names": {}, "norm": {}}
|
||||||
|
_index_state[system] = st
|
||||||
if not force and st["names"] and time.time() - st["ts"] < _INDEX_TTL:
|
if not force and st["names"] and time.time() - st["ts"] < _INDEX_TTL:
|
||||||
return st["names"]
|
return st["names"]
|
||||||
path = _index_path()
|
path = _index_path(system)
|
||||||
if not force and path.exists():
|
if not force and path.exists():
|
||||||
try:
|
try:
|
||||||
# Load the on-disk index so it doubles as a fallback if the network
|
# Load the on-disk index so it doubles as a fallback if the network
|
||||||
# is down, but only return it immediately while it's still fresh.
|
# is down, but only return it immediately while it's still fresh.
|
||||||
names = json.loads(path.read_text(encoding="utf-8"))
|
names = json.loads(path.read_text(encoding="utf-8"))
|
||||||
_set_index(names)
|
_set_index(names, system)
|
||||||
if time.time() - path.stat().st_mtime < _INDEX_TTL:
|
if time.time() - path.stat().st_mtime < _INDEX_TTL:
|
||||||
return names
|
return names
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
data = _http_get(_TREE_URL, timeout=30)
|
data = _http_get(_libretro_tree_url(system), timeout=30)
|
||||||
if data:
|
if data:
|
||||||
try:
|
try:
|
||||||
tree = json.loads(data).get("tree", [])
|
tree = json.loads(data).get("tree", [])
|
||||||
@@ -313,7 +332,7 @@ def _load_index(force=False):
|
|||||||
prefix = kind + "/"
|
prefix = kind + "/"
|
||||||
if p.startswith(prefix) and p.endswith(".png"):
|
if p.startswith(prefix) and p.endswith(".png"):
|
||||||
names[kind].append(p[len(prefix):-4])
|
names[kind].append(p[len(prefix):-4])
|
||||||
_set_index(names)
|
_set_index(names, system)
|
||||||
try:
|
try:
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
path.write_text(json.dumps(names), encoding="utf-8")
|
path.write_text(json.dumps(names), encoding="utf-8")
|
||||||
@@ -325,9 +344,10 @@ def _load_index(force=False):
|
|||||||
return st["names"] or {kind: [] for kind in _LIBRETRO_KINDS}
|
return st["names"] or {kind: [] for kind in _LIBRETRO_KINDS}
|
||||||
|
|
||||||
|
|
||||||
def _set_index(names):
|
def _set_index(names, system="games"):
|
||||||
"""Store the raw filename index plus a pre-normalized copy for fast lookup."""
|
"""Store the raw filename index plus a pre-normalized copy for fast lookup."""
|
||||||
st = _index_state
|
st = _index_state.get(system) or {"ts": 0.0, "names": {}, "norm": {}}
|
||||||
|
_index_state[system] = st
|
||||||
st["names"] = names
|
st["names"] = names
|
||||||
st["norm"] = {k: [(_norm(n), n) for n in v] for k, v in names.items()}
|
st["norm"] = {k: [(_norm(n), n) for n in v] for k, v in names.items()}
|
||||||
st["ts"] = time.time()
|
st["ts"] = time.time()
|
||||||
@@ -343,11 +363,12 @@ def _plain(s):
|
|||||||
return re.sub(r"\s*\([^()]*\)", "", s).strip()
|
return re.sub(r"\s*\([^()]*\)", "", s).strip()
|
||||||
|
|
||||||
|
|
||||||
def _index_best(name, kind):
|
def _index_best(name, kind, system="games"):
|
||||||
"""Return the best-matching repo filename for `name`, or None."""
|
"""Return the best-matching repo filename for `name`, or None."""
|
||||||
st = _index_state
|
st = _index_state.get(system) or {"ts": 0.0, "names": {}, "norm": {}}
|
||||||
if not st["norm"]:
|
if not st["norm"]:
|
||||||
_load_index()
|
_load_index(system=system)
|
||||||
|
st = _index_state.get(system) or st
|
||||||
pairs = st["norm"].get(kind, [])
|
pairs = st["norm"].get(kind, [])
|
||||||
db = _norm(name)
|
db = _norm(name)
|
||||||
if not pairs or len(db) < 5:
|
if not pairs or len(db) < 5:
|
||||||
@@ -384,24 +405,25 @@ def _index_best(name, kind):
|
|||||||
return best
|
return best
|
||||||
|
|
||||||
|
|
||||||
def _fetch_libretro(name, timeout, indexed_only=False):
|
def _fetch_libretro(name, timeout, system="games", indexed_only=False):
|
||||||
# 0. Shipped offline bundle first (fast, no network).
|
# 0. Shipped offline bundle first (fast, no network) — PSP only for now.
|
||||||
data = fetch_libretro_local(name)
|
if system == "games":
|
||||||
if data is not None:
|
data = fetch_libretro_local(name)
|
||||||
return data
|
if data is not None:
|
||||||
|
return data
|
||||||
if not indexed_only:
|
if not indexed_only:
|
||||||
# 1. Cheap: exact + progressively stripped names (no index needed).
|
# 1. Cheap: exact + progressively stripped names (no index needed).
|
||||||
for kind in _LIBRETRO_KINDS:
|
for kind in _LIBRETRO_KINDS:
|
||||||
for cand in _candidate_names(name):
|
for cand in _candidate_names(name):
|
||||||
data = _fetch_png(cand, kind, timeout)
|
data = _fetch_png(cand, kind, timeout, system)
|
||||||
if data is not None:
|
if data is not None:
|
||||||
return data
|
return data
|
||||||
# 2. Fuzzy/exact via the precomputed index (covers exact names too, so the
|
# 2. Fuzzy/exact via the precomputed index (covers exact names too, so the
|
||||||
# bulk loader skips the wasteful exact pass entirely).
|
# bulk loader skips the wasteful exact pass entirely).
|
||||||
for kind in _LIBRETRO_KINDS:
|
for kind in _LIBRETRO_KINDS:
|
||||||
best = _index_best(name, kind)
|
best = _index_best(name, kind, system)
|
||||||
if best:
|
if best:
|
||||||
data = _fetch_png(best, kind, timeout)
|
data = _fetch_png(best, kind, timeout, system)
|
||||||
if data is not None:
|
if data is not None:
|
||||||
return data
|
return data
|
||||||
return None
|
return None
|
||||||
@@ -466,6 +488,33 @@ def _ovgdb_name_list():
|
|||||||
return _ovgdb_names
|
return _ovgdb_names
|
||||||
|
|
||||||
|
|
||||||
|
_ovgdb_index = None
|
||||||
|
|
||||||
|
|
||||||
|
def _load_ovgdb_index():
|
||||||
|
"""Load {key: {"title", "url"}} written by app/openvgdb.py, or {}.
|
||||||
|
|
||||||
|
Looks in the runtime covers dir first, then the shipped ``data/covers``
|
||||||
|
copy (so a fresh install still shows readable titles without re-running
|
||||||
|
the full OpenVGDB scrape).
|
||||||
|
"""
|
||||||
|
global _ovgdb_index
|
||||||
|
if _ovgdb_index is not None:
|
||||||
|
return _ovgdb_index
|
||||||
|
_ovgdb_index = {}
|
||||||
|
for path in (
|
||||||
|
config.COVERS_DIR / "openvgdb-index.json",
|
||||||
|
config.LEGACY_COVERS_DIR / "openvgdb-index.json",
|
||||||
|
):
|
||||||
|
if path.exists():
|
||||||
|
try:
|
||||||
|
_ovgdb_index = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
break
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
_ovgdb_index = {}
|
||||||
|
return _ovgdb_index
|
||||||
|
|
||||||
|
|
||||||
def fetch_openvgdb_local(name):
|
def fetch_openvgdb_local(name):
|
||||||
"""Return a bundled OpenVGDB cover image for `name`, or None."""
|
"""Return a bundled OpenVGDB cover image for `name`, or None."""
|
||||||
names = _ovgdb_name_list()
|
names = _ovgdb_name_list()
|
||||||
@@ -549,7 +598,7 @@ def _screenscraper_box_urls(jeu):
|
|||||||
return urls
|
return urls
|
||||||
|
|
||||||
|
|
||||||
def _fetch_screenscraper(name, timeout):
|
def _fetch_screenscraper(name, timeout, system="games"):
|
||||||
creds = sources.load()
|
creds = sources.load()
|
||||||
devid = creds.get("screenscraper_devid")
|
devid = creds.get("screenscraper_devid")
|
||||||
devpassword = creds.get("screenscraper_devpassword")
|
devpassword = creds.get("screenscraper_devpassword")
|
||||||
@@ -560,7 +609,7 @@ def _fetch_screenscraper(name, timeout):
|
|||||||
"devpassword": devpassword,
|
"devpassword": devpassword,
|
||||||
"softname": creds.get("screenscraper_softname") or "psp-preserve",
|
"softname": creds.get("screenscraper_softname") or "psp-preserve",
|
||||||
"output": "json",
|
"output": "json",
|
||||||
"systemeid": _SS_PSP_ID,
|
"systemeid": _platform(system)["ss_id"],
|
||||||
"romtype": "rom",
|
"romtype": "rom",
|
||||||
"romnom": name,
|
"romnom": name,
|
||||||
}
|
}
|
||||||
@@ -806,19 +855,20 @@ def fetch_cover(name, release_id, system, timeout=10, quick=False):
|
|||||||
if data is not None:
|
if data is not None:
|
||||||
log.info("cover %r <- manual", name)
|
log.info("cover %r <- manual", name)
|
||||||
elif system != "videos":
|
elif system != "videos":
|
||||||
# 2. Games: libretro (full box art) first, then the shipped OpenVGDB
|
# 2. Games/PS1: libretro (full box art) first, then the shipped
|
||||||
# bundle (offline), then ScreenScraper (full box art, non-quick),
|
# OpenVGDB bundle (PSP only, offline), then ScreenScraper (non-quick),
|
||||||
# then Wikipedia (keyless, non-quick), then IGDB last — IGDB only
|
# then Wikipedia (keyless, non-quick), then IGDB last.
|
||||||
# stores cropped 3:4 key art.
|
data = _fetch_libretro(name, t, system=system, indexed_only=quick)
|
||||||
data = _fetch_libretro(name, t, indexed_only=quick)
|
|
||||||
if data is not None:
|
if data is not None:
|
||||||
log.info("cover %r <- libretro", name)
|
log.info("cover %r <- libretro", name)
|
||||||
else:
|
else:
|
||||||
data = fetch_openvgdb_local(name)
|
data = None
|
||||||
if data is not None:
|
if system == "games":
|
||||||
log.info("cover %r <- openvgdb", name)
|
data = fetch_openvgdb_local(name)
|
||||||
elif not quick and time.time() < deadline:
|
if data is not None:
|
||||||
data = _fetch_screenscraper(name, t)
|
log.info("cover %r <- openvgdb", name)
|
||||||
|
if data is None and not quick and time.time() < deadline:
|
||||||
|
data = _fetch_screenscraper(name, t, system=system)
|
||||||
if data is not None:
|
if data is not None:
|
||||||
log.info("cover %r <- screenscraper", name)
|
log.info("cover %r <- screenscraper", name)
|
||||||
if data is None and not quick and time.time() < deadline:
|
if data is None and not quick and time.time() < deadline:
|
||||||
@@ -826,11 +876,12 @@ def fetch_cover(name, release_id, system, timeout=10, quick=False):
|
|||||||
if data is not None:
|
if data is not None:
|
||||||
log.info("cover %r <- wikipedia", name)
|
log.info("cover %r <- wikipedia", name)
|
||||||
if data is None and time.time() < deadline:
|
if data is None and time.time() < deadline:
|
||||||
data = _fetch_igdb_cover(name, t)
|
data = _fetch_igdb_cover(name, t, system=system)
|
||||||
if data is not None:
|
if data is not None:
|
||||||
# IGDB only has cropped 3:4 key art; composite it into the
|
if system == "games":
|
||||||
# full PSP cover template (header bar + artwork).
|
# IGDB only has cropped 3:4 PSP key art; composite it
|
||||||
data = compose_game_cover(data)
|
# into the full PSP cover template (header + artwork).
|
||||||
|
data = compose_game_cover(data)
|
||||||
log.info("cover %r <- igdb", name)
|
log.info("cover %r <- igdb", name)
|
||||||
|
|
||||||
if data is None:
|
if data is None:
|
||||||
@@ -918,6 +969,46 @@ def _search_wikipedia(query, limit=15, timeout=10):
|
|||||||
return results
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def _search_google_images(query, limit=15, timeout=10):
|
||||||
|
"""Return [{title, source, thumb_url, image_url}] from Google Images via the
|
||||||
|
Custom Search JSON API. Requires google_api_key + google_cx; keyless -> [].
|
||||||
|
"""
|
||||||
|
key = sources.load().get("google_api_key")
|
||||||
|
cx = sources.load().get("google_cx")
|
||||||
|
if not key or not cx:
|
||||||
|
return []
|
||||||
|
url = _GOOGLE_CSE_API + "?" + urllib.parse.urlencode({
|
||||||
|
"key": key,
|
||||||
|
"cx": cx,
|
||||||
|
"q": query,
|
||||||
|
"searchType": "image",
|
||||||
|
"num": min(max(limit, 1), 10),
|
||||||
|
"imgSize": "xlarge",
|
||||||
|
"safe": "off",
|
||||||
|
})
|
||||||
|
data = _http_get(url, timeout=timeout)
|
||||||
|
if not data:
|
||||||
|
return []
|
||||||
|
try:
|
||||||
|
items = json.loads(data).get("items") or []
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return []
|
||||||
|
out = []
|
||||||
|
for it in items:
|
||||||
|
link = it.get("link") or ""
|
||||||
|
thumb = ((it.get("image") or {}).get("thumbnailLink")) or link
|
||||||
|
if not link:
|
||||||
|
continue
|
||||||
|
out.append({
|
||||||
|
"id": None,
|
||||||
|
"title": it.get("title") or query,
|
||||||
|
"thumb_url": thumb,
|
||||||
|
"image_url": link,
|
||||||
|
"source": "google",
|
||||||
|
})
|
||||||
|
return out[:limit]
|
||||||
|
|
||||||
|
|
||||||
def _fetch_wikipedia_cover(name, timeout):
|
def _fetch_wikipedia_cover(name, timeout):
|
||||||
"""Return a game cover image from Wikipedia's File: namespace (keyless)."""
|
"""Return a game cover image from Wikipedia's File: namespace (keyless)."""
|
||||||
base = plain_title(name)
|
base = plain_title(name)
|
||||||
@@ -995,8 +1086,8 @@ def _wiki_file_search(query, limit, timeout):
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def _search_libretro(query, limit):
|
def _search_libretro(query, limit, system="games"):
|
||||||
names = _load_index().get("Named_Boxarts", [])
|
names = _load_index(system=system).get("Named_Boxarts", [])
|
||||||
db = _norm(query)
|
db = _norm(query)
|
||||||
results = []
|
results = []
|
||||||
if not db:
|
if not db:
|
||||||
@@ -1022,21 +1113,60 @@ def _search_libretro(query, limit):
|
|||||||
results.append({
|
results.append({
|
||||||
"title": n,
|
"title": n,
|
||||||
"source": "libretro",
|
"source": "libretro",
|
||||||
"thumb_url": cover_url(n),
|
"thumb_url": cover_url(n, system=system),
|
||||||
"image_url": cover_url(n),
|
"image_url": cover_url(n, system=system),
|
||||||
})
|
})
|
||||||
return results
|
return results
|
||||||
|
|
||||||
|
|
||||||
|
def _search_openvgdb(query, limit):
|
||||||
|
"""Return [{title, source, thumb_url, image_url}] from the offline OpenVGDB
|
||||||
|
bundle, using the display titles stored in openvgdb-index.json."""
|
||||||
|
names = _ovgdb_name_list()
|
||||||
|
if not names:
|
||||||
|
return []
|
||||||
|
index = _load_ovgdb_index()
|
||||||
|
db = _norm(query)
|
||||||
|
if not db:
|
||||||
|
return []
|
||||||
|
scored = []
|
||||||
|
for key in names:
|
||||||
|
if key == db:
|
||||||
|
score = 5
|
||||||
|
elif len(key) >= 4 and key.startswith(db):
|
||||||
|
score = 4
|
||||||
|
elif len(key) >= 4 and db in key:
|
||||||
|
score = 3
|
||||||
|
elif len(db) >= 4 and len(key) >= 4 and key in db:
|
||||||
|
score = 2
|
||||||
|
else:
|
||||||
|
continue
|
||||||
|
info = index.get(key) or {}
|
||||||
|
title = info.get("title") or key
|
||||||
|
scored.append((score, title, key))
|
||||||
|
scored.sort(key=lambda t: (-t[0], t[1]))
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"id": None,
|
||||||
|
"title": title,
|
||||||
|
"source": "openvgdb",
|
||||||
|
"thumb_url": "",
|
||||||
|
"image_url": "",
|
||||||
|
"_key": key,
|
||||||
|
}
|
||||||
|
for _score, title, key in scored[:limit]
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
# Game metadata (details sidebar)
|
# Game metadata (details sidebar)
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
_META_TTL = 86400.0
|
_META_TTL = 86400.0
|
||||||
|
|
||||||
|
|
||||||
def get_game_metadata(title, timeout=10):
|
def get_game_metadata(title, timeout=10, system="games"):
|
||||||
"""Return game metadata from IGDB."""
|
"""Return game metadata from IGDB (system picks the platform)."""
|
||||||
return get_game_metadata_igdb(title, timeout)
|
return get_game_metadata_igdb(title, timeout, system)
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
@@ -1161,28 +1291,30 @@ def _igdb_game_meta(g):
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
def _igdb_search(query, limit=20, timeout=10):
|
def _igdb_search(query, limit=20, timeout=10, system="games"):
|
||||||
cid, _ = _igdb_creds()
|
cid, _ = _igdb_creds()
|
||||||
if not cid:
|
if not cid:
|
||||||
return []
|
return []
|
||||||
term = plain_title(query) or query
|
term = plain_title(query) or query
|
||||||
|
platform_id = _platform(system)["igdb_id"]
|
||||||
q = (
|
q = (
|
||||||
f'search "{term}"; '
|
f'search "{term}"; '
|
||||||
"fields name, cover.image_id, first_release_date, summary, genres.name, "
|
"fields name, cover.image_id, first_release_date, summary, genres.name, "
|
||||||
"involved_companies.company.name, involved_companies.developer, involved_companies.publisher, "
|
"involved_companies.company.name, involved_companies.developer, involved_companies.publisher, "
|
||||||
"rating, videos.video_id; "
|
"rating, videos.video_id; "
|
||||||
f"where platforms = ({_IGDB_PSP_ID}); "
|
f"where platforms = ({platform_id}); "
|
||||||
f"limit {max(limit, 1)};"
|
f"limit {max(limit, 1)};"
|
||||||
)
|
)
|
||||||
return _igdb_post(q, timeout) or []
|
return _igdb_post(q, timeout) or []
|
||||||
|
|
||||||
|
|
||||||
def _fetch_igdb_cover(name, timeout):
|
def _fetch_igdb_cover(name, timeout, system="games"):
|
||||||
cid, _ = _igdb_creds()
|
cid, _ = _igdb_creds()
|
||||||
if not cid:
|
if not cid:
|
||||||
return None
|
return None
|
||||||
term = plain_title(name) or name
|
term = plain_title(name) or name
|
||||||
q = f'search "{term}"; fields cover.image_id; where platforms = ({_IGDB_PSP_ID}); limit 10;'
|
platform_id = _platform(system)["igdb_id"]
|
||||||
|
q = f'search "{term}"; fields cover.image_id; where platforms = ({platform_id}); limit 10;'
|
||||||
games = _igdb_post(q, timeout) or []
|
games = _igdb_post(q, timeout) or []
|
||||||
tried = set()
|
tried = set()
|
||||||
for g in games:
|
for g in games:
|
||||||
@@ -1196,9 +1328,9 @@ def _fetch_igdb_cover(name, timeout):
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
def search_games_igdb(query, limit=20, timeout=10):
|
def search_games_igdb(query, limit=20, timeout=10, system="games"):
|
||||||
out = []
|
out = []
|
||||||
for g in _igdb_search(query, limit, timeout):
|
for g in _igdb_search(query, limit, timeout, system):
|
||||||
cover = (g.get("cover") or {}).get("image_id") or ""
|
cover = (g.get("cover") or {}).get("image_id") or ""
|
||||||
out.append({
|
out.append({
|
||||||
"id": g.get("id"),
|
"id": g.get("id"),
|
||||||
@@ -1213,8 +1345,8 @@ def search_games_igdb(query, limit=20, timeout=10):
|
|||||||
return out[:limit]
|
return out[:limit]
|
||||||
|
|
||||||
|
|
||||||
def get_game_metadata_igdb(title, timeout=10):
|
def get_game_metadata_igdb(title, timeout=10, system="games"):
|
||||||
games = _igdb_search(title, limit=10, timeout=timeout)
|
games = _igdb_search(title, limit=10, timeout=timeout, system=system)
|
||||||
if not games:
|
if not games:
|
||||||
return None
|
return None
|
||||||
games.sort(key=lambda g: -_title_score(title, g.get("name") or ""))
|
games.sort(key=lambda g: -_title_score(title, g.get("name") or ""))
|
||||||
@@ -1334,11 +1466,19 @@ def get_movie_metadata(title, timeout=10):
|
|||||||
# Manual linking (pick the correct movie/game when the auto-match is wrong)
|
# Manual linking (pick the correct movie/game when the auto-match is wrong)
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
def search_movies(query, limit=20, timeout=10):
|
def search_movies(query, limit=20, timeout=10):
|
||||||
"""Return [{id, title, year, overview, thumb_url, source}] for a movie search.
|
"""Return movie candidates for a UMD video: TMDB first, then Wikipedia
|
||||||
|
images and Google Images (both keyless-optional). A Wikipedia page-title
|
||||||
TheMovieDB first; keyless Wikipedia page titles as a fallback.
|
fallback is added only when nothing else matched (metadata-only linking).
|
||||||
"""
|
"""
|
||||||
out = []
|
out = []
|
||||||
|
seen = set()
|
||||||
|
|
||||||
|
def add(r):
|
||||||
|
k = (r.get("source"), _norm(r.get("title") or ""))
|
||||||
|
if k[1] and k not in seen:
|
||||||
|
seen.add(k)
|
||||||
|
out.append(r)
|
||||||
|
|
||||||
key = sources.load().get("tmdb_api_key")
|
key = sources.load().get("tmdb_api_key")
|
||||||
if key:
|
if key:
|
||||||
url = _TMDB_SEARCH + "?" + urllib.parse.urlencode({"api_key": key, "query": query})
|
url = _TMDB_SEARCH + "?" + urllib.parse.urlencode({"api_key": key, "query": query})
|
||||||
@@ -1348,16 +1488,31 @@ def search_movies(query, limit=20, timeout=10):
|
|||||||
results = json.loads(data).get("results") or []
|
results = json.loads(data).get("results") or []
|
||||||
except Exception:
|
except Exception:
|
||||||
results = []
|
results = []
|
||||||
for r in results:
|
# Cap TMDB results so Wikipedia/Google image picks still fit.
|
||||||
|
for r in results[: max(1, min(limit, 6))]:
|
||||||
poster = r.get("poster_path")
|
poster = r.get("poster_path")
|
||||||
out.append({
|
add({
|
||||||
"id": r.get("id"),
|
"id": r.get("id"),
|
||||||
"title": r.get("title") or r.get("name") or "",
|
"title": r.get("title") or r.get("name") or "",
|
||||||
"year": (r.get("release_date") or "")[:4],
|
"year": (r.get("release_date") or "")[:4],
|
||||||
"overview": (r.get("overview") or "").strip(),
|
"overview": (r.get("overview") or "").strip(),
|
||||||
"thumb_url": (_TMDB_IMAGE + "w185" + poster) if poster else "",
|
"thumb_url": (_TMDB_IMAGE + "w185" + poster) if poster else "",
|
||||||
|
"image_url": (_TMDB_IMAGE + "w500" + poster) if poster else "",
|
||||||
"source": "tmdb",
|
"source": "tmdb",
|
||||||
})
|
})
|
||||||
|
|
||||||
|
# Wikipedia image search (keyless): real poster/cover files.
|
||||||
|
for r in _search_wikipedia(query, min(limit, 10), timeout):
|
||||||
|
r["id"] = None
|
||||||
|
r["movie_title"] = query
|
||||||
|
add(r)
|
||||||
|
|
||||||
|
# Google Images (Custom Search JSON API, when a key + cx are configured).
|
||||||
|
for r in _search_google_images(query, min(limit, 10), timeout):
|
||||||
|
r["movie_title"] = query
|
||||||
|
add(r)
|
||||||
|
|
||||||
|
# Last resort: Wikipedia page titles (metadata-only, no image).
|
||||||
if not out:
|
if not out:
|
||||||
for page in _wiki_search(query, limit):
|
for page in _wiki_search(query, limit):
|
||||||
out.append({
|
out.append({
|
||||||
@@ -1366,6 +1521,8 @@ def search_movies(query, limit=20, timeout=10):
|
|||||||
"year": "",
|
"year": "",
|
||||||
"overview": "",
|
"overview": "",
|
||||||
"thumb_url": "",
|
"thumb_url": "",
|
||||||
|
"image_url": "",
|
||||||
|
"movie_title": page,
|
||||||
"source": "wikipedia",
|
"source": "wikipedia",
|
||||||
})
|
})
|
||||||
return out[:limit]
|
return out[:limit]
|
||||||
@@ -1411,40 +1568,71 @@ def movie_metadata_from_wikipedia(page_title, timeout=10):
|
|||||||
|
|
||||||
|
|
||||||
def search_metadata_candidates(system, query, limit=20, timeout=10):
|
def search_metadata_candidates(system, query, limit=20, timeout=10):
|
||||||
"""Search candidates the user can link an item to."""
|
"""Search candidates the user can link an item to.
|
||||||
|
|
||||||
|
Games/PS1: IGDB (rich metadata) first, then libretro box art (and the
|
||||||
|
PSP-only OpenVGDB bundle) so covers can be scraped even without an IGDB key.
|
||||||
|
"""
|
||||||
if system == "videos":
|
if system == "videos":
|
||||||
return search_movies(query, limit, timeout)
|
return search_movies(query, limit, timeout)
|
||||||
results = search_games_igdb(query, limit, timeout)
|
results = []
|
||||||
if results:
|
seen = set()
|
||||||
return results
|
|
||||||
# No IGDB results (or no key): fall back to libretro catalog names so the
|
def add(r):
|
||||||
# user can still link to the correct entry (title-only).
|
key = (r.get("source"), _norm(r.get("title") or ""))
|
||||||
results = _search_libretro(query, limit)
|
if key[1] and key not in seen:
|
||||||
for r in results:
|
seen.add(key)
|
||||||
r["source"] = "libretro"
|
results.append(r)
|
||||||
r["id"] = None
|
|
||||||
return results
|
# Cap IGDB so libretro/OpenVGDB cover picks still fit in the list.
|
||||||
|
for r in search_games_igdb(query, max(1, min(limit, 6)), timeout, system):
|
||||||
|
add(r)
|
||||||
|
for r in _search_libretro(query, limit, system):
|
||||||
|
r.setdefault("id", None)
|
||||||
|
add(r)
|
||||||
|
if system == "games":
|
||||||
|
for r in _search_openvgdb(query, limit):
|
||||||
|
add(r)
|
||||||
|
return results[:limit]
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_meta(title=""):
|
||||||
|
return {
|
||||||
|
"title": title,
|
||||||
|
"release_date": "",
|
||||||
|
"overview": "",
|
||||||
|
"genres": [],
|
||||||
|
"rating": None,
|
||||||
|
"youtube": "",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def link_metadata(system, candidate, timeout=10):
|
def link_metadata(system, candidate, timeout=10):
|
||||||
"""Fetch full metadata for a search candidate the user picked."""
|
"""Fetch full metadata for a search candidate the user picked."""
|
||||||
if candidate.get("source") == "wikipedia":
|
src = candidate.get("source")
|
||||||
return movie_metadata_from_wikipedia(candidate.get("title") or "", timeout)
|
if src == "wikipedia":
|
||||||
if candidate.get("source") == "libretro":
|
title = candidate.get("movie_title") or candidate.get("title") or ""
|
||||||
# Catalog-name link only: no metadata backend, just the correct title.
|
meta = movie_metadata_from_wikipedia(title, timeout) if title else None
|
||||||
return {
|
if meta:
|
||||||
"title": candidate.get("title") or "",
|
return meta
|
||||||
"release_date": "",
|
if system == "videos":
|
||||||
"overview": "",
|
return get_movie_metadata(title, timeout) or _empty_meta(title)
|
||||||
"genres": [],
|
return _empty_meta(title)
|
||||||
"rating": None,
|
if src in ("libretro", "openvgdb"):
|
||||||
"youtube": "",
|
# Cover repositories have no metadata backend; try IGDB by title so the
|
||||||
}
|
# user still gets synopsis/genre/etc. when a key is configured.
|
||||||
|
title = candidate.get("title") or ""
|
||||||
|
if title:
|
||||||
|
meta = get_game_metadata_igdb(title, timeout, system)
|
||||||
|
if meta:
|
||||||
|
return meta
|
||||||
|
return _empty_meta(title)
|
||||||
if system == "videos":
|
if system == "videos":
|
||||||
if candidate.get("id") is not None:
|
if src == "tmdb" and candidate.get("id") is not None:
|
||||||
return movie_metadata_by_id(candidate["id"], timeout)
|
return movie_metadata_by_id(candidate["id"], timeout)
|
||||||
return movie_metadata_from_wikipedia(candidate.get("title") or "", timeout)
|
title = candidate.get("movie_title") or candidate.get("title") or ""
|
||||||
if candidate.get("id") is not None and candidate.get("source") == "igdb":
|
return get_movie_metadata(title, timeout) or _empty_meta(title)
|
||||||
|
if src == "igdb" and candidate.get("id") is not None:
|
||||||
return game_metadata_by_id_igdb(candidate["id"], timeout)
|
return game_metadata_by_id_igdb(candidate["id"], timeout)
|
||||||
return None
|
return _empty_meta(candidate.get("title") or "")
|
||||||
|
|
||||||
|
|||||||
+72
-16
@@ -1,23 +1,29 @@
|
|||||||
"""Parser for clrmamepro (.dat) files.
|
"""Parser for clrmamepro / Logiqx XML (.dat) files.
|
||||||
|
|
||||||
Handles the No-Intro / Redump text DAT format::
|
Supports both DAT layouts the project uses:
|
||||||
|
|
||||||
clrmamepro (
|
* clrmamepro text DAT (No-Intro): parenthesised blocks, e.g.::
|
||||||
name "Sony - PlayStation Portable"
|
|
||||||
version "2021.10.24"
|
|
||||||
)
|
|
||||||
game (
|
|
||||||
name "WipEout Pure (Europe)"
|
|
||||||
region "Europe"
|
|
||||||
serial "ULES-00011"
|
|
||||||
rom ( name "WipEout Pure (Europe).iso" size 123 crc ABCD1234 md5 ... sha1 ... )
|
|
||||||
)
|
|
||||||
|
|
||||||
``parse_dat_text`` returns ``(header, entries)``. Each entry is a dict with
|
clrmamepro (
|
||||||
scalar keys (``name``, ``description``, ``region``, ``serial``, ...) and
|
name "Sony - PlayStation Portable"
|
||||||
repeated nested blocks exposed as lists under their block key (``rom``).
|
version "2021.10.24"
|
||||||
|
)
|
||||||
|
game (
|
||||||
|
name "WipEout Pure (Europe)"
|
||||||
|
region "Europe"
|
||||||
|
serial "ULES-00011"
|
||||||
|
rom ( name "WipEout Pure (Europe).iso" size 123 crc ABCD1234 md5 ... sha1 ... )
|
||||||
|
)
|
||||||
|
|
||||||
|
* Logiqx XML DAT (Redump): ``<game>`` elements with ``<rom>`` children.
|
||||||
|
|
||||||
|
Both ``parse_dat_text`` and ``parse_dat_xml`` return ``(header, entries)``.
|
||||||
|
Each entry is a dict with scalar keys (``name``, ``description``, ``region``,
|
||||||
|
``serial``, ...) and repeated nested blocks exposed as lists under their block
|
||||||
|
key (``rom``).
|
||||||
"""
|
"""
|
||||||
import re
|
import re
|
||||||
|
import xml.etree.ElementTree as ET
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
_TOKEN_RE = re.compile(
|
_TOKEN_RE = re.compile(
|
||||||
@@ -124,5 +130,55 @@ def parse_dat_text(text):
|
|||||||
return header, entries
|
return header, entries
|
||||||
|
|
||||||
|
|
||||||
|
def _xml_text(element, key):
|
||||||
|
child = element.find(key)
|
||||||
|
if child is not None and child.text:
|
||||||
|
return child.text.strip()
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def parse_dat_xml(text):
|
||||||
|
"""Parse a Logiqx XML DAT (the format redump.org uses).
|
||||||
|
|
||||||
|
Returns ``(header, entries)`` in the same shape as :func:`parse_dat_text`.
|
||||||
|
"""
|
||||||
|
root = ET.fromstring(text)
|
||||||
|
|
||||||
|
header = {}
|
||||||
|
header_el = root.find("header")
|
||||||
|
if header_el is not None:
|
||||||
|
for child in header_el:
|
||||||
|
if child.text:
|
||||||
|
header[child.tag] = child.text.strip()
|
||||||
|
|
||||||
|
entries = []
|
||||||
|
for game in root.iter():
|
||||||
|
if game.tag not in ("game", "machine"):
|
||||||
|
continue
|
||||||
|
entry = {"name": (game.get("name") or "").strip()}
|
||||||
|
for key in ("description", "serial", "region", "category"):
|
||||||
|
value = _xml_text(game, key)
|
||||||
|
if value:
|
||||||
|
entry[key] = value
|
||||||
|
entry["rom"] = [
|
||||||
|
{
|
||||||
|
"name": rom.get("name", ""),
|
||||||
|
"size": rom.get("size"),
|
||||||
|
"crc": rom.get("crc"),
|
||||||
|
"md5": rom.get("md5"),
|
||||||
|
"sha1": rom.get("sha1"),
|
||||||
|
"serial": rom.get("serial"),
|
||||||
|
}
|
||||||
|
for rom in game.findall("rom")
|
||||||
|
]
|
||||||
|
entries.append(entry)
|
||||||
|
|
||||||
|
return header, entries
|
||||||
|
|
||||||
|
|
||||||
def parse_dat_file(path):
|
def parse_dat_file(path):
|
||||||
return parse_dat_text(Path(path).read_text(encoding="utf-8-sig"))
|
text = Path(path).read_text(encoding="utf-8-sig")
|
||||||
|
stripped = text.lstrip()
|
||||||
|
if stripped.startswith("<?xml") or stripped.startswith("<"):
|
||||||
|
return parse_dat_xml(text)
|
||||||
|
return parse_dat_text(text)
|
||||||
|
|||||||
@@ -101,6 +101,20 @@ CREATE TABLE IF NOT EXISTS save_sigs (
|
|||||||
sig TEXT NOT NULL,
|
sig TEXT NOT NULL,
|
||||||
PRIMARY KEY (save_id, size, mtime_ns)
|
PRIMARY KEY (save_id, size, mtime_ns)
|
||||||
);
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS ps1_entries (
|
||||||
|
id INTEGER PRIMARY KEY,
|
||||||
|
source_path TEXT NOT NULL UNIQUE,
|
||||||
|
title TEXT,
|
||||||
|
serial TEXT,
|
||||||
|
region TEXT,
|
||||||
|
disc INTEGER NOT NULL DEFAULT 1,
|
||||||
|
disc_total INTEGER NOT NULL DEFAULT 1,
|
||||||
|
output_eboot TEXT,
|
||||||
|
status TEXT NOT NULL DEFAULT 'pending',
|
||||||
|
error TEXT,
|
||||||
|
updated_at TEXT NOT NULL
|
||||||
|
);
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+287
-18
@@ -14,13 +14,13 @@ from PySide6.QtCore import QBuffer, QByteArray, QObject, QRect, QSettings, QSize
|
|||||||
from PySide6.QtGui import QColor, QDesktopServices, QFont, QIcon, QImage, QPainter, QPixmap
|
from PySide6.QtGui import QColor, QDesktopServices, QFont, QIcon, QImage, QPainter, QPixmap
|
||||||
from PySide6.QtWidgets import (
|
from PySide6.QtWidgets import (
|
||||||
QAbstractItemView, QApplication, QCheckBox, QComboBox, QDialog, QDialogButtonBox,
|
QAbstractItemView, QApplication, QCheckBox, QComboBox, QDialog, QDialogButtonBox,
|
||||||
QFileDialog, QFormLayout, QGridLayout, QHBoxLayout, QHeaderView, QLabel, QLineEdit,
|
QFileDialog, QFormLayout, QGridLayout, QHBoxLayout, QHeaderView, QInputDialog, QLabel, QLineEdit,
|
||||||
QListWidget, QListWidgetItem, QMainWindow, QMenu, QMessageBox, QPlainTextEdit,
|
QListWidget, QListWidgetItem, QMainWindow, QMenu, QMessageBox, QPlainTextEdit,
|
||||||
QProgressBar, QProgressDialog, QPushButton, QScrollArea, QSpinBox, QSplitter, QStackedWidget,
|
QProgressBar, QProgressDialog, QPushButton, QScrollArea, QSpinBox, QSplitter, QStackedWidget,
|
||||||
QTabWidget, QTableWidget, QTableWidgetItem, QVBoxLayout, QWidget,
|
QTabWidget, QTableWidget, QTableWidgetItem, QVBoxLayout, QWidget,
|
||||||
)
|
)
|
||||||
|
|
||||||
from . import config, covers, info, iso_reader, launcher, matcher, namer, pipeline, psp, queue, saves, scanner, sources, store, transfers
|
from . import config, covers, info, iso_reader, launcher, matcher, namer, pipeline, ps1, psp, queue, saves, scanner, sources, store, transfers
|
||||||
from .db import get_connection, init_db
|
from .db import get_connection, init_db
|
||||||
|
|
||||||
log = logging.getLogger("gui")
|
log = logging.getLogger("gui")
|
||||||
@@ -559,6 +559,7 @@ class MetadataWorker(QObject):
|
|||||||
class GameDetailsPanel(QWidget):
|
class GameDetailsPanel(QWidget):
|
||||||
edit_requested = Signal() # emitted when the user wants to hand-tag the item
|
edit_requested = Signal() # emitted when the user wants to hand-tag the item
|
||||||
link_requested = Signal() # emitted when the user wants to pick a movie/game to link
|
link_requested = Signal() # emitted when the user wants to pick a movie/game to link
|
||||||
|
convert_requested = Signal() # emitted when the user wants to convert a PS1 disc to EBOOT
|
||||||
copy_requested = Signal() # emitted when the user wants to copy the current item to the PSP
|
copy_requested = Signal() # emitted when the user wants to copy the current item to the PSP
|
||||||
delete_requested = Signal() # emitted when the user wants to delete the current item
|
delete_requested = Signal() # emitted when the user wants to delete the current item
|
||||||
play_requested = Signal() # emitted when the user wants to launch the item in PPSSPP
|
play_requested = Signal() # emitted when the user wants to launch the item in PPSSPP
|
||||||
@@ -569,6 +570,7 @@ class GameDetailsPanel(QWidget):
|
|||||||
self.setFixedWidth(300)
|
self.setFixedWidth(300)
|
||||||
self._youtube_url = ""
|
self._youtube_url = ""
|
||||||
self._meta = None
|
self._meta = None
|
||||||
|
self._copy_available = False
|
||||||
|
|
||||||
outer = QVBoxLayout(self)
|
outer = QVBoxLayout(self)
|
||||||
outer.setContentsMargins(12, 12, 0, 12)
|
outer.setContentsMargins(12, 12, 0, 12)
|
||||||
@@ -627,6 +629,12 @@ class GameDetailsPanel(QWidget):
|
|||||||
self.play_btn.setToolTip("Launch this game in PPSSPP")
|
self.play_btn.setToolTip("Launch this game in PPSSPP")
|
||||||
self.layout.addWidget(self.play_btn)
|
self.layout.addWidget(self.play_btn)
|
||||||
|
|
||||||
|
self.convert_btn = QPushButton("\u2699 Convert to EBOOT")
|
||||||
|
self.convert_btn.clicked.connect(self.convert_requested.emit)
|
||||||
|
self.convert_btn.setVisible(False)
|
||||||
|
self.convert_btn.setToolTip("Convert this PS1 disc to an EBOOT.PBP")
|
||||||
|
self.layout.addWidget(self.convert_btn)
|
||||||
|
|
||||||
btn_grid = QGridLayout()
|
btn_grid = QGridLayout()
|
||||||
btn_grid.setHorizontalSpacing(6)
|
btn_grid.setHorizontalSpacing(6)
|
||||||
btn_grid.setVerticalSpacing(6)
|
btn_grid.setVerticalSpacing(6)
|
||||||
@@ -658,14 +666,28 @@ class GameDetailsPanel(QWidget):
|
|||||||
|
|
||||||
def set_item(self, item):
|
def set_item(self, item):
|
||||||
self.system = item.get("system")
|
self.system = item.get("system")
|
||||||
|
self._ps1_kind = item.get("ps1_kind")
|
||||||
path = item.get("path") or ""
|
path = item.get("path") or ""
|
||||||
self.play_btn.setEnabled(item.get("system") == "games" and bool(path) and Path(path).suffix.lower() == ".iso")
|
is_ps1 = item.get("system") == "ps1"
|
||||||
|
is_ps1_disc = is_ps1 and item.get("ps1_kind") == "disc"
|
||||||
|
self.play_btn.setEnabled(
|
||||||
|
(item.get("system") == "games" and bool(path) and Path(path).suffix.lower() == ".iso")
|
||||||
|
or (is_ps1 and item.get("ps1_kind") == "eboot" and bool(path))
|
||||||
|
)
|
||||||
|
self.play_btn.setVisible(not is_ps1_disc)
|
||||||
|
self.convert_btn.setVisible(is_ps1_disc)
|
||||||
|
self.copy_btn.setEnabled(self._copy_available and not is_ps1_disc)
|
||||||
|
self.copy_btn.setToolTip(
|
||||||
|
"Copy this EBOOT to /PSP/GAME" if is_ps1 else "Copy this disc to the connected PSP"
|
||||||
|
)
|
||||||
self.del_psp_btn.setEnabled(bool(item.get("on_psp")))
|
self.del_psp_btn.setEnabled(bool(item.get("on_psp")))
|
||||||
self.title.setText(item.get("title_override") or item.get("name") or "Unknown")
|
self.title.setText(item.get("title_override") or item.get("name") or "Unknown")
|
||||||
parts = []
|
parts = []
|
||||||
if item.get("serial"):
|
if item.get("serial"):
|
||||||
parts.append(item["serial"])
|
parts.append(item["serial"])
|
||||||
if item.get("region"):
|
if is_ps1 and item.get("ps1_status"):
|
||||||
|
parts.append(item["ps1_status"])
|
||||||
|
elif item.get("region"):
|
||||||
parts.append(item["region"])
|
parts.append(item["region"])
|
||||||
self.subtitle.setText(" · ".join(parts))
|
self.subtitle.setText(" · ".join(parts))
|
||||||
self._clear_meta()
|
self._clear_meta()
|
||||||
@@ -676,7 +698,8 @@ class GameDetailsPanel(QWidget):
|
|||||||
self._set_cover(item.get("release_id"), item.get("system"))
|
self._set_cover(item.get("release_id"), item.get("system"))
|
||||||
|
|
||||||
def set_copy_available(self, available):
|
def set_copy_available(self, available):
|
||||||
self.copy_btn.setEnabled(bool(available))
|
self._copy_available = bool(available)
|
||||||
|
self.copy_btn.setEnabled(self._copy_available and getattr(self, "_ps1_kind", None) != "disc")
|
||||||
|
|
||||||
def set_psp_installed(self, on_psp):
|
def set_psp_installed(self, on_psp):
|
||||||
self.del_psp_btn.setEnabled(bool(on_psp))
|
self.del_psp_btn.setEnabled(bool(on_psp))
|
||||||
@@ -824,8 +847,17 @@ class ScrapeSearchWorker(QObject):
|
|||||||
def run(self):
|
def run(self):
|
||||||
found = covers.search_metadata_candidates(self.system, self.query, limit=20, timeout=10)
|
found = covers.search_metadata_candidates(self.system, self.query, limit=20, timeout=10)
|
||||||
for r in found:
|
for r in found:
|
||||||
url = r.get("thumb_url")
|
src = r.get("source")
|
||||||
r["thumb_bytes"] = covers.download_image(url, timeout=8) if url else None
|
if src == "openvgdb":
|
||||||
|
r["thumb_bytes"] = covers.fetch_openvgdb_local(r.get("title") or "")
|
||||||
|
elif src == "libretro":
|
||||||
|
data = covers.fetch_libretro_local(r.get("title") or "")
|
||||||
|
r["thumb_bytes"] = data if data else (
|
||||||
|
covers.download_image(r.get("thumb_url"), timeout=8) if r.get("thumb_url") else None
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
url = r.get("thumb_url")
|
||||||
|
r["thumb_bytes"] = covers.download_image(url, timeout=8) if url else None
|
||||||
self.results.emit(found)
|
self.results.emit(found)
|
||||||
self.finished.emit()
|
self.finished.emit()
|
||||||
|
|
||||||
@@ -849,16 +881,26 @@ class ScrapeFetchWorker(QObject):
|
|||||||
|
|
||||||
def _scrape_cover(system, candidate, rid, timeout=20):
|
def _scrape_cover(system, candidate, rid, timeout=20):
|
||||||
"""Download the cover for a metadata candidate; returns bytes or None."""
|
"""Download the cover for a metadata candidate; returns bytes or None."""
|
||||||
|
src = candidate.get("source")
|
||||||
|
title = candidate.get("title") or ""
|
||||||
|
# Cover repositories ship offline bundles — read them directly (no network).
|
||||||
|
if src == "openvgdb" and title:
|
||||||
|
data = covers.fetch_openvgdb_local(title)
|
||||||
|
if data:
|
||||||
|
return data
|
||||||
|
if src == "libretro" and title:
|
||||||
|
data = covers.fetch_libretro_local(title)
|
||||||
|
if data:
|
||||||
|
return data
|
||||||
url = candidate.get("image_url")
|
url = candidate.get("image_url")
|
||||||
if url:
|
if url:
|
||||||
data = covers.download_image(url, timeout)
|
data = covers.download_image(url, timeout)
|
||||||
if data:
|
if data:
|
||||||
if system == "videos":
|
if system == "videos":
|
||||||
return compose_umd_cover(data)
|
return compose_umd_cover(data)
|
||||||
if candidate.get("source") == "igdb":
|
if candidate.get("source") == "igdb" and system == "games":
|
||||||
return covers.compose_game_cover(data)
|
return covers.compose_game_cover(data)
|
||||||
return data
|
return data
|
||||||
title = candidate.get("title") or ""
|
|
||||||
if title:
|
if title:
|
||||||
path = covers.fetch_cover(title, rid, system, timeout=timeout)
|
path = covers.fetch_cover(title, rid, system, timeout=timeout)
|
||||||
if path is not None:
|
if path is not None:
|
||||||
@@ -955,6 +997,8 @@ class ScrapeDialog(QDialog):
|
|||||||
icon = QIcon(pm)
|
icon = QIcon(pm)
|
||||||
year = r.get("year") or (r.get("release_date") or "")[:4]
|
year = r.get("year") or (r.get("release_date") or "")[:4]
|
||||||
label = f"{r['title']} ({year})" if year else r["title"]
|
label = f"{r['title']} ({year})" if year else r["title"]
|
||||||
|
if r.get("source") not in (None, "igdb", "tmdb"):
|
||||||
|
label += f" [{r['source']}]"
|
||||||
item = QListWidgetItem(icon, label)
|
item = QListWidgetItem(icon, label)
|
||||||
if r.get("overview"):
|
if r.get("overview"):
|
||||||
item.setToolTip(r["overview"])
|
item.setToolTip(r["overview"])
|
||||||
@@ -1011,7 +1055,7 @@ class SettingsPage(QWidget):
|
|||||||
|
|
||||||
def __init__(self, games_dir, videos_dir, store_dir, ppsspp_dir, cloud_dir, ppsspp_exe,
|
def __init__(self, games_dir, videos_dir, store_dir, ppsspp_dir, cloud_dir, ppsspp_exe,
|
||||||
store_server_url, store_token, store_archive_id, store_archive_sig, store_archive_user,
|
store_server_url, store_token, store_archive_id, store_archive_sig, store_archive_user,
|
||||||
iso_category, parent=None):
|
iso_category, ps1_dir, ps1_output_dir, ps1_base_pbp, parent=None):
|
||||||
super().__init__(parent)
|
super().__init__(parent)
|
||||||
outer = QVBoxLayout(self)
|
outer = QVBoxLayout(self)
|
||||||
outer.setContentsMargins(0, 0, 0, 0)
|
outer.setContentsMargins(0, 0, 0, 0)
|
||||||
@@ -1025,6 +1069,9 @@ class SettingsPage(QWidget):
|
|||||||
page, form = self._new_form()
|
page, form = self._new_form()
|
||||||
self.games_edit = self._dir_row(form, "Games directory:", games_dir)
|
self.games_edit = self._dir_row(form, "Games directory:", games_dir)
|
||||||
self.videos_edit = self._dir_row(form, "Videos directory:", videos_dir)
|
self.videos_edit = self._dir_row(form, "Videos directory:", videos_dir)
|
||||||
|
self.ps1_edit = self._dir_row(form, "PS1 directory:", ps1_dir)
|
||||||
|
self.ps1_out_edit = self._dir_row(form, "PS1 output directory:", ps1_output_dir)
|
||||||
|
self.ps1_base_edit = self._file_row(form, "PS1 base.pbp (PSN classic):", ps1_base_pbp)
|
||||||
self.auto_scan = QCheckBox("Scan directories automatically on startup")
|
self.auto_scan = QCheckBox("Scan directories automatically on startup")
|
||||||
self.auto_scan.setChecked(str(s.value("auto_scan", False)).lower() == "true")
|
self.auto_scan.setChecked(str(s.value("auto_scan", False)).lower() == "true")
|
||||||
form.addRow(self.auto_scan)
|
form.addRow(self.auto_scan)
|
||||||
@@ -1094,6 +1141,8 @@ class SettingsPage(QWidget):
|
|||||||
|
|
||||||
form.addRow(QLabel("<b>Videos — metadata & posters</b>"))
|
form.addRow(QLabel("<b>Videos — metadata & posters</b>"))
|
||||||
add_field("TheMovieDB API key:", "tmdb_api_key", password=True)
|
add_field("TheMovieDB API key:", "tmdb_api_key", password=True)
|
||||||
|
add_field("Google API key:", "google_api_key", password=True)
|
||||||
|
add_field("Google Search Engine ID (cx):", "google_cx")
|
||||||
|
|
||||||
hint = QLabel(
|
hint = QLabel(
|
||||||
"Manual UMD covers: drop images into "
|
"Manual UMD covers: drop images into "
|
||||||
@@ -1156,7 +1205,7 @@ class SettingsPage(QWidget):
|
|||||||
return edit
|
return edit
|
||||||
|
|
||||||
def _browse_file(self, edit):
|
def _browse_file(self, edit):
|
||||||
path, _ = QFileDialog.getOpenFileName(self, "Select PPSSPP executable")
|
path, _ = QFileDialog.getOpenFileName(self, "Select file")
|
||||||
if path:
|
if path:
|
||||||
edit.setText(path)
|
edit.setText(path)
|
||||||
|
|
||||||
@@ -1210,6 +1259,15 @@ class SettingsPage(QWidget):
|
|||||||
def iso_category(self):
|
def iso_category(self):
|
||||||
return self.iso_category_edit.text().strip()
|
return self.iso_category_edit.text().strip()
|
||||||
|
|
||||||
|
def ps1_dir(self):
|
||||||
|
return self.ps1_edit.text().strip()
|
||||||
|
|
||||||
|
def ps1_output_dir(self):
|
||||||
|
return self.ps1_out_edit.text().strip()
|
||||||
|
|
||||||
|
def ps1_base_pbp(self):
|
||||||
|
return self.ps1_base_edit.text().strip()
|
||||||
|
|
||||||
def _save(self):
|
def _save(self):
|
||||||
sources.save({key: edit.text() for key, edit in self.fields.items()})
|
sources.save({key: edit.text() for key, edit in self.fields.items()})
|
||||||
covers.clear_miss_cache()
|
covers.clear_miss_cache()
|
||||||
@@ -2989,7 +3047,7 @@ class PspSyncPage(QWidget):
|
|||||||
self._workers = []
|
self._workers = []
|
||||||
|
|
||||||
|
|
||||||
def build_library_items():
|
def build_library_items(ps1_dir="", output_dir=""):
|
||||||
"""Build the library item list from the DB with a single joined query."""
|
"""Build the library item list from the DB with a single joined query."""
|
||||||
items = []
|
items = []
|
||||||
seen = set()
|
seen = set()
|
||||||
@@ -3058,16 +3116,59 @@ def build_library_items():
|
|||||||
conn.commit()
|
conn.commit()
|
||||||
finally:
|
finally:
|
||||||
conn.close()
|
conn.close()
|
||||||
|
items.extend(_ps1_library_items(ps1_dir, output_dir, overrides, dates))
|
||||||
return items
|
return items
|
||||||
|
|
||||||
|
|
||||||
|
def _ps1_library_items(ps1_dir, output_dir, overrides, dates):
|
||||||
|
"""PS1 discs (pending) + converted EBOOTs, merged into the library list."""
|
||||||
|
if not ps1_dir:
|
||||||
|
return []
|
||||||
|
try:
|
||||||
|
entries = ps1.scan_library(ps1_dir, output_dir)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return []
|
||||||
|
out = []
|
||||||
|
for e in entries:
|
||||||
|
# A converted disc is represented by its EBOOT entry (one row per game).
|
||||||
|
if e["kind"] == "disc" and e.get("status") == "converted":
|
||||||
|
continue
|
||||||
|
name = e["name"]
|
||||||
|
rid = covers.stable_key(name)
|
||||||
|
path = Path(e["path"])
|
||||||
|
try:
|
||||||
|
size = path.stat().st_size if path.exists() else 0
|
||||||
|
except OSError:
|
||||||
|
size = 0
|
||||||
|
out.append({
|
||||||
|
"name": name,
|
||||||
|
"release_id": rid,
|
||||||
|
"system": "ps1",
|
||||||
|
"canonical": name,
|
||||||
|
"serial": e.get("serial") or "",
|
||||||
|
"region": None,
|
||||||
|
"title_override": overrides.get(("ps1", str(rid))),
|
||||||
|
"release_date": dates.get((str(rid), "ps1")) or "",
|
||||||
|
"path": str(path),
|
||||||
|
"size": size,
|
||||||
|
"ps1_kind": e["kind"],
|
||||||
|
"ps1_status": e.get("status") or "pending",
|
||||||
|
})
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
class LibraryWorker(QObject):
|
class LibraryWorker(QObject):
|
||||||
result = Signal(list)
|
result = Signal(list)
|
||||||
finished = Signal()
|
finished = Signal()
|
||||||
|
|
||||||
|
def __init__(self, ps1_dir="", output_dir=""):
|
||||||
|
super().__init__()
|
||||||
|
self.ps1_dir = ps1_dir
|
||||||
|
self.output_dir = output_dir
|
||||||
|
|
||||||
def run(self):
|
def run(self):
|
||||||
try:
|
try:
|
||||||
items = build_library_items()
|
items = build_library_items(self.ps1_dir, self.output_dir)
|
||||||
except Exception: # noqa: BLE001
|
except Exception: # noqa: BLE001
|
||||||
items = []
|
items = []
|
||||||
self.result.emit(items)
|
self.result.emit(items)
|
||||||
@@ -3220,6 +3321,113 @@ class SavesPage(QWidget):
|
|||||||
self._workers = []
|
self._workers = []
|
||||||
|
|
||||||
|
|
||||||
|
class Ps1ScanWorker(QObject):
|
||||||
|
results = Signal(list)
|
||||||
|
failed = Signal(str)
|
||||||
|
|
||||||
|
def __init__(self, ps1_dir, output_dir):
|
||||||
|
super().__init__()
|
||||||
|
self.ps1_dir = ps1_dir
|
||||||
|
self.output_dir = output_dir
|
||||||
|
|
||||||
|
def run(self):
|
||||||
|
try:
|
||||||
|
self.results.emit(ps1.scan_library(self.ps1_dir, self.output_dir))
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
self.failed.emit(str(exc))
|
||||||
|
|
||||||
|
|
||||||
|
class Ps1Page(QWidget):
|
||||||
|
"""PS1 tab: list raw source discs, converted EBOOTs and PSN games."""
|
||||||
|
|
||||||
|
convert_requested = Signal(dict)
|
||||||
|
|
||||||
|
def __init__(self, parent=None):
|
||||||
|
super().__init__(parent)
|
||||||
|
self._ps1_dir = ""
|
||||||
|
self._output_dir = ""
|
||||||
|
self._entries = []
|
||||||
|
self._thread = None
|
||||||
|
self._worker = None
|
||||||
|
|
||||||
|
layout = QVBoxLayout(self)
|
||||||
|
bar = QHBoxLayout()
|
||||||
|
self.status = QLabel("PS1 discs — set the PS1 folder in Settings → General.")
|
||||||
|
bar.addWidget(self.status, 1)
|
||||||
|
self.refresh_btn = QPushButton("Refresh")
|
||||||
|
self.refresh_btn.clicked.connect(self.refresh)
|
||||||
|
self.convert_btn = QPushButton("Convert selected")
|
||||||
|
self.convert_btn.clicked.connect(self._convert_selected)
|
||||||
|
bar.addWidget(self.refresh_btn)
|
||||||
|
bar.addWidget(self.convert_btn)
|
||||||
|
layout.addLayout(bar)
|
||||||
|
|
||||||
|
self.table = QTableWidget(0, 4)
|
||||||
|
self.table.setHorizontalHeaderLabels(["Type", "Name", "Serial", "Status"])
|
||||||
|
self.table.setSelectionBehavior(QAbstractItemView.SelectRows)
|
||||||
|
self.table.setEditTriggers(QAbstractItemView.NoEditTriggers)
|
||||||
|
self.table.verticalHeader().setVisible(False)
|
||||||
|
self.table.horizontalHeader().setSectionResizeMode(1, QHeaderView.Stretch)
|
||||||
|
layout.addWidget(self.table, 1)
|
||||||
|
|
||||||
|
def set_dirs(self, ps1_dir, output_dir):
|
||||||
|
self._ps1_dir = ps1_dir
|
||||||
|
self._output_dir = output_dir
|
||||||
|
|
||||||
|
def refresh(self):
|
||||||
|
if not self._ps1_dir:
|
||||||
|
self.status.setText("PS1 folder not set — configure it in Settings → General.")
|
||||||
|
self.table.setRowCount(0)
|
||||||
|
self._entries = []
|
||||||
|
return
|
||||||
|
if self._thread is not None and _thread_running(self._thread):
|
||||||
|
return
|
||||||
|
self.status.setText("Scanning PS1 folders…")
|
||||||
|
self._worker = Ps1ScanWorker(self._ps1_dir, self._output_dir)
|
||||||
|
self._thread = QThread(self)
|
||||||
|
self._worker.moveToThread(self._thread)
|
||||||
|
self._thread.started.connect(self._worker.run)
|
||||||
|
self._worker.results.connect(self._on_results)
|
||||||
|
self._worker.failed.connect(self._on_failed)
|
||||||
|
self._worker.results.connect(self._thread.quit)
|
||||||
|
self._worker.failed.connect(self._thread.quit)
|
||||||
|
self._thread.start()
|
||||||
|
|
||||||
|
def _on_results(self, entries):
|
||||||
|
self._entries = entries
|
||||||
|
self.table.setRowCount(len(entries))
|
||||||
|
for row, e in enumerate(entries):
|
||||||
|
kind = "Disc" if e["kind"] == "disc" else "EBOOT"
|
||||||
|
self.table.setItem(row, 0, QTableWidgetItem(kind))
|
||||||
|
self.table.setItem(row, 1, QTableWidgetItem(e["name"]))
|
||||||
|
self.table.setItem(row, 2, QTableWidgetItem(e.get("serial") or ""))
|
||||||
|
self.table.setItem(row, 3, QTableWidgetItem(e.get("status") or "pending"))
|
||||||
|
pending = sum(1 for e in entries if e["kind"] == "disc" and e["status"] != "converted")
|
||||||
|
self.status.setText(f"{len(entries)} item(s) — {pending} ready to convert")
|
||||||
|
|
||||||
|
def _on_failed(self, message):
|
||||||
|
self.status.setText(f"PS1 scan failed: {message}")
|
||||||
|
|
||||||
|
def _convert_selected(self):
|
||||||
|
row = self.table.currentRow()
|
||||||
|
if row < 0 or row >= len(self._entries):
|
||||||
|
return
|
||||||
|
e = self._entries[row]
|
||||||
|
if e["kind"] != "disc":
|
||||||
|
QMessageBox.information(self, "PS1", "Select a raw PS1 disc (Disc row), not an EBOOT.")
|
||||||
|
return
|
||||||
|
if e["status"] == "converted":
|
||||||
|
QMessageBox.information(self, "PS1", "This disc already has a converted EBOOT.")
|
||||||
|
return
|
||||||
|
game_id = e.get("serial") or ""
|
||||||
|
if not game_id:
|
||||||
|
text, ok = QInputDialog.getText(self, "PS1", "Game ID (e.g. SLUS00594):")
|
||||||
|
if not ok or not text.strip():
|
||||||
|
return
|
||||||
|
game_id = text.strip()
|
||||||
|
self.convert_requested.emit({"source": e["path"], "title": e["name"], "game_id": game_id})
|
||||||
|
|
||||||
|
|
||||||
class MainWindow(QMainWindow):
|
class MainWindow(QMainWindow):
|
||||||
def __init__(self):
|
def __init__(self):
|
||||||
super().__init__()
|
super().__init__()
|
||||||
@@ -3259,6 +3467,9 @@ class MainWindow(QMainWindow):
|
|||||||
self._auto_sync_saves = str(self.settings.value("auto_sync_saves", "false")).lower() == "true"
|
self._auto_sync_saves = str(self.settings.value("auto_sync_saves", "false")).lower() == "true"
|
||||||
self._ppsspp_exe = str(self.settings.value("ppsspp_exe", "") or "")
|
self._ppsspp_exe = str(self.settings.value("ppsspp_exe", "") or "")
|
||||||
self._iso_category = str(self.settings.value("iso_category", "") or "")
|
self._iso_category = str(self.settings.value("iso_category", "") or "")
|
||||||
|
self._ps1_dir = self._first_existing(config.PS1_ROOTS) or str(self.settings.value("ps1_dir", "") or "")
|
||||||
|
self._ps1_output_dir = str(self.settings.value("ps1_output_dir", str(config.PS1_OUTPUT_DIR)) or "")
|
||||||
|
self._ps1_base_pbp = str(self.settings.value("ps1_base_pbp", "") or "")
|
||||||
log.info("games dir: %s", self._games_dir or "(none)")
|
log.info("games dir: %s", self._games_dir or "(none)")
|
||||||
log.info("videos dir: %s", self._videos_dir or "(none)")
|
log.info("videos dir: %s", self._videos_dir or "(none)")
|
||||||
log.info("store: %s", self._store_server_url or self._store_dir or "(none)")
|
log.info("store: %s", self._store_server_url or self._store_dir or "(none)")
|
||||||
@@ -3291,6 +3502,10 @@ class MainWindow(QMainWindow):
|
|||||||
self.saves_page = SavesPage()
|
self.saves_page = SavesPage()
|
||||||
self.saves_page.sync_requested.connect(self._on_saves_sync)
|
self.saves_page.sync_requested.connect(self._on_saves_sync)
|
||||||
self.tabs.addTab(self.saves_page, "Saves")
|
self.tabs.addTab(self.saves_page, "Saves")
|
||||||
|
self.ps1_page = Ps1Page()
|
||||||
|
self.ps1_page.set_dirs(self._ps1_dir, self._ps1_output_dir)
|
||||||
|
self.ps1_page.convert_requested.connect(self._on_ps1_convert)
|
||||||
|
self.tabs.addTab(self.ps1_page, "PS1")
|
||||||
self.tabs.addTab(self._build_settings_tab(), "Settings")
|
self.tabs.addTab(self._build_settings_tab(), "Settings")
|
||||||
self.tabs.currentChanged.connect(self._on_tab_changed)
|
self.tabs.currentChanged.connect(self._on_tab_changed)
|
||||||
self._on_psp_device_changed(self.psp_page.root())
|
self._on_psp_device_changed(self.psp_page.root())
|
||||||
@@ -3321,7 +3536,7 @@ class MainWindow(QMainWindow):
|
|||||||
self.library_search.textChanged.connect(self._apply_library_filter)
|
self.library_search.textChanged.connect(self._apply_library_filter)
|
||||||
bar.addWidget(self.library_search, 1)
|
bar.addWidget(self.library_search, 1)
|
||||||
self.library_type = QComboBox()
|
self.library_type = QComboBox()
|
||||||
self.library_type.addItems(["All types", "Games", "Videos"])
|
self.library_type.addItems(["All types", "Games", "Videos", "PS1"])
|
||||||
self.library_type.currentTextChanged.connect(self._apply_library_filter)
|
self.library_type.currentTextChanged.connect(self._apply_library_filter)
|
||||||
bar.addWidget(self.library_type)
|
bar.addWidget(self.library_type)
|
||||||
self.library_psp = QComboBox()
|
self.library_psp = QComboBox()
|
||||||
@@ -3371,6 +3586,7 @@ class MainWindow(QMainWindow):
|
|||||||
self.cover_grid.scrape_requested.connect(self._scrape_grid_item)
|
self.cover_grid.scrape_requested.connect(self._scrape_grid_item)
|
||||||
self.details_panel.edit_requested.connect(self._edit_details)
|
self.details_panel.edit_requested.connect(self._edit_details)
|
||||||
self.details_panel.link_requested.connect(self._scrape_item)
|
self.details_panel.link_requested.connect(self._scrape_item)
|
||||||
|
self.details_panel.convert_requested.connect(self._convert_current_item)
|
||||||
self.details_panel.copy_requested.connect(self._copy_current_to_psp)
|
self.details_panel.copy_requested.connect(self._copy_current_to_psp)
|
||||||
self.details_panel.delete_requested.connect(self._delete_current)
|
self.details_panel.delete_requested.connect(self._delete_current)
|
||||||
self.details_panel.delete_psp_requested.connect(self._delete_current_from_psp)
|
self.details_panel.delete_psp_requested.connect(self._delete_current_from_psp)
|
||||||
@@ -3461,7 +3677,8 @@ class MainWindow(QMainWindow):
|
|||||||
self.settings_page = SettingsPage(self._games_dir, self._videos_dir, self._store_dir,
|
self.settings_page = SettingsPage(self._games_dir, self._videos_dir, self._store_dir,
|
||||||
self._ppsspp_saves_dir, self._cloud_saves_dir, self._ppsspp_exe,
|
self._ppsspp_saves_dir, self._cloud_saves_dir, self._ppsspp_exe,
|
||||||
self._store_server_url, self._store_token, self._store_archive_id,
|
self._store_server_url, self._store_token, self._store_archive_id,
|
||||||
self._store_archive_sig, self._store_archive_user, self._iso_category)
|
self._store_archive_sig, self._store_archive_user, self._iso_category,
|
||||||
|
self._ps1_dir, self._ps1_output_dir, self._ps1_base_pbp)
|
||||||
self.settings_page.saved.connect(self._on_settings_saved)
|
self.settings_page.saved.connect(self._on_settings_saved)
|
||||||
return self.settings_page
|
return self.settings_page
|
||||||
|
|
||||||
@@ -3490,6 +3707,13 @@ class MainWindow(QMainWindow):
|
|||||||
self._auto_sync_saves = self.settings_page.auto_sync_saves()
|
self._auto_sync_saves = self.settings_page.auto_sync_saves()
|
||||||
self._ppsspp_exe = self.settings_page.ppsspp_exe()
|
self._ppsspp_exe = self.settings_page.ppsspp_exe()
|
||||||
self._iso_category = self.settings_page.iso_category()
|
self._iso_category = self.settings_page.iso_category()
|
||||||
|
self._ps1_dir = self.settings_page.ps1_dir()
|
||||||
|
self._ps1_output_dir = self.settings_page.ps1_output_dir()
|
||||||
|
self._ps1_base_pbp = self.settings_page.ps1_base_pbp()
|
||||||
|
self.settings.setValue("ps1_dir", self._ps1_dir)
|
||||||
|
self.settings.setValue("ps1_output_dir", self._ps1_output_dir)
|
||||||
|
self.settings.setValue("ps1_base_pbp", self._ps1_base_pbp)
|
||||||
|
self.ps1_page.set_dirs(self._ps1_dir, self._ps1_output_dir)
|
||||||
self.settings.setValue("ppsspp_saves_dir", self._ppsspp_saves_dir)
|
self.settings.setValue("ppsspp_saves_dir", self._ppsspp_saves_dir)
|
||||||
self.settings.setValue("cloud_saves_dir", self._cloud_saves_dir)
|
self.settings.setValue("cloud_saves_dir", self._cloud_saves_dir)
|
||||||
self.settings.setValue("auto_sync_saves", self._auto_sync_saves)
|
self.settings.setValue("auto_sync_saves", self._auto_sync_saves)
|
||||||
@@ -3499,7 +3723,8 @@ class MainWindow(QMainWindow):
|
|||||||
self._update_dir_status()
|
self._update_dir_status()
|
||||||
|
|
||||||
def _on_tab_changed(self, index):
|
def _on_tab_changed(self, index):
|
||||||
pass # store builds eagerly and caches its view; no per-tab work needed
|
if hasattr(self, "ps1_page") and index == self.tabs.indexOf(self.ps1_page):
|
||||||
|
self.ps1_page.refresh()
|
||||||
|
|
||||||
def _update_saves_locations(self):
|
def _update_saves_locations(self):
|
||||||
if not hasattr(self, "saves_page"):
|
if not hasattr(self, "saves_page"):
|
||||||
@@ -3530,7 +3755,7 @@ class MainWindow(QMainWindow):
|
|||||||
self.statusBar().showMessage("Loading library…")
|
self.statusBar().showMessage("Loading library…")
|
||||||
log.info("loading library…")
|
log.info("loading library…")
|
||||||
thread = QThread(self)
|
thread = QThread(self)
|
||||||
worker = LibraryWorker()
|
worker = LibraryWorker(self._ps1_dir, self._ps1_output_dir)
|
||||||
worker.moveToThread(thread)
|
worker.moveToThread(thread)
|
||||||
thread.started.connect(worker.run)
|
thread.started.connect(worker.run)
|
||||||
worker.result.connect(self._on_library_loaded)
|
worker.result.connect(self._on_library_loaded)
|
||||||
@@ -3568,6 +3793,8 @@ class MainWindow(QMainWindow):
|
|||||||
continue
|
continue
|
||||||
if typ == "Videos" and it["system"] != "videos":
|
if typ == "Videos" and it["system"] != "videos":
|
||||||
continue
|
continue
|
||||||
|
if typ == "PS1" and it["system"] != "ps1":
|
||||||
|
continue
|
||||||
if q and q not in (it["canonical"] or it["name"]).lower():
|
if q and q not in (it["canonical"] or it["name"]).lower():
|
||||||
continue
|
continue
|
||||||
if psp_f == "On PSP" and not it.get("on_psp"):
|
if psp_f == "On PSP" and not it.get("on_psp"):
|
||||||
@@ -3593,7 +3820,8 @@ class MainWindow(QMainWindow):
|
|||||||
title.setIcon(self._psp_dot_icon())
|
title.setIcon(self._psp_dot_icon())
|
||||||
title.setToolTip("Installed on PSP")
|
title.setToolTip("Installed on PSP")
|
||||||
self.library_table.setItem(row, 0, title)
|
self.library_table.setItem(row, 0, title)
|
||||||
self.library_table.setItem(row, 1, QTableWidgetItem("Game" if it["system"] == "games" else "Video"))
|
kind_label = {"games": "Game", "videos": "Video", "ps1": "PS1"}.get(it["system"], "Game")
|
||||||
|
self.library_table.setItem(row, 1, QTableWidgetItem(kind_label))
|
||||||
self.library_table.setItem(row, 2, QTableWidgetItem(it.get("region") or ""))
|
self.library_table.setItem(row, 2, QTableWidgetItem(it.get("region") or ""))
|
||||||
self.library_table.setItem(row, 3, QTableWidgetItem(it.get("serial") or ""))
|
self.library_table.setItem(row, 3, QTableWidgetItem(it.get("serial") or ""))
|
||||||
size = it.get("size")
|
size = it.get("size")
|
||||||
@@ -3716,6 +3944,10 @@ class MainWindow(QMainWindow):
|
|||||||
meta["manual"] = True
|
meta["manual"] = True
|
||||||
self.details_panel.set_item(item)
|
self.details_panel.set_item(item)
|
||||||
self.details_panel.set_metadata(meta)
|
self.details_panel.set_metadata(meta)
|
||||||
|
# Always refresh the details panel's cover box: a cover-only scrape
|
||||||
|
# (no metadata backend) skips set_item above, and `rid` may fall back
|
||||||
|
# to a stable key when the release has no database id.
|
||||||
|
self.details_panel._set_cover(rid, system)
|
||||||
self.cover_grid.update_item_label(item)
|
self.cover_grid.update_item_label(item)
|
||||||
self.cover_grid.refresh_item(item)
|
self.cover_grid.refresh_item(item)
|
||||||
|
|
||||||
@@ -3724,6 +3956,18 @@ class MainWindow(QMainWindow):
|
|||||||
self.details_panel.set_item(it)
|
self.details_panel.set_item(it)
|
||||||
self._scrape_item()
|
self._scrape_item()
|
||||||
|
|
||||||
|
def _convert_current_item(self):
|
||||||
|
item = getattr(self, "_current_item", None)
|
||||||
|
if item is None or item.get("system") != "ps1" or item.get("ps1_kind") != "disc":
|
||||||
|
return
|
||||||
|
game_id = item.get("serial") or ""
|
||||||
|
if not game_id:
|
||||||
|
text, ok = QInputDialog.getText(self, "PS1", "Game ID (e.g. SLUS00594):")
|
||||||
|
if not ok or not text.strip():
|
||||||
|
return
|
||||||
|
game_id = text.strip()
|
||||||
|
self._on_ps1_convert({"source": item.get("path"), "title": item.get("name"), "game_id": game_id})
|
||||||
|
|
||||||
def _on_store_download(self, items):
|
def _on_store_download(self, items):
|
||||||
if not items:
|
if not items:
|
||||||
return
|
return
|
||||||
@@ -3805,6 +4049,8 @@ class MainWindow(QMainWindow):
|
|||||||
return
|
return
|
||||||
if item.get("system") == "videos":
|
if item.get("system") == "videos":
|
||||||
return # UMD Video discs can't be played in PPSSPP
|
return # UMD Video discs can't be played in PPSSPP
|
||||||
|
if item.get("system") == "ps1" and item.get("ps1_kind") != "eboot":
|
||||||
|
return # raw PS1 discs are converted, not played
|
||||||
path = item.get("path")
|
path = item.get("path")
|
||||||
if not path or not Path(path).exists():
|
if not path or not Path(path).exists():
|
||||||
QMessageBox.warning(self, "Missing file", f"File not found:\n{path}")
|
QMessageBox.warning(self, "Missing file", f"File not found:\n{path}")
|
||||||
@@ -3877,6 +4123,15 @@ class MainWindow(QMainWindow):
|
|||||||
psp_root = Path(self._psp_root)
|
psp_root = Path(self._psp_root)
|
||||||
if system == "videos":
|
if system == "videos":
|
||||||
dest_dir = psp_root / "ISO" / "VIDEO"
|
dest_dir = psp_root / "ISO" / "VIDEO"
|
||||||
|
elif system == "ps1":
|
||||||
|
# PS1 EBOOTs install as a whole game folder under /PSP/GAME.
|
||||||
|
src_folder = Path(item["path"]).parent
|
||||||
|
dest_dir = psp_root / "PSP" / "GAME"
|
||||||
|
log.info("copying ps1 game to psp: %s -> %s", src_folder, dest_dir / src_folder.name)
|
||||||
|
self.queue.enqueue(transfers.TreeCopySource(str(src_folder)), dest_dir,
|
||||||
|
{"dest_name": src_folder.name}, item.get("name") or src_folder.name,
|
||||||
|
kind="psp-install")
|
||||||
|
return True
|
||||||
else:
|
else:
|
||||||
cat = "".join(c for c in (self._iso_category or "").strip() if c not in "/\\")
|
cat = "".join(c for c in (self._iso_category or "").strip() if c not in "/\\")
|
||||||
dest_dir = (psp_root / "ISO" / cat) if cat else (psp_root / "ISO")
|
dest_dir = (psp_root / "ISO" / cat) if cat else (psp_root / "ISO")
|
||||||
@@ -3984,6 +4239,17 @@ class MainWindow(QMainWindow):
|
|||||||
if hasattr(self, "psp_page"):
|
if hasattr(self, "psp_page"):
|
||||||
self.psp_page.refresh_inventory()
|
self.psp_page.refresh_inventory()
|
||||||
|
|
||||||
|
def _on_ps1_convert(self, item):
|
||||||
|
if not self._ps1_base_pbp or not Path(self._ps1_base_pbp).is_file():
|
||||||
|
QMessageBox.warning(self, "PS1", "Set a valid PS1 base.pbp path in Settings → General first.")
|
||||||
|
return
|
||||||
|
if not item.get("game_id"):
|
||||||
|
QMessageBox.warning(self, "PS1", "No game ID for this disc.")
|
||||||
|
return
|
||||||
|
out_dir = Path(self._ps1_output_dir) if self._ps1_output_dir else Path.cwd() / "ps1_converted"
|
||||||
|
source = transfers.Ps1ConvertSource(self._ps1_base_pbp)
|
||||||
|
self.queue.enqueue(source, out_dir, item, item.get("title") or item["source"], kind="ps1-convert")
|
||||||
|
|
||||||
def _on_queue_job_done(self, job):
|
def _on_queue_job_done(self, job):
|
||||||
if self._closing:
|
if self._closing:
|
||||||
return
|
return
|
||||||
@@ -4004,6 +4270,9 @@ class MainWindow(QMainWindow):
|
|||||||
QTimer.singleShot(0, self._refresh_psp)
|
QTimer.singleShot(0, self._refresh_psp)
|
||||||
if kind == "save-sync":
|
if kind == "save-sync":
|
||||||
QTimer.singleShot(0, self._refresh_saves)
|
QTimer.singleShot(0, self._refresh_saves)
|
||||||
|
if kind == "ps1-convert":
|
||||||
|
QTimer.singleShot(0, self.ps1_page.refresh)
|
||||||
|
QTimer.singleShot(0, self.load_library)
|
||||||
|
|
||||||
# -- Common ------------------------------------------------------------ #
|
# -- Common ------------------------------------------------------------ #
|
||||||
@staticmethod
|
@staticmethod
|
||||||
|
|||||||
+1
-1
@@ -197,7 +197,7 @@ def get_or_fetch(release_id, system, fetch_title):
|
|||||||
if system == "videos":
|
if system == "videos":
|
||||||
meta = covers.get_movie_metadata(fetch_title)
|
meta = covers.get_movie_metadata(fetch_title)
|
||||||
else:
|
else:
|
||||||
meta = covers.get_game_metadata(fetch_title)
|
meta = covers.get_game_metadata(fetch_title, system=system)
|
||||||
except Exception: # noqa: BLE001 - network/API failures must not crash the UI
|
except Exception: # noqa: BLE001 - network/API failures must not crash the UI
|
||||||
meta = None
|
meta = None
|
||||||
if meta:
|
if meta:
|
||||||
|
|||||||
@@ -24,6 +24,9 @@ def detect_system(dat_name):
|
|||||||
return "videos"
|
return "videos"
|
||||||
if "PlayStation Portable" in dat_name:
|
if "PlayStation Portable" in dat_name:
|
||||||
return "games"
|
return "games"
|
||||||
|
# Redump PS1: "Sony - PlayStation" (assumes no PS2/PS3 DATs are present).
|
||||||
|
if "PlayStation" in dat_name:
|
||||||
|
return "ps1"
|
||||||
return "unknown"
|
return "unknown"
|
||||||
|
|
||||||
|
|
||||||
@@ -37,6 +40,12 @@ def ingest_dat(conn, path):
|
|||||||
existing = conn.execute("SELECT id FROM dats WHERE name = ?", (dat_name,)).fetchone()
|
existing = conn.execute("SELECT id FROM dats WHERE name = ?", (dat_name,)).fetchone()
|
||||||
if existing:
|
if existing:
|
||||||
dat_id = existing["id"]
|
dat_id = existing["id"]
|
||||||
|
# Files that resolved to a release in this DAT must be re-identified
|
||||||
|
# against the new catalog (the release ids are about to be recycled).
|
||||||
|
conn.execute(
|
||||||
|
"UPDATE files SET release_id = NULL WHERE release_id IN (SELECT id FROM releases WHERE dat_id = ?)",
|
||||||
|
(dat_id,),
|
||||||
|
)
|
||||||
conn.execute("DELETE FROM roms WHERE release_id IN (SELECT id FROM releases WHERE dat_id = ?)", (dat_id,))
|
conn.execute("DELETE FROM roms WHERE release_id IN (SELECT id FROM releases WHERE dat_id = ?)", (dat_id,))
|
||||||
conn.execute("DELETE FROM releases WHERE dat_id = ?", (dat_id,))
|
conn.execute("DELETE FROM releases WHERE dat_id = ?", (dat_id,))
|
||||||
conn.execute(
|
conn.execute(
|
||||||
|
|||||||
+13
-3
@@ -15,8 +15,8 @@ _ARTICLE_RE = re.compile(r"^(.+),\s*(the|a|an)(?:\s+-\s+(.+))?$")
|
|||||||
|
|
||||||
def _normalize(name):
|
def _normalize(name):
|
||||||
name = name.lower()
|
name = name.lower()
|
||||||
name = re.sub(r"\.(iso|cso|pbp|img|bin)$", "", name) # strip disc extension
|
name = re.sub(r"\.(iso|cso|pbp|img|bin|cue)$", "", name) # strip disc extension
|
||||||
name = re.sub(r"^\s*\d+\s*-\s*", "", name) # leading No-Intro catalog number
|
name = re.sub(r"^\s*\d+\s*-\s*", "", name) # legacy No-Intro "#### - Name" catalog number
|
||||||
name = re.sub(r"\([^)]*\)", " ", name) # parenthetical tags
|
name = re.sub(r"\([^)]*\)", " ", name) # parenthetical tags
|
||||||
name = " ".join(name.split())
|
name = " ".join(name.split())
|
||||||
m = _ARTICLE_RE.match(name)
|
m = _ARTICLE_RE.match(name)
|
||||||
@@ -141,7 +141,7 @@ def match(conn, record):
|
|||||||
if row is not None:
|
if row is not None:
|
||||||
return f"hash:{field}", dict(row)
|
return f"hash:{field}", dict(row)
|
||||||
|
|
||||||
# 2. Local supplement catalog (dumps not present in No-Intro).
|
# 2. Local supplement catalog (dumps not present in the DATs).
|
||||||
for entry in _load_supplement():
|
for entry in _load_supplement():
|
||||||
for field in ("crc32", "sha1", "md5"):
|
for field in ("crc32", "sha1", "md5"):
|
||||||
value = entry.get(field)
|
value = entry.get(field)
|
||||||
@@ -181,6 +181,16 @@ def match(conn, record):
|
|||||||
return "unknown", None
|
return "unknown", None
|
||||||
top = [cand for ratio, cand in scored if ratio >= best_ratio - 0.001]
|
top = [cand for ratio, cand in scored if ratio >= best_ratio - 0.001]
|
||||||
top = _narrow(conn, top, size)
|
top = _narrow(conn, top, size)
|
||||||
|
# The same disc can exist in multiple catalogs (e.g. No-Intro + Redump);
|
||||||
|
# collapse identical entries so they don't read as an ambiguous match.
|
||||||
|
seen = set()
|
||||||
|
deduped = []
|
||||||
|
for cand in top:
|
||||||
|
key = (cand.get("name"), size)
|
||||||
|
if key not in seen:
|
||||||
|
seen.add(key)
|
||||||
|
deduped.append(cand)
|
||||||
|
top = deduped
|
||||||
if len(top) == 1:
|
if len(top) == 1:
|
||||||
return f"fuzzy:{best_ratio:.2f}", top[0]
|
return f"fuzzy:{best_ratio:.2f}", top[0]
|
||||||
return "unknown", None
|
return "unknown", None
|
||||||
|
|||||||
+2
-2
@@ -62,7 +62,7 @@ def _reorder_article(title):
|
|||||||
return title
|
return title
|
||||||
|
|
||||||
|
|
||||||
def canonical_name(release, serial):
|
def canonical_name(release, serial, ext=".iso"):
|
||||||
name = release["name"]
|
name = release["name"]
|
||||||
region = release.get("region")
|
region = release.get("region")
|
||||||
if region:
|
if region:
|
||||||
@@ -78,4 +78,4 @@ def canonical_name(release, serial):
|
|||||||
parts.append(f"({serial})")
|
parts.append(f"({serial})")
|
||||||
name = _ILLEGAL.sub(" ", " ".join(parts)).strip()
|
name = _ILLEGAL.sub(" ", " ".join(parts)).strip()
|
||||||
name = re.sub(r"\s+", " ", name)
|
name = re.sub(r"\s+", " ", name)
|
||||||
return f"{name}.iso"
|
return f"{name}{ext}"
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ tags removed). The GUI reads this bundle with no network access.
|
|||||||
"""
|
"""
|
||||||
import argparse
|
import argparse
|
||||||
import concurrent.futures
|
import concurrent.futures
|
||||||
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import re
|
import re
|
||||||
@@ -94,6 +95,19 @@ def build_index(sqlite_path):
|
|||||||
return best
|
return best
|
||||||
|
|
||||||
|
|
||||||
|
def write_index(index, path=None):
|
||||||
|
"""Persist {key: {"title", "url"}} so the GUI can show readable titles when
|
||||||
|
searching the offline bundle (see covers._search_openvgdb)."""
|
||||||
|
path = Path(path) if path else (config.COVERS_DIR / "openvgdb-index.json")
|
||||||
|
data = {k: {"title": t, "url": u} for k, (t, u, _r) in index.items()}
|
||||||
|
try:
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
path.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
|
||||||
|
except OSError as exc: # noqa: BLE001
|
||||||
|
log.warning("could not write index %s: %s", path, exc)
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
def _existing_keys():
|
def _existing_keys():
|
||||||
"""Titles already present in the bundle (for resume)."""
|
"""Titles already present in the bundle (for resume)."""
|
||||||
if not BUNDLE.exists():
|
if not BUNDLE.exists():
|
||||||
@@ -152,6 +166,7 @@ def scrape(limit=None, workers=8, rate=3.0):
|
|||||||
|
|
||||||
index = build_index(sqlite_path)
|
index = build_index(sqlite_path)
|
||||||
log.info("indexed %d unique PSP covers", len(index))
|
log.info("indexed %d unique PSP covers", len(index))
|
||||||
|
write_index(index)
|
||||||
|
|
||||||
existing = _existing_keys()
|
existing = _existing_keys()
|
||||||
todo = [(k, v) for k, v in sorted(index.items()) if k not in existing]
|
todo = [(k, v) for k, v in sorted(index.items()) if k not in existing]
|
||||||
|
|||||||
+158
@@ -0,0 +1,158 @@
|
|||||||
|
"""PS1 library helpers: discover source discs and converted EBOOT.PBPs.
|
||||||
|
|
||||||
|
This is the read/scan side of PS1 support (the conversion side lives in
|
||||||
|
``app.ps1_convert``). It finds raw PS1 disc images (``.cue`` sets and
|
||||||
|
standalone ``.iso``/``.img``), reads each disc's boot serial, and lists
|
||||||
|
converted ``EBOOT.PBP`` files in the output folder.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import List, Optional
|
||||||
|
|
||||||
|
from . import config
|
||||||
|
|
||||||
|
RAW_SECTOR = 2352
|
||||||
|
COOKED_SECTOR = 2048
|
||||||
|
EBOOT_NAME = "EBOOT.PBP"
|
||||||
|
|
||||||
|
# SYSTEM.CNF boot line: cdrom:\SLPS_017.62;1 (backslash optional in some rips)
|
||||||
|
_SERIAL_RE = re.compile(rb"cdrom:\\?([A-Za-z]{4})[_-]?(\d{3})\.(\d{2})")
|
||||||
|
|
||||||
|
|
||||||
|
def _sector_size(path: Path) -> int:
|
||||||
|
size = path.stat().st_size
|
||||||
|
if size and size % RAW_SECTOR == 0:
|
||||||
|
return RAW_SECTOR
|
||||||
|
return COOKED_SECTOR
|
||||||
|
|
||||||
|
|
||||||
|
def read_serial(image_path: Path) -> Optional[str]:
|
||||||
|
"""Read the boot serial (e.g. ``SLPS01762``) from a PS1 disc image.
|
||||||
|
|
||||||
|
Works on both raw 2352-byte/sector images (BIN/IMG) and cooked 2048-byte
|
||||||
|
images (ISO). Returns ``None`` if the boot line can't be found.
|
||||||
|
"""
|
||||||
|
image_path = Path(image_path)
|
||||||
|
if not image_path.is_file():
|
||||||
|
return None
|
||||||
|
sector_size = _sector_size(image_path)
|
||||||
|
data_off = 24 if sector_size == RAW_SECTOR else 0
|
||||||
|
|
||||||
|
buf = bytearray()
|
||||||
|
with open(image_path, "rb") as fh:
|
||||||
|
for _ in range(128): # SYSTEM.CNF is always near the start of the disc
|
||||||
|
sector = fh.read(sector_size)
|
||||||
|
if len(sector) < sector_size:
|
||||||
|
break
|
||||||
|
buf += sector[data_off:data_off + COOKED_SECTOR]
|
||||||
|
|
||||||
|
m = _SERIAL_RE.search(bytes(buf))
|
||||||
|
if m:
|
||||||
|
return (m.group(1) + m.group(2) + m.group(3)).decode().upper()
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def cue_data_track(cue_path: Path) -> Optional[Path]:
|
||||||
|
"""Return the first data-track image referenced by a cue sheet."""
|
||||||
|
from . import ps1_convert
|
||||||
|
|
||||||
|
try:
|
||||||
|
disc = ps1_convert.parse_cue(cue_path.read_text(encoding="utf-8", errors="replace"))
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
track = disc.data_track
|
||||||
|
if track is None:
|
||||||
|
return None
|
||||||
|
return (cue_path.parent / track.file) if track.file else None
|
||||||
|
|
||||||
|
|
||||||
|
def iter_source_discs(root: Path):
|
||||||
|
"""Yield PS1 source discs under ``root`` (cue sheets + standalone images)."""
|
||||||
|
root = Path(root)
|
||||||
|
if not root.is_dir():
|
||||||
|
return
|
||||||
|
|
||||||
|
for path in sorted(root.rglob("*")):
|
||||||
|
if not path.is_file():
|
||||||
|
continue
|
||||||
|
suffix = path.suffix.lower()
|
||||||
|
if suffix == ".cue":
|
||||||
|
yield path
|
||||||
|
elif suffix in (".iso", ".img"):
|
||||||
|
# A standalone image: no cue sheet next to it (same stem).
|
||||||
|
if not (path.parent / f"{path.stem}.cue").exists():
|
||||||
|
yield path
|
||||||
|
|
||||||
|
|
||||||
|
def find_eboots(root: Path) -> List[Path]:
|
||||||
|
"""List converted/PSN ``EBOOT.PBP`` files under ``root``."""
|
||||||
|
root = Path(root)
|
||||||
|
if not root.is_dir():
|
||||||
|
return []
|
||||||
|
return [p for p in sorted(root.rglob("*")) if p.is_file() and p.name.upper() == EBOOT_NAME]
|
||||||
|
|
||||||
|
|
||||||
|
def eboot_title(pbp_path: Path) -> str:
|
||||||
|
"""Read the game title embedded in an EBOOT.PBP (falls back to folder name)."""
|
||||||
|
from . import iso_reader, ps1_convert
|
||||||
|
|
||||||
|
try:
|
||||||
|
header = ps1_convert.read_pbp_header(pbp_path)
|
||||||
|
idx = ps1_convert.SECTION_OFFSET["sfo"] // 4
|
||||||
|
sfo = ps1_convert.read_pbp_section(pbp_path, header, idx)
|
||||||
|
parsed = iso_reader.parse_sfo(sfo)
|
||||||
|
title = parsed.get("TITLE") or parsed.get("TITLE_0") or parsed.get("TITLE_00")
|
||||||
|
if title:
|
||||||
|
return str(title)
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return pbp_path.parent.name
|
||||||
|
|
||||||
|
|
||||||
|
def scan_library(ps1_dir, output_dir) -> List[dict]:
|
||||||
|
"""Return a combined view of PS1 source discs and converted EBOOTs.
|
||||||
|
|
||||||
|
Each entry is a dict with ``kind`` ("disc" or "eboot"), ``path``, ``name``,
|
||||||
|
``serial`` and ``status`` ("pending" / "converted").
|
||||||
|
"""
|
||||||
|
ps1_dir = Path(ps1_dir)
|
||||||
|
output_dir = Path(output_dir)
|
||||||
|
|
||||||
|
eboots = find_eboots(output_dir)
|
||||||
|
eboot_names = {e.name.upper() for e in eboots}
|
||||||
|
eboot_titles = {eboot_title(e).lower() for e in eboots}
|
||||||
|
|
||||||
|
entries: List[dict] = []
|
||||||
|
for src in iter_source_discs(ps1_dir):
|
||||||
|
if src.suffix.lower() == ".cue":
|
||||||
|
image = cue_data_track(src) or src
|
||||||
|
else:
|
||||||
|
image = src
|
||||||
|
serial = read_serial(image)
|
||||||
|
title = src.stem
|
||||||
|
# A disc is "converted" when an EBOOT's embedded title or filename
|
||||||
|
# matches the source disc's title/serial.
|
||||||
|
status = "pending"
|
||||||
|
if title.lower() in eboot_titles or (serial and any(serial.lower() in t for t in eboot_titles)):
|
||||||
|
status = "converted"
|
||||||
|
entries.append({
|
||||||
|
"kind": "disc",
|
||||||
|
"path": str(src),
|
||||||
|
"name": title,
|
||||||
|
"serial": serial or "",
|
||||||
|
"status": status,
|
||||||
|
})
|
||||||
|
|
||||||
|
for e in eboots:
|
||||||
|
entries.append({
|
||||||
|
"kind": "eboot",
|
||||||
|
"path": str(e),
|
||||||
|
"name": eboot_title(e),
|
||||||
|
"serial": "",
|
||||||
|
"status": "converted",
|
||||||
|
})
|
||||||
|
|
||||||
|
return entries
|
||||||
@@ -0,0 +1,899 @@
|
|||||||
|
"""PS1 -> PSP EBOOT.PBP conversion (pure Python).
|
||||||
|
|
||||||
|
Converts a PlayStation 1 disc image (``.iso``, ``.img``, or a ``.cue`` +
|
||||||
|
``.bin`` set) into the ``EBOOT.PBP`` "PS1 classic" container that PSP POPS
|
||||||
|
firmware can launch.
|
||||||
|
|
||||||
|
This is an independent, MIT-clean reimplementation of the popstation-md
|
||||||
|
algorithm (same format that PSXPackager/RupertAvery writes), implemented from
|
||||||
|
the published container format rather than by copying any GPL source.
|
||||||
|
|
||||||
|
Format reference (offsets are byte-exact):
|
||||||
|
* PBP header - 40 bytes: b"\\x00PBP" magic + version + 8 uint32 section offsets
|
||||||
|
(SFO, ICON0, ICON1, PIC0, PIC1, SND0, DATA.PSP, DATA.PSAR).
|
||||||
|
* PSAR - the DATA.PSAR section holding the disc image. Starts with
|
||||||
|
b"PSISOIMG0000", game id at +0x400, TOC at +0x800,
|
||||||
|
ISO block index at +0x4000, disc data at +0x100000.
|
||||||
|
|
||||||
|
A ``base.pbp`` (any PSN PS1 classic ``EBOOT.PBP``) is REQUIRED to supply two
|
||||||
|
blobs that we cannot synthesise:
|
||||||
|
|
||||||
|
* DATA.PSP - the encrypted POPS loader (header section 8).
|
||||||
|
* STARTDAT - the trailing encrypted block (boot.png + PGD).
|
||||||
|
|
||||||
|
Without a base.pbp the resulting EBOOT will not boot on real hardware; the
|
||||||
|
converter refuses to run without one.
|
||||||
|
|
||||||
|
Compression: only level 0 (uncompressed) is implemented. LZRC (levels 1-9) is a
|
||||||
|
planned fast-follow.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import io
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import struct
|
||||||
|
import zlib
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterator, List, Optional, Tuple
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Constants
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
PBP_MAGIC = b"\x00PBP"
|
||||||
|
PBP_HEADER_SIZE = 0x28
|
||||||
|
PBP_HEADER_U32 = 10 # magic + version + 8 section offsets
|
||||||
|
|
||||||
|
# Section index -> header offset (in bytes) within the 40-byte header.
|
||||||
|
SECTION_OFFSET = {
|
||||||
|
"sfo": 0x08,
|
||||||
|
"icon0": 0x0C,
|
||||||
|
"icon1": 0x10,
|
||||||
|
"pic0": 0x14,
|
||||||
|
"pic1": 0x18,
|
||||||
|
"snd0": 0x1C,
|
||||||
|
"data_psp": 0x20,
|
||||||
|
"psar": 0x24,
|
||||||
|
}
|
||||||
|
SECTION_ORDER = ["sfo", "icon0", "icon1", "pic0", "pic1", "snd0", "data_psp", "psar"]
|
||||||
|
|
||||||
|
# PSAR (DATA.PSAR) layout.
|
||||||
|
PSAR_MAGIC = b"PSISOIMG0000"
|
||||||
|
PSAR_GAMEID_OFFSET = 0x400
|
||||||
|
PSAR_TOC_OFFSET = 0x800
|
||||||
|
PSAR_INDEX_OFFSET = 0x4000
|
||||||
|
PSAR_ISO_OFFSET = 0x100000
|
||||||
|
PSAR_ALIGNMENT = 0x10000
|
||||||
|
|
||||||
|
# One disc "block" in the ISO index = 16 raw 2352-byte sectors.
|
||||||
|
RAW_SECTOR = 0x930 # 2352
|
||||||
|
COOKED_SECTOR = 0x800 # 2048
|
||||||
|
BLOCK_SIZE = RAW_SECTOR * 16 # 0x9300
|
||||||
|
INDEX_ENTRY_SIZE = 32 # offset u32 + length u32 + 6 unused u32
|
||||||
|
|
||||||
|
SYNC_PATTERN = b"\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00"
|
||||||
|
|
||||||
|
# SFO field formats (PSP PARAM.SFO).
|
||||||
|
FMT_U32 = 0x0004
|
||||||
|
FMT_STRING = 0x0204
|
||||||
|
|
||||||
|
SFO_DEFAULTS = {
|
||||||
|
"BOOTABLE": (FMT_U32, 1),
|
||||||
|
"CATEGORY": (FMT_STRING, "ME"),
|
||||||
|
"DISC_VERSION": (FMT_STRING, "1.00"),
|
||||||
|
"LICENSE": (FMT_STRING, "Library programs Copyright(C) Sony ComputerEntertainment Inc."),
|
||||||
|
"PARENTAL_LEVEL": (FMT_U32, 1),
|
||||||
|
"PSP_SYSTEM_VER": (FMT_STRING, "3.71"),
|
||||||
|
"REGION": (FMT_U32, 0x8000),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
class ConversionError(Exception):
|
||||||
|
"""Raised when a disc cannot be converted."""
|
||||||
|
|
||||||
|
|
||||||
|
class Cancelled(Exception):
|
||||||
|
"""Raised internally when the caller requests cancellation."""
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# CUE sheet parsing
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class CueTrack:
|
||||||
|
number: int
|
||||||
|
mode: str # "MODE1/2048", "MODE1/2352", "MODE2/2352", "AUDIO"
|
||||||
|
file: str
|
||||||
|
file_type: str # "BINARY" | "WAVE" | ...
|
||||||
|
indexes: List[Tuple[int, int]] = field(default_factory=list) # (index, msf)
|
||||||
|
sector_size: int = 0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def pregap_sectors(self) -> int:
|
||||||
|
"""Sectors between INDEX 00 and INDEX 01 (0 if no pregap)."""
|
||||||
|
idx = dict(self.indexes)
|
||||||
|
if 0 in idx and 1 in idx:
|
||||||
|
return max(0, idx[1] - idx[0])
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class CueDisc:
|
||||||
|
tracks: List[CueTrack]
|
||||||
|
|
||||||
|
@property
|
||||||
|
def data_track(self) -> Optional[CueTrack]:
|
||||||
|
for t in self.tracks:
|
||||||
|
if t.mode != "AUDIO":
|
||||||
|
return t
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _msf_to_sectors(msf: str) -> int:
|
||||||
|
m, s, f = (int(x) for x in msf.split(":"))
|
||||||
|
return (m * 60 + s) * 75 + f
|
||||||
|
|
||||||
|
|
||||||
|
def _sector_size_for_mode(mode: str) -> int:
|
||||||
|
if mode in ("MODE1/2048",):
|
||||||
|
return COOKED_SECTOR
|
||||||
|
if mode in ("MODE1/2352", "MODE2/2352", "AUDIO"):
|
||||||
|
return RAW_SECTOR
|
||||||
|
# CDG / MODE2/2336 etc. fall back to raw.
|
||||||
|
return RAW_SECTOR
|
||||||
|
|
||||||
|
|
||||||
|
def parse_cue(text: str) -> CueDisc:
|
||||||
|
"""Parse a CUE sheet into a :class:`CueDisc`.
|
||||||
|
|
||||||
|
Handles the common PS1 layouts: one or more ``FILE`` lines, ``TRACK``
|
||||||
|
lines (``MODE1/2048``, ``MODE1/2352``, ``MODE2/2352``, ``AUDIO``) and
|
||||||
|
``INDEX`` lines (``INDEX 00`` pregap + ``INDEX 01`` start).
|
||||||
|
"""
|
||||||
|
tracks: List[CueTrack] = []
|
||||||
|
current_file: Optional[str] = None
|
||||||
|
current_type = "BINARY"
|
||||||
|
current_track: Optional[CueTrack] = None
|
||||||
|
|
||||||
|
file_re = re.compile(r'^FILE\s+"([^"]+)"\s+(\w+)', re.I)
|
||||||
|
track_re = re.compile(r'^\s*TRACK\s+(\d{1,2})\s+(\S+)', re.I)
|
||||||
|
index_re = re.compile(r'^\s*INDEX\s+(\d{1,2})\s+(\d{1,2}:\d{2}:\d{2})', re.I)
|
||||||
|
|
||||||
|
for raw in text.splitlines():
|
||||||
|
line = raw.strip()
|
||||||
|
if not line or line.startswith(("REM", "CATALOG", "CDTEXTFILE", "PERFORMER", "TITLE", "ISRC", "FLAGS", "POSTGAP", "PREGAP")):
|
||||||
|
if line.startswith("PREGAP"):
|
||||||
|
# PREGAP <mm:ss:ff> is equivalent to an INDEX 00 offset.
|
||||||
|
m = re.search(r'PREGAP\s+(\d{1,2}:\d{2}:\d{2})', line, re.I)
|
||||||
|
if m and current_track is not None:
|
||||||
|
current_track.indexes.append((0, _msf_to_sectors(m.group(1))))
|
||||||
|
continue
|
||||||
|
|
||||||
|
m = file_re.match(line)
|
||||||
|
if m:
|
||||||
|
current_file = m.group(1)
|
||||||
|
current_type = m.group(2).upper()
|
||||||
|
current_track = None
|
||||||
|
continue
|
||||||
|
|
||||||
|
m = track_re.match(line)
|
||||||
|
if m:
|
||||||
|
number = int(m.group(1))
|
||||||
|
mode = m.group(2).upper()
|
||||||
|
current_track = CueTrack(
|
||||||
|
number=number,
|
||||||
|
mode=mode,
|
||||||
|
file=current_file or "",
|
||||||
|
file_type=current_type,
|
||||||
|
sector_size=_sector_size_for_mode(mode),
|
||||||
|
)
|
||||||
|
tracks.append(current_track)
|
||||||
|
continue
|
||||||
|
|
||||||
|
m = index_re.match(line)
|
||||||
|
if m and current_track is not None:
|
||||||
|
current_track.indexes.append((int(m.group(1)), _msf_to_sectors(m.group(2))))
|
||||||
|
|
||||||
|
if not tracks:
|
||||||
|
raise ConversionError("CUE sheet contains no tracks")
|
||||||
|
for t in tracks:
|
||||||
|
if not t.file:
|
||||||
|
raise ConversionError(f"track {t.number} has no FILE")
|
||||||
|
return CueDisc(tracks)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Sector normalisation (cooked 2048 -> raw 2352 Mode 2 Form 1)
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _to_bcd(value: int) -> int:
|
||||||
|
return ((value // 10) << 4) | (value % 10)
|
||||||
|
|
||||||
|
|
||||||
|
def _sectors_to_msf(sectors: int) -> Tuple[int, int, int]:
|
||||||
|
"""Convert a sector count to (minutes, seconds, frames) at 75 frames/s."""
|
||||||
|
total_seconds = sectors // 75
|
||||||
|
return total_seconds // 60, total_seconds % 60, sectors % 75
|
||||||
|
|
||||||
|
|
||||||
|
def _expand_cooked_sector(data: bytes, lba: int) -> bytes:
|
||||||
|
"""Expand a 2048-byte cooked sector to a 2352-byte raw Mode2 Form1 sector.
|
||||||
|
|
||||||
|
Rebuilds the 12-byte sync pattern, the 4-byte MSF header (BCD), the
|
||||||
|
8-byte subheader, and appends 280 bytes of EDC/ECC (zero-filled; the PSP
|
||||||
|
POPS loader does not verify EDC/ECC for converted games).
|
||||||
|
"""
|
||||||
|
assert len(data) == COOKED_SECTOR
|
||||||
|
frame = lba + 150 # 150-sector lead-in offset
|
||||||
|
msf = bytes([
|
||||||
|
_to_bcd((frame // 75 // 60) % 100),
|
||||||
|
_to_bcd((frame // 75) % 60),
|
||||||
|
_to_bcd(frame % 75),
|
||||||
|
0x02, # mode 2
|
||||||
|
])
|
||||||
|
# Subheader: file=2, channel=0, submode=0, coding=0; repeated once.
|
||||||
|
subheader = b"\x00\x00\x08\x00" * 2
|
||||||
|
return SYNC_PATTERN + msf + subheader + data + (b"\x00" * 280)
|
||||||
|
|
||||||
|
|
||||||
|
def iter_raw_sectors(
|
||||||
|
path: Path,
|
||||||
|
sector_size: int,
|
||||||
|
pregap_sectors: int = 0,
|
||||||
|
progress_cb=None,
|
||||||
|
cancel_event=None,
|
||||||
|
) -> Iterator[bytes]:
|
||||||
|
"""Yield 2352-byte raw sectors for one track file.
|
||||||
|
|
||||||
|
``sector_size`` is 2048 (cooked; expanded on the fly) or 2352 (raw).
|
||||||
|
``pregap_sectors`` leading zero sectors are emitted first.
|
||||||
|
"""
|
||||||
|
def _cancelled():
|
||||||
|
return cancel_event is not None and cancel_event.is_set()
|
||||||
|
|
||||||
|
if pregap_sectors:
|
||||||
|
for _ in range(pregap_sectors):
|
||||||
|
yield b"\x00" * RAW_SECTOR
|
||||||
|
|
||||||
|
size = path.stat().st_size
|
||||||
|
with open(path, "rb") as fh:
|
||||||
|
lba = 0
|
||||||
|
done = 0
|
||||||
|
while True:
|
||||||
|
if _cancelled():
|
||||||
|
raise Cancelled()
|
||||||
|
chunk = fh.read(sector_size)
|
||||||
|
if not chunk:
|
||||||
|
break
|
||||||
|
if len(chunk) != sector_size:
|
||||||
|
chunk = chunk.ljust(sector_size, b"\x00")
|
||||||
|
if sector_size == COOKED_SECTOR:
|
||||||
|
yield _expand_cooked_sector(chunk, lba)
|
||||||
|
else:
|
||||||
|
yield chunk
|
||||||
|
lba += 1
|
||||||
|
done += sector_size
|
||||||
|
if progress_cb:
|
||||||
|
progress_cb(done, size)
|
||||||
|
if progress_cb:
|
||||||
|
progress_cb(size, size)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# PARAM.SFO writer
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def build_param_sfo(entries: List[Tuple[str, int, object]]) -> bytes:
|
||||||
|
"""Build a PSP PARAM.SFO from ``(key, fmt, value)`` entries.
|
||||||
|
|
||||||
|
``fmt`` is ``FMT_U32`` (uint32) or ``FMT_STRING`` (utf-8 string).
|
||||||
|
Layout matches the PSP format: header, index table, key table, data table.
|
||||||
|
"""
|
||||||
|
header_size = 20
|
||||||
|
index_size = len(entries) * 16
|
||||||
|
key_table_start = header_size + index_size
|
||||||
|
|
||||||
|
key_table = b"".join(key.encode("utf-8") + b"\x00" for key, _, _ in entries)
|
||||||
|
key_pad = (4 - (len(key_table) % 4)) % 4
|
||||||
|
data_table_start = key_table_start + len(key_table) + key_pad
|
||||||
|
|
||||||
|
data_table = b""
|
||||||
|
index_table = b""
|
||||||
|
key_offset = 0
|
||||||
|
data_offset = 0
|
||||||
|
for key, fmt, value in entries:
|
||||||
|
if fmt == FMT_U32:
|
||||||
|
raw = struct.pack("<I", int(value))
|
||||||
|
length = 4
|
||||||
|
max_len = 4
|
||||||
|
elif fmt == FMT_STRING:
|
||||||
|
s = str(value)
|
||||||
|
raw = s.encode("utf-8") + b"\x00"
|
||||||
|
length = len(s)
|
||||||
|
max_len = len(s) + 1
|
||||||
|
if len(raw) < max_len:
|
||||||
|
raw = raw.ljust(max_len, b"\x00")
|
||||||
|
else:
|
||||||
|
raise ConversionError(f"unsupported SFO format 0x{fmt:04x}")
|
||||||
|
|
||||||
|
index_table += struct.pack(
|
||||||
|
"<HHIII", key_offset, fmt, length, max_len, data_offset
|
||||||
|
)
|
||||||
|
data_table += raw
|
||||||
|
key_offset += len(key) + 1
|
||||||
|
data_offset += len(raw)
|
||||||
|
|
||||||
|
return (
|
||||||
|
b"\x00PSF"
|
||||||
|
+ struct.pack("<IIII", 0x00000101, key_table_start, data_table_start, len(entries))
|
||||||
|
+ index_table
|
||||||
|
+ key_table
|
||||||
|
+ (b"\x00" * key_pad)
|
||||||
|
+ data_table
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def default_ps1_sfo(title: str, game_id: str) -> bytes:
|
||||||
|
"""A complete PARAM.SFO for a converted PS1 game."""
|
||||||
|
entries = []
|
||||||
|
for key, (fmt, value) in SFO_DEFAULTS.items():
|
||||||
|
entries.append((key, fmt, value))
|
||||||
|
entries.append(("DISC_ID", FMT_STRING, game_id))
|
||||||
|
entries.append(("TITLE", FMT_STRING, title))
|
||||||
|
return build_param_sfo(entries)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Minimal PNG generator (no external image deps)
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _png(width: int, height: int, rgb: Tuple[int, int, int]) -> bytes:
|
||||||
|
def chunk(typ: bytes, data: bytes) -> bytes:
|
||||||
|
payload = typ + data
|
||||||
|
return struct.pack(">I", len(data)) + payload + struct.pack(">I", zlib.crc32(payload) & 0xFFFFFFFF)
|
||||||
|
|
||||||
|
ihdr = struct.pack(">IIBBBBB", width, height, 8, 2, 0, 0, 0) # 8-bit RGB
|
||||||
|
raw = b"".join(b"\x00" + bytes(rgb) * width for _ in range(height))
|
||||||
|
return (
|
||||||
|
b"\x89PNG\r\n\x1a\n"
|
||||||
|
+ chunk(b"IHDR", ihdr)
|
||||||
|
+ chunk(b"IDAT", zlib.compress(raw, 9))
|
||||||
|
+ chunk(b"IEND", b"")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def default_icon0_png() -> bytes:
|
||||||
|
return _png(144, 80, (24, 24, 24))
|
||||||
|
|
||||||
|
|
||||||
|
def default_pic1_png() -> bytes:
|
||||||
|
return _png(480, 272, (10, 10, 16))
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# PBP reading (used both for base.pbp extraction and self-test validation)
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def read_pbp_header(path: Path) -> List[int]:
|
||||||
|
with open(path, "rb") as fh:
|
||||||
|
data = fh.read(PBP_HEADER_SIZE)
|
||||||
|
if len(data) < PBP_HEADER_SIZE or data[:4] != PBP_MAGIC:
|
||||||
|
raise ConversionError(f"{path} is not a PBP file (bad magic)")
|
||||||
|
return list(struct.unpack("<10I", data))
|
||||||
|
|
||||||
|
|
||||||
|
def read_pbp_section(path: Path, header: List[int], index: int) -> bytes:
|
||||||
|
start = header[index]
|
||||||
|
end = header[index + 1] if index + 1 < len(header) else None
|
||||||
|
with open(path, "rb") as fh:
|
||||||
|
fh.seek(start)
|
||||||
|
data = fh.read() if end is None else fh.read(end - start)
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def extract_data_psp(base_pbp: Path) -> bytes:
|
||||||
|
"""Extract the encrypted POPS loader (DATA.PSP section) from a base PBP."""
|
||||||
|
header = read_pbp_header(base_pbp)
|
||||||
|
idx = SECTION_OFFSET["data_psp"] // 4 # header array index (2 + section index)
|
||||||
|
data = read_pbp_section(base_pbp, header, idx)
|
||||||
|
if not data:
|
||||||
|
raise ConversionError(f"{base_pbp} has no DATA.PSP section")
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def extract_startdat(base_pbp: Path) -> bytes:
|
||||||
|
"""Extract the trailing STARTDAT block (boot.png + PGD) from a base PBP.
|
||||||
|
|
||||||
|
The STARTDAT offset is discovered the same way the reference writer does:
|
||||||
|
read the u32 at ``psar + 0x0C``, add ``0x50000``, and expect b"STARTDAT"
|
||||||
|
there. Everything from that marker to EOF is copied verbatim.
|
||||||
|
"""
|
||||||
|
header = read_pbp_header(base_pbp)
|
||||||
|
psar_offset = header[SECTION_OFFSET["psar"] // 4] # header array index
|
||||||
|
with open(base_pbp, "rb") as fh:
|
||||||
|
fh.seek(psar_offset + 0x0C)
|
||||||
|
(x,) = struct.unpack("<I", fh.read(4))
|
||||||
|
marker_off = x + 0x50000
|
||||||
|
fh.seek(marker_off)
|
||||||
|
if fh.read(8) != b"STARTDAT":
|
||||||
|
raise ConversionError(f"cannot find STARTDAT in {base_pbp} (not a valid PS1 base PBP?)")
|
||||||
|
fh.seek(marker_off)
|
||||||
|
return fh.read()
|
||||||
|
raise ConversionError("unreachable")
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# PBP writer
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _format_psar_game_id(game_id: str) -> str:
|
||||||
|
"""Format a PS1 serial for the PSAR header field.
|
||||||
|
|
||||||
|
The PBP's PARAM.SFO uses the plain 9-char id (``SLPS01762``), while the
|
||||||
|
PSAR header stores the region-delineated form (``_SLPS_01762``).
|
||||||
|
"""
|
||||||
|
m = re.match(r"^([A-Za-z]{4})[_-]?(\d{5})$", game_id)
|
||||||
|
if m:
|
||||||
|
return f"_{m.group(1).upper()}_{m.group(2)}"
|
||||||
|
return game_id
|
||||||
|
|
||||||
|
|
||||||
|
def _pad_to(out, target: int) -> None:
|
||||||
|
"""Write zero bytes until the stream reaches ``target`` (absolute offset)."""
|
||||||
|
delta = target - out.tell()
|
||||||
|
if delta < 0:
|
||||||
|
raise ConversionError(f"internal error: overshot offset by {-delta} bytes")
|
||||||
|
if delta:
|
||||||
|
out.write(b"\x00" * delta)
|
||||||
|
|
||||||
|
|
||||||
|
# Track type values used in the PS1 TOC (PSAR +0x800).
|
||||||
|
_TOC_DATA = 0x41
|
||||||
|
_TOC_AUDIO = 0x01
|
||||||
|
_TOC_A0 = 0xA0
|
||||||
|
_TOC_A1 = 0xA1
|
||||||
|
_TOC_A2 = 0xA2
|
||||||
|
_TOC_ENTRY = 10 # bytes per TOC entry
|
||||||
|
_TOC_LEAD_IN = 150 # 2-second lead-in, in sectors
|
||||||
|
|
||||||
|
|
||||||
|
def build_toc(disc: "DiscImage") -> bytes:
|
||||||
|
"""Generate the PS1 table-of-contents stored at PSAR +0x800.
|
||||||
|
|
||||||
|
Emits the standard CD-style TOC: an A0 entry (first track), an A1 entry
|
||||||
|
(last track), an A2 entry (lead-out), then one 10-byte entry per track.
|
||||||
|
Track addresses are BCD-encoded MSF; a 150-sector (2 s) lead-in is added
|
||||||
|
to the relative address field, matching the reference implementation.
|
||||||
|
"""
|
||||||
|
tracks = disc.tracks_info()
|
||||||
|
total = disc.total_sectors()
|
||||||
|
entries = [bytearray(_TOC_ENTRY) for _ in range(len(tracks) + 3)]
|
||||||
|
|
||||||
|
first_num, first_data, _ = tracks[0]
|
||||||
|
last_num, last_data, _ = tracks[-1]
|
||||||
|
|
||||||
|
e = entries[0]
|
||||||
|
e[0] = _TOC_DATA if first_data else _TOC_AUDIO
|
||||||
|
e[2] = _TOC_A0
|
||||||
|
e[7] = _to_bcd(first_num)
|
||||||
|
e[8] = _to_bcd(0x20)
|
||||||
|
|
||||||
|
e = entries[1]
|
||||||
|
e[0] = _TOC_DATA if last_data else _TOC_AUDIO
|
||||||
|
e[2] = _TOC_A1
|
||||||
|
e[7] = _to_bcd(last_num)
|
||||||
|
|
||||||
|
e = entries[2]
|
||||||
|
e[0] = _TOC_AUDIO
|
||||||
|
e[2] = _TOC_A2
|
||||||
|
mm, ss, ff = _sectors_to_msf(total)
|
||||||
|
e[7], e[8], e[9] = _to_bcd(mm), _to_bcd(ss), _to_bcd(ff)
|
||||||
|
|
||||||
|
for i, (num, is_data, start) in enumerate(tracks):
|
||||||
|
e = entries[3 + i]
|
||||||
|
e[0] = _TOC_DATA if is_data else _TOC_AUDIO
|
||||||
|
e[2] = _to_bcd(num)
|
||||||
|
mm, ss, ff = _sectors_to_msf(start)
|
||||||
|
e[3], e[4], e[5] = _to_bcd(mm), _to_bcd(ss), _to_bcd(ff)
|
||||||
|
mm, ss, ff = _sectors_to_msf(start + _TOC_LEAD_IN)
|
||||||
|
e[7], e[8], e[9] = _to_bcd(mm), _to_bcd(ss), _to_bcd(ff)
|
||||||
|
|
||||||
|
return b"".join(bytes(e) for e in entries)
|
||||||
|
|
||||||
|
|
||||||
|
def _build_psar_header(disc: "DiscImage", game_id: str, title: str) -> bytes:
|
||||||
|
"""Build the fixed PSAR header region 0x400..0x4000, byte-identical to the
|
||||||
|
popstation level-0 writer.
|
||||||
|
|
||||||
|
Layout (offsets relative to the PSAR start):
|
||||||
|
* 0x400 data1 (0xE20 bytes): game id at +0, TOC at +0x400, 0x10 at +0x7FE
|
||||||
|
* 0x1220 p2 offset (u32) = padded ISO size + 0x100000 + 0x2D31
|
||||||
|
* 0x1224 data2 (0x2DDC bytes): FF 07 at +4, title at +8, 0x03 at +0x88
|
||||||
|
"""
|
||||||
|
data1_size = 0xE20
|
||||||
|
data2_size = 0x2DDC
|
||||||
|
p2_delta = 0x2D31
|
||||||
|
|
||||||
|
block = bytearray(data1_size + 4 + data2_size) # 0x3C00 = 0x400..0x4000
|
||||||
|
|
||||||
|
gid = _format_psar_game_id(game_id).encode("ascii")[:11]
|
||||||
|
block[: len(gid)] = gid
|
||||||
|
|
||||||
|
toc = build_toc(disc)
|
||||||
|
block[0x400: 0x400 + len(toc)] = toc
|
||||||
|
block[0x7FE] = 0x10
|
||||||
|
|
||||||
|
p2 = disc.padded_size() + PSAR_ISO_OFFSET + p2_delta
|
||||||
|
block[data1_size: data1_size + 4] = struct.pack("<I", p2)
|
||||||
|
|
||||||
|
data2 = data1_size + 4
|
||||||
|
block[data2 + 4: data2 + 6] = b"\xFF\x07"
|
||||||
|
title_bytes = title.encode("ascii", "replace")[:0x80]
|
||||||
|
block[data2 + 8: data2 + 8 + len(title_bytes)] = title_bytes
|
||||||
|
block[data2 + 0x88] = 0x03
|
||||||
|
|
||||||
|
return bytes(block)
|
||||||
|
|
||||||
|
|
||||||
|
def _write_psar(
|
||||||
|
out: io.BufferedWriter,
|
||||||
|
disc: "DiscImage",
|
||||||
|
game_id: str,
|
||||||
|
title: str,
|
||||||
|
progress_cb=None,
|
||||||
|
cancel_event=None,
|
||||||
|
) -> None:
|
||||||
|
"""Write the DATA.PSAR section (uncompressed, single-disc).
|
||||||
|
|
||||||
|
All offsets are relative to the start of the PSAR section.
|
||||||
|
"""
|
||||||
|
base = out.tell()
|
||||||
|
iso_size = disc.padded_size()
|
||||||
|
|
||||||
|
# 0x000: PSAR signature; 0x00C: disc size hint.
|
||||||
|
out.write(PSAR_MAGIC)
|
||||||
|
out.write(struct.pack("<I", iso_size + PSAR_ISO_OFFSET))
|
||||||
|
_pad_to(out, base + PSAR_GAMEID_OFFSET)
|
||||||
|
|
||||||
|
out.write(_build_psar_header(disc, game_id, title))
|
||||||
|
_pad_to(out, base + PSAR_INDEX_OFFSET)
|
||||||
|
|
||||||
|
# ISO block index: (offset, length) per 0x9300 block. Uncompressed =>
|
||||||
|
# length is always BLOCK_SIZE and offsets increase linearly.
|
||||||
|
block_count = iso_size // BLOCK_SIZE
|
||||||
|
offset = 0
|
||||||
|
for _ in range(block_count):
|
||||||
|
out.write(struct.pack("<II", offset, BLOCK_SIZE))
|
||||||
|
out.write(b"\x00" * (INDEX_ENTRY_SIZE - 8))
|
||||||
|
offset += BLOCK_SIZE
|
||||||
|
|
||||||
|
_pad_to(out, base + PSAR_ISO_OFFSET)
|
||||||
|
|
||||||
|
# Disc data, block by block.
|
||||||
|
buf = bytearray()
|
||||||
|
for sector in disc.iter_raw(progress_cb=progress_cb, cancel_event=cancel_event):
|
||||||
|
buf.extend(sector)
|
||||||
|
if len(buf) == BLOCK_SIZE:
|
||||||
|
out.write(buf)
|
||||||
|
buf = bytearray()
|
||||||
|
if buf:
|
||||||
|
out.write(buf.ljust(BLOCK_SIZE, b"\x00"))
|
||||||
|
|
||||||
|
|
||||||
|
class DiscImage:
|
||||||
|
"""A normalised view of a PS1 disc as a stream of raw 2352-byte sectors."""
|
||||||
|
|
||||||
|
def __init__(self, cue: Optional[CueDisc], path: Path):
|
||||||
|
self.cue = cue
|
||||||
|
self.path = path
|
||||||
|
|
||||||
|
def total_sectors(self) -> int:
|
||||||
|
if self.cue is None:
|
||||||
|
size = self.path.stat().st_size
|
||||||
|
if size % RAW_SECTOR == 0:
|
||||||
|
return size // RAW_SECTOR
|
||||||
|
return (size + COOKED_SECTOR - 1) // COOKED_SECTOR
|
||||||
|
total = 0
|
||||||
|
base_dir = self.path.parent
|
||||||
|
for track in self.cue.tracks:
|
||||||
|
fpath = base_dir / track.file
|
||||||
|
if fpath.is_file():
|
||||||
|
total += track.pregap_sectors + (fpath.stat().st_size // track.sector_size)
|
||||||
|
return total
|
||||||
|
|
||||||
|
def padded_size(self) -> int:
|
||||||
|
size = self.total_sectors() * RAW_SECTOR
|
||||||
|
if size % BLOCK_SIZE:
|
||||||
|
size += BLOCK_SIZE - (size % BLOCK_SIZE)
|
||||||
|
return size
|
||||||
|
|
||||||
|
def tracks_info(self) -> List[Tuple[int, bool, int]]:
|
||||||
|
"""Return ``(track_number, is_data, absolute_start_sector)`` per track.
|
||||||
|
|
||||||
|
``absolute_start_sector`` is the disc-wide sector of each track's
|
||||||
|
INDEX 01 (i.e. accounting for every preceding track's pregap + data).
|
||||||
|
A cue-less single image is treated as one data track starting at 0.
|
||||||
|
"""
|
||||||
|
if self.cue is None:
|
||||||
|
return [(1, True, 0)]
|
||||||
|
|
||||||
|
base_dir = self.path.parent
|
||||||
|
out: List[Tuple[int, bool, int]] = []
|
||||||
|
cursor = 0
|
||||||
|
for track in self.cue.tracks:
|
||||||
|
idx01 = dict(track.indexes).get(1, 0)
|
||||||
|
out.append((track.number, track.mode != "AUDIO", cursor + idx01))
|
||||||
|
fpath = base_dir / track.file
|
||||||
|
data_sectors = fpath.stat().st_size // track.sector_size if fpath.is_file() else 0
|
||||||
|
cursor += track.pregap_sectors + data_sectors
|
||||||
|
return out
|
||||||
|
|
||||||
|
def iter_raw(self, progress_cb=None, cancel_event=None) -> Iterator[bytes]:
|
||||||
|
if self.cue is None:
|
||||||
|
yield from iter_raw_sectors(
|
||||||
|
self.path,
|
||||||
|
RAW_SECTOR if self.path.stat().st_size % RAW_SECTOR == 0 else COOKED_SECTOR,
|
||||||
|
progress_cb=progress_cb,
|
||||||
|
cancel_event=cancel_event,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
base_dir = self.path.parent
|
||||||
|
for track in self.cue.tracks:
|
||||||
|
fpath = base_dir / track.file
|
||||||
|
if not fpath.is_file():
|
||||||
|
raise ConversionError(f"track file missing: {fpath}")
|
||||||
|
yield from iter_raw_sectors(
|
||||||
|
fpath,
|
||||||
|
track.sector_size,
|
||||||
|
pregap_sectors=track.pregap_sectors,
|
||||||
|
progress_cb=progress_cb,
|
||||||
|
cancel_event=cancel_event,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _aligned_up(value: int, alignment: int) -> int:
|
||||||
|
return value + ((alignment - (value % alignment)) % alignment)
|
||||||
|
|
||||||
|
|
||||||
|
def convert(
|
||||||
|
source: Path,
|
||||||
|
title: str,
|
||||||
|
game_id: str,
|
||||||
|
base_pbp: Path,
|
||||||
|
out_path: Path,
|
||||||
|
progress_cb=None,
|
||||||
|
cancel_event=None,
|
||||||
|
) -> Path:
|
||||||
|
"""Convert a PS1 disc image to ``out_path`` (an ``EBOOT.PBP``).
|
||||||
|
|
||||||
|
``source`` may be a ``.cue`` (multi-track supported) or a single
|
||||||
|
``.iso``/``.img``/``.bin``. ``base_pbp`` must be a PSN PS1 classic EBOOT.
|
||||||
|
"""
|
||||||
|
if not base_pbp.is_file():
|
||||||
|
raise ConversionError(f"base.pbp not found: {base_pbp}")
|
||||||
|
if not source.is_file():
|
||||||
|
raise ConversionError(f"disc image not found: {source}")
|
||||||
|
|
||||||
|
game_id = game_id.strip()
|
||||||
|
if not game_id:
|
||||||
|
raise ConversionError("a game id (e.g. SLUS00594) is required")
|
||||||
|
|
||||||
|
if source.suffix.lower() == ".cue":
|
||||||
|
disc: DiscImage = DiscImage(parse_cue(source.read_text(encoding="utf-8", errors="replace")), source)
|
||||||
|
else:
|
||||||
|
disc = DiscImage(None, source)
|
||||||
|
|
||||||
|
# Resources required from the base PBP.
|
||||||
|
data_psp = extract_data_psp(base_pbp)
|
||||||
|
startdat = extract_startdat(base_pbp)
|
||||||
|
|
||||||
|
sfo = default_ps1_sfo(title, game_id)
|
||||||
|
icon0 = default_icon0_png()
|
||||||
|
pic1 = default_pic1_png()
|
||||||
|
|
||||||
|
# Sections in header order; empty sections keep the previous offset.
|
||||||
|
sections = {
|
||||||
|
"sfo": sfo,
|
||||||
|
"icon0": icon0,
|
||||||
|
"icon1": b"",
|
||||||
|
"pic0": b"",
|
||||||
|
"pic1": pic1,
|
||||||
|
"snd0": b"",
|
||||||
|
"data_psp": data_psp,
|
||||||
|
# psar written separately, sized after everything else.
|
||||||
|
"psar": None,
|
||||||
|
}
|
||||||
|
|
||||||
|
header = [0] * PBP_HEADER_U32
|
||||||
|
header[0] = struct.unpack("<I", PBP_MAGIC)[0]
|
||||||
|
header[1] = 0x00010000
|
||||||
|
|
||||||
|
current = PBP_HEADER_SIZE
|
||||||
|
for i, key in enumerate(SECTION_ORDER):
|
||||||
|
header[2 + i] = current
|
||||||
|
if key != "psar":
|
||||||
|
current += len(sections[key])
|
||||||
|
|
||||||
|
# DATA.PSAR must start on a 0x10000 boundary. It is the final section, so
|
||||||
|
# its length is implicit (everything up to STARTDAT/EOF).
|
||||||
|
psar_offset = _aligned_up(header[2 + SECTION_ORDER.index("psar")], PSAR_ALIGNMENT)
|
||||||
|
header[2 + SECTION_ORDER.index("psar")] = psar_offset
|
||||||
|
|
||||||
|
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
tmp = out_path.with_suffix(out_path.suffix + ".tmp")
|
||||||
|
try:
|
||||||
|
with open(tmp, "wb") as fh:
|
||||||
|
fh.write(struct.pack("<10I", *header))
|
||||||
|
for key in SECTION_ORDER:
|
||||||
|
if key == "psar":
|
||||||
|
break
|
||||||
|
if sections[key]:
|
||||||
|
fh.write(sections[key])
|
||||||
|
|
||||||
|
# Pad up to the aligned PSAR offset.
|
||||||
|
while fh.tell() < psar_offset:
|
||||||
|
fh.write(b"\x00")
|
||||||
|
|
||||||
|
_write_psar(fh, disc, game_id, title, progress_cb=progress_cb, cancel_event=cancel_event)
|
||||||
|
|
||||||
|
# STARTDAT goes at the very end.
|
||||||
|
fh.write(startdat)
|
||||||
|
os.replace(tmp, out_path)
|
||||||
|
except BaseException:
|
||||||
|
try:
|
||||||
|
tmp.unlink(missing_ok=True)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
raise
|
||||||
|
return out_path
|
||||||
|
|
||||||
|
|
||||||
|
def convert_to_folder(
|
||||||
|
source: Path,
|
||||||
|
title: str,
|
||||||
|
game_id: str,
|
||||||
|
base_pbp: Path,
|
||||||
|
out_dir: Path,
|
||||||
|
progress_cb=None,
|
||||||
|
cancel_event=None,
|
||||||
|
) -> Path:
|
||||||
|
"""Convert a PS1 disc into a game folder ready for /PSP/GAME.
|
||||||
|
|
||||||
|
Writes ``<out_dir>/<GAME_ID>/EBOOT.PBP`` — the same single-file output the
|
||||||
|
popstation tool produces (uncompressed/unencrypted level-0 PSAR). No
|
||||||
|
KEYS.BIN or DOCUMENT.DAT is written: popstation doesn't emit them and CFW
|
||||||
|
(ARK-4 POPS) doesn't need them.
|
||||||
|
"""
|
||||||
|
folder_name = re.sub(r"[^A-Za-z0-9]", "", game_id).upper() or "PS1GAME"
|
||||||
|
folder = Path(out_dir) / folder_name
|
||||||
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
convert(source, title, game_id, base_pbp, folder / "EBOOT.PBP",
|
||||||
|
progress_cb=progress_cb, cancel_event=cancel_event)
|
||||||
|
return folder
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
# Self-test / CLI
|
||||||
|
# --------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _self_test() -> None:
|
||||||
|
"""Validate the parts that do not need hardware: SFO round-trip, PNG
|
||||||
|
generation, PBP header round-trip, and cue parsing."""
|
||||||
|
import tempfile
|
||||||
|
|
||||||
|
# CUE parsing.
|
||||||
|
cue = parse_cue(
|
||||||
|
'FILE "Game (Track 1).bin" BINARY\n'
|
||||||
|
" TRACK 01 MODE2/2352\n"
|
||||||
|
" INDEX 01 00:00:00\n"
|
||||||
|
" TRACK 02 AUDIO\n"
|
||||||
|
" INDEX 00 00:02:00\n"
|
||||||
|
" INDEX 01 00:04:00\n"
|
||||||
|
'FILE "Game (Track 2).bin" BINARY\n'
|
||||||
|
" TRACK 03 AUDIO\n"
|
||||||
|
" INDEX 01 00:00:00\n"
|
||||||
|
)
|
||||||
|
assert len(cue.tracks) == 3, "expected 3 tracks"
|
||||||
|
assert cue.tracks[0].mode == "MODE2/2352"
|
||||||
|
assert cue.tracks[1].pregap_sectors == 150, "expected 2s (150 sector) pregap"
|
||||||
|
assert cue.data_track is cue.tracks[0]
|
||||||
|
|
||||||
|
# TOC generation: 1 data (10 sectors) + 1 audio track (20 sectors + 2s pregap).
|
||||||
|
with tempfile.TemporaryDirectory() as td:
|
||||||
|
d = Path(td)
|
||||||
|
(d / "t1.bin").write_bytes(b"\x00" * (10 * RAW_SECTOR))
|
||||||
|
(d / "t2.bin").write_bytes(b"\x00" * (20 * RAW_SECTOR))
|
||||||
|
cue2 = parse_cue(
|
||||||
|
'FILE "t1.bin" BINARY\n TRACK 01 MODE2/2352\n INDEX 01 00:00:00\n'
|
||||||
|
'FILE "t2.bin" BINARY\n TRACK 02 AUDIO\n INDEX 00 00:00:00\n INDEX 01 00:02:00\n'
|
||||||
|
)
|
||||||
|
toc = build_toc(DiscImage(cue2, d / "t.cue"))
|
||||||
|
assert len(toc) == (2 + 3) * 10, len(toc)
|
||||||
|
assert toc[0] == 0x41 and toc[2] == 0xA0 and toc[7] == 0x01 # A0: first track
|
||||||
|
assert toc[10] == 0x01 and toc[12] == 0xA1 and toc[17] == 0x02 # A1: last track
|
||||||
|
assert toc[22] == 0xA2 # A2: lead-out
|
||||||
|
assert toc[30] == 0x41 and toc[32] == 0x01 # track 1: data, number 1
|
||||||
|
assert toc[40] == 0x01 and toc[42] == 0x02 # track 2: audio, number 2
|
||||||
|
# track 2 relative address = start(160) + 150 lead-in = 310 = 00:04:10
|
||||||
|
assert toc[47] == 0x00 and toc[48] == 0x04 and toc[49] == 0x10
|
||||||
|
|
||||||
|
# SFO round-trip through the project's own parser.
|
||||||
|
from . import iso_reader # local import to keep the module importable standalone
|
||||||
|
|
||||||
|
sfo = default_ps1_sfo("Final Fantasy VII", "SCUS94163")
|
||||||
|
parsed = iso_reader.parse_sfo(sfo)
|
||||||
|
assert parsed.get("TITLE") == "Final Fantasy VII", parsed
|
||||||
|
assert parsed.get("DISC_ID") == "SCUS94163", parsed
|
||||||
|
assert parsed.get("CATEGORY") == "ME", parsed
|
||||||
|
assert parsed.get("BOOTABLE") == 1, parsed
|
||||||
|
|
||||||
|
# PNG generation produces valid PNG signatures + IHDR dimensions.
|
||||||
|
icon = default_icon0_png()
|
||||||
|
assert icon[:8] == b"\x89PNG\r\n\x1a\n"
|
||||||
|
assert struct.unpack(">II", icon[16:24]) == (144, 80)
|
||||||
|
pic = default_pic1_png()
|
||||||
|
assert struct.unpack(">II", pic[16:24]) == (480, 272)
|
||||||
|
|
||||||
|
# PBP header round-trip: write a synthetic PBP and read it back.
|
||||||
|
with tempfile.TemporaryDirectory() as td:
|
||||||
|
p = Path(td) / "test.pbp"
|
||||||
|
header = [0] * 10
|
||||||
|
header[0] = struct.unpack("<I", PBP_MAGIC)[0]
|
||||||
|
header[1] = 0x00010000
|
||||||
|
header[2] = 0x28
|
||||||
|
header[3] = 0x28 + len(sfo)
|
||||||
|
with open(p, "wb") as fh:
|
||||||
|
fh.write(struct.pack("<10I", *header))
|
||||||
|
fh.write(sfo)
|
||||||
|
assert read_pbp_header(p)[0] == header[0]
|
||||||
|
assert read_pbp_section(p, header, 2) == sfo
|
||||||
|
|
||||||
|
print("ps1_convert self-test OK")
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: Optional[List[str]] = None) -> int:
|
||||||
|
ap = argparse.ArgumentParser(description="Convert a PS1 disc image to a PSP EBOOT.PBP")
|
||||||
|
ap.add_argument("--source", help="path to .cue or a single .iso/.img/.bin")
|
||||||
|
ap.add_argument("--title", help="game title shown on the PSP")
|
||||||
|
ap.add_argument("--game-id", help="PS1 game id, e.g. SLUS00594")
|
||||||
|
ap.add_argument("--base-pbp", help="path to a PSN PS1 classic EBOOT.PBP (required)")
|
||||||
|
ap.add_argument("--out", help="output EBOOT.PBP path (single file)")
|
||||||
|
ap.add_argument("--out-dir", help="output folder: writes <dir>/<GAME_ID>/EBOOT.PBP")
|
||||||
|
ap.add_argument("--test", action="store_true", help="run self-test and exit")
|
||||||
|
args = ap.parse_args(argv)
|
||||||
|
|
||||||
|
if args.test:
|
||||||
|
_self_test()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
if not all([args.source, args.title, args.game_id, args.base_pbp]) or (not args.out and not args.out_dir):
|
||||||
|
ap.error("--source --title --game-id --base-pbp and --out (or --out-dir) are all required (or use --test)")
|
||||||
|
|
||||||
|
try:
|
||||||
|
if args.out_dir:
|
||||||
|
folder = convert_to_folder(
|
||||||
|
Path(args.source), args.title, args.game_id,
|
||||||
|
Path(args.base_pbp), Path(args.out_dir),
|
||||||
|
)
|
||||||
|
print(f"wrote {folder}")
|
||||||
|
else:
|
||||||
|
convert(
|
||||||
|
Path(args.source), args.title, args.game_id,
|
||||||
|
Path(args.base_pbp), Path(args.out),
|
||||||
|
)
|
||||||
|
print(f"wrote {args.out}")
|
||||||
|
return 0
|
||||||
|
except ConversionError as exc:
|
||||||
|
print(f"error: {exc}")
|
||||||
|
return 1
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -18,6 +18,8 @@ FIELDS = [
|
|||||||
"igdb_client_id",
|
"igdb_client_id",
|
||||||
"igdb_client_secret",
|
"igdb_client_secret",
|
||||||
"tmdb_api_key",
|
"tmdb_api_key",
|
||||||
|
"google_api_key",
|
||||||
|
"google_cx",
|
||||||
"screenscraper_devid",
|
"screenscraper_devid",
|
||||||
"screenscraper_devpassword",
|
"screenscraper_devpassword",
|
||||||
"screenscraper_ssid",
|
"screenscraper_ssid",
|
||||||
@@ -29,6 +31,8 @@ _ENV_MAP = {
|
|||||||
"igdb_client_id": "IGDB_CLIENT_ID",
|
"igdb_client_id": "IGDB_CLIENT_ID",
|
||||||
"igdb_client_secret": "IGDB_CLIENT_SECRET",
|
"igdb_client_secret": "IGDB_CLIENT_SECRET",
|
||||||
"tmdb_api_key": "TMDB_API_KEY",
|
"tmdb_api_key": "TMDB_API_KEY",
|
||||||
|
"google_api_key": "GOOGLE_API_KEY",
|
||||||
|
"google_cx": "GOOGLE_CX",
|
||||||
"screenscraper_devid": "SCREENSCRAPER_DEVID",
|
"screenscraper_devid": "SCREENSCRAPER_DEVID",
|
||||||
"screenscraper_devpassword": "SCREENSCRAPER_DEVPASSWORD",
|
"screenscraper_devpassword": "SCREENSCRAPER_DEVPASSWORD",
|
||||||
"screenscraper_ssid": "SCREENSCRAPER_SSID",
|
"screenscraper_ssid": "SCREENSCRAPER_SSID",
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ archives, a remote HTTPS store server, etc.
|
|||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
|
import re
|
||||||
import shutil
|
import shutil
|
||||||
import tempfile
|
import tempfile
|
||||||
import urllib.parse
|
import urllib.parse
|
||||||
@@ -200,6 +201,54 @@ class FileCopySource(Source):
|
|||||||
return dest
|
return dest
|
||||||
|
|
||||||
|
|
||||||
|
class Ps1ConvertSource(Source):
|
||||||
|
"""Convert a PS1 disc image (.cue/.iso) into an EBOOT.PBP.
|
||||||
|
|
||||||
|
``item`` must carry ``source`` (path to the .cue or single image),
|
||||||
|
``title`` and ``game_id``. ``base_pbp`` is a PSN PS1 classic EBOOT used to
|
||||||
|
supply the encrypted DATA.PSP loader and STARTDAT block.
|
||||||
|
"""
|
||||||
|
|
||||||
|
label = "PS1 convert"
|
||||||
|
|
||||||
|
def __init__(self, base_pbp):
|
||||||
|
self.base_pbp = Path(base_pbp)
|
||||||
|
|
||||||
|
def scan(self, root=None):
|
||||||
|
return []
|
||||||
|
|
||||||
|
def download(self, item, dest_dir, progress_cb=None, cancel_event=None):
|
||||||
|
dest_dir = Path(dest_dir)
|
||||||
|
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
source = Path(item["source"])
|
||||||
|
title = str(item.get("title") or source.stem)
|
||||||
|
game_id = item.get("game_id")
|
||||||
|
if not game_id:
|
||||||
|
raise RuntimeError("missing game_id for PS1 conversion")
|
||||||
|
|
||||||
|
folder_name = re.sub(r"[^A-Za-z0-9]", "", game_id).upper() or "PS1GAME"
|
||||||
|
folder = dest_dir / folder_name
|
||||||
|
eboot = folder / "EBOOT.PBP"
|
||||||
|
if eboot.exists():
|
||||||
|
raise FileExistsError(str(eboot))
|
||||||
|
|
||||||
|
def cb(done, total):
|
||||||
|
if progress_cb is not None and total:
|
||||||
|
progress_cb("Converting", min(done / total, 1.0), "")
|
||||||
|
|
||||||
|
from . import ps1_convert
|
||||||
|
|
||||||
|
try:
|
||||||
|
ps1_convert.convert_to_folder(
|
||||||
|
source, title, game_id, self.base_pbp, dest_dir,
|
||||||
|
progress_cb=cb, cancel_event=cancel_event,
|
||||||
|
)
|
||||||
|
except ps1_convert.Cancelled:
|
||||||
|
raise TransferCancelled() from None
|
||||||
|
return folder
|
||||||
|
|
||||||
|
|
||||||
class TreeCopySource(Source):
|
class TreeCopySource(Source):
|
||||||
"""Copy a whole directory (save-data folder) into a destination directory."""
|
"""Copy a whole directory (save-data folder) into a destination directory."""
|
||||||
|
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
clrmamepro (
|
clrmamepro (
|
||||||
name "Sony - PlayStation Portable"
|
name "Sony - PlayStation Portable (No-Intro)"
|
||||||
description "Sony - PlayStation Portable"
|
description "Sony - PlayStation Portable"
|
||||||
version "2021.10.24"
|
version "2021.10.24"
|
||||||
homepage "http://github.com/robloach/libretro-dats"
|
homepage "http://github.com/robloach/libretro-dats"
|
||||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user