Compare commits
14
Commits
00f9e24c79
...
02b4adb1c1
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
02b4adb1c1 | ||
|
|
a9c371c221 | ||
|
|
4d5c48cb94 | ||
|
|
777d316c0f | ||
|
|
625e2f8718 | ||
|
|
7d8b449846 | ||
|
|
27e482d344 | ||
|
|
5d619e5f59 | ||
|
|
743d5e8e90 | ||
|
|
bb1b8dea66 | ||
|
|
2d599141f9 | ||
|
|
922c01d17c | ||
|
|
128b62e5b8 | ||
|
|
da3429d918 |
@@ -6,3 +6,7 @@ data/*.sqlite
|
||||
data/*.sqlite-*
|
||||
data/credentials.json
|
||||
session-transcript-2026-09-01.md
|
||||
|
||||
# Copyrighted PSN EBOOTs (base.pbp) used only for local PS1 conversion testing
|
||||
data/*.PBP
|
||||
data/*.pbp
|
||||
|
||||
@@ -3,10 +3,11 @@
|
||||

|
||||
|
||||
Native Linux GUI (PySide6) for preserving PSP UMD **games** and **videos** as
|
||||
`.iso` files. It scans your discs, verifies them against a full catalog
|
||||
(No-Intro DATs), reports what's missing, and retags/renames discs to a canonical
|
||||
`.iso` files. It scans your discs, verifies them against the full Redump catalog
|
||||
(Redump DATs), reports what's missing, and retags/renames discs to a canonical
|
||||
scheme. It also fetches cover art and metadata, syncs discs and save-data with a
|
||||
real PSP, and can pull missing discs from your own Redump 7z archive collection.
|
||||
real PSP, can pull missing discs from your own Redump 7z archive collection, and
|
||||
converts PlayStation 1 ISO/CUE/BIN rips into PSP `EBOOT.PBP` homebrew.
|
||||
|
||||

|
||||
* Use default app theme for Qt.
|
||||
@@ -25,6 +26,9 @@ real PSP, and can pull missing discs from your own Redump 7z archive collection.
|
||||
from the PSP, and dump a UMD to the library.
|
||||
- **Saves** — two-way sync of save-data between the PSP, PPSSPP, and a cloud
|
||||
folder.
|
||||
- **PS1** — scan a folder of PlayStation 1 discs (ISO/BIN/CUE), read each disc's
|
||||
serial, convert to PSP `EBOOT.PBP` with a pure-Python popstation converter,
|
||||
and track already-converted games.
|
||||
- **Settings** — directories, cover-source API keys, store/server config, and
|
||||
misc options.
|
||||
|
||||
@@ -52,8 +56,11 @@ python3 -m app.main
|
||||
## First-time setup
|
||||
|
||||
1. Put your DAT files in `data/dats/`:
|
||||
- games: No-Intro `Sony - PlayStation Portable`
|
||||
- videos: No-Intro `Sony - PlayStation Portable (UMD Video)`
|
||||
- games: Redump `Sony - PlayStation Portable` (authoritative hashes)
|
||||
**and** No-Intro `Sony - PlayStation Portable` (supplies the serials
|
||||
Redump lacks — keeps serial-based matching working).
|
||||
- videos: No-Intro `Sony - PlayStation Portable (UMD Video)` (Redump has
|
||||
no UMD Video DAT).
|
||||
2. Ingest them into the local SQLite library:
|
||||
|
||||
```bash
|
||||
@@ -62,8 +69,12 @@ python3 -m app.main
|
||||
|
||||
3. Open the GUI and point **Settings → General → Games/Videos directory** at
|
||||
your disc folders (or set `PSP_GAMES_ROOTS` / `PSP_VIDEO_ROOTS`).
|
||||
4. (Optional) PS1 conversion — copy any PSN PS1 classic `EBOOT.PBP` to
|
||||
`data/base.PBP` (it supplies Sony's encrypted loader). Then set the **PS1
|
||||
directory**, **PS1 output directory** and **PS1 base.pbp** in
|
||||
**Settings → General** and use the **PS1** tab (see [PS1 support](#ps1-support)).
|
||||
|
||||
Discs that aren't in the No-Intro DATs can be added to `data/supplement.json`
|
||||
Discs that aren't in the Redump DATs can be added to `data/supplement.json`
|
||||
(`name`, `region`, `crc32`, `sha1`, `md5`) so the tool still recognizes them.
|
||||
|
||||
## Nomenclature
|
||||
@@ -75,13 +86,15 @@ Discs are matched by hash first (most precise), then serial, then filename.
|
||||
## Directory layout
|
||||
|
||||
- `data/dats/` — clrmamepro DAT files (games + videos).
|
||||
- `data/supplement.json` — extra discs not in No-Intro.
|
||||
- `data/supplement.json` — extra discs not in Redump.
|
||||
- `data/credentials.json` — cover/metadata API keys (IGDB, TheMovieDB,
|
||||
ScreenScraper). Environment variables take precedence.
|
||||
ScreenScraper, Google Custom Search). Environment variables take precedence.
|
||||
- `data/manual_covers/games|videos/` — your own cover images.
|
||||
- `data/covers/` — checked-in assets: the UMD video/game cover templates,
|
||||
`libretro-index.json` catalog, `libretro-covers.zip` + `openvgdb-covers.zip`
|
||||
offline cover bundles, and the legacy cover cache.
|
||||
`libretro-index.json` + `openvgdb-index.json` catalogs, `libretro-covers.zip` +
|
||||
`openvgdb-covers.zip` offline cover bundles, and the legacy cover cache.
|
||||
- `data/base.PBP` — a PSN PS1 classic EBOOT (any game) used as the encrypted
|
||||
loader when converting PS1 discs to `EBOOT.PBP` (git-ignored, copyrighted).
|
||||
- State dir (`~/.local/state/psp-preserve/` by default, or `$XDG_STATE_HOME/psp-preserve`):
|
||||
- `library.sqlite` — the catalog + library database.
|
||||
- `covers/` — downloaded cover cache.
|
||||
@@ -104,6 +117,8 @@ QSettings under `PSPPreserve`) taking precedence where noted.
|
||||
| `PSP_COVERS_DIR` | `$PSP_STATE_DIR/covers` | cover cache |
|
||||
| `PSP_GAMES_ROOTS` | `/data/umd_games` | comma-separated game disc roots |
|
||||
| `PSP_VIDEO_ROOTS` | `/data/umd_videos` | comma-separated video disc roots |
|
||||
| `PSP_PS1_ROOTS` | `/data/ps1` | comma-separated PS1 disc roots |
|
||||
| `PSP_PS1_OUTPUT_DIR` | `/data/ps1_converted` | converted EBOOT.PBP output folder |
|
||||
| `PSP_SCAN_WORKERS` | `1` | parallel hashing workers |
|
||||
| `PSP_STORE_DIR` | `/media/downloads/Sony - PlayStation Portable` | local 7z store folder |
|
||||
| `PSP_STORE_SERVER` | `https://psp.example.com` | store server base URL |
|
||||
@@ -117,9 +132,9 @@ QSettings under `PSPPreserve`) taking precedence where noted.
|
||||
| `PSP_OVGDB_COVERS_URL` | *(empty)* | direct URL for `openvgdb-covers.zip` (else derived from store server) |
|
||||
|
||||
Cover/metadata keys can also be set as env vars: `IGDB_CLIENT_ID`,
|
||||
`IGDB_CLIENT_SECRET`, `TMDB_API_KEY`, `SCREENSCRAPER_DEVID`,
|
||||
`SCREENSCRAPER_DEVPASSWORD`, `SCREENSCRAPER_SSID`, `SCREENSCRAPER_SSPASSWORD`,
|
||||
`SCREENSCRAPER_SOFTNAME`.
|
||||
`IGDB_CLIENT_SECRET`, `TMDB_API_KEY`, `GOOGLE_API_KEY`, `GOOGLE_CX`,
|
||||
`SCREENSCRAPER_DEVID`, `SCREENSCRAPER_DEVPASSWORD`, `SCREENSCRAPER_SSID`,
|
||||
`SCREENSCRAPER_SSPASSWORD`, `SCREENSCRAPER_SOFTNAME`.
|
||||
|
||||
## Cover art
|
||||
|
||||
@@ -139,8 +154,11 @@ Games are resolved in this order (offline-first, then network):
|
||||
3:4 key art, so the app composites it into the PSP game cover template
|
||||
(`data/covers/UMD_GAME_COVER_TEMPLATE.png`) to restore the full cover.
|
||||
|
||||
UMD **videos** have no public box-art API: they use your manual covers, or a
|
||||
TheMovieDB poster composited into `data/covers/UMD_COVER_TEMPLATE.png`.
|
||||
UMD **videos** have no public box-art API. The manual **Scrape…** dialog offers
|
||||
three sources: TheMovieDB posters, Wikipedia image search (keyless), and Google
|
||||
Images (needs a Google API key + Search Engine ID in **Settings → Cover
|
||||
sources**). The pick is composited into `data/covers/UMD_COVER_TEMPLATE.png`.
|
||||
Manual covers in `data/manual_covers/videos/` always win.
|
||||
|
||||
### Offline bundles & auto-download
|
||||
|
||||
@@ -158,6 +176,29 @@ python3 -m app.openvgdb # OpenVGDB/GameFAQs -> openvgdb-covers.zip
|
||||
|
||||
(`app.openvgdb` needs `curl_cffi` to pass GameFAQs' Cloudflare challenge.)
|
||||
|
||||
## PS1 support
|
||||
|
||||
The **PS1** tab scans a folder of PlayStation 1 rips and converts them into PSP
|
||||
`EBOOT.PBP` files with a pure-Python popstation converter (no external tools).
|
||||
PS1 discs (pending) and converted EBOOTs also show up in the **Library** tab
|
||||
(filter by **PS1**), where you can convert a disc or copy an EBOOT to the PSP.
|
||||
|
||||
- **Source formats** — `.iso`, `.img`, `.bin` (with a matching `.cue` for
|
||||
multi-track discs).
|
||||
- **Serial detection** — reads `SYSTEM.CNF` to derive the game ID (e.g.
|
||||
`SLUS-00001`) and writes it into `PARAM.SFO` + the PSAR header.
|
||||
- **base.pbp** — conversion needs a PSN PS1 classic `EBOOT.PBP` (any game) to
|
||||
supply Sony's encrypted `DATA.PSP` loader. Copy one to `data/base.PBP` and
|
||||
point **Settings → General → PS1 base.pbp** at it.
|
||||
- **Output** — a folder `<out>/<Game ID>/` containing a single `EBOOT.PBP`
|
||||
(the popstation way: uncompressed level-0 PSAR, no `KEYS.BIN`/`DOCUMENT.DAT`),
|
||||
ready to copy to `<PSP>/PSP/GAME/<Game ID>/`. Compatible with CFW POPS
|
||||
(ARK-4).
|
||||
|
||||
Configure the folders in **Settings → General** (or `PSP_PS1_ROOTS` /
|
||||
`PSP_PS1_OUTPUT_DIR`). Current limits: single-disc games only (no multi-disc
|
||||
PBP), and images are stored uncompressed (LZRC compression is on the roadmap).
|
||||
|
||||
## Store — archive.org and local folder
|
||||
|
||||
The **Store** tab downloads missing discs from a Redump 7z archive collection
|
||||
@@ -289,6 +330,10 @@ python3 -m app.ingest --dats data/dats --db data/library.sqlite
|
||||
# (re)build the offline cover bundles (see Cover art)
|
||||
python3 -m app.libretro
|
||||
python3 -m app.openvgdb
|
||||
|
||||
# convert one PS1 disc to EBOOT.PBP (CLI)
|
||||
python3 -m app.ps1_convert --source /path/to/game.cue --title "Game" \
|
||||
--game-id SLUS00001 --base-pbp data/base.PBP --out /out/Game.pbp
|
||||
```
|
||||
|
||||
`python3 -m app.scan --json` prints machine-readable results, and `--quick`
|
||||
|
||||
@@ -50,10 +50,17 @@ ARCHIVE_LOGGED_IN_USER = os.environ.get("PSP_ARCHIVE_LOGGED_IN_USER", "")
|
||||
GAMES_ROOTS = [Path(p) for p in os.environ.get("PSP_GAMES_ROOTS", "/data/umd_games").split(",") if p.strip()]
|
||||
VIDEO_ROOTS = [Path(p) for p in os.environ.get("PSP_VIDEO_ROOTS", "/data/umd_videos").split(",") if p.strip()]
|
||||
|
||||
# PS1: source folder (raw ISO/BIN/CUE) and converted EBOOT.PBP output folder.
|
||||
PS1_ROOTS = [Path(p) for p in os.environ.get("PSP_PS1_ROOTS", "/data/ps1").split(",") if p.strip()]
|
||||
PS1_OUTPUT_DIR = Path(os.environ.get("PSP_PS1_OUTPUT_DIR", "/data/ps1_converted"))
|
||||
|
||||
# Only .iso discs are tracked; everything below is explicitly ignored.
|
||||
DISC_EXTENSIONS = {".iso"}
|
||||
IGNORED_EXTENSIONS = {".pkg", ".pbp", ".cso"}
|
||||
|
||||
# PS1 source images (scanned separately from PSP .iso discs).
|
||||
PS1_EXTENSIONS = {".cue", ".iso", ".img", ".bin"}
|
||||
|
||||
# Parallel hashing workers for the scanner. Keep at 1 on spinning disks
|
||||
# (concurrent reads cause seek thrash); raise it for SSD / fast arrays.
|
||||
SCAN_WORKERS = int(os.environ.get("PSP_SCAN_WORKERS", "1"))
|
||||
|
||||
+281
-93
@@ -24,18 +24,34 @@ from .db import get_connection
|
||||
|
||||
log = logging.getLogger("covers")
|
||||
|
||||
_LIBRETRO_BASE = "https://raw.githubusercontent.com/libretro-thumbnails/Sony_-_PlayStation_Portable/master"
|
||||
_LIBRETRO_KINDS = ("Named_Boxarts",)
|
||||
_TREE_URL = "https://api.github.com/repos/libretro-thumbnails/Sony_-_PlayStation_Portable/git/trees/master?recursive=1"
|
||||
_PLATFORMS = {
|
||||
"games": {"libretro_repo": "Sony_-_PlayStation_Portable", "igdb_id": 38, "ss_id": "61"},
|
||||
"ps1": {"libretro_repo": "Sony_-_PlayStation", "igdb_id": 7, "ss_id": "57"},
|
||||
}
|
||||
|
||||
|
||||
def _platform(system):
|
||||
"""Platform parameters (libretro repo, IGDB id, ScreenScraper id)."""
|
||||
return _PLATFORMS.get(system) or _PLATFORMS["games"]
|
||||
|
||||
|
||||
def _libretro_base(system="games"):
|
||||
return ("https://raw.githubusercontent.com/libretro-thumbnails/"
|
||||
f"{_platform(system)['libretro_repo']}/master")
|
||||
|
||||
|
||||
def _libretro_tree_url(system="games"):
|
||||
return ("https://api.github.com/repos/libretro-thumbnails/"
|
||||
f"{_platform(system)['libretro_repo']}/git/trees/master?recursive=1")
|
||||
|
||||
_SS_API = "https://api.screenscraper.fr/api2/jeuInfos.php"
|
||||
_SS_PSP_ID = "61" # Sony PSP
|
||||
_IGDB_TOKEN_URL = "https://id.twitch.tv/oauth2/token"
|
||||
_IGDB_API = "https://api.igdb.com/v4/games"
|
||||
_IGDB_IMAGE = "https://images.igdb.com/igdb/image/upload/"
|
||||
_IGDB_PSP_ID = 38 # Sony PSP
|
||||
_TMDB_SEARCH = "https://api.themoviedb.org/3/search/movie"
|
||||
_TMDB_IMAGE = "https://image.tmdb.org/t/p/"
|
||||
_GOOGLE_CSE_API = "https://www.googleapis.com/customsearch/v1"
|
||||
_MANUAL_DIR = config.DATA_DIR / "manual_covers"
|
||||
_OVGDB_ZIP = config.DATA_DIR / "covers" / "openvgdb-covers.zip"
|
||||
_LIBRETRO_ZIP = config.DATA_DIR / "covers" / "libretro-covers.zip"
|
||||
@@ -51,7 +67,7 @@ _igdb_token_state = {"token": "", "expires": 0.0} # IGDB OAuth2 token cache
|
||||
_igdb_lock = threading.Lock()
|
||||
_igdb_last = 0.0 # monotonic clock of the last IGDB API call
|
||||
|
||||
_index_state = {"ts": 0.0, "names": {}, "norm": {}}
|
||||
_index_state = {} # system -> {"ts", "names", "norm"}
|
||||
_INDEX_TTL = 86400.0 # refresh the libretro file index once a day
|
||||
|
||||
_neg_cache = {} # cache key -> time.time() of the last failed fetch
|
||||
@@ -186,9 +202,9 @@ def ensure_cover_bundles(server_url=""):
|
||||
return downloaded
|
||||
|
||||
|
||||
def cover_url(name, kind="Named_Boxarts"):
|
||||
def cover_url(name, kind="Named_Boxarts", system="games"):
|
||||
"""Public libretro URL of an image for a catalog name."""
|
||||
return f"{_LIBRETRO_BASE}/{kind}/{urllib.parse.quote(name)}.png"
|
||||
return f"{_libretro_base(system)}/{kind}/{urllib.parse.quote(name)}.png"
|
||||
|
||||
|
||||
def cover_path(release_id, system):
|
||||
@@ -260,7 +276,7 @@ def _candidate_names(name):
|
||||
yield s
|
||||
|
||||
|
||||
def _fetch_png(name, kind, timeout):
|
||||
def _fetch_png(name, kind, timeout, system="games"):
|
||||
"""Fetch a real image from libretro, following symlink text bodies.
|
||||
|
||||
GitHub serves a repo symlink as a tiny text body containing the target
|
||||
@@ -268,7 +284,7 @@ def _fetch_png(name, kind, timeout):
|
||||
"""
|
||||
target = name
|
||||
for _ in range(_MAX_HOPS + 1):
|
||||
data = _http_get(cover_url(target, kind), timeout)
|
||||
data = _http_get(cover_url(target, kind, system), timeout)
|
||||
if data is None:
|
||||
return None
|
||||
if _is_image(data):
|
||||
@@ -283,26 +299,29 @@ def _fetch_png(name, kind, timeout):
|
||||
# --------------------------------------------------------------------------- #
|
||||
# libretro filename index + fuzzy matching
|
||||
# --------------------------------------------------------------------------- #
|
||||
def _index_path():
|
||||
return config.COVERS_DIR / "libretro-index.json"
|
||||
def _index_path(system="games"):
|
||||
if system == "games":
|
||||
return config.COVERS_DIR / "libretro-index.json"
|
||||
return config.COVERS_DIR / f"libretro-index-{system}.json"
|
||||
|
||||
|
||||
def _load_index(force=False):
|
||||
st = _index_state
|
||||
def _load_index(force=False, system="games"):
|
||||
st = _index_state.get(system) or {"ts": 0.0, "names": {}, "norm": {}}
|
||||
_index_state[system] = st
|
||||
if not force and st["names"] and time.time() - st["ts"] < _INDEX_TTL:
|
||||
return st["names"]
|
||||
path = _index_path()
|
||||
path = _index_path(system)
|
||||
if not force and path.exists():
|
||||
try:
|
||||
# Load the on-disk index so it doubles as a fallback if the network
|
||||
# is down, but only return it immediately while it's still fresh.
|
||||
names = json.loads(path.read_text(encoding="utf-8"))
|
||||
_set_index(names)
|
||||
_set_index(names, system)
|
||||
if time.time() - path.stat().st_mtime < _INDEX_TTL:
|
||||
return names
|
||||
except Exception:
|
||||
pass
|
||||
data = _http_get(_TREE_URL, timeout=30)
|
||||
data = _http_get(_libretro_tree_url(system), timeout=30)
|
||||
if data:
|
||||
try:
|
||||
tree = json.loads(data).get("tree", [])
|
||||
@@ -313,7 +332,7 @@ def _load_index(force=False):
|
||||
prefix = kind + "/"
|
||||
if p.startswith(prefix) and p.endswith(".png"):
|
||||
names[kind].append(p[len(prefix):-4])
|
||||
_set_index(names)
|
||||
_set_index(names, system)
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(names), encoding="utf-8")
|
||||
@@ -325,9 +344,10 @@ def _load_index(force=False):
|
||||
return st["names"] or {kind: [] for kind in _LIBRETRO_KINDS}
|
||||
|
||||
|
||||
def _set_index(names):
|
||||
def _set_index(names, system="games"):
|
||||
"""Store the raw filename index plus a pre-normalized copy for fast lookup."""
|
||||
st = _index_state
|
||||
st = _index_state.get(system) or {"ts": 0.0, "names": {}, "norm": {}}
|
||||
_index_state[system] = st
|
||||
st["names"] = names
|
||||
st["norm"] = {k: [(_norm(n), n) for n in v] for k, v in names.items()}
|
||||
st["ts"] = time.time()
|
||||
@@ -343,11 +363,12 @@ def _plain(s):
|
||||
return re.sub(r"\s*\([^()]*\)", "", s).strip()
|
||||
|
||||
|
||||
def _index_best(name, kind):
|
||||
def _index_best(name, kind, system="games"):
|
||||
"""Return the best-matching repo filename for `name`, or None."""
|
||||
st = _index_state
|
||||
st = _index_state.get(system) or {"ts": 0.0, "names": {}, "norm": {}}
|
||||
if not st["norm"]:
|
||||
_load_index()
|
||||
_load_index(system=system)
|
||||
st = _index_state.get(system) or st
|
||||
pairs = st["norm"].get(kind, [])
|
||||
db = _norm(name)
|
||||
if not pairs or len(db) < 5:
|
||||
@@ -384,24 +405,25 @@ def _index_best(name, kind):
|
||||
return best
|
||||
|
||||
|
||||
def _fetch_libretro(name, timeout, indexed_only=False):
|
||||
# 0. Shipped offline bundle first (fast, no network).
|
||||
data = fetch_libretro_local(name)
|
||||
if data is not None:
|
||||
return data
|
||||
def _fetch_libretro(name, timeout, system="games", indexed_only=False):
|
||||
# 0. Shipped offline bundle first (fast, no network) — PSP only for now.
|
||||
if system == "games":
|
||||
data = fetch_libretro_local(name)
|
||||
if data is not None:
|
||||
return data
|
||||
if not indexed_only:
|
||||
# 1. Cheap: exact + progressively stripped names (no index needed).
|
||||
for kind in _LIBRETRO_KINDS:
|
||||
for cand in _candidate_names(name):
|
||||
data = _fetch_png(cand, kind, timeout)
|
||||
data = _fetch_png(cand, kind, timeout, system)
|
||||
if data is not None:
|
||||
return data
|
||||
# 2. Fuzzy/exact via the precomputed index (covers exact names too, so the
|
||||
# bulk loader skips the wasteful exact pass entirely).
|
||||
for kind in _LIBRETRO_KINDS:
|
||||
best = _index_best(name, kind)
|
||||
best = _index_best(name, kind, system)
|
||||
if best:
|
||||
data = _fetch_png(best, kind, timeout)
|
||||
data = _fetch_png(best, kind, timeout, system)
|
||||
if data is not None:
|
||||
return data
|
||||
return None
|
||||
@@ -466,6 +488,33 @@ def _ovgdb_name_list():
|
||||
return _ovgdb_names
|
||||
|
||||
|
||||
_ovgdb_index = None
|
||||
|
||||
|
||||
def _load_ovgdb_index():
|
||||
"""Load {key: {"title", "url"}} written by app/openvgdb.py, or {}.
|
||||
|
||||
Looks in the runtime covers dir first, then the shipped ``data/covers``
|
||||
copy (so a fresh install still shows readable titles without re-running
|
||||
the full OpenVGDB scrape).
|
||||
"""
|
||||
global _ovgdb_index
|
||||
if _ovgdb_index is not None:
|
||||
return _ovgdb_index
|
||||
_ovgdb_index = {}
|
||||
for path in (
|
||||
config.COVERS_DIR / "openvgdb-index.json",
|
||||
config.LEGACY_COVERS_DIR / "openvgdb-index.json",
|
||||
):
|
||||
if path.exists():
|
||||
try:
|
||||
_ovgdb_index = json.loads(path.read_text(encoding="utf-8"))
|
||||
break
|
||||
except Exception: # noqa: BLE001
|
||||
_ovgdb_index = {}
|
||||
return _ovgdb_index
|
||||
|
||||
|
||||
def fetch_openvgdb_local(name):
|
||||
"""Return a bundled OpenVGDB cover image for `name`, or None."""
|
||||
names = _ovgdb_name_list()
|
||||
@@ -549,7 +598,7 @@ def _screenscraper_box_urls(jeu):
|
||||
return urls
|
||||
|
||||
|
||||
def _fetch_screenscraper(name, timeout):
|
||||
def _fetch_screenscraper(name, timeout, system="games"):
|
||||
creds = sources.load()
|
||||
devid = creds.get("screenscraper_devid")
|
||||
devpassword = creds.get("screenscraper_devpassword")
|
||||
@@ -560,7 +609,7 @@ def _fetch_screenscraper(name, timeout):
|
||||
"devpassword": devpassword,
|
||||
"softname": creds.get("screenscraper_softname") or "psp-preserve",
|
||||
"output": "json",
|
||||
"systemeid": _SS_PSP_ID,
|
||||
"systemeid": _platform(system)["ss_id"],
|
||||
"romtype": "rom",
|
||||
"romnom": name,
|
||||
}
|
||||
@@ -806,19 +855,20 @@ def fetch_cover(name, release_id, system, timeout=10, quick=False):
|
||||
if data is not None:
|
||||
log.info("cover %r <- manual", name)
|
||||
elif system != "videos":
|
||||
# 2. Games: libretro (full box art) first, then the shipped OpenVGDB
|
||||
# bundle (offline), then ScreenScraper (full box art, non-quick),
|
||||
# then Wikipedia (keyless, non-quick), then IGDB last — IGDB only
|
||||
# stores cropped 3:4 key art.
|
||||
data = _fetch_libretro(name, t, indexed_only=quick)
|
||||
# 2. Games/PS1: libretro (full box art) first, then the shipped
|
||||
# OpenVGDB bundle (PSP only, offline), then ScreenScraper (non-quick),
|
||||
# then Wikipedia (keyless, non-quick), then IGDB last.
|
||||
data = _fetch_libretro(name, t, system=system, indexed_only=quick)
|
||||
if data is not None:
|
||||
log.info("cover %r <- libretro", name)
|
||||
else:
|
||||
data = fetch_openvgdb_local(name)
|
||||
if data is not None:
|
||||
log.info("cover %r <- openvgdb", name)
|
||||
elif not quick and time.time() < deadline:
|
||||
data = _fetch_screenscraper(name, t)
|
||||
data = None
|
||||
if system == "games":
|
||||
data = fetch_openvgdb_local(name)
|
||||
if data is not None:
|
||||
log.info("cover %r <- openvgdb", name)
|
||||
if data is None and not quick and time.time() < deadline:
|
||||
data = _fetch_screenscraper(name, t, system=system)
|
||||
if data is not None:
|
||||
log.info("cover %r <- screenscraper", name)
|
||||
if data is None and not quick and time.time() < deadline:
|
||||
@@ -826,11 +876,12 @@ def fetch_cover(name, release_id, system, timeout=10, quick=False):
|
||||
if data is not None:
|
||||
log.info("cover %r <- wikipedia", name)
|
||||
if data is None and time.time() < deadline:
|
||||
data = _fetch_igdb_cover(name, t)
|
||||
data = _fetch_igdb_cover(name, t, system=system)
|
||||
if data is not None:
|
||||
# IGDB only has cropped 3:4 key art; composite it into the
|
||||
# full PSP cover template (header bar + artwork).
|
||||
data = compose_game_cover(data)
|
||||
if system == "games":
|
||||
# IGDB only has cropped 3:4 PSP key art; composite it
|
||||
# into the full PSP cover template (header + artwork).
|
||||
data = compose_game_cover(data)
|
||||
log.info("cover %r <- igdb", name)
|
||||
|
||||
if data is None:
|
||||
@@ -918,6 +969,46 @@ def _search_wikipedia(query, limit=15, timeout=10):
|
||||
return results
|
||||
|
||||
|
||||
def _search_google_images(query, limit=15, timeout=10):
|
||||
"""Return [{title, source, thumb_url, image_url}] from Google Images via the
|
||||
Custom Search JSON API. Requires google_api_key + google_cx; keyless -> [].
|
||||
"""
|
||||
key = sources.load().get("google_api_key")
|
||||
cx = sources.load().get("google_cx")
|
||||
if not key or not cx:
|
||||
return []
|
||||
url = _GOOGLE_CSE_API + "?" + urllib.parse.urlencode({
|
||||
"key": key,
|
||||
"cx": cx,
|
||||
"q": query,
|
||||
"searchType": "image",
|
||||
"num": min(max(limit, 1), 10),
|
||||
"imgSize": "xlarge",
|
||||
"safe": "off",
|
||||
})
|
||||
data = _http_get(url, timeout=timeout)
|
||||
if not data:
|
||||
return []
|
||||
try:
|
||||
items = json.loads(data).get("items") or []
|
||||
except Exception: # noqa: BLE001
|
||||
return []
|
||||
out = []
|
||||
for it in items:
|
||||
link = it.get("link") or ""
|
||||
thumb = ((it.get("image") or {}).get("thumbnailLink")) or link
|
||||
if not link:
|
||||
continue
|
||||
out.append({
|
||||
"id": None,
|
||||
"title": it.get("title") or query,
|
||||
"thumb_url": thumb,
|
||||
"image_url": link,
|
||||
"source": "google",
|
||||
})
|
||||
return out[:limit]
|
||||
|
||||
|
||||
def _fetch_wikipedia_cover(name, timeout):
|
||||
"""Return a game cover image from Wikipedia's File: namespace (keyless)."""
|
||||
base = plain_title(name)
|
||||
@@ -995,8 +1086,8 @@ def _wiki_file_search(query, limit, timeout):
|
||||
return out
|
||||
|
||||
|
||||
def _search_libretro(query, limit):
|
||||
names = _load_index().get("Named_Boxarts", [])
|
||||
def _search_libretro(query, limit, system="games"):
|
||||
names = _load_index(system=system).get("Named_Boxarts", [])
|
||||
db = _norm(query)
|
||||
results = []
|
||||
if not db:
|
||||
@@ -1022,21 +1113,60 @@ def _search_libretro(query, limit):
|
||||
results.append({
|
||||
"title": n,
|
||||
"source": "libretro",
|
||||
"thumb_url": cover_url(n),
|
||||
"image_url": cover_url(n),
|
||||
"thumb_url": cover_url(n, system=system),
|
||||
"image_url": cover_url(n, system=system),
|
||||
})
|
||||
return results
|
||||
|
||||
|
||||
def _search_openvgdb(query, limit):
|
||||
"""Return [{title, source, thumb_url, image_url}] from the offline OpenVGDB
|
||||
bundle, using the display titles stored in openvgdb-index.json."""
|
||||
names = _ovgdb_name_list()
|
||||
if not names:
|
||||
return []
|
||||
index = _load_ovgdb_index()
|
||||
db = _norm(query)
|
||||
if not db:
|
||||
return []
|
||||
scored = []
|
||||
for key in names:
|
||||
if key == db:
|
||||
score = 5
|
||||
elif len(key) >= 4 and key.startswith(db):
|
||||
score = 4
|
||||
elif len(key) >= 4 and db in key:
|
||||
score = 3
|
||||
elif len(db) >= 4 and len(key) >= 4 and key in db:
|
||||
score = 2
|
||||
else:
|
||||
continue
|
||||
info = index.get(key) or {}
|
||||
title = info.get("title") or key
|
||||
scored.append((score, title, key))
|
||||
scored.sort(key=lambda t: (-t[0], t[1]))
|
||||
return [
|
||||
{
|
||||
"id": None,
|
||||
"title": title,
|
||||
"source": "openvgdb",
|
||||
"thumb_url": "",
|
||||
"image_url": "",
|
||||
"_key": key,
|
||||
}
|
||||
for _score, title, key in scored[:limit]
|
||||
]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Game metadata (details sidebar)
|
||||
# --------------------------------------------------------------------------- #
|
||||
_META_TTL = 86400.0
|
||||
|
||||
|
||||
def get_game_metadata(title, timeout=10):
|
||||
"""Return game metadata from IGDB."""
|
||||
return get_game_metadata_igdb(title, timeout)
|
||||
def get_game_metadata(title, timeout=10, system="games"):
|
||||
"""Return game metadata from IGDB (system picks the platform)."""
|
||||
return get_game_metadata_igdb(title, timeout, system)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
@@ -1161,28 +1291,30 @@ def _igdb_game_meta(g):
|
||||
}
|
||||
|
||||
|
||||
def _igdb_search(query, limit=20, timeout=10):
|
||||
def _igdb_search(query, limit=20, timeout=10, system="games"):
|
||||
cid, _ = _igdb_creds()
|
||||
if not cid:
|
||||
return []
|
||||
term = plain_title(query) or query
|
||||
platform_id = _platform(system)["igdb_id"]
|
||||
q = (
|
||||
f'search "{term}"; '
|
||||
"fields name, cover.image_id, first_release_date, summary, genres.name, "
|
||||
"involved_companies.company.name, involved_companies.developer, involved_companies.publisher, "
|
||||
"rating, videos.video_id; "
|
||||
f"where platforms = ({_IGDB_PSP_ID}); "
|
||||
f"where platforms = ({platform_id}); "
|
||||
f"limit {max(limit, 1)};"
|
||||
)
|
||||
return _igdb_post(q, timeout) or []
|
||||
|
||||
|
||||
def _fetch_igdb_cover(name, timeout):
|
||||
def _fetch_igdb_cover(name, timeout, system="games"):
|
||||
cid, _ = _igdb_creds()
|
||||
if not cid:
|
||||
return None
|
||||
term = plain_title(name) or name
|
||||
q = f'search "{term}"; fields cover.image_id; where platforms = ({_IGDB_PSP_ID}); limit 10;'
|
||||
platform_id = _platform(system)["igdb_id"]
|
||||
q = f'search "{term}"; fields cover.image_id; where platforms = ({platform_id}); limit 10;'
|
||||
games = _igdb_post(q, timeout) or []
|
||||
tried = set()
|
||||
for g in games:
|
||||
@@ -1196,9 +1328,9 @@ def _fetch_igdb_cover(name, timeout):
|
||||
return None
|
||||
|
||||
|
||||
def search_games_igdb(query, limit=20, timeout=10):
|
||||
def search_games_igdb(query, limit=20, timeout=10, system="games"):
|
||||
out = []
|
||||
for g in _igdb_search(query, limit, timeout):
|
||||
for g in _igdb_search(query, limit, timeout, system):
|
||||
cover = (g.get("cover") or {}).get("image_id") or ""
|
||||
out.append({
|
||||
"id": g.get("id"),
|
||||
@@ -1213,8 +1345,8 @@ def search_games_igdb(query, limit=20, timeout=10):
|
||||
return out[:limit]
|
||||
|
||||
|
||||
def get_game_metadata_igdb(title, timeout=10):
|
||||
games = _igdb_search(title, limit=10, timeout=timeout)
|
||||
def get_game_metadata_igdb(title, timeout=10, system="games"):
|
||||
games = _igdb_search(title, limit=10, timeout=timeout, system=system)
|
||||
if not games:
|
||||
return None
|
||||
games.sort(key=lambda g: -_title_score(title, g.get("name") or ""))
|
||||
@@ -1334,11 +1466,19 @@ def get_movie_metadata(title, timeout=10):
|
||||
# Manual linking (pick the correct movie/game when the auto-match is wrong)
|
||||
# --------------------------------------------------------------------------- #
|
||||
def search_movies(query, limit=20, timeout=10):
|
||||
"""Return [{id, title, year, overview, thumb_url, source}] for a movie search.
|
||||
|
||||
TheMovieDB first; keyless Wikipedia page titles as a fallback.
|
||||
"""Return movie candidates for a UMD video: TMDB first, then Wikipedia
|
||||
images and Google Images (both keyless-optional). A Wikipedia page-title
|
||||
fallback is added only when nothing else matched (metadata-only linking).
|
||||
"""
|
||||
out = []
|
||||
seen = set()
|
||||
|
||||
def add(r):
|
||||
k = (r.get("source"), _norm(r.get("title") or ""))
|
||||
if k[1] and k not in seen:
|
||||
seen.add(k)
|
||||
out.append(r)
|
||||
|
||||
key = sources.load().get("tmdb_api_key")
|
||||
if key:
|
||||
url = _TMDB_SEARCH + "?" + urllib.parse.urlencode({"api_key": key, "query": query})
|
||||
@@ -1348,16 +1488,31 @@ def search_movies(query, limit=20, timeout=10):
|
||||
results = json.loads(data).get("results") or []
|
||||
except Exception:
|
||||
results = []
|
||||
for r in results:
|
||||
# Cap TMDB results so Wikipedia/Google image picks still fit.
|
||||
for r in results[: max(1, min(limit, 6))]:
|
||||
poster = r.get("poster_path")
|
||||
out.append({
|
||||
add({
|
||||
"id": r.get("id"),
|
||||
"title": r.get("title") or r.get("name") or "",
|
||||
"year": (r.get("release_date") or "")[:4],
|
||||
"overview": (r.get("overview") or "").strip(),
|
||||
"thumb_url": (_TMDB_IMAGE + "w185" + poster) if poster else "",
|
||||
"image_url": (_TMDB_IMAGE + "w500" + poster) if poster else "",
|
||||
"source": "tmdb",
|
||||
})
|
||||
|
||||
# Wikipedia image search (keyless): real poster/cover files.
|
||||
for r in _search_wikipedia(query, min(limit, 10), timeout):
|
||||
r["id"] = None
|
||||
r["movie_title"] = query
|
||||
add(r)
|
||||
|
||||
# Google Images (Custom Search JSON API, when a key + cx are configured).
|
||||
for r in _search_google_images(query, min(limit, 10), timeout):
|
||||
r["movie_title"] = query
|
||||
add(r)
|
||||
|
||||
# Last resort: Wikipedia page titles (metadata-only, no image).
|
||||
if not out:
|
||||
for page in _wiki_search(query, limit):
|
||||
out.append({
|
||||
@@ -1366,6 +1521,8 @@ def search_movies(query, limit=20, timeout=10):
|
||||
"year": "",
|
||||
"overview": "",
|
||||
"thumb_url": "",
|
||||
"image_url": "",
|
||||
"movie_title": page,
|
||||
"source": "wikipedia",
|
||||
})
|
||||
return out[:limit]
|
||||
@@ -1411,40 +1568,71 @@ def movie_metadata_from_wikipedia(page_title, timeout=10):
|
||||
|
||||
|
||||
def search_metadata_candidates(system, query, limit=20, timeout=10):
|
||||
"""Search candidates the user can link an item to."""
|
||||
"""Search candidates the user can link an item to.
|
||||
|
||||
Games/PS1: IGDB (rich metadata) first, then libretro box art (and the
|
||||
PSP-only OpenVGDB bundle) so covers can be scraped even without an IGDB key.
|
||||
"""
|
||||
if system == "videos":
|
||||
return search_movies(query, limit, timeout)
|
||||
results = search_games_igdb(query, limit, timeout)
|
||||
if results:
|
||||
return results
|
||||
# No IGDB results (or no key): fall back to libretro catalog names so the
|
||||
# user can still link to the correct entry (title-only).
|
||||
results = _search_libretro(query, limit)
|
||||
for r in results:
|
||||
r["source"] = "libretro"
|
||||
r["id"] = None
|
||||
return results
|
||||
results = []
|
||||
seen = set()
|
||||
|
||||
def add(r):
|
||||
key = (r.get("source"), _norm(r.get("title") or ""))
|
||||
if key[1] and key not in seen:
|
||||
seen.add(key)
|
||||
results.append(r)
|
||||
|
||||
# Cap IGDB so libretro/OpenVGDB cover picks still fit in the list.
|
||||
for r in search_games_igdb(query, max(1, min(limit, 6)), timeout, system):
|
||||
add(r)
|
||||
for r in _search_libretro(query, limit, system):
|
||||
r.setdefault("id", None)
|
||||
add(r)
|
||||
if system == "games":
|
||||
for r in _search_openvgdb(query, limit):
|
||||
add(r)
|
||||
return results[:limit]
|
||||
|
||||
|
||||
def _empty_meta(title=""):
|
||||
return {
|
||||
"title": title,
|
||||
"release_date": "",
|
||||
"overview": "",
|
||||
"genres": [],
|
||||
"rating": None,
|
||||
"youtube": "",
|
||||
}
|
||||
|
||||
|
||||
def link_metadata(system, candidate, timeout=10):
|
||||
"""Fetch full metadata for a search candidate the user picked."""
|
||||
if candidate.get("source") == "wikipedia":
|
||||
return movie_metadata_from_wikipedia(candidate.get("title") or "", timeout)
|
||||
if candidate.get("source") == "libretro":
|
||||
# Catalog-name link only: no metadata backend, just the correct title.
|
||||
return {
|
||||
"title": candidate.get("title") or "",
|
||||
"release_date": "",
|
||||
"overview": "",
|
||||
"genres": [],
|
||||
"rating": None,
|
||||
"youtube": "",
|
||||
}
|
||||
src = candidate.get("source")
|
||||
if src == "wikipedia":
|
||||
title = candidate.get("movie_title") or candidate.get("title") or ""
|
||||
meta = movie_metadata_from_wikipedia(title, timeout) if title else None
|
||||
if meta:
|
||||
return meta
|
||||
if system == "videos":
|
||||
return get_movie_metadata(title, timeout) or _empty_meta(title)
|
||||
return _empty_meta(title)
|
||||
if src in ("libretro", "openvgdb"):
|
||||
# Cover repositories have no metadata backend; try IGDB by title so the
|
||||
# user still gets synopsis/genre/etc. when a key is configured.
|
||||
title = candidate.get("title") or ""
|
||||
if title:
|
||||
meta = get_game_metadata_igdb(title, timeout, system)
|
||||
if meta:
|
||||
return meta
|
||||
return _empty_meta(title)
|
||||
if system == "videos":
|
||||
if candidate.get("id") is not None:
|
||||
if src == "tmdb" and candidate.get("id") is not None:
|
||||
return movie_metadata_by_id(candidate["id"], timeout)
|
||||
return movie_metadata_from_wikipedia(candidate.get("title") or "", timeout)
|
||||
if candidate.get("id") is not None and candidate.get("source") == "igdb":
|
||||
title = candidate.get("movie_title") or candidate.get("title") or ""
|
||||
return get_movie_metadata(title, timeout) or _empty_meta(title)
|
||||
if src == "igdb" and candidate.get("id") is not None:
|
||||
return game_metadata_by_id_igdb(candidate["id"], timeout)
|
||||
return None
|
||||
return _empty_meta(candidate.get("title") or "")
|
||||
|
||||
|
||||
+72
-16
@@ -1,23 +1,29 @@
|
||||
"""Parser for clrmamepro (.dat) files.
|
||||
"""Parser for clrmamepro / Logiqx XML (.dat) files.
|
||||
|
||||
Handles the No-Intro / Redump text DAT format::
|
||||
Supports both DAT layouts the project uses:
|
||||
|
||||
clrmamepro (
|
||||
name "Sony - PlayStation Portable"
|
||||
version "2021.10.24"
|
||||
)
|
||||
game (
|
||||
name "WipEout Pure (Europe)"
|
||||
region "Europe"
|
||||
serial "ULES-00011"
|
||||
rom ( name "WipEout Pure (Europe).iso" size 123 crc ABCD1234 md5 ... sha1 ... )
|
||||
)
|
||||
* clrmamepro text DAT (No-Intro): parenthesised blocks, e.g.::
|
||||
|
||||
``parse_dat_text`` returns ``(header, entries)``. Each entry is a dict with
|
||||
scalar keys (``name``, ``description``, ``region``, ``serial``, ...) and
|
||||
repeated nested blocks exposed as lists under their block key (``rom``).
|
||||
clrmamepro (
|
||||
name "Sony - PlayStation Portable"
|
||||
version "2021.10.24"
|
||||
)
|
||||
game (
|
||||
name "WipEout Pure (Europe)"
|
||||
region "Europe"
|
||||
serial "ULES-00011"
|
||||
rom ( name "WipEout Pure (Europe).iso" size 123 crc ABCD1234 md5 ... sha1 ... )
|
||||
)
|
||||
|
||||
* Logiqx XML DAT (Redump): ``<game>`` elements with ``<rom>`` children.
|
||||
|
||||
Both ``parse_dat_text`` and ``parse_dat_xml`` return ``(header, entries)``.
|
||||
Each entry is a dict with scalar keys (``name``, ``description``, ``region``,
|
||||
``serial``, ...) and repeated nested blocks exposed as lists under their block
|
||||
key (``rom``).
|
||||
"""
|
||||
import re
|
||||
import xml.etree.ElementTree as ET
|
||||
from pathlib import Path
|
||||
|
||||
_TOKEN_RE = re.compile(
|
||||
@@ -124,5 +130,55 @@ def parse_dat_text(text):
|
||||
return header, entries
|
||||
|
||||
|
||||
def _xml_text(element, key):
|
||||
child = element.find(key)
|
||||
if child is not None and child.text:
|
||||
return child.text.strip()
|
||||
return None
|
||||
|
||||
|
||||
def parse_dat_xml(text):
|
||||
"""Parse a Logiqx XML DAT (the format redump.org uses).
|
||||
|
||||
Returns ``(header, entries)`` in the same shape as :func:`parse_dat_text`.
|
||||
"""
|
||||
root = ET.fromstring(text)
|
||||
|
||||
header = {}
|
||||
header_el = root.find("header")
|
||||
if header_el is not None:
|
||||
for child in header_el:
|
||||
if child.text:
|
||||
header[child.tag] = child.text.strip()
|
||||
|
||||
entries = []
|
||||
for game in root.iter():
|
||||
if game.tag not in ("game", "machine"):
|
||||
continue
|
||||
entry = {"name": (game.get("name") or "").strip()}
|
||||
for key in ("description", "serial", "region", "category"):
|
||||
value = _xml_text(game, key)
|
||||
if value:
|
||||
entry[key] = value
|
||||
entry["rom"] = [
|
||||
{
|
||||
"name": rom.get("name", ""),
|
||||
"size": rom.get("size"),
|
||||
"crc": rom.get("crc"),
|
||||
"md5": rom.get("md5"),
|
||||
"sha1": rom.get("sha1"),
|
||||
"serial": rom.get("serial"),
|
||||
}
|
||||
for rom in game.findall("rom")
|
||||
]
|
||||
entries.append(entry)
|
||||
|
||||
return header, entries
|
||||
|
||||
|
||||
def parse_dat_file(path):
|
||||
return parse_dat_text(Path(path).read_text(encoding="utf-8-sig"))
|
||||
text = Path(path).read_text(encoding="utf-8-sig")
|
||||
stripped = text.lstrip()
|
||||
if stripped.startswith("<?xml") or stripped.startswith("<"):
|
||||
return parse_dat_xml(text)
|
||||
return parse_dat_text(text)
|
||||
|
||||
@@ -101,6 +101,20 @@ CREATE TABLE IF NOT EXISTS save_sigs (
|
||||
sig TEXT NOT NULL,
|
||||
PRIMARY KEY (save_id, size, mtime_ns)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS ps1_entries (
|
||||
id INTEGER PRIMARY KEY,
|
||||
source_path TEXT NOT NULL UNIQUE,
|
||||
title TEXT,
|
||||
serial TEXT,
|
||||
region TEXT,
|
||||
disc INTEGER NOT NULL DEFAULT 1,
|
||||
disc_total INTEGER NOT NULL DEFAULT 1,
|
||||
output_eboot TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'pending',
|
||||
error TEXT,
|
||||
updated_at TEXT NOT NULL
|
||||
);
|
||||
"""
|
||||
|
||||
|
||||
|
||||
+287
-18
@@ -14,13 +14,13 @@ from PySide6.QtCore import QBuffer, QByteArray, QObject, QRect, QSettings, QSize
|
||||
from PySide6.QtGui import QColor, QDesktopServices, QFont, QIcon, QImage, QPainter, QPixmap
|
||||
from PySide6.QtWidgets import (
|
||||
QAbstractItemView, QApplication, QCheckBox, QComboBox, QDialog, QDialogButtonBox,
|
||||
QFileDialog, QFormLayout, QGridLayout, QHBoxLayout, QHeaderView, QLabel, QLineEdit,
|
||||
QFileDialog, QFormLayout, QGridLayout, QHBoxLayout, QHeaderView, QInputDialog, QLabel, QLineEdit,
|
||||
QListWidget, QListWidgetItem, QMainWindow, QMenu, QMessageBox, QPlainTextEdit,
|
||||
QProgressBar, QProgressDialog, QPushButton, QScrollArea, QSpinBox, QSplitter, QStackedWidget,
|
||||
QTabWidget, QTableWidget, QTableWidgetItem, QVBoxLayout, QWidget,
|
||||
)
|
||||
|
||||
from . import config, covers, info, iso_reader, launcher, matcher, namer, pipeline, psp, queue, saves, scanner, sources, store, transfers
|
||||
from . import config, covers, info, iso_reader, launcher, matcher, namer, pipeline, ps1, psp, queue, saves, scanner, sources, store, transfers
|
||||
from .db import get_connection, init_db
|
||||
|
||||
log = logging.getLogger("gui")
|
||||
@@ -559,6 +559,7 @@ class MetadataWorker(QObject):
|
||||
class GameDetailsPanel(QWidget):
|
||||
edit_requested = Signal() # emitted when the user wants to hand-tag the item
|
||||
link_requested = Signal() # emitted when the user wants to pick a movie/game to link
|
||||
convert_requested = Signal() # emitted when the user wants to convert a PS1 disc to EBOOT
|
||||
copy_requested = Signal() # emitted when the user wants to copy the current item to the PSP
|
||||
delete_requested = Signal() # emitted when the user wants to delete the current item
|
||||
play_requested = Signal() # emitted when the user wants to launch the item in PPSSPP
|
||||
@@ -569,6 +570,7 @@ class GameDetailsPanel(QWidget):
|
||||
self.setFixedWidth(300)
|
||||
self._youtube_url = ""
|
||||
self._meta = None
|
||||
self._copy_available = False
|
||||
|
||||
outer = QVBoxLayout(self)
|
||||
outer.setContentsMargins(12, 12, 0, 12)
|
||||
@@ -627,6 +629,12 @@ class GameDetailsPanel(QWidget):
|
||||
self.play_btn.setToolTip("Launch this game in PPSSPP")
|
||||
self.layout.addWidget(self.play_btn)
|
||||
|
||||
self.convert_btn = QPushButton("\u2699 Convert to EBOOT")
|
||||
self.convert_btn.clicked.connect(self.convert_requested.emit)
|
||||
self.convert_btn.setVisible(False)
|
||||
self.convert_btn.setToolTip("Convert this PS1 disc to an EBOOT.PBP")
|
||||
self.layout.addWidget(self.convert_btn)
|
||||
|
||||
btn_grid = QGridLayout()
|
||||
btn_grid.setHorizontalSpacing(6)
|
||||
btn_grid.setVerticalSpacing(6)
|
||||
@@ -658,14 +666,28 @@ class GameDetailsPanel(QWidget):
|
||||
|
||||
def set_item(self, item):
|
||||
self.system = item.get("system")
|
||||
self._ps1_kind = item.get("ps1_kind")
|
||||
path = item.get("path") or ""
|
||||
self.play_btn.setEnabled(item.get("system") == "games" and bool(path) and Path(path).suffix.lower() == ".iso")
|
||||
is_ps1 = item.get("system") == "ps1"
|
||||
is_ps1_disc = is_ps1 and item.get("ps1_kind") == "disc"
|
||||
self.play_btn.setEnabled(
|
||||
(item.get("system") == "games" and bool(path) and Path(path).suffix.lower() == ".iso")
|
||||
or (is_ps1 and item.get("ps1_kind") == "eboot" and bool(path))
|
||||
)
|
||||
self.play_btn.setVisible(not is_ps1_disc)
|
||||
self.convert_btn.setVisible(is_ps1_disc)
|
||||
self.copy_btn.setEnabled(self._copy_available and not is_ps1_disc)
|
||||
self.copy_btn.setToolTip(
|
||||
"Copy this EBOOT to /PSP/GAME" if is_ps1 else "Copy this disc to the connected PSP"
|
||||
)
|
||||
self.del_psp_btn.setEnabled(bool(item.get("on_psp")))
|
||||
self.title.setText(item.get("title_override") or item.get("name") or "Unknown")
|
||||
parts = []
|
||||
if item.get("serial"):
|
||||
parts.append(item["serial"])
|
||||
if item.get("region"):
|
||||
if is_ps1 and item.get("ps1_status"):
|
||||
parts.append(item["ps1_status"])
|
||||
elif item.get("region"):
|
||||
parts.append(item["region"])
|
||||
self.subtitle.setText(" · ".join(parts))
|
||||
self._clear_meta()
|
||||
@@ -676,7 +698,8 @@ class GameDetailsPanel(QWidget):
|
||||
self._set_cover(item.get("release_id"), item.get("system"))
|
||||
|
||||
def set_copy_available(self, available):
|
||||
self.copy_btn.setEnabled(bool(available))
|
||||
self._copy_available = bool(available)
|
||||
self.copy_btn.setEnabled(self._copy_available and getattr(self, "_ps1_kind", None) != "disc")
|
||||
|
||||
def set_psp_installed(self, on_psp):
|
||||
self.del_psp_btn.setEnabled(bool(on_psp))
|
||||
@@ -824,8 +847,17 @@ class ScrapeSearchWorker(QObject):
|
||||
def run(self):
|
||||
found = covers.search_metadata_candidates(self.system, self.query, limit=20, timeout=10)
|
||||
for r in found:
|
||||
url = r.get("thumb_url")
|
||||
r["thumb_bytes"] = covers.download_image(url, timeout=8) if url else None
|
||||
src = r.get("source")
|
||||
if src == "openvgdb":
|
||||
r["thumb_bytes"] = covers.fetch_openvgdb_local(r.get("title") or "")
|
||||
elif src == "libretro":
|
||||
data = covers.fetch_libretro_local(r.get("title") or "")
|
||||
r["thumb_bytes"] = data if data else (
|
||||
covers.download_image(r.get("thumb_url"), timeout=8) if r.get("thumb_url") else None
|
||||
)
|
||||
else:
|
||||
url = r.get("thumb_url")
|
||||
r["thumb_bytes"] = covers.download_image(url, timeout=8) if url else None
|
||||
self.results.emit(found)
|
||||
self.finished.emit()
|
||||
|
||||
@@ -849,16 +881,26 @@ class ScrapeFetchWorker(QObject):
|
||||
|
||||
def _scrape_cover(system, candidate, rid, timeout=20):
|
||||
"""Download the cover for a metadata candidate; returns bytes or None."""
|
||||
src = candidate.get("source")
|
||||
title = candidate.get("title") or ""
|
||||
# Cover repositories ship offline bundles — read them directly (no network).
|
||||
if src == "openvgdb" and title:
|
||||
data = covers.fetch_openvgdb_local(title)
|
||||
if data:
|
||||
return data
|
||||
if src == "libretro" and title:
|
||||
data = covers.fetch_libretro_local(title)
|
||||
if data:
|
||||
return data
|
||||
url = candidate.get("image_url")
|
||||
if url:
|
||||
data = covers.download_image(url, timeout)
|
||||
if data:
|
||||
if system == "videos":
|
||||
return compose_umd_cover(data)
|
||||
if candidate.get("source") == "igdb":
|
||||
if candidate.get("source") == "igdb" and system == "games":
|
||||
return covers.compose_game_cover(data)
|
||||
return data
|
||||
title = candidate.get("title") or ""
|
||||
if title:
|
||||
path = covers.fetch_cover(title, rid, system, timeout=timeout)
|
||||
if path is not None:
|
||||
@@ -955,6 +997,8 @@ class ScrapeDialog(QDialog):
|
||||
icon = QIcon(pm)
|
||||
year = r.get("year") or (r.get("release_date") or "")[:4]
|
||||
label = f"{r['title']} ({year})" if year else r["title"]
|
||||
if r.get("source") not in (None, "igdb", "tmdb"):
|
||||
label += f" [{r['source']}]"
|
||||
item = QListWidgetItem(icon, label)
|
||||
if r.get("overview"):
|
||||
item.setToolTip(r["overview"])
|
||||
@@ -1011,7 +1055,7 @@ class SettingsPage(QWidget):
|
||||
|
||||
def __init__(self, games_dir, videos_dir, store_dir, ppsspp_dir, cloud_dir, ppsspp_exe,
|
||||
store_server_url, store_token, store_archive_id, store_archive_sig, store_archive_user,
|
||||
iso_category, parent=None):
|
||||
iso_category, ps1_dir, ps1_output_dir, ps1_base_pbp, parent=None):
|
||||
super().__init__(parent)
|
||||
outer = QVBoxLayout(self)
|
||||
outer.setContentsMargins(0, 0, 0, 0)
|
||||
@@ -1025,6 +1069,9 @@ class SettingsPage(QWidget):
|
||||
page, form = self._new_form()
|
||||
self.games_edit = self._dir_row(form, "Games directory:", games_dir)
|
||||
self.videos_edit = self._dir_row(form, "Videos directory:", videos_dir)
|
||||
self.ps1_edit = self._dir_row(form, "PS1 directory:", ps1_dir)
|
||||
self.ps1_out_edit = self._dir_row(form, "PS1 output directory:", ps1_output_dir)
|
||||
self.ps1_base_edit = self._file_row(form, "PS1 base.pbp (PSN classic):", ps1_base_pbp)
|
||||
self.auto_scan = QCheckBox("Scan directories automatically on startup")
|
||||
self.auto_scan.setChecked(str(s.value("auto_scan", False)).lower() == "true")
|
||||
form.addRow(self.auto_scan)
|
||||
@@ -1094,6 +1141,8 @@ class SettingsPage(QWidget):
|
||||
|
||||
form.addRow(QLabel("<b>Videos — metadata & posters</b>"))
|
||||
add_field("TheMovieDB API key:", "tmdb_api_key", password=True)
|
||||
add_field("Google API key:", "google_api_key", password=True)
|
||||
add_field("Google Search Engine ID (cx):", "google_cx")
|
||||
|
||||
hint = QLabel(
|
||||
"Manual UMD covers: drop images into "
|
||||
@@ -1156,7 +1205,7 @@ class SettingsPage(QWidget):
|
||||
return edit
|
||||
|
||||
def _browse_file(self, edit):
|
||||
path, _ = QFileDialog.getOpenFileName(self, "Select PPSSPP executable")
|
||||
path, _ = QFileDialog.getOpenFileName(self, "Select file")
|
||||
if path:
|
||||
edit.setText(path)
|
||||
|
||||
@@ -1210,6 +1259,15 @@ class SettingsPage(QWidget):
|
||||
def iso_category(self):
|
||||
return self.iso_category_edit.text().strip()
|
||||
|
||||
def ps1_dir(self):
|
||||
return self.ps1_edit.text().strip()
|
||||
|
||||
def ps1_output_dir(self):
|
||||
return self.ps1_out_edit.text().strip()
|
||||
|
||||
def ps1_base_pbp(self):
|
||||
return self.ps1_base_edit.text().strip()
|
||||
|
||||
def _save(self):
|
||||
sources.save({key: edit.text() for key, edit in self.fields.items()})
|
||||
covers.clear_miss_cache()
|
||||
@@ -2989,7 +3047,7 @@ class PspSyncPage(QWidget):
|
||||
self._workers = []
|
||||
|
||||
|
||||
def build_library_items():
|
||||
def build_library_items(ps1_dir="", output_dir=""):
|
||||
"""Build the library item list from the DB with a single joined query."""
|
||||
items = []
|
||||
seen = set()
|
||||
@@ -3058,16 +3116,59 @@ def build_library_items():
|
||||
conn.commit()
|
||||
finally:
|
||||
conn.close()
|
||||
items.extend(_ps1_library_items(ps1_dir, output_dir, overrides, dates))
|
||||
return items
|
||||
|
||||
|
||||
def _ps1_library_items(ps1_dir, output_dir, overrides, dates):
|
||||
"""PS1 discs (pending) + converted EBOOTs, merged into the library list."""
|
||||
if not ps1_dir:
|
||||
return []
|
||||
try:
|
||||
entries = ps1.scan_library(ps1_dir, output_dir)
|
||||
except Exception: # noqa: BLE001
|
||||
return []
|
||||
out = []
|
||||
for e in entries:
|
||||
# A converted disc is represented by its EBOOT entry (one row per game).
|
||||
if e["kind"] == "disc" and e.get("status") == "converted":
|
||||
continue
|
||||
name = e["name"]
|
||||
rid = covers.stable_key(name)
|
||||
path = Path(e["path"])
|
||||
try:
|
||||
size = path.stat().st_size if path.exists() else 0
|
||||
except OSError:
|
||||
size = 0
|
||||
out.append({
|
||||
"name": name,
|
||||
"release_id": rid,
|
||||
"system": "ps1",
|
||||
"canonical": name,
|
||||
"serial": e.get("serial") or "",
|
||||
"region": None,
|
||||
"title_override": overrides.get(("ps1", str(rid))),
|
||||
"release_date": dates.get((str(rid), "ps1")) or "",
|
||||
"path": str(path),
|
||||
"size": size,
|
||||
"ps1_kind": e["kind"],
|
||||
"ps1_status": e.get("status") or "pending",
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
class LibraryWorker(QObject):
|
||||
result = Signal(list)
|
||||
finished = Signal()
|
||||
|
||||
def __init__(self, ps1_dir="", output_dir=""):
|
||||
super().__init__()
|
||||
self.ps1_dir = ps1_dir
|
||||
self.output_dir = output_dir
|
||||
|
||||
def run(self):
|
||||
try:
|
||||
items = build_library_items()
|
||||
items = build_library_items(self.ps1_dir, self.output_dir)
|
||||
except Exception: # noqa: BLE001
|
||||
items = []
|
||||
self.result.emit(items)
|
||||
@@ -3220,6 +3321,113 @@ class SavesPage(QWidget):
|
||||
self._workers = []
|
||||
|
||||
|
||||
class Ps1ScanWorker(QObject):
|
||||
results = Signal(list)
|
||||
failed = Signal(str)
|
||||
|
||||
def __init__(self, ps1_dir, output_dir):
|
||||
super().__init__()
|
||||
self.ps1_dir = ps1_dir
|
||||
self.output_dir = output_dir
|
||||
|
||||
def run(self):
|
||||
try:
|
||||
self.results.emit(ps1.scan_library(self.ps1_dir, self.output_dir))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
self.failed.emit(str(exc))
|
||||
|
||||
|
||||
class Ps1Page(QWidget):
|
||||
"""PS1 tab: list raw source discs, converted EBOOTs and PSN games."""
|
||||
|
||||
convert_requested = Signal(dict)
|
||||
|
||||
def __init__(self, parent=None):
|
||||
super().__init__(parent)
|
||||
self._ps1_dir = ""
|
||||
self._output_dir = ""
|
||||
self._entries = []
|
||||
self._thread = None
|
||||
self._worker = None
|
||||
|
||||
layout = QVBoxLayout(self)
|
||||
bar = QHBoxLayout()
|
||||
self.status = QLabel("PS1 discs — set the PS1 folder in Settings → General.")
|
||||
bar.addWidget(self.status, 1)
|
||||
self.refresh_btn = QPushButton("Refresh")
|
||||
self.refresh_btn.clicked.connect(self.refresh)
|
||||
self.convert_btn = QPushButton("Convert selected")
|
||||
self.convert_btn.clicked.connect(self._convert_selected)
|
||||
bar.addWidget(self.refresh_btn)
|
||||
bar.addWidget(self.convert_btn)
|
||||
layout.addLayout(bar)
|
||||
|
||||
self.table = QTableWidget(0, 4)
|
||||
self.table.setHorizontalHeaderLabels(["Type", "Name", "Serial", "Status"])
|
||||
self.table.setSelectionBehavior(QAbstractItemView.SelectRows)
|
||||
self.table.setEditTriggers(QAbstractItemView.NoEditTriggers)
|
||||
self.table.verticalHeader().setVisible(False)
|
||||
self.table.horizontalHeader().setSectionResizeMode(1, QHeaderView.Stretch)
|
||||
layout.addWidget(self.table, 1)
|
||||
|
||||
def set_dirs(self, ps1_dir, output_dir):
|
||||
self._ps1_dir = ps1_dir
|
||||
self._output_dir = output_dir
|
||||
|
||||
def refresh(self):
|
||||
if not self._ps1_dir:
|
||||
self.status.setText("PS1 folder not set — configure it in Settings → General.")
|
||||
self.table.setRowCount(0)
|
||||
self._entries = []
|
||||
return
|
||||
if self._thread is not None and _thread_running(self._thread):
|
||||
return
|
||||
self.status.setText("Scanning PS1 folders…")
|
||||
self._worker = Ps1ScanWorker(self._ps1_dir, self._output_dir)
|
||||
self._thread = QThread(self)
|
||||
self._worker.moveToThread(self._thread)
|
||||
self._thread.started.connect(self._worker.run)
|
||||
self._worker.results.connect(self._on_results)
|
||||
self._worker.failed.connect(self._on_failed)
|
||||
self._worker.results.connect(self._thread.quit)
|
||||
self._worker.failed.connect(self._thread.quit)
|
||||
self._thread.start()
|
||||
|
||||
def _on_results(self, entries):
|
||||
self._entries = entries
|
||||
self.table.setRowCount(len(entries))
|
||||
for row, e in enumerate(entries):
|
||||
kind = "Disc" if e["kind"] == "disc" else "EBOOT"
|
||||
self.table.setItem(row, 0, QTableWidgetItem(kind))
|
||||
self.table.setItem(row, 1, QTableWidgetItem(e["name"]))
|
||||
self.table.setItem(row, 2, QTableWidgetItem(e.get("serial") or ""))
|
||||
self.table.setItem(row, 3, QTableWidgetItem(e.get("status") or "pending"))
|
||||
pending = sum(1 for e in entries if e["kind"] == "disc" and e["status"] != "converted")
|
||||
self.status.setText(f"{len(entries)} item(s) — {pending} ready to convert")
|
||||
|
||||
def _on_failed(self, message):
|
||||
self.status.setText(f"PS1 scan failed: {message}")
|
||||
|
||||
def _convert_selected(self):
|
||||
row = self.table.currentRow()
|
||||
if row < 0 or row >= len(self._entries):
|
||||
return
|
||||
e = self._entries[row]
|
||||
if e["kind"] != "disc":
|
||||
QMessageBox.information(self, "PS1", "Select a raw PS1 disc (Disc row), not an EBOOT.")
|
||||
return
|
||||
if e["status"] == "converted":
|
||||
QMessageBox.information(self, "PS1", "This disc already has a converted EBOOT.")
|
||||
return
|
||||
game_id = e.get("serial") or ""
|
||||
if not game_id:
|
||||
text, ok = QInputDialog.getText(self, "PS1", "Game ID (e.g. SLUS00594):")
|
||||
if not ok or not text.strip():
|
||||
return
|
||||
game_id = text.strip()
|
||||
self.convert_requested.emit({"source": e["path"], "title": e["name"], "game_id": game_id})
|
||||
|
||||
|
||||
class MainWindow(QMainWindow):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
@@ -3259,6 +3467,9 @@ class MainWindow(QMainWindow):
|
||||
self._auto_sync_saves = str(self.settings.value("auto_sync_saves", "false")).lower() == "true"
|
||||
self._ppsspp_exe = str(self.settings.value("ppsspp_exe", "") or "")
|
||||
self._iso_category = str(self.settings.value("iso_category", "") or "")
|
||||
self._ps1_dir = self._first_existing(config.PS1_ROOTS) or str(self.settings.value("ps1_dir", "") or "")
|
||||
self._ps1_output_dir = str(self.settings.value("ps1_output_dir", str(config.PS1_OUTPUT_DIR)) or "")
|
||||
self._ps1_base_pbp = str(self.settings.value("ps1_base_pbp", "") or "")
|
||||
log.info("games dir: %s", self._games_dir or "(none)")
|
||||
log.info("videos dir: %s", self._videos_dir or "(none)")
|
||||
log.info("store: %s", self._store_server_url or self._store_dir or "(none)")
|
||||
@@ -3291,6 +3502,10 @@ class MainWindow(QMainWindow):
|
||||
self.saves_page = SavesPage()
|
||||
self.saves_page.sync_requested.connect(self._on_saves_sync)
|
||||
self.tabs.addTab(self.saves_page, "Saves")
|
||||
self.ps1_page = Ps1Page()
|
||||
self.ps1_page.set_dirs(self._ps1_dir, self._ps1_output_dir)
|
||||
self.ps1_page.convert_requested.connect(self._on_ps1_convert)
|
||||
self.tabs.addTab(self.ps1_page, "PS1")
|
||||
self.tabs.addTab(self._build_settings_tab(), "Settings")
|
||||
self.tabs.currentChanged.connect(self._on_tab_changed)
|
||||
self._on_psp_device_changed(self.psp_page.root())
|
||||
@@ -3321,7 +3536,7 @@ class MainWindow(QMainWindow):
|
||||
self.library_search.textChanged.connect(self._apply_library_filter)
|
||||
bar.addWidget(self.library_search, 1)
|
||||
self.library_type = QComboBox()
|
||||
self.library_type.addItems(["All types", "Games", "Videos"])
|
||||
self.library_type.addItems(["All types", "Games", "Videos", "PS1"])
|
||||
self.library_type.currentTextChanged.connect(self._apply_library_filter)
|
||||
bar.addWidget(self.library_type)
|
||||
self.library_psp = QComboBox()
|
||||
@@ -3371,6 +3586,7 @@ class MainWindow(QMainWindow):
|
||||
self.cover_grid.scrape_requested.connect(self._scrape_grid_item)
|
||||
self.details_panel.edit_requested.connect(self._edit_details)
|
||||
self.details_panel.link_requested.connect(self._scrape_item)
|
||||
self.details_panel.convert_requested.connect(self._convert_current_item)
|
||||
self.details_panel.copy_requested.connect(self._copy_current_to_psp)
|
||||
self.details_panel.delete_requested.connect(self._delete_current)
|
||||
self.details_panel.delete_psp_requested.connect(self._delete_current_from_psp)
|
||||
@@ -3461,7 +3677,8 @@ class MainWindow(QMainWindow):
|
||||
self.settings_page = SettingsPage(self._games_dir, self._videos_dir, self._store_dir,
|
||||
self._ppsspp_saves_dir, self._cloud_saves_dir, self._ppsspp_exe,
|
||||
self._store_server_url, self._store_token, self._store_archive_id,
|
||||
self._store_archive_sig, self._store_archive_user, self._iso_category)
|
||||
self._store_archive_sig, self._store_archive_user, self._iso_category,
|
||||
self._ps1_dir, self._ps1_output_dir, self._ps1_base_pbp)
|
||||
self.settings_page.saved.connect(self._on_settings_saved)
|
||||
return self.settings_page
|
||||
|
||||
@@ -3490,6 +3707,13 @@ class MainWindow(QMainWindow):
|
||||
self._auto_sync_saves = self.settings_page.auto_sync_saves()
|
||||
self._ppsspp_exe = self.settings_page.ppsspp_exe()
|
||||
self._iso_category = self.settings_page.iso_category()
|
||||
self._ps1_dir = self.settings_page.ps1_dir()
|
||||
self._ps1_output_dir = self.settings_page.ps1_output_dir()
|
||||
self._ps1_base_pbp = self.settings_page.ps1_base_pbp()
|
||||
self.settings.setValue("ps1_dir", self._ps1_dir)
|
||||
self.settings.setValue("ps1_output_dir", self._ps1_output_dir)
|
||||
self.settings.setValue("ps1_base_pbp", self._ps1_base_pbp)
|
||||
self.ps1_page.set_dirs(self._ps1_dir, self._ps1_output_dir)
|
||||
self.settings.setValue("ppsspp_saves_dir", self._ppsspp_saves_dir)
|
||||
self.settings.setValue("cloud_saves_dir", self._cloud_saves_dir)
|
||||
self.settings.setValue("auto_sync_saves", self._auto_sync_saves)
|
||||
@@ -3499,7 +3723,8 @@ class MainWindow(QMainWindow):
|
||||
self._update_dir_status()
|
||||
|
||||
def _on_tab_changed(self, index):
|
||||
pass # store builds eagerly and caches its view; no per-tab work needed
|
||||
if hasattr(self, "ps1_page") and index == self.tabs.indexOf(self.ps1_page):
|
||||
self.ps1_page.refresh()
|
||||
|
||||
def _update_saves_locations(self):
|
||||
if not hasattr(self, "saves_page"):
|
||||
@@ -3530,7 +3755,7 @@ class MainWindow(QMainWindow):
|
||||
self.statusBar().showMessage("Loading library…")
|
||||
log.info("loading library…")
|
||||
thread = QThread(self)
|
||||
worker = LibraryWorker()
|
||||
worker = LibraryWorker(self._ps1_dir, self._ps1_output_dir)
|
||||
worker.moveToThread(thread)
|
||||
thread.started.connect(worker.run)
|
||||
worker.result.connect(self._on_library_loaded)
|
||||
@@ -3568,6 +3793,8 @@ class MainWindow(QMainWindow):
|
||||
continue
|
||||
if typ == "Videos" and it["system"] != "videos":
|
||||
continue
|
||||
if typ == "PS1" and it["system"] != "ps1":
|
||||
continue
|
||||
if q and q not in (it["canonical"] or it["name"]).lower():
|
||||
continue
|
||||
if psp_f == "On PSP" and not it.get("on_psp"):
|
||||
@@ -3593,7 +3820,8 @@ class MainWindow(QMainWindow):
|
||||
title.setIcon(self._psp_dot_icon())
|
||||
title.setToolTip("Installed on PSP")
|
||||
self.library_table.setItem(row, 0, title)
|
||||
self.library_table.setItem(row, 1, QTableWidgetItem("Game" if it["system"] == "games" else "Video"))
|
||||
kind_label = {"games": "Game", "videos": "Video", "ps1": "PS1"}.get(it["system"], "Game")
|
||||
self.library_table.setItem(row, 1, QTableWidgetItem(kind_label))
|
||||
self.library_table.setItem(row, 2, QTableWidgetItem(it.get("region") or ""))
|
||||
self.library_table.setItem(row, 3, QTableWidgetItem(it.get("serial") or ""))
|
||||
size = it.get("size")
|
||||
@@ -3716,6 +3944,10 @@ class MainWindow(QMainWindow):
|
||||
meta["manual"] = True
|
||||
self.details_panel.set_item(item)
|
||||
self.details_panel.set_metadata(meta)
|
||||
# Always refresh the details panel's cover box: a cover-only scrape
|
||||
# (no metadata backend) skips set_item above, and `rid` may fall back
|
||||
# to a stable key when the release has no database id.
|
||||
self.details_panel._set_cover(rid, system)
|
||||
self.cover_grid.update_item_label(item)
|
||||
self.cover_grid.refresh_item(item)
|
||||
|
||||
@@ -3724,6 +3956,18 @@ class MainWindow(QMainWindow):
|
||||
self.details_panel.set_item(it)
|
||||
self._scrape_item()
|
||||
|
||||
def _convert_current_item(self):
|
||||
item = getattr(self, "_current_item", None)
|
||||
if item is None or item.get("system") != "ps1" or item.get("ps1_kind") != "disc":
|
||||
return
|
||||
game_id = item.get("serial") or ""
|
||||
if not game_id:
|
||||
text, ok = QInputDialog.getText(self, "PS1", "Game ID (e.g. SLUS00594):")
|
||||
if not ok or not text.strip():
|
||||
return
|
||||
game_id = text.strip()
|
||||
self._on_ps1_convert({"source": item.get("path"), "title": item.get("name"), "game_id": game_id})
|
||||
|
||||
def _on_store_download(self, items):
|
||||
if not items:
|
||||
return
|
||||
@@ -3805,6 +4049,8 @@ class MainWindow(QMainWindow):
|
||||
return
|
||||
if item.get("system") == "videos":
|
||||
return # UMD Video discs can't be played in PPSSPP
|
||||
if item.get("system") == "ps1" and item.get("ps1_kind") != "eboot":
|
||||
return # raw PS1 discs are converted, not played
|
||||
path = item.get("path")
|
||||
if not path or not Path(path).exists():
|
||||
QMessageBox.warning(self, "Missing file", f"File not found:\n{path}")
|
||||
@@ -3877,6 +4123,15 @@ class MainWindow(QMainWindow):
|
||||
psp_root = Path(self._psp_root)
|
||||
if system == "videos":
|
||||
dest_dir = psp_root / "ISO" / "VIDEO"
|
||||
elif system == "ps1":
|
||||
# PS1 EBOOTs install as a whole game folder under /PSP/GAME.
|
||||
src_folder = Path(item["path"]).parent
|
||||
dest_dir = psp_root / "PSP" / "GAME"
|
||||
log.info("copying ps1 game to psp: %s -> %s", src_folder, dest_dir / src_folder.name)
|
||||
self.queue.enqueue(transfers.TreeCopySource(str(src_folder)), dest_dir,
|
||||
{"dest_name": src_folder.name}, item.get("name") or src_folder.name,
|
||||
kind="psp-install")
|
||||
return True
|
||||
else:
|
||||
cat = "".join(c for c in (self._iso_category or "").strip() if c not in "/\\")
|
||||
dest_dir = (psp_root / "ISO" / cat) if cat else (psp_root / "ISO")
|
||||
@@ -3984,6 +4239,17 @@ class MainWindow(QMainWindow):
|
||||
if hasattr(self, "psp_page"):
|
||||
self.psp_page.refresh_inventory()
|
||||
|
||||
def _on_ps1_convert(self, item):
|
||||
if not self._ps1_base_pbp or not Path(self._ps1_base_pbp).is_file():
|
||||
QMessageBox.warning(self, "PS1", "Set a valid PS1 base.pbp path in Settings → General first.")
|
||||
return
|
||||
if not item.get("game_id"):
|
||||
QMessageBox.warning(self, "PS1", "No game ID for this disc.")
|
||||
return
|
||||
out_dir = Path(self._ps1_output_dir) if self._ps1_output_dir else Path.cwd() / "ps1_converted"
|
||||
source = transfers.Ps1ConvertSource(self._ps1_base_pbp)
|
||||
self.queue.enqueue(source, out_dir, item, item.get("title") or item["source"], kind="ps1-convert")
|
||||
|
||||
def _on_queue_job_done(self, job):
|
||||
if self._closing:
|
||||
return
|
||||
@@ -4004,6 +4270,9 @@ class MainWindow(QMainWindow):
|
||||
QTimer.singleShot(0, self._refresh_psp)
|
||||
if kind == "save-sync":
|
||||
QTimer.singleShot(0, self._refresh_saves)
|
||||
if kind == "ps1-convert":
|
||||
QTimer.singleShot(0, self.ps1_page.refresh)
|
||||
QTimer.singleShot(0, self.load_library)
|
||||
|
||||
# -- Common ------------------------------------------------------------ #
|
||||
@staticmethod
|
||||
|
||||
+1
-1
@@ -197,7 +197,7 @@ def get_or_fetch(release_id, system, fetch_title):
|
||||
if system == "videos":
|
||||
meta = covers.get_movie_metadata(fetch_title)
|
||||
else:
|
||||
meta = covers.get_game_metadata(fetch_title)
|
||||
meta = covers.get_game_metadata(fetch_title, system=system)
|
||||
except Exception: # noqa: BLE001 - network/API failures must not crash the UI
|
||||
meta = None
|
||||
if meta:
|
||||
|
||||
@@ -24,6 +24,9 @@ def detect_system(dat_name):
|
||||
return "videos"
|
||||
if "PlayStation Portable" in dat_name:
|
||||
return "games"
|
||||
# Redump PS1: "Sony - PlayStation" (assumes no PS2/PS3 DATs are present).
|
||||
if "PlayStation" in dat_name:
|
||||
return "ps1"
|
||||
return "unknown"
|
||||
|
||||
|
||||
@@ -37,6 +40,12 @@ def ingest_dat(conn, path):
|
||||
existing = conn.execute("SELECT id FROM dats WHERE name = ?", (dat_name,)).fetchone()
|
||||
if existing:
|
||||
dat_id = existing["id"]
|
||||
# Files that resolved to a release in this DAT must be re-identified
|
||||
# against the new catalog (the release ids are about to be recycled).
|
||||
conn.execute(
|
||||
"UPDATE files SET release_id = NULL WHERE release_id IN (SELECT id FROM releases WHERE dat_id = ?)",
|
||||
(dat_id,),
|
||||
)
|
||||
conn.execute("DELETE FROM roms WHERE release_id IN (SELECT id FROM releases WHERE dat_id = ?)", (dat_id,))
|
||||
conn.execute("DELETE FROM releases WHERE dat_id = ?", (dat_id,))
|
||||
conn.execute(
|
||||
|
||||
+13
-3
@@ -15,8 +15,8 @@ _ARTICLE_RE = re.compile(r"^(.+),\s*(the|a|an)(?:\s+-\s+(.+))?$")
|
||||
|
||||
def _normalize(name):
|
||||
name = name.lower()
|
||||
name = re.sub(r"\.(iso|cso|pbp|img|bin)$", "", name) # strip disc extension
|
||||
name = re.sub(r"^\s*\d+\s*-\s*", "", name) # leading No-Intro catalog number
|
||||
name = re.sub(r"\.(iso|cso|pbp|img|bin|cue)$", "", name) # strip disc extension
|
||||
name = re.sub(r"^\s*\d+\s*-\s*", "", name) # legacy No-Intro "#### - Name" catalog number
|
||||
name = re.sub(r"\([^)]*\)", " ", name) # parenthetical tags
|
||||
name = " ".join(name.split())
|
||||
m = _ARTICLE_RE.match(name)
|
||||
@@ -141,7 +141,7 @@ def match(conn, record):
|
||||
if row is not None:
|
||||
return f"hash:{field}", dict(row)
|
||||
|
||||
# 2. Local supplement catalog (dumps not present in No-Intro).
|
||||
# 2. Local supplement catalog (dumps not present in the DATs).
|
||||
for entry in _load_supplement():
|
||||
for field in ("crc32", "sha1", "md5"):
|
||||
value = entry.get(field)
|
||||
@@ -181,6 +181,16 @@ def match(conn, record):
|
||||
return "unknown", None
|
||||
top = [cand for ratio, cand in scored if ratio >= best_ratio - 0.001]
|
||||
top = _narrow(conn, top, size)
|
||||
# The same disc can exist in multiple catalogs (e.g. No-Intro + Redump);
|
||||
# collapse identical entries so they don't read as an ambiguous match.
|
||||
seen = set()
|
||||
deduped = []
|
||||
for cand in top:
|
||||
key = (cand.get("name"), size)
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
deduped.append(cand)
|
||||
top = deduped
|
||||
if len(top) == 1:
|
||||
return f"fuzzy:{best_ratio:.2f}", top[0]
|
||||
return "unknown", None
|
||||
|
||||
+2
-2
@@ -62,7 +62,7 @@ def _reorder_article(title):
|
||||
return title
|
||||
|
||||
|
||||
def canonical_name(release, serial):
|
||||
def canonical_name(release, serial, ext=".iso"):
|
||||
name = release["name"]
|
||||
region = release.get("region")
|
||||
if region:
|
||||
@@ -78,4 +78,4 @@ def canonical_name(release, serial):
|
||||
parts.append(f"({serial})")
|
||||
name = _ILLEGAL.sub(" ", " ".join(parts)).strip()
|
||||
name = re.sub(r"\s+", " ", name)
|
||||
return f"{name}.iso"
|
||||
return f"{name}{ext}"
|
||||
|
||||
@@ -17,6 +17,7 @@ tags removed). The GUI reads this bundle with no network access.
|
||||
"""
|
||||
import argparse
|
||||
import concurrent.futures
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
@@ -94,6 +95,19 @@ def build_index(sqlite_path):
|
||||
return best
|
||||
|
||||
|
||||
def write_index(index, path=None):
|
||||
"""Persist {key: {"title", "url"}} so the GUI can show readable titles when
|
||||
searching the offline bundle (see covers._search_openvgdb)."""
|
||||
path = Path(path) if path else (config.COVERS_DIR / "openvgdb-index.json")
|
||||
data = {k: {"title": t, "url": u} for k, (t, u, _r) in index.items()}
|
||||
try:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(json.dumps(data, ensure_ascii=False), encoding="utf-8")
|
||||
except OSError as exc: # noqa: BLE001
|
||||
log.warning("could not write index %s: %s", path, exc)
|
||||
return path
|
||||
|
||||
|
||||
def _existing_keys():
|
||||
"""Titles already present in the bundle (for resume)."""
|
||||
if not BUNDLE.exists():
|
||||
@@ -152,6 +166,7 @@ def scrape(limit=None, workers=8, rate=3.0):
|
||||
|
||||
index = build_index(sqlite_path)
|
||||
log.info("indexed %d unique PSP covers", len(index))
|
||||
write_index(index)
|
||||
|
||||
existing = _existing_keys()
|
||||
todo = [(k, v) for k, v in sorted(index.items()) if k not in existing]
|
||||
|
||||
+158
@@ -0,0 +1,158 @@
|
||||
"""PS1 library helpers: discover source discs and converted EBOOT.PBPs.
|
||||
|
||||
This is the read/scan side of PS1 support (the conversion side lives in
|
||||
``app.ps1_convert``). It finds raw PS1 disc images (``.cue`` sets and
|
||||
standalone ``.iso``/``.img``), reads each disc's boot serial, and lists
|
||||
converted ``EBOOT.PBP`` files in the output folder.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
|
||||
from . import config
|
||||
|
||||
RAW_SECTOR = 2352
|
||||
COOKED_SECTOR = 2048
|
||||
EBOOT_NAME = "EBOOT.PBP"
|
||||
|
||||
# SYSTEM.CNF boot line: cdrom:\SLPS_017.62;1 (backslash optional in some rips)
|
||||
_SERIAL_RE = re.compile(rb"cdrom:\\?([A-Za-z]{4})[_-]?(\d{3})\.(\d{2})")
|
||||
|
||||
|
||||
def _sector_size(path: Path) -> int:
|
||||
size = path.stat().st_size
|
||||
if size and size % RAW_SECTOR == 0:
|
||||
return RAW_SECTOR
|
||||
return COOKED_SECTOR
|
||||
|
||||
|
||||
def read_serial(image_path: Path) -> Optional[str]:
|
||||
"""Read the boot serial (e.g. ``SLPS01762``) from a PS1 disc image.
|
||||
|
||||
Works on both raw 2352-byte/sector images (BIN/IMG) and cooked 2048-byte
|
||||
images (ISO). Returns ``None`` if the boot line can't be found.
|
||||
"""
|
||||
image_path = Path(image_path)
|
||||
if not image_path.is_file():
|
||||
return None
|
||||
sector_size = _sector_size(image_path)
|
||||
data_off = 24 if sector_size == RAW_SECTOR else 0
|
||||
|
||||
buf = bytearray()
|
||||
with open(image_path, "rb") as fh:
|
||||
for _ in range(128): # SYSTEM.CNF is always near the start of the disc
|
||||
sector = fh.read(sector_size)
|
||||
if len(sector) < sector_size:
|
||||
break
|
||||
buf += sector[data_off:data_off + COOKED_SECTOR]
|
||||
|
||||
m = _SERIAL_RE.search(bytes(buf))
|
||||
if m:
|
||||
return (m.group(1) + m.group(2) + m.group(3)).decode().upper()
|
||||
return None
|
||||
|
||||
|
||||
def cue_data_track(cue_path: Path) -> Optional[Path]:
|
||||
"""Return the first data-track image referenced by a cue sheet."""
|
||||
from . import ps1_convert
|
||||
|
||||
try:
|
||||
disc = ps1_convert.parse_cue(cue_path.read_text(encoding="utf-8", errors="replace"))
|
||||
except Exception:
|
||||
return None
|
||||
track = disc.data_track
|
||||
if track is None:
|
||||
return None
|
||||
return (cue_path.parent / track.file) if track.file else None
|
||||
|
||||
|
||||
def iter_source_discs(root: Path):
|
||||
"""Yield PS1 source discs under ``root`` (cue sheets + standalone images)."""
|
||||
root = Path(root)
|
||||
if not root.is_dir():
|
||||
return
|
||||
|
||||
for path in sorted(root.rglob("*")):
|
||||
if not path.is_file():
|
||||
continue
|
||||
suffix = path.suffix.lower()
|
||||
if suffix == ".cue":
|
||||
yield path
|
||||
elif suffix in (".iso", ".img"):
|
||||
# A standalone image: no cue sheet next to it (same stem).
|
||||
if not (path.parent / f"{path.stem}.cue").exists():
|
||||
yield path
|
||||
|
||||
|
||||
def find_eboots(root: Path) -> List[Path]:
|
||||
"""List converted/PSN ``EBOOT.PBP`` files under ``root``."""
|
||||
root = Path(root)
|
||||
if not root.is_dir():
|
||||
return []
|
||||
return [p for p in sorted(root.rglob("*")) if p.is_file() and p.name.upper() == EBOOT_NAME]
|
||||
|
||||
|
||||
def eboot_title(pbp_path: Path) -> str:
|
||||
"""Read the game title embedded in an EBOOT.PBP (falls back to folder name)."""
|
||||
from . import iso_reader, ps1_convert
|
||||
|
||||
try:
|
||||
header = ps1_convert.read_pbp_header(pbp_path)
|
||||
idx = ps1_convert.SECTION_OFFSET["sfo"] // 4
|
||||
sfo = ps1_convert.read_pbp_section(pbp_path, header, idx)
|
||||
parsed = iso_reader.parse_sfo(sfo)
|
||||
title = parsed.get("TITLE") or parsed.get("TITLE_0") or parsed.get("TITLE_00")
|
||||
if title:
|
||||
return str(title)
|
||||
except Exception:
|
||||
pass
|
||||
return pbp_path.parent.name
|
||||
|
||||
|
||||
def scan_library(ps1_dir, output_dir) -> List[dict]:
|
||||
"""Return a combined view of PS1 source discs and converted EBOOTs.
|
||||
|
||||
Each entry is a dict with ``kind`` ("disc" or "eboot"), ``path``, ``name``,
|
||||
``serial`` and ``status`` ("pending" / "converted").
|
||||
"""
|
||||
ps1_dir = Path(ps1_dir)
|
||||
output_dir = Path(output_dir)
|
||||
|
||||
eboots = find_eboots(output_dir)
|
||||
eboot_names = {e.name.upper() for e in eboots}
|
||||
eboot_titles = {eboot_title(e).lower() for e in eboots}
|
||||
|
||||
entries: List[dict] = []
|
||||
for src in iter_source_discs(ps1_dir):
|
||||
if src.suffix.lower() == ".cue":
|
||||
image = cue_data_track(src) or src
|
||||
else:
|
||||
image = src
|
||||
serial = read_serial(image)
|
||||
title = src.stem
|
||||
# A disc is "converted" when an EBOOT's embedded title or filename
|
||||
# matches the source disc's title/serial.
|
||||
status = "pending"
|
||||
if title.lower() in eboot_titles or (serial and any(serial.lower() in t for t in eboot_titles)):
|
||||
status = "converted"
|
||||
entries.append({
|
||||
"kind": "disc",
|
||||
"path": str(src),
|
||||
"name": title,
|
||||
"serial": serial or "",
|
||||
"status": status,
|
||||
})
|
||||
|
||||
for e in eboots:
|
||||
entries.append({
|
||||
"kind": "eboot",
|
||||
"path": str(e),
|
||||
"name": eboot_title(e),
|
||||
"serial": "",
|
||||
"status": "converted",
|
||||
})
|
||||
|
||||
return entries
|
||||
@@ -0,0 +1,899 @@
|
||||
"""PS1 -> PSP EBOOT.PBP conversion (pure Python).
|
||||
|
||||
Converts a PlayStation 1 disc image (``.iso``, ``.img``, or a ``.cue`` +
|
||||
``.bin`` set) into the ``EBOOT.PBP`` "PS1 classic" container that PSP POPS
|
||||
firmware can launch.
|
||||
|
||||
This is an independent, MIT-clean reimplementation of the popstation-md
|
||||
algorithm (same format that PSXPackager/RupertAvery writes), implemented from
|
||||
the published container format rather than by copying any GPL source.
|
||||
|
||||
Format reference (offsets are byte-exact):
|
||||
* PBP header - 40 bytes: b"\\x00PBP" magic + version + 8 uint32 section offsets
|
||||
(SFO, ICON0, ICON1, PIC0, PIC1, SND0, DATA.PSP, DATA.PSAR).
|
||||
* PSAR - the DATA.PSAR section holding the disc image. Starts with
|
||||
b"PSISOIMG0000", game id at +0x400, TOC at +0x800,
|
||||
ISO block index at +0x4000, disc data at +0x100000.
|
||||
|
||||
A ``base.pbp`` (any PSN PS1 classic ``EBOOT.PBP``) is REQUIRED to supply two
|
||||
blobs that we cannot synthesise:
|
||||
|
||||
* DATA.PSP - the encrypted POPS loader (header section 8).
|
||||
* STARTDAT - the trailing encrypted block (boot.png + PGD).
|
||||
|
||||
Without a base.pbp the resulting EBOOT will not boot on real hardware; the
|
||||
converter refuses to run without one.
|
||||
|
||||
Compression: only level 0 (uncompressed) is implemented. LZRC (levels 1-9) is a
|
||||
planned fast-follow.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import io
|
||||
import os
|
||||
import re
|
||||
import struct
|
||||
import zlib
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Iterator, List, Optional, Tuple
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Constants
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
PBP_MAGIC = b"\x00PBP"
|
||||
PBP_HEADER_SIZE = 0x28
|
||||
PBP_HEADER_U32 = 10 # magic + version + 8 section offsets
|
||||
|
||||
# Section index -> header offset (in bytes) within the 40-byte header.
|
||||
SECTION_OFFSET = {
|
||||
"sfo": 0x08,
|
||||
"icon0": 0x0C,
|
||||
"icon1": 0x10,
|
||||
"pic0": 0x14,
|
||||
"pic1": 0x18,
|
||||
"snd0": 0x1C,
|
||||
"data_psp": 0x20,
|
||||
"psar": 0x24,
|
||||
}
|
||||
SECTION_ORDER = ["sfo", "icon0", "icon1", "pic0", "pic1", "snd0", "data_psp", "psar"]
|
||||
|
||||
# PSAR (DATA.PSAR) layout.
|
||||
PSAR_MAGIC = b"PSISOIMG0000"
|
||||
PSAR_GAMEID_OFFSET = 0x400
|
||||
PSAR_TOC_OFFSET = 0x800
|
||||
PSAR_INDEX_OFFSET = 0x4000
|
||||
PSAR_ISO_OFFSET = 0x100000
|
||||
PSAR_ALIGNMENT = 0x10000
|
||||
|
||||
# One disc "block" in the ISO index = 16 raw 2352-byte sectors.
|
||||
RAW_SECTOR = 0x930 # 2352
|
||||
COOKED_SECTOR = 0x800 # 2048
|
||||
BLOCK_SIZE = RAW_SECTOR * 16 # 0x9300
|
||||
INDEX_ENTRY_SIZE = 32 # offset u32 + length u32 + 6 unused u32
|
||||
|
||||
SYNC_PATTERN = b"\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x00"
|
||||
|
||||
# SFO field formats (PSP PARAM.SFO).
|
||||
FMT_U32 = 0x0004
|
||||
FMT_STRING = 0x0204
|
||||
|
||||
SFO_DEFAULTS = {
|
||||
"BOOTABLE": (FMT_U32, 1),
|
||||
"CATEGORY": (FMT_STRING, "ME"),
|
||||
"DISC_VERSION": (FMT_STRING, "1.00"),
|
||||
"LICENSE": (FMT_STRING, "Library programs Copyright(C) Sony ComputerEntertainment Inc."),
|
||||
"PARENTAL_LEVEL": (FMT_U32, 1),
|
||||
"PSP_SYSTEM_VER": (FMT_STRING, "3.71"),
|
||||
"REGION": (FMT_U32, 0x8000),
|
||||
}
|
||||
|
||||
|
||||
class ConversionError(Exception):
|
||||
"""Raised when a disc cannot be converted."""
|
||||
|
||||
|
||||
class Cancelled(Exception):
|
||||
"""Raised internally when the caller requests cancellation."""
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# CUE sheet parsing
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
@dataclass
|
||||
class CueTrack:
|
||||
number: int
|
||||
mode: str # "MODE1/2048", "MODE1/2352", "MODE2/2352", "AUDIO"
|
||||
file: str
|
||||
file_type: str # "BINARY" | "WAVE" | ...
|
||||
indexes: List[Tuple[int, int]] = field(default_factory=list) # (index, msf)
|
||||
sector_size: int = 0
|
||||
|
||||
@property
|
||||
def pregap_sectors(self) -> int:
|
||||
"""Sectors between INDEX 00 and INDEX 01 (0 if no pregap)."""
|
||||
idx = dict(self.indexes)
|
||||
if 0 in idx and 1 in idx:
|
||||
return max(0, idx[1] - idx[0])
|
||||
return 0
|
||||
|
||||
|
||||
@dataclass
|
||||
class CueDisc:
|
||||
tracks: List[CueTrack]
|
||||
|
||||
@property
|
||||
def data_track(self) -> Optional[CueTrack]:
|
||||
for t in self.tracks:
|
||||
if t.mode != "AUDIO":
|
||||
return t
|
||||
return None
|
||||
|
||||
|
||||
def _msf_to_sectors(msf: str) -> int:
|
||||
m, s, f = (int(x) for x in msf.split(":"))
|
||||
return (m * 60 + s) * 75 + f
|
||||
|
||||
|
||||
def _sector_size_for_mode(mode: str) -> int:
|
||||
if mode in ("MODE1/2048",):
|
||||
return COOKED_SECTOR
|
||||
if mode in ("MODE1/2352", "MODE2/2352", "AUDIO"):
|
||||
return RAW_SECTOR
|
||||
# CDG / MODE2/2336 etc. fall back to raw.
|
||||
return RAW_SECTOR
|
||||
|
||||
|
||||
def parse_cue(text: str) -> CueDisc:
|
||||
"""Parse a CUE sheet into a :class:`CueDisc`.
|
||||
|
||||
Handles the common PS1 layouts: one or more ``FILE`` lines, ``TRACK``
|
||||
lines (``MODE1/2048``, ``MODE1/2352``, ``MODE2/2352``, ``AUDIO``) and
|
||||
``INDEX`` lines (``INDEX 00`` pregap + ``INDEX 01`` start).
|
||||
"""
|
||||
tracks: List[CueTrack] = []
|
||||
current_file: Optional[str] = None
|
||||
current_type = "BINARY"
|
||||
current_track: Optional[CueTrack] = None
|
||||
|
||||
file_re = re.compile(r'^FILE\s+"([^"]+)"\s+(\w+)', re.I)
|
||||
track_re = re.compile(r'^\s*TRACK\s+(\d{1,2})\s+(\S+)', re.I)
|
||||
index_re = re.compile(r'^\s*INDEX\s+(\d{1,2})\s+(\d{1,2}:\d{2}:\d{2})', re.I)
|
||||
|
||||
for raw in text.splitlines():
|
||||
line = raw.strip()
|
||||
if not line or line.startswith(("REM", "CATALOG", "CDTEXTFILE", "PERFORMER", "TITLE", "ISRC", "FLAGS", "POSTGAP", "PREGAP")):
|
||||
if line.startswith("PREGAP"):
|
||||
# PREGAP <mm:ss:ff> is equivalent to an INDEX 00 offset.
|
||||
m = re.search(r'PREGAP\s+(\d{1,2}:\d{2}:\d{2})', line, re.I)
|
||||
if m and current_track is not None:
|
||||
current_track.indexes.append((0, _msf_to_sectors(m.group(1))))
|
||||
continue
|
||||
|
||||
m = file_re.match(line)
|
||||
if m:
|
||||
current_file = m.group(1)
|
||||
current_type = m.group(2).upper()
|
||||
current_track = None
|
||||
continue
|
||||
|
||||
m = track_re.match(line)
|
||||
if m:
|
||||
number = int(m.group(1))
|
||||
mode = m.group(2).upper()
|
||||
current_track = CueTrack(
|
||||
number=number,
|
||||
mode=mode,
|
||||
file=current_file or "",
|
||||
file_type=current_type,
|
||||
sector_size=_sector_size_for_mode(mode),
|
||||
)
|
||||
tracks.append(current_track)
|
||||
continue
|
||||
|
||||
m = index_re.match(line)
|
||||
if m and current_track is not None:
|
||||
current_track.indexes.append((int(m.group(1)), _msf_to_sectors(m.group(2))))
|
||||
|
||||
if not tracks:
|
||||
raise ConversionError("CUE sheet contains no tracks")
|
||||
for t in tracks:
|
||||
if not t.file:
|
||||
raise ConversionError(f"track {t.number} has no FILE")
|
||||
return CueDisc(tracks)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Sector normalisation (cooked 2048 -> raw 2352 Mode 2 Form 1)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def _to_bcd(value: int) -> int:
|
||||
return ((value // 10) << 4) | (value % 10)
|
||||
|
||||
|
||||
def _sectors_to_msf(sectors: int) -> Tuple[int, int, int]:
|
||||
"""Convert a sector count to (minutes, seconds, frames) at 75 frames/s."""
|
||||
total_seconds = sectors // 75
|
||||
return total_seconds // 60, total_seconds % 60, sectors % 75
|
||||
|
||||
|
||||
def _expand_cooked_sector(data: bytes, lba: int) -> bytes:
|
||||
"""Expand a 2048-byte cooked sector to a 2352-byte raw Mode2 Form1 sector.
|
||||
|
||||
Rebuilds the 12-byte sync pattern, the 4-byte MSF header (BCD), the
|
||||
8-byte subheader, and appends 280 bytes of EDC/ECC (zero-filled; the PSP
|
||||
POPS loader does not verify EDC/ECC for converted games).
|
||||
"""
|
||||
assert len(data) == COOKED_SECTOR
|
||||
frame = lba + 150 # 150-sector lead-in offset
|
||||
msf = bytes([
|
||||
_to_bcd((frame // 75 // 60) % 100),
|
||||
_to_bcd((frame // 75) % 60),
|
||||
_to_bcd(frame % 75),
|
||||
0x02, # mode 2
|
||||
])
|
||||
# Subheader: file=2, channel=0, submode=0, coding=0; repeated once.
|
||||
subheader = b"\x00\x00\x08\x00" * 2
|
||||
return SYNC_PATTERN + msf + subheader + data + (b"\x00" * 280)
|
||||
|
||||
|
||||
def iter_raw_sectors(
|
||||
path: Path,
|
||||
sector_size: int,
|
||||
pregap_sectors: int = 0,
|
||||
progress_cb=None,
|
||||
cancel_event=None,
|
||||
) -> Iterator[bytes]:
|
||||
"""Yield 2352-byte raw sectors for one track file.
|
||||
|
||||
``sector_size`` is 2048 (cooked; expanded on the fly) or 2352 (raw).
|
||||
``pregap_sectors`` leading zero sectors are emitted first.
|
||||
"""
|
||||
def _cancelled():
|
||||
return cancel_event is not None and cancel_event.is_set()
|
||||
|
||||
if pregap_sectors:
|
||||
for _ in range(pregap_sectors):
|
||||
yield b"\x00" * RAW_SECTOR
|
||||
|
||||
size = path.stat().st_size
|
||||
with open(path, "rb") as fh:
|
||||
lba = 0
|
||||
done = 0
|
||||
while True:
|
||||
if _cancelled():
|
||||
raise Cancelled()
|
||||
chunk = fh.read(sector_size)
|
||||
if not chunk:
|
||||
break
|
||||
if len(chunk) != sector_size:
|
||||
chunk = chunk.ljust(sector_size, b"\x00")
|
||||
if sector_size == COOKED_SECTOR:
|
||||
yield _expand_cooked_sector(chunk, lba)
|
||||
else:
|
||||
yield chunk
|
||||
lba += 1
|
||||
done += sector_size
|
||||
if progress_cb:
|
||||
progress_cb(done, size)
|
||||
if progress_cb:
|
||||
progress_cb(size, size)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# PARAM.SFO writer
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def build_param_sfo(entries: List[Tuple[str, int, object]]) -> bytes:
|
||||
"""Build a PSP PARAM.SFO from ``(key, fmt, value)`` entries.
|
||||
|
||||
``fmt`` is ``FMT_U32`` (uint32) or ``FMT_STRING`` (utf-8 string).
|
||||
Layout matches the PSP format: header, index table, key table, data table.
|
||||
"""
|
||||
header_size = 20
|
||||
index_size = len(entries) * 16
|
||||
key_table_start = header_size + index_size
|
||||
|
||||
key_table = b"".join(key.encode("utf-8") + b"\x00" for key, _, _ in entries)
|
||||
key_pad = (4 - (len(key_table) % 4)) % 4
|
||||
data_table_start = key_table_start + len(key_table) + key_pad
|
||||
|
||||
data_table = b""
|
||||
index_table = b""
|
||||
key_offset = 0
|
||||
data_offset = 0
|
||||
for key, fmt, value in entries:
|
||||
if fmt == FMT_U32:
|
||||
raw = struct.pack("<I", int(value))
|
||||
length = 4
|
||||
max_len = 4
|
||||
elif fmt == FMT_STRING:
|
||||
s = str(value)
|
||||
raw = s.encode("utf-8") + b"\x00"
|
||||
length = len(s)
|
||||
max_len = len(s) + 1
|
||||
if len(raw) < max_len:
|
||||
raw = raw.ljust(max_len, b"\x00")
|
||||
else:
|
||||
raise ConversionError(f"unsupported SFO format 0x{fmt:04x}")
|
||||
|
||||
index_table += struct.pack(
|
||||
"<HHIII", key_offset, fmt, length, max_len, data_offset
|
||||
)
|
||||
data_table += raw
|
||||
key_offset += len(key) + 1
|
||||
data_offset += len(raw)
|
||||
|
||||
return (
|
||||
b"\x00PSF"
|
||||
+ struct.pack("<IIII", 0x00000101, key_table_start, data_table_start, len(entries))
|
||||
+ index_table
|
||||
+ key_table
|
||||
+ (b"\x00" * key_pad)
|
||||
+ data_table
|
||||
)
|
||||
|
||||
|
||||
def default_ps1_sfo(title: str, game_id: str) -> bytes:
|
||||
"""A complete PARAM.SFO for a converted PS1 game."""
|
||||
entries = []
|
||||
for key, (fmt, value) in SFO_DEFAULTS.items():
|
||||
entries.append((key, fmt, value))
|
||||
entries.append(("DISC_ID", FMT_STRING, game_id))
|
||||
entries.append(("TITLE", FMT_STRING, title))
|
||||
return build_param_sfo(entries)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Minimal PNG generator (no external image deps)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def _png(width: int, height: int, rgb: Tuple[int, int, int]) -> bytes:
|
||||
def chunk(typ: bytes, data: bytes) -> bytes:
|
||||
payload = typ + data
|
||||
return struct.pack(">I", len(data)) + payload + struct.pack(">I", zlib.crc32(payload) & 0xFFFFFFFF)
|
||||
|
||||
ihdr = struct.pack(">IIBBBBB", width, height, 8, 2, 0, 0, 0) # 8-bit RGB
|
||||
raw = b"".join(b"\x00" + bytes(rgb) * width for _ in range(height))
|
||||
return (
|
||||
b"\x89PNG\r\n\x1a\n"
|
||||
+ chunk(b"IHDR", ihdr)
|
||||
+ chunk(b"IDAT", zlib.compress(raw, 9))
|
||||
+ chunk(b"IEND", b"")
|
||||
)
|
||||
|
||||
|
||||
def default_icon0_png() -> bytes:
|
||||
return _png(144, 80, (24, 24, 24))
|
||||
|
||||
|
||||
def default_pic1_png() -> bytes:
|
||||
return _png(480, 272, (10, 10, 16))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# PBP reading (used both for base.pbp extraction and self-test validation)
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def read_pbp_header(path: Path) -> List[int]:
|
||||
with open(path, "rb") as fh:
|
||||
data = fh.read(PBP_HEADER_SIZE)
|
||||
if len(data) < PBP_HEADER_SIZE or data[:4] != PBP_MAGIC:
|
||||
raise ConversionError(f"{path} is not a PBP file (bad magic)")
|
||||
return list(struct.unpack("<10I", data))
|
||||
|
||||
|
||||
def read_pbp_section(path: Path, header: List[int], index: int) -> bytes:
|
||||
start = header[index]
|
||||
end = header[index + 1] if index + 1 < len(header) else None
|
||||
with open(path, "rb") as fh:
|
||||
fh.seek(start)
|
||||
data = fh.read() if end is None else fh.read(end - start)
|
||||
return data
|
||||
|
||||
|
||||
def extract_data_psp(base_pbp: Path) -> bytes:
|
||||
"""Extract the encrypted POPS loader (DATA.PSP section) from a base PBP."""
|
||||
header = read_pbp_header(base_pbp)
|
||||
idx = SECTION_OFFSET["data_psp"] // 4 # header array index (2 + section index)
|
||||
data = read_pbp_section(base_pbp, header, idx)
|
||||
if not data:
|
||||
raise ConversionError(f"{base_pbp} has no DATA.PSP section")
|
||||
return data
|
||||
|
||||
|
||||
def extract_startdat(base_pbp: Path) -> bytes:
|
||||
"""Extract the trailing STARTDAT block (boot.png + PGD) from a base PBP.
|
||||
|
||||
The STARTDAT offset is discovered the same way the reference writer does:
|
||||
read the u32 at ``psar + 0x0C``, add ``0x50000``, and expect b"STARTDAT"
|
||||
there. Everything from that marker to EOF is copied verbatim.
|
||||
"""
|
||||
header = read_pbp_header(base_pbp)
|
||||
psar_offset = header[SECTION_OFFSET["psar"] // 4] # header array index
|
||||
with open(base_pbp, "rb") as fh:
|
||||
fh.seek(psar_offset + 0x0C)
|
||||
(x,) = struct.unpack("<I", fh.read(4))
|
||||
marker_off = x + 0x50000
|
||||
fh.seek(marker_off)
|
||||
if fh.read(8) != b"STARTDAT":
|
||||
raise ConversionError(f"cannot find STARTDAT in {base_pbp} (not a valid PS1 base PBP?)")
|
||||
fh.seek(marker_off)
|
||||
return fh.read()
|
||||
raise ConversionError("unreachable")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# PBP writer
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def _format_psar_game_id(game_id: str) -> str:
|
||||
"""Format a PS1 serial for the PSAR header field.
|
||||
|
||||
The PBP's PARAM.SFO uses the plain 9-char id (``SLPS01762``), while the
|
||||
PSAR header stores the region-delineated form (``_SLPS_01762``).
|
||||
"""
|
||||
m = re.match(r"^([A-Za-z]{4})[_-]?(\d{5})$", game_id)
|
||||
if m:
|
||||
return f"_{m.group(1).upper()}_{m.group(2)}"
|
||||
return game_id
|
||||
|
||||
|
||||
def _pad_to(out, target: int) -> None:
|
||||
"""Write zero bytes until the stream reaches ``target`` (absolute offset)."""
|
||||
delta = target - out.tell()
|
||||
if delta < 0:
|
||||
raise ConversionError(f"internal error: overshot offset by {-delta} bytes")
|
||||
if delta:
|
||||
out.write(b"\x00" * delta)
|
||||
|
||||
|
||||
# Track type values used in the PS1 TOC (PSAR +0x800).
|
||||
_TOC_DATA = 0x41
|
||||
_TOC_AUDIO = 0x01
|
||||
_TOC_A0 = 0xA0
|
||||
_TOC_A1 = 0xA1
|
||||
_TOC_A2 = 0xA2
|
||||
_TOC_ENTRY = 10 # bytes per TOC entry
|
||||
_TOC_LEAD_IN = 150 # 2-second lead-in, in sectors
|
||||
|
||||
|
||||
def build_toc(disc: "DiscImage") -> bytes:
|
||||
"""Generate the PS1 table-of-contents stored at PSAR +0x800.
|
||||
|
||||
Emits the standard CD-style TOC: an A0 entry (first track), an A1 entry
|
||||
(last track), an A2 entry (lead-out), then one 10-byte entry per track.
|
||||
Track addresses are BCD-encoded MSF; a 150-sector (2 s) lead-in is added
|
||||
to the relative address field, matching the reference implementation.
|
||||
"""
|
||||
tracks = disc.tracks_info()
|
||||
total = disc.total_sectors()
|
||||
entries = [bytearray(_TOC_ENTRY) for _ in range(len(tracks) + 3)]
|
||||
|
||||
first_num, first_data, _ = tracks[0]
|
||||
last_num, last_data, _ = tracks[-1]
|
||||
|
||||
e = entries[0]
|
||||
e[0] = _TOC_DATA if first_data else _TOC_AUDIO
|
||||
e[2] = _TOC_A0
|
||||
e[7] = _to_bcd(first_num)
|
||||
e[8] = _to_bcd(0x20)
|
||||
|
||||
e = entries[1]
|
||||
e[0] = _TOC_DATA if last_data else _TOC_AUDIO
|
||||
e[2] = _TOC_A1
|
||||
e[7] = _to_bcd(last_num)
|
||||
|
||||
e = entries[2]
|
||||
e[0] = _TOC_AUDIO
|
||||
e[2] = _TOC_A2
|
||||
mm, ss, ff = _sectors_to_msf(total)
|
||||
e[7], e[8], e[9] = _to_bcd(mm), _to_bcd(ss), _to_bcd(ff)
|
||||
|
||||
for i, (num, is_data, start) in enumerate(tracks):
|
||||
e = entries[3 + i]
|
||||
e[0] = _TOC_DATA if is_data else _TOC_AUDIO
|
||||
e[2] = _to_bcd(num)
|
||||
mm, ss, ff = _sectors_to_msf(start)
|
||||
e[3], e[4], e[5] = _to_bcd(mm), _to_bcd(ss), _to_bcd(ff)
|
||||
mm, ss, ff = _sectors_to_msf(start + _TOC_LEAD_IN)
|
||||
e[7], e[8], e[9] = _to_bcd(mm), _to_bcd(ss), _to_bcd(ff)
|
||||
|
||||
return b"".join(bytes(e) for e in entries)
|
||||
|
||||
|
||||
def _build_psar_header(disc: "DiscImage", game_id: str, title: str) -> bytes:
|
||||
"""Build the fixed PSAR header region 0x400..0x4000, byte-identical to the
|
||||
popstation level-0 writer.
|
||||
|
||||
Layout (offsets relative to the PSAR start):
|
||||
* 0x400 data1 (0xE20 bytes): game id at +0, TOC at +0x400, 0x10 at +0x7FE
|
||||
* 0x1220 p2 offset (u32) = padded ISO size + 0x100000 + 0x2D31
|
||||
* 0x1224 data2 (0x2DDC bytes): FF 07 at +4, title at +8, 0x03 at +0x88
|
||||
"""
|
||||
data1_size = 0xE20
|
||||
data2_size = 0x2DDC
|
||||
p2_delta = 0x2D31
|
||||
|
||||
block = bytearray(data1_size + 4 + data2_size) # 0x3C00 = 0x400..0x4000
|
||||
|
||||
gid = _format_psar_game_id(game_id).encode("ascii")[:11]
|
||||
block[: len(gid)] = gid
|
||||
|
||||
toc = build_toc(disc)
|
||||
block[0x400: 0x400 + len(toc)] = toc
|
||||
block[0x7FE] = 0x10
|
||||
|
||||
p2 = disc.padded_size() + PSAR_ISO_OFFSET + p2_delta
|
||||
block[data1_size: data1_size + 4] = struct.pack("<I", p2)
|
||||
|
||||
data2 = data1_size + 4
|
||||
block[data2 + 4: data2 + 6] = b"\xFF\x07"
|
||||
title_bytes = title.encode("ascii", "replace")[:0x80]
|
||||
block[data2 + 8: data2 + 8 + len(title_bytes)] = title_bytes
|
||||
block[data2 + 0x88] = 0x03
|
||||
|
||||
return bytes(block)
|
||||
|
||||
|
||||
def _write_psar(
|
||||
out: io.BufferedWriter,
|
||||
disc: "DiscImage",
|
||||
game_id: str,
|
||||
title: str,
|
||||
progress_cb=None,
|
||||
cancel_event=None,
|
||||
) -> None:
|
||||
"""Write the DATA.PSAR section (uncompressed, single-disc).
|
||||
|
||||
All offsets are relative to the start of the PSAR section.
|
||||
"""
|
||||
base = out.tell()
|
||||
iso_size = disc.padded_size()
|
||||
|
||||
# 0x000: PSAR signature; 0x00C: disc size hint.
|
||||
out.write(PSAR_MAGIC)
|
||||
out.write(struct.pack("<I", iso_size + PSAR_ISO_OFFSET))
|
||||
_pad_to(out, base + PSAR_GAMEID_OFFSET)
|
||||
|
||||
out.write(_build_psar_header(disc, game_id, title))
|
||||
_pad_to(out, base + PSAR_INDEX_OFFSET)
|
||||
|
||||
# ISO block index: (offset, length) per 0x9300 block. Uncompressed =>
|
||||
# length is always BLOCK_SIZE and offsets increase linearly.
|
||||
block_count = iso_size // BLOCK_SIZE
|
||||
offset = 0
|
||||
for _ in range(block_count):
|
||||
out.write(struct.pack("<II", offset, BLOCK_SIZE))
|
||||
out.write(b"\x00" * (INDEX_ENTRY_SIZE - 8))
|
||||
offset += BLOCK_SIZE
|
||||
|
||||
_pad_to(out, base + PSAR_ISO_OFFSET)
|
||||
|
||||
# Disc data, block by block.
|
||||
buf = bytearray()
|
||||
for sector in disc.iter_raw(progress_cb=progress_cb, cancel_event=cancel_event):
|
||||
buf.extend(sector)
|
||||
if len(buf) == BLOCK_SIZE:
|
||||
out.write(buf)
|
||||
buf = bytearray()
|
||||
if buf:
|
||||
out.write(buf.ljust(BLOCK_SIZE, b"\x00"))
|
||||
|
||||
|
||||
class DiscImage:
|
||||
"""A normalised view of a PS1 disc as a stream of raw 2352-byte sectors."""
|
||||
|
||||
def __init__(self, cue: Optional[CueDisc], path: Path):
|
||||
self.cue = cue
|
||||
self.path = path
|
||||
|
||||
def total_sectors(self) -> int:
|
||||
if self.cue is None:
|
||||
size = self.path.stat().st_size
|
||||
if size % RAW_SECTOR == 0:
|
||||
return size // RAW_SECTOR
|
||||
return (size + COOKED_SECTOR - 1) // COOKED_SECTOR
|
||||
total = 0
|
||||
base_dir = self.path.parent
|
||||
for track in self.cue.tracks:
|
||||
fpath = base_dir / track.file
|
||||
if fpath.is_file():
|
||||
total += track.pregap_sectors + (fpath.stat().st_size // track.sector_size)
|
||||
return total
|
||||
|
||||
def padded_size(self) -> int:
|
||||
size = self.total_sectors() * RAW_SECTOR
|
||||
if size % BLOCK_SIZE:
|
||||
size += BLOCK_SIZE - (size % BLOCK_SIZE)
|
||||
return size
|
||||
|
||||
def tracks_info(self) -> List[Tuple[int, bool, int]]:
|
||||
"""Return ``(track_number, is_data, absolute_start_sector)`` per track.
|
||||
|
||||
``absolute_start_sector`` is the disc-wide sector of each track's
|
||||
INDEX 01 (i.e. accounting for every preceding track's pregap + data).
|
||||
A cue-less single image is treated as one data track starting at 0.
|
||||
"""
|
||||
if self.cue is None:
|
||||
return [(1, True, 0)]
|
||||
|
||||
base_dir = self.path.parent
|
||||
out: List[Tuple[int, bool, int]] = []
|
||||
cursor = 0
|
||||
for track in self.cue.tracks:
|
||||
idx01 = dict(track.indexes).get(1, 0)
|
||||
out.append((track.number, track.mode != "AUDIO", cursor + idx01))
|
||||
fpath = base_dir / track.file
|
||||
data_sectors = fpath.stat().st_size // track.sector_size if fpath.is_file() else 0
|
||||
cursor += track.pregap_sectors + data_sectors
|
||||
return out
|
||||
|
||||
def iter_raw(self, progress_cb=None, cancel_event=None) -> Iterator[bytes]:
|
||||
if self.cue is None:
|
||||
yield from iter_raw_sectors(
|
||||
self.path,
|
||||
RAW_SECTOR if self.path.stat().st_size % RAW_SECTOR == 0 else COOKED_SECTOR,
|
||||
progress_cb=progress_cb,
|
||||
cancel_event=cancel_event,
|
||||
)
|
||||
return
|
||||
|
||||
base_dir = self.path.parent
|
||||
for track in self.cue.tracks:
|
||||
fpath = base_dir / track.file
|
||||
if not fpath.is_file():
|
||||
raise ConversionError(f"track file missing: {fpath}")
|
||||
yield from iter_raw_sectors(
|
||||
fpath,
|
||||
track.sector_size,
|
||||
pregap_sectors=track.pregap_sectors,
|
||||
progress_cb=progress_cb,
|
||||
cancel_event=cancel_event,
|
||||
)
|
||||
|
||||
|
||||
def _aligned_up(value: int, alignment: int) -> int:
|
||||
return value + ((alignment - (value % alignment)) % alignment)
|
||||
|
||||
|
||||
def convert(
|
||||
source: Path,
|
||||
title: str,
|
||||
game_id: str,
|
||||
base_pbp: Path,
|
||||
out_path: Path,
|
||||
progress_cb=None,
|
||||
cancel_event=None,
|
||||
) -> Path:
|
||||
"""Convert a PS1 disc image to ``out_path`` (an ``EBOOT.PBP``).
|
||||
|
||||
``source`` may be a ``.cue`` (multi-track supported) or a single
|
||||
``.iso``/``.img``/``.bin``. ``base_pbp`` must be a PSN PS1 classic EBOOT.
|
||||
"""
|
||||
if not base_pbp.is_file():
|
||||
raise ConversionError(f"base.pbp not found: {base_pbp}")
|
||||
if not source.is_file():
|
||||
raise ConversionError(f"disc image not found: {source}")
|
||||
|
||||
game_id = game_id.strip()
|
||||
if not game_id:
|
||||
raise ConversionError("a game id (e.g. SLUS00594) is required")
|
||||
|
||||
if source.suffix.lower() == ".cue":
|
||||
disc: DiscImage = DiscImage(parse_cue(source.read_text(encoding="utf-8", errors="replace")), source)
|
||||
else:
|
||||
disc = DiscImage(None, source)
|
||||
|
||||
# Resources required from the base PBP.
|
||||
data_psp = extract_data_psp(base_pbp)
|
||||
startdat = extract_startdat(base_pbp)
|
||||
|
||||
sfo = default_ps1_sfo(title, game_id)
|
||||
icon0 = default_icon0_png()
|
||||
pic1 = default_pic1_png()
|
||||
|
||||
# Sections in header order; empty sections keep the previous offset.
|
||||
sections = {
|
||||
"sfo": sfo,
|
||||
"icon0": icon0,
|
||||
"icon1": b"",
|
||||
"pic0": b"",
|
||||
"pic1": pic1,
|
||||
"snd0": b"",
|
||||
"data_psp": data_psp,
|
||||
# psar written separately, sized after everything else.
|
||||
"psar": None,
|
||||
}
|
||||
|
||||
header = [0] * PBP_HEADER_U32
|
||||
header[0] = struct.unpack("<I", PBP_MAGIC)[0]
|
||||
header[1] = 0x00010000
|
||||
|
||||
current = PBP_HEADER_SIZE
|
||||
for i, key in enumerate(SECTION_ORDER):
|
||||
header[2 + i] = current
|
||||
if key != "psar":
|
||||
current += len(sections[key])
|
||||
|
||||
# DATA.PSAR must start on a 0x10000 boundary. It is the final section, so
|
||||
# its length is implicit (everything up to STARTDAT/EOF).
|
||||
psar_offset = _aligned_up(header[2 + SECTION_ORDER.index("psar")], PSAR_ALIGNMENT)
|
||||
header[2 + SECTION_ORDER.index("psar")] = psar_offset
|
||||
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = out_path.with_suffix(out_path.suffix + ".tmp")
|
||||
try:
|
||||
with open(tmp, "wb") as fh:
|
||||
fh.write(struct.pack("<10I", *header))
|
||||
for key in SECTION_ORDER:
|
||||
if key == "psar":
|
||||
break
|
||||
if sections[key]:
|
||||
fh.write(sections[key])
|
||||
|
||||
# Pad up to the aligned PSAR offset.
|
||||
while fh.tell() < psar_offset:
|
||||
fh.write(b"\x00")
|
||||
|
||||
_write_psar(fh, disc, game_id, title, progress_cb=progress_cb, cancel_event=cancel_event)
|
||||
|
||||
# STARTDAT goes at the very end.
|
||||
fh.write(startdat)
|
||||
os.replace(tmp, out_path)
|
||||
except BaseException:
|
||||
try:
|
||||
tmp.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
return out_path
|
||||
|
||||
|
||||
def convert_to_folder(
|
||||
source: Path,
|
||||
title: str,
|
||||
game_id: str,
|
||||
base_pbp: Path,
|
||||
out_dir: Path,
|
||||
progress_cb=None,
|
||||
cancel_event=None,
|
||||
) -> Path:
|
||||
"""Convert a PS1 disc into a game folder ready for /PSP/GAME.
|
||||
|
||||
Writes ``<out_dir>/<GAME_ID>/EBOOT.PBP`` — the same single-file output the
|
||||
popstation tool produces (uncompressed/unencrypted level-0 PSAR). No
|
||||
KEYS.BIN or DOCUMENT.DAT is written: popstation doesn't emit them and CFW
|
||||
(ARK-4 POPS) doesn't need them.
|
||||
"""
|
||||
folder_name = re.sub(r"[^A-Za-z0-9]", "", game_id).upper() or "PS1GAME"
|
||||
folder = Path(out_dir) / folder_name
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
convert(source, title, game_id, base_pbp, folder / "EBOOT.PBP",
|
||||
progress_cb=progress_cb, cancel_event=cancel_event)
|
||||
return folder
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------
|
||||
# Self-test / CLI
|
||||
# --------------------------------------------------------------------------
|
||||
|
||||
def _self_test() -> None:
|
||||
"""Validate the parts that do not need hardware: SFO round-trip, PNG
|
||||
generation, PBP header round-trip, and cue parsing."""
|
||||
import tempfile
|
||||
|
||||
# CUE parsing.
|
||||
cue = parse_cue(
|
||||
'FILE "Game (Track 1).bin" BINARY\n'
|
||||
" TRACK 01 MODE2/2352\n"
|
||||
" INDEX 01 00:00:00\n"
|
||||
" TRACK 02 AUDIO\n"
|
||||
" INDEX 00 00:02:00\n"
|
||||
" INDEX 01 00:04:00\n"
|
||||
'FILE "Game (Track 2).bin" BINARY\n'
|
||||
" TRACK 03 AUDIO\n"
|
||||
" INDEX 01 00:00:00\n"
|
||||
)
|
||||
assert len(cue.tracks) == 3, "expected 3 tracks"
|
||||
assert cue.tracks[0].mode == "MODE2/2352"
|
||||
assert cue.tracks[1].pregap_sectors == 150, "expected 2s (150 sector) pregap"
|
||||
assert cue.data_track is cue.tracks[0]
|
||||
|
||||
# TOC generation: 1 data (10 sectors) + 1 audio track (20 sectors + 2s pregap).
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
d = Path(td)
|
||||
(d / "t1.bin").write_bytes(b"\x00" * (10 * RAW_SECTOR))
|
||||
(d / "t2.bin").write_bytes(b"\x00" * (20 * RAW_SECTOR))
|
||||
cue2 = parse_cue(
|
||||
'FILE "t1.bin" BINARY\n TRACK 01 MODE2/2352\n INDEX 01 00:00:00\n'
|
||||
'FILE "t2.bin" BINARY\n TRACK 02 AUDIO\n INDEX 00 00:00:00\n INDEX 01 00:02:00\n'
|
||||
)
|
||||
toc = build_toc(DiscImage(cue2, d / "t.cue"))
|
||||
assert len(toc) == (2 + 3) * 10, len(toc)
|
||||
assert toc[0] == 0x41 and toc[2] == 0xA0 and toc[7] == 0x01 # A0: first track
|
||||
assert toc[10] == 0x01 and toc[12] == 0xA1 and toc[17] == 0x02 # A1: last track
|
||||
assert toc[22] == 0xA2 # A2: lead-out
|
||||
assert toc[30] == 0x41 and toc[32] == 0x01 # track 1: data, number 1
|
||||
assert toc[40] == 0x01 and toc[42] == 0x02 # track 2: audio, number 2
|
||||
# track 2 relative address = start(160) + 150 lead-in = 310 = 00:04:10
|
||||
assert toc[47] == 0x00 and toc[48] == 0x04 and toc[49] == 0x10
|
||||
|
||||
# SFO round-trip through the project's own parser.
|
||||
from . import iso_reader # local import to keep the module importable standalone
|
||||
|
||||
sfo = default_ps1_sfo("Final Fantasy VII", "SCUS94163")
|
||||
parsed = iso_reader.parse_sfo(sfo)
|
||||
assert parsed.get("TITLE") == "Final Fantasy VII", parsed
|
||||
assert parsed.get("DISC_ID") == "SCUS94163", parsed
|
||||
assert parsed.get("CATEGORY") == "ME", parsed
|
||||
assert parsed.get("BOOTABLE") == 1, parsed
|
||||
|
||||
# PNG generation produces valid PNG signatures + IHDR dimensions.
|
||||
icon = default_icon0_png()
|
||||
assert icon[:8] == b"\x89PNG\r\n\x1a\n"
|
||||
assert struct.unpack(">II", icon[16:24]) == (144, 80)
|
||||
pic = default_pic1_png()
|
||||
assert struct.unpack(">II", pic[16:24]) == (480, 272)
|
||||
|
||||
# PBP header round-trip: write a synthetic PBP and read it back.
|
||||
with tempfile.TemporaryDirectory() as td:
|
||||
p = Path(td) / "test.pbp"
|
||||
header = [0] * 10
|
||||
header[0] = struct.unpack("<I", PBP_MAGIC)[0]
|
||||
header[1] = 0x00010000
|
||||
header[2] = 0x28
|
||||
header[3] = 0x28 + len(sfo)
|
||||
with open(p, "wb") as fh:
|
||||
fh.write(struct.pack("<10I", *header))
|
||||
fh.write(sfo)
|
||||
assert read_pbp_header(p)[0] == header[0]
|
||||
assert read_pbp_section(p, header, 2) == sfo
|
||||
|
||||
print("ps1_convert self-test OK")
|
||||
|
||||
|
||||
def main(argv: Optional[List[str]] = None) -> int:
|
||||
ap = argparse.ArgumentParser(description="Convert a PS1 disc image to a PSP EBOOT.PBP")
|
||||
ap.add_argument("--source", help="path to .cue or a single .iso/.img/.bin")
|
||||
ap.add_argument("--title", help="game title shown on the PSP")
|
||||
ap.add_argument("--game-id", help="PS1 game id, e.g. SLUS00594")
|
||||
ap.add_argument("--base-pbp", help="path to a PSN PS1 classic EBOOT.PBP (required)")
|
||||
ap.add_argument("--out", help="output EBOOT.PBP path (single file)")
|
||||
ap.add_argument("--out-dir", help="output folder: writes <dir>/<GAME_ID>/EBOOT.PBP")
|
||||
ap.add_argument("--test", action="store_true", help="run self-test and exit")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.test:
|
||||
_self_test()
|
||||
return 0
|
||||
|
||||
if not all([args.source, args.title, args.game_id, args.base_pbp]) or (not args.out and not args.out_dir):
|
||||
ap.error("--source --title --game-id --base-pbp and --out (or --out-dir) are all required (or use --test)")
|
||||
|
||||
try:
|
||||
if args.out_dir:
|
||||
folder = convert_to_folder(
|
||||
Path(args.source), args.title, args.game_id,
|
||||
Path(args.base_pbp), Path(args.out_dir),
|
||||
)
|
||||
print(f"wrote {folder}")
|
||||
else:
|
||||
convert(
|
||||
Path(args.source), args.title, args.game_id,
|
||||
Path(args.base_pbp), Path(args.out),
|
||||
)
|
||||
print(f"wrote {args.out}")
|
||||
return 0
|
||||
except ConversionError as exc:
|
||||
print(f"error: {exc}")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -18,6 +18,8 @@ FIELDS = [
|
||||
"igdb_client_id",
|
||||
"igdb_client_secret",
|
||||
"tmdb_api_key",
|
||||
"google_api_key",
|
||||
"google_cx",
|
||||
"screenscraper_devid",
|
||||
"screenscraper_devpassword",
|
||||
"screenscraper_ssid",
|
||||
@@ -29,6 +31,8 @@ _ENV_MAP = {
|
||||
"igdb_client_id": "IGDB_CLIENT_ID",
|
||||
"igdb_client_secret": "IGDB_CLIENT_SECRET",
|
||||
"tmdb_api_key": "TMDB_API_KEY",
|
||||
"google_api_key": "GOOGLE_API_KEY",
|
||||
"google_cx": "GOOGLE_CX",
|
||||
"screenscraper_devid": "SCREENSCRAPER_DEVID",
|
||||
"screenscraper_devpassword": "SCREENSCRAPER_DEVPASSWORD",
|
||||
"screenscraper_ssid": "SCREENSCRAPER_SSID",
|
||||
|
||||
@@ -6,6 +6,7 @@ archives, a remote HTTPS store server, etc.
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
import urllib.parse
|
||||
@@ -200,6 +201,54 @@ class FileCopySource(Source):
|
||||
return dest
|
||||
|
||||
|
||||
class Ps1ConvertSource(Source):
|
||||
"""Convert a PS1 disc image (.cue/.iso) into an EBOOT.PBP.
|
||||
|
||||
``item`` must carry ``source`` (path to the .cue or single image),
|
||||
``title`` and ``game_id``. ``base_pbp`` is a PSN PS1 classic EBOOT used to
|
||||
supply the encrypted DATA.PSP loader and STARTDAT block.
|
||||
"""
|
||||
|
||||
label = "PS1 convert"
|
||||
|
||||
def __init__(self, base_pbp):
|
||||
self.base_pbp = Path(base_pbp)
|
||||
|
||||
def scan(self, root=None):
|
||||
return []
|
||||
|
||||
def download(self, item, dest_dir, progress_cb=None, cancel_event=None):
|
||||
dest_dir = Path(dest_dir)
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
source = Path(item["source"])
|
||||
title = str(item.get("title") or source.stem)
|
||||
game_id = item.get("game_id")
|
||||
if not game_id:
|
||||
raise RuntimeError("missing game_id for PS1 conversion")
|
||||
|
||||
folder_name = re.sub(r"[^A-Za-z0-9]", "", game_id).upper() or "PS1GAME"
|
||||
folder = dest_dir / folder_name
|
||||
eboot = folder / "EBOOT.PBP"
|
||||
if eboot.exists():
|
||||
raise FileExistsError(str(eboot))
|
||||
|
||||
def cb(done, total):
|
||||
if progress_cb is not None and total:
|
||||
progress_cb("Converting", min(done / total, 1.0), "")
|
||||
|
||||
from . import ps1_convert
|
||||
|
||||
try:
|
||||
ps1_convert.convert_to_folder(
|
||||
source, title, game_id, self.base_pbp, dest_dir,
|
||||
progress_cb=cb, cancel_event=cancel_event,
|
||||
)
|
||||
except ps1_convert.Cancelled:
|
||||
raise TransferCancelled() from None
|
||||
return folder
|
||||
|
||||
|
||||
class TreeCopySource(Source):
|
||||
"""Copy a whole directory (save-data folder) into a destination directory."""
|
||||
|
||||
|
||||
File diff suppressed because one or more lines are too long
+1
-1
@@ -1,5 +1,5 @@
|
||||
clrmamepro (
|
||||
name "Sony - PlayStation Portable"
|
||||
name "Sony - PlayStation Portable (No-Intro)"
|
||||
description "Sony - PlayStation Portable"
|
||||
version "2021.10.24"
|
||||
homepage "http://github.com/robloach/libretro-dats"
|
||||
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user