Commit Graph
22 Commits
Author SHA1 Message Date
Patryk GenschandClaude Opus 5.5 597733d8d7 feat: type_events in diff and tracker, joined with emulator signal literals
Diff gains a type_events axis keyed by (script type, event). The tracker
lists each type's original events (vftable list, or the nearest class
list up the constructor chain for Piklib 6.1) and where the emulator names
them: the type's class, Variable.java or other engine code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:49:26 +02:00
Patryk GenschandClaude Opus 5.5 a3a572c9db fix(tracker): sanitize the Behaviour factory misparse before 6.1 mapping
Pinokio's 6.1 factory parse attributes every type to CMC_Behaviour; the
constructor-runner walk ran before that known failure was discarded and
gave INTEGER the BEHAVIOUR runner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:30:42 +02:00
Patryk GenschandClaude Opus 5.5 11f4b6fc37 feat: map Piklib 6.1 types through constructor-created runners
Piklib 6.1 predates CMC_RunnerFactory; each class constructor creates its
own runner. The extractor records class -> runner from those call sites
(runner_map.by_class) and the tracker walks the snapshot's constructor
chain from a type's class to the nearest one creating a runner, so 6.1
variable and condition types no longer fall into pseudo-types either.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:59:58 +02:00
Patryk GenschandClaude Opus 5.5 edae0929d7 feat(tracker): signatures and per-type validity from method facts
Original entries carry the signature read from the method body and the
per-type verdict of shared runners: a method the original skips for a
type via obj.getType() is no longer attributed to it, and a trivial
virtual is flagged as a no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:56:20 +02:00
Patryk GenschandClaude Opus 5.5 87fcefa877 feat(tracker): map script types through runner_map
Types follow CMC_RunnerFactory when the snapshot has it, so the shared
variable runner no longer forms a VARIABLE pseudo-type. Base CMC_Runner
verbs are listed once under * and referenced via inherits; type 0 becomes
the @ scope, matched against CodeParser.dispatchSpecialCall.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:37:17 +02:00
Patryk GenschandClaude Opus 5.5 0796924f6d feat(compat): keep game aliases and library names from batch snapshots
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:24:15 +02:00
Patryk GenschandClaude Opus 5.5 10c81c4390 feat(tracker): Atlas joining DLL snapshots with emulator registrations
Matches original script APIs against MethodSpec registrations in
Rex-EMoolator and keeps review evidence separate from registration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:24:15 +02:00
Patryk GenschandClaude Opus 5.5 b0013c5114 feat(acquire): resumable batch extraction queue and binary inventory
Inventory classifies the DLL repository by SHA; the batch queue runs each
engine build in a fresh PyGhidra process with a timeout, fingerprints the
extractor environment and keeps per-attempt state and logs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:24:15 +02:00
Patryk Gensch ce44e30c60 feat: add full supported API details for variants and enhance tests 2026-06-19 00:20:46 +02:00
Patryk Gensch 0666692198 feat: add detailed method changes in compatibility analysis and update tests 2026-06-19 00:15:04 +02:00
Patryk Gensch 429aafb6ce feat: enhance compatibility table with file name handling and UI updates 2026-06-19 00:07:18 +02:00
Patryk Gensch 2c785dc87c feat: add compatibility table endpoint and rendering for game libraries 2026-06-18 23:32:31 +02:00
Patryk GenschandClaude Opus 4.8 ba9db82a4c worker: optional PyGhidra back-end for Ghidra 11.4+/12.x (no Jython)
The .py extractor runs fine under PyGhidra in the GUI; only `analyzeHeadless`
doesn't init PyGhidra. Add an env-gated CPython path so modern Ghidra works headless:

- ghidra.run_extractor_pyghidra(): runs the same GhidraScript via pyghidra.run_script
  (boots Ghidra in-process, imports+analyses, getScriptArgs()=[out_path]); run_extractor
  dispatches to it when AMS_USE_PYGHIDRA is set. No script changes needed.
- worker image installs pyghidra + sets GHIDRA_INSTALL_DIR; compose exposes
  AMS_USE_PYGHIDRA (default off). Jython path stays the default and untouched.
- README documents both variants (Jython <=11.3.x vs PyGhidra 11.4+/12.x).
- test: AMS_USE_PYGHIDRA routes to the PyGhidra back-end (clear error if pkg missing).

35/35 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 18:03:04 +02:00
Patryk GenschandClaude Opus 4.8 b0d3d22445 Body normalisation: per-method similarity score + leaf delta
Turns the dispatch axis from a binary changed/unchanged into a "how much" measure
of code change — the original goal. ams.normalize compares two body fingerprints
(the ordered leaf-call anchors) with difflib after collapsing consecutive-duplicate
anchors (a load-twice codegen artefact), yielding a 0-100 similarity and the exact
leaves that appeared/vanished.

Every dispatch `changed` entry now carries body={similarity, added, removed}, and the
block carries a summary={shared, identical, changed, mean_similarity}.

Golden pair (cross-compiler): 470 shared bodies, 131 identical, mean 66% similar;
Animo SHOW/HIDE/PAUSE/RESUME come out 100% despite MSVC6 vs MSVC8, LOAD 50% with the
swapped leaves spelled out.

- normalize.py: canonical / body_similarity / body_delta
- diff: _dispatch_diff enriches changed with body + adds summary
- render: METHOD BODIES shows %, leaf delta, summary line
- UI: similarity % + leaf delta + axis summary
- tests: 5 new -> 34/34

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 13:23:15 +02:00
Patryk GenschandClaude Opus 4.8 27399a52b1 Method dispatch axis: map id -> body via Runner::run switch
Recovers how a script method id maps to its implementation, the foundation for
body-level normalisation. Each CMC_*_Runner::run is a switch(id) (vtable slot 17);
every case is the method body — inline (MSVC6) or a tail-call to a separate
show()/load() (MSVC8). The extractor parses the jump table at the disassembly
level (Ghidra's decompiler jump-table recovery silently dropped the big runners),
fingerprints each case by its ordered CALL anchors (Class::method / vtbl+0xNN),
and expands thin wrappers one level so MSVC8 lines up with MSVC6.

Validated on the golden pair: Animo SHOW..RESUME (id 1-4) yield identical leaves
(getAnimo + vtbl+0xa0/0xa4/0x4c/0x50) across both compilers. Coverage 30/32
runners; Piklib 475 / BlooMoo 619 dispatch rows.

- extract_engine_surface.py: extract_method_dispatch (schema_version -> 4)
- snapshots regenerated with the method_dispatch axis
- ams: Snapshot.method_dispatch; diff axis keyed (owner,id) on [impl,calls] with
  method-name join; render METHOD BODIES section; cli --only dispatch; owner filter
- UI: "Ciała metod" diff axis + browse tab
- tests: body-change unit + cross-compiler vtbl assertion -> 29/29

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 13:15:58 +02:00
Patryk GenschandClaude Opus 4.8 38be932abc Similar versions: surface-overlap metric + endpoint + UI panel
Ranks catalogued engine versions by how much of their CMC_* surface they share,
which (unlike a binary fuzzy hash) stays meaningful across compilers — the golden
pair PIKLIB8/MSVC6 vs bloomoodll/MSVC8 scores 85%.

- similarity.py: jaccard, surface_similarity (per-axis + pooled overall),
  fuzzy_similarity (ssdeep via ppdeep, secondary signal)
- service.similar_snapshots + GET /snapshots/{id}/similar?min=N (SimilarHit)
- UI: "Podobne wersje" panel in the snapshot browser (overlap bar + ⇄ diff)
- tests: 6 new (jaccard, identical/disjoint, golden pair 0<x<100, fuzzy,
  endpoint + min filter) -> 28/28

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 12:33:50 +02:00
Patryk GenschandClaude Opus 4.8 30b2b1011e UI: upload panel — POST /jobs with live status polling
Adds a "+ wgraj" control to the sidebar that uploads an ISO/ZIP/DLL to the
acquisition endpoint and tracks the job to completion, then refreshes the
version list so the new snapshot appears without a reload.

- index.html: upload form + #jobs panel in the sidebar
- app.js: submitUpload() (FormData → POST /jobs), pollJobs() (2.5s while any
  job is queued/started; finished → load(); failed → inline error)
- style.css: mini-btn / upload form / job rows + queued/started badges

Verified: node --check clean; uvicorn serves /ui assets 200 and GET /jobs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 12:28:35 +02:00
Patryk GenschandClaude Opus 4.8 f4aa7caaa9 Containerise: Postgres + Redis/RQ + API + Ghidra worker
Brings up the documented target architecture as a docker-compose stack — a
modular monolith with the Ghidra step split into its own async worker.

- worker/: RQ queue (lazy redis import) + run_acquisition task (Job status
  queued→started→finished/failed, drives ams.acquire with sink=db)
- Job model + JobOut schema; Snapshot.data is JSONB on Postgres
- POST/GET /jobs: stream an upload to a shared volume, enqueue, poll status
- docker/api.Dockerfile (slim) + docker/worker.Dockerfile (JDK21 + Ghidra
  fetched at build, overridable via GHIDRA_URL) + docker-compose.yml
- ghidra.py: AMS_GHIDRA_SCRIPTS override for in-container script path
- pyproject: [worker] extra (rq/redis/psycopg), python-multipart in [api]
- tests: 4 new (task success/failure + endpoint enqueue/503) -> 22/22

Verified: API image builds, container serves /health + /ui + /jobs; compose
config validates. Worker image (downloads ~1 GB Ghidra) not built here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 12:24:47 +02:00
Patryk GenschandClaude Opus 4.8 6797ad5ddb Add ISO/ZIP acquisition pipeline (ams.acquire worker)
Closes the chain from a game file to a catalog entry: unpack an ISO/ZIP,
content-identify the engine DLL (CMC_ObjectsContainer marker in RTTI, so a
renamed file is still found), hash it (sha256 + md5 + optional ssdeep via
ppdeep), run Ghidra headless with the extractor, enrich and import the snapshot.

- unpack.py: bsdtar (ISO9660 + ZIP) with a pure-Python zipfile fallback
- identify.py: content-based engine-DLL picker + hashing
- ghidra.py: analyzeHeadless launcher discovery + post-script run
- pipeline.py: orchestration with injectable extract_fn; sink db|http|none
- cli.py: python -m ams.acquire (incl. --identify-only dry run)
- tests: 7 new (forged PE markers + stubbed extractor) -> 18/18

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 12:11:56 +02:00
Patryk GenschandClaude Opus 4.8 4542763936 Add Command Center web UI (no build step)
Static HTML/CSS/JS served by FastAPI (mounted at /ui, / redirects there),
talking to the existing JSON API — no node/npm, no bundler.

- games/versions sidebar with A/B version selectors
- visual 4-axis diff (types/methods/events/fields, +/- struct_layout) with
  +/-/~ rows, per-axis counts, class (owner) filter, moved-methods section
- single-snapshot browser (tabs + live filter)
- app.py mounts StaticFiles(html=True) last so API routes win; / -> /ui/

Smoke-tested live on uvicorn: /, /ui/ and assets serve 200; UI wiring drives
the same /games and /diff endpoints verified end-to-end. app.js passes
`node --check`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 23:37:03 +02:00
Patryk GenschandClaude Opus 4.8 8386196653 Add FastAPI catalog backend (games/snapshots/diff) + tests
Modular-monolith backend over SQLAlchemy (SQLite by default, Postgres-ready
via DATABASE_URL). The full snapshot.json is stored verbatim; diffing reads it
back through the ams.diff engine, so the DB never mirrors the snapshot schema.

- ams.api.db/models/schemas/service : Game 1-N Snapshot, sha256-deduped upsert
- routes: POST/GET /games, POST/GET /snapshots (import, deduped), GET /diff
          (?old&new[&owner]) running compute_diff on stored snapshots, /health
- ams.api.importer : bulk CLI loader (python -m ams.api.importer --game ...)
- run: uvicorn ams.api.app:create_app --factory

11 tests pass (6 diff + 5 API via TestClient over the golden pair). Smoke-tested
live on uvicorn: import -> /snapshots -> /diff returns the BlooMoo deltas.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 22:27:24 +02:00
Patryk GenschandClaude Opus 4.8 6885bbee3d Add snapshot diff engine (ams package) + tests
Standalone CLI that diffs two engine-surface snapshots across all four axes,
the foundation the FastAPI/DB layer will sit on.

- ams.snapshot : typed access to a snapshot.json
- ams.diff     : keyed set-diff per axis (added/removed/changed) + cross-owner
                 method-move detection; types keyed by (script_name,
                 via_module_iface) so the dual MULTIARRAY stays stable;
                 filter_by_owner for per-class focus
- ams.render   : human-readable report (+/-/~), owner-grouped
- ams.cli      : python -m ams OLD NEW [--owner C] [--only ...] [--json]

6 tests pass, incl. an integration test over the committed golden pair
(asserts BlooMoo adds GRBUFFER/INTERNET, MOUSE grows 104->128, Animo gains
GETFPS, Animo script fields unchanged).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 22:18:04 +02:00