Commit Graph
50 Commits
Author SHA1 Message Date
Patryk GenschandClaude Opus 5.5 be5a701db8 fix(extractor): read a 6.1 case's facts from its member when run has no case region
Pinokio's unoptimised run(int, CXVector*) now parses as a switch, but the
decompiler gives Application's cases no region; a case that forwards to a
named runner member is read from that member, as name dispatch did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 21:34:36 +01:00
Patryk GenschandClaude Opus 5.5 7d85a69274 feat(tracker): report a scene module under the engine it runs in
Binaries from a module snapshot carry host_family (from module_of); the
roadmap lists a module type's presence by that engine family.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 21:00:23 +01:00
Patryk GenschandClaude Opus 5.5 4e65268a1e feat(extractor): extract scene modules (SCENE:DLLS=) like engines
A module DLL has no engine factory: its CMC_ExternObject::resolve compares
TYPE (CXString::equals or operator==), allocates and constructs. The
runner of each type is the vftable its constructor or a virtual installs
whose prepare slot chains to CMC_Runner::prepareMthHashSet (defined when
analysis left it out); registrations, dispatch, signatures, per-type
events and fields then come from the engine passes, limited to the
module's own runners. Snapshots record module_of (the host engine DLL).

An unoptimised run() keeps the switch index in a stack slot; its bounds
are followed through that slot only when the register match finds none.
The batch queue extracts plugins too, the inventory knows MATRIX, QMAN,
INERTIA and BOARD, and the tracker marks module-mapped types.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:58:40 +01:00
Patryk GenschandClaude Opus 5.5 f71cd9a90f fix(roadmap): credit a base method the emulator has on the object's own type
A script call ANIMO^GETNAME is served by AnimoVariable's own GETNAME, so
it no longer counts toward the missing base `*` GETNAME.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:38:23 +01:00
Patryk GenschandClaude Opus 5.5 60a2c65e1f data: script usage of 34 game releases and the usage-ranked roadmap
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:37:10 +01:00
Patryk GenschandClaude Opus 5.5 ebbeeea144 feat(usage): count how game scripts use each type and rank roadmap gaps by it
ams.usage reads game discs with 7-Zip (bsdtar misses the files on some
images), decrypts CNV/DEF/CLASS like the emulator's ScriptDecypher and
counts method calls, event handlers, properties and objects per declared
type. CLONE copies, database cursors, global objects, CLASS instances and
string-named targets are resolved; the rest stays per method name.

ams.roadmap --usage orders every gap by the games and script places that
use it, adds a priority table and lists direct usage the original's known
API does not explain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 20:37:10 +01:00
Patryk GenschandClaude Opus 5.5 96c39d542a data: re-extract 34 engine builds with per-type fields and Piklib 6.1 ids (schema 7)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 21:29:24 +02:00
Patryk GenschandClaude Opus 5.5 b16c8ecf33 docs: generated emulator roadmap and schema 7 results
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 21:29:24 +02:00
Patryk GenschandClaude Opus 5.5 d6d6010f4e fix(extractor): keep every Piklib 6.1 name and read ids of unoptimised builds
getMethodId ids are now completed by run(CXString) names instead of
replacing them, so a partial parse never drops methods. Pinokio's
unoptimised build stores the id into a stack local, which the register-
only and no-reference filters rejected (225 -> 103 methods).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 21:03:10 +02:00
Patryk GenschandClaude Opus 5.5 95ad4edb5c feat: ams.roadmap renders the emulator's gaps per scope from an Atlas export
Methods without a registration (with the original signature), emulator
stubs, and events/properties the emulator never names, each with the
engine families whose DLLs have it so core API sorts before late ones.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:35:42 +02:00
Patryk GenschandClaude Opus 5.5 835cd4f729 feat: Piklib 6.1 method ids, name dispatch and signatures
Later 6.1 builds give every runner getMethodId(CXString); its
equalsIgnoreCase chain now yields name -> id, so their run(int) switch
rows join to names. Runners without a switch (earliest 6.1) get rows
from run(CXString) by name, joined by name in the tracker. Method facts
run for 6.1 too, with its ABI: run(int, CXVector*), methods as runner
members taking only the params, and new CMC_<T> as the result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:34:33 +02:00
Patryk GenschandClaude Opus 5.5 aee8f17417 feat: type_fields in diff and tracker, joined with emulator property reads
The tracker lists each scope's original properties (constructor chain,
or class fields up the chain for Piklib 6.1 and singletons such as
SCENE) and where the emulator reads them: a CNVParser.createVariable
case arm, an attribute read in the type's class or Variable.java, or
other code. The switch-arm parser is shared with the @ statements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:30:36 +02:00
Patryk GenschandClaude Opus 5.5 9d5a600d6b feat(extractor): properties per script type along the constructor chain
type_fields walks each type's factory constructor through its base
constructors, base first, so unnamed classes get their properties
(VECTOR VALUE/SIZE, FILTER ACTION, PATH NODES, MOVIE FILENAME/POSITION)
and INTEGER/STRING inherit CMC_Variable's TOINI/NETNOTIFY. Property
names come only from pushed or register-loaded literals: a vftable store
whose bytes look printable produced BlooMoo's X5/Gs field names. Schema 7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:30:36 +02:00
Patryk GenschandClaude Opus 5.5 6217fde83c data: re-extract 34 engine builds with per-type events (schema 6)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 20:14:57 +02:00
Patryk GenschandClaude Opus 5.5 597733d8d7 feat: type_events in diff and tracker, joined with emulator signal literals
Diff gains a type_events axis keyed by (script type, event). The tracker
lists each type's original events (vftable list, or the nearest class
list up the constructor chain for Piklib 6.1) and where the emulator names
them: the type's class, Variable.java or other engine code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:49:26 +02:00
Patryk GenschandClaude Opus 5.5 cb03de47ac feat(extractor): events per script type through the type's vftable
type_events reads getBehavioursList from each type's own vftable, so
unnamed classes (FILTER, FIFO/LIFO, MOVIE) get their events and
INTEGER/STRING/... resolve to CMC_Variable's list without guessing. The
slot is voted by same-class overrides because the folded base body
carries unrelated labels. Type vftables are recorded for every engine.
Schema 6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:49:26 +02:00
Patryk GenschandClaude Opus 5.5 f85e2a14ef data: re-extract 34 engine builds with runner_map and method facts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:30:42 +02:00
Patryk GenschandClaude Opus 5.5 a3a572c9db fix(tracker): sanitize the Behaviour factory misparse before 6.1 mapping
Pinokio's 6.1 factory parse attributes every type to CMC_Behaviour; the
constructor-runner walk ran before that known failure was discarded and
gave INTEGER the BEHAVIOUR runner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:30:42 +02:00
Patryk GenschandClaude Opus 5.5 f8178a08f3 docs: runner_map and method facts; tools/ghidra_probe.py for fast iteration
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 19:01:41 +02:00
Patryk GenschandClaude Opus 5.5 11f4b6fc37 feat: map Piklib 6.1 types through constructor-created runners
Piklib 6.1 predates CMC_RunnerFactory; each class constructor creates its
own runner. The extractor records class -> runner from those call sites
(runner_map.by_class) and the tracker walks the snapshot's constructor
chain from a type's class to the nearest one creating a runner, so 6.1
variable and condition types no longer fall into pseudo-types either.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:59:58 +02:00
Patryk GenschandClaude Opus 5.5 edae0929d7 feat(tracker): signatures and per-type validity from method facts
Original entries carry the signature read from the method body and the
per-type verdict of shared runners: a method the original skips for a
type via obj.getType() is no longer attributed to it, and a trivial
virtual is flagged as a no-op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:56:20 +02:00
Patryk GenschandClaude Opus 5.5 5f3006cdf0 feat(extractor): method signatures and per-type validity from P-code
Each dispatch row now records the argument types a method reads through
CMC_Runner::getParameter*, the CMC_VariableFactory allocator its result
comes from, the virtuals it calls on the object and the type ids it
branches on. Runner helpers are followed with roles bound by storage,
since MSVC8 bodies without a signature drop an unused this.

For runners several script types share, per_type says whether a method
applies to each type: only an explicit obj.getType() branch excludes one,
and trivial vftable entries mark a no-op. Types record their vftable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:55:12 +02:00
Patryk GenschandClaude Opus 5.5 87fcefa877 feat(tracker): map script types through runner_map
Types follow CMC_RunnerFactory when the snapshot has it, so the shared
variable runner no longer forms a VARIABLE pseudo-type. Base CMC_Runner
verbs are listed once under * and referenced via inherits; type 0 becomes
the @ scope, matched against CodeParser.dispatchSpecialCall.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:37:17 +02:00
Patryk GenschandClaude Opus 5.5 5c597a3a52 feat(extractor): runner_map axis from CMC_RunnerFactory
Script dispatch is getRunner(obj.getType())->run(id), so map type ids to
runners instead of guessing runner names from C++ classes. Type ids come
from the base-constructor argument, the map from emulating getRunner, and
runner vtables from RunnerFactory::initialize; vtable slots are read off
the labelled CMC_Runner vftable because they move between releases.

This recovers runners without symbols (VECTOR, FILTER, STATICFILTER,
PATH, FIFO/LIFO, MOVIE and the @ built-ins of type 0) and lets several
script types share one runner (INTEGER/STRING/BOOL/DOUBLE/EXPRESSION).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:37:17 +02:00
Patryk GenschandClaude Opus 5.5 3132a5222a data: snapshots of 34 engine builds and WORLD/KOLOROWANKA evidence
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:24:15 +02:00
Patryk GenschandClaude Opus 5.5 0796924f6d feat(compat): keep game aliases and library names from batch snapshots
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:24:15 +02:00
Patryk GenschandClaude Opus 5.5 10c81c4390 feat(tracker): Atlas joining DLL snapshots with emulator registrations
Matches original script APIs against MethodSpec registrations in
Rex-EMoolator and keeps review evidence separate from registration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:24:15 +02:00
Patryk GenschandClaude Opus 5.5 b0013c5114 feat(acquire): resumable batch extraction queue and binary inventory
Inventory classifies the DLL repository by SHA; the batch queue runs each
engine build in a fresh PyGhidra process with a timeout, fingerprints the
extractor environment and keeps per-attempt state and logs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:24:15 +02:00
Patryk GenschandClaude Opus 5.5 bddb5949a6 fix(extractor): read registration literals from raw bytes first
Analyzer-typed string data can split short or overlapping literals; the
operand points at the literal start, so trust the bytes and require a
terminator within the bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 18:24:15 +02:00
Patryk Gensch ce44e30c60 feat: add full supported API details for variants and enhance tests 2026-06-19 00:20:46 +02:00
Patryk Gensch 0666692198 feat: add detailed method changes in compatibility analysis and update tests 2026-06-19 00:15:04 +02:00
Patryk Gensch 429aafb6ce feat: enhance compatibility table with file name handling and UI updates 2026-06-19 00:07:18 +02:00
Patryk Gensch 2c785dc87c feat: add compatibility table endpoint and rendering for game libraries 2026-06-18 23:32:31 +02:00
Patryk GenschandClaude Opus 4.8 8875540186 Extract Piklib 6.1 methods by name from Runner::run
6.1 has no prepareMthHashSet and no id-switch: CMC_*_Runner::run takes the method
*name* (CXString) and dispatches via a `CXString(tmp,"name"); equalsIgnoreCase(name)`
chain. extract_methods now falls back (only when the hashset pass finds nothing) to
scanning run() for that pattern, recovering the names (no numeric ids).

6.1 now yields 186 methods (Animo: show/hide/play/setFPS/...); dispatch stays 0 since
the string chain isn't a jump table. 7.1/8.x untouched (they have prepareMthHashSet).

Note: 6.1 names are camelCase/lowercase vs 7.1+ UPPERCASE (engine compares case-
insensitively), so a cross-version method diff needs case folding to be clean.

38/38 tests (test_versions updated: 6.1 methods present with id=None).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 20:25:20 +02:00
Patryk GenschandClaude Opus 4.8 67cbc32a2c Support Piklib 6.1/7.1: CMC_Scene::resolve factory + tag-based types
Earlier text-script engines (Piklib 6.1/7.1, added text scripts in 6.1) keep the
type factory on CMC_Scene::resolve, not CMC_ObjectsContainer::resolve — so the
extractor bailed with "resolve not found". find_factory() now tries both anchors.

6.1's factory is also tag-based: each branch is operator==(NAME) -> new(0x74) ->
store tag -> jmp, with the ctor in a separate tag switch (no inline ctor). extract_types
gains a pre-emit: when the next operator== arrives still armed, it records the pending
type by name (size known, ctor/cpp_class not). The 8.x inline-ctor factory clears `armed`
first, so it's untouched (golden pair unchanged).

Per-version reality: 6.1 = 23 types / 0 methods (no prepareMthHashSet yet) / 103 events
/ 80 fields; 7.1 = 26 / 322 / 102 / 86 / 288 dispatch (full); type names line up across
6.1->7.1->8.x so version diffs work.

- snapshots/PIKLib61 + PIKLIB71 added as golden fixtures (evolution chain)
- tests/test_versions.py: 6.1 partial surface, 7.1 full, 61->71 diff -> 38/38

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 19:53:47 +02:00
Patryk GenschandClaude Opus 4.8 ba9db82a4c worker: optional PyGhidra back-end for Ghidra 11.4+/12.x (no Jython)
The .py extractor runs fine under PyGhidra in the GUI; only `analyzeHeadless`
doesn't init PyGhidra. Add an env-gated CPython path so modern Ghidra works headless:

- ghidra.run_extractor_pyghidra(): runs the same GhidraScript via pyghidra.run_script
  (boots Ghidra in-process, imports+analyses, getScriptArgs()=[out_path]); run_extractor
  dispatches to it when AMS_USE_PYGHIDRA is set. No script changes needed.
- worker image installs pyghidra + sets GHIDRA_INSTALL_DIR; compose exposes
  AMS_USE_PYGHIDRA (default off). Jython path stays the default and untouched.
- README documents both variants (Jython <=11.3.x vs PyGhidra 11.4+/12.x).
- test: AMS_USE_PYGHIDRA routes to the PyGhidra back-end (clear error if pkg missing).

35/35 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 18:03:04 +02:00
Patryk GenschandClaude Opus 4.8 aa65beb7c1 worker: pin Jython-era Ghidra (11.2.1) for headless .py post-script
Ghidra 11.4+/12.x dropped the bundled Jython, so the .py extractor fails headless
with "Ghidra was not started with PyGhidra. Python is not available" — analysis
succeeds but the post-script never runs, so no snapshot is produced. Default
GHIDRA_URL now points at 11.2.1 (Jython); README documents the constraint and the
PyGhidra path for staying on 12.x. Keeps the local Dockerfile fixes (pip upgrade,
non-editable install).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 17:57:17 +02:00
Patryk GenschandClaude Opus 4.8 b77419f4d2 docs: add Quickstart (local, docker compose + Ghidra worker) to README
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 17:00:14 +02:00
Patryk GenschandClaude Opus 4.8 be733cf8b7 Harden dispatch switch parsing (id-gap drobiazgi)
Resolves the method-id gaps surfaced by the dispatch axis, all real switch-shape
edge cases rather than numbering bugs:

- default holes: ids the runner doesn't implement route to the `JA default` block
  (tail-call to base CMC_Runner::run); capture that target and drop those cases
  (was emitting false Sound 5/6, Scene 10-15, Array 26-31)
- sign-extension: high-base switches (CMC_NetPeer id 257+) encode the base as
  `LEA/ADD idx, 0xFFFFFEFF` (-257); _s32 sign-extends on both the scalar and the
  text path (Ghidra prints big displacements unsigned, small ones signed)
- two-level (byte-indexed) switches: sparse runners (Image) use
  `MOVZX r,byte[i+byteTable]` (MSVC8) / `MOV rl,byte[i+byteTable]` (MSVC6) then
  `JMP [r*4+ptrTable]`; decode target = ptrTable[byteTable[i]], taking base/count
  from the byte-table's index register (differs from the JMP index reg on MSVC6)
- _executable() guard + id clamp: never emit a non-code "case"

Result: Piklib 500 rows / BlooMoo 561, garbage 0, dispatch<->methods consistent.
The lone genuinely-nameless method is CMC_Animo id 14 (a bool getter prepareMthHashSet
doesn't register) - a real engine property, correctly absent from the methods axis.

FUN_ ctor names are not recoverable (no symbols/mangled strings/RTTI in the binary
for FILTER/MOVIE/VECTOR/PATH/FIFO/LIFO/STATICFILTER); cpp_class=None stays.

Snapshots regenerated; 34/34 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 16:56:24 +02:00
Patryk GenschandClaude Opus 4.8 b0d3d22445 Body normalisation: per-method similarity score + leaf delta
Turns the dispatch axis from a binary changed/unchanged into a "how much" measure
of code change — the original goal. ams.normalize compares two body fingerprints
(the ordered leaf-call anchors) with difflib after collapsing consecutive-duplicate
anchors (a load-twice codegen artefact), yielding a 0-100 similarity and the exact
leaves that appeared/vanished.

Every dispatch `changed` entry now carries body={similarity, added, removed}, and the
block carries a summary={shared, identical, changed, mean_similarity}.

Golden pair (cross-compiler): 470 shared bodies, 131 identical, mean 66% similar;
Animo SHOW/HIDE/PAUSE/RESUME come out 100% despite MSVC6 vs MSVC8, LOAD 50% with the
swapped leaves spelled out.

- normalize.py: canonical / body_similarity / body_delta
- diff: _dispatch_diff enriches changed with body + adds summary
- render: METHOD BODIES shows %, leaf delta, summary line
- UI: similarity % + leaf delta + axis summary
- tests: 5 new -> 34/34

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 13:23:15 +02:00
Patryk GenschandClaude Opus 4.8 27399a52b1 Method dispatch axis: map id -> body via Runner::run switch
Recovers how a script method id maps to its implementation, the foundation for
body-level normalisation. Each CMC_*_Runner::run is a switch(id) (vtable slot 17);
every case is the method body — inline (MSVC6) or a tail-call to a separate
show()/load() (MSVC8). The extractor parses the jump table at the disassembly
level (Ghidra's decompiler jump-table recovery silently dropped the big runners),
fingerprints each case by its ordered CALL anchors (Class::method / vtbl+0xNN),
and expands thin wrappers one level so MSVC8 lines up with MSVC6.

Validated on the golden pair: Animo SHOW..RESUME (id 1-4) yield identical leaves
(getAnimo + vtbl+0xa0/0xa4/0x4c/0x50) across both compilers. Coverage 30/32
runners; Piklib 475 / BlooMoo 619 dispatch rows.

- extract_engine_surface.py: extract_method_dispatch (schema_version -> 4)
- snapshots regenerated with the method_dispatch axis
- ams: Snapshot.method_dispatch; diff axis keyed (owner,id) on [impl,calls] with
  method-name join; render METHOD BODIES section; cli --only dispatch; owner filter
- UI: "Ciała metod" diff axis + browse tab
- tests: body-change unit + cross-compiler vtbl assertion -> 29/29

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 13:15:58 +02:00
Patryk GenschandClaude Opus 4.8 38be932abc Similar versions: surface-overlap metric + endpoint + UI panel
Ranks catalogued engine versions by how much of their CMC_* surface they share,
which (unlike a binary fuzzy hash) stays meaningful across compilers — the golden
pair PIKLIB8/MSVC6 vs bloomoodll/MSVC8 scores 85%.

- similarity.py: jaccard, surface_similarity (per-axis + pooled overall),
  fuzzy_similarity (ssdeep via ppdeep, secondary signal)
- service.similar_snapshots + GET /snapshots/{id}/similar?min=N (SimilarHit)
- UI: "Podobne wersje" panel in the snapshot browser (overlap bar + ⇄ diff)
- tests: 6 new (jaccard, identical/disjoint, golden pair 0<x<100, fuzzy,
  endpoint + min filter) -> 28/28

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 12:33:50 +02:00
Patryk GenschandClaude Opus 4.8 30b2b1011e UI: upload panel — POST /jobs with live status polling
Adds a "+ wgraj" control to the sidebar that uploads an ISO/ZIP/DLL to the
acquisition endpoint and tracks the job to completion, then refreshes the
version list so the new snapshot appears without a reload.

- index.html: upload form + #jobs panel in the sidebar
- app.js: submitUpload() (FormData → POST /jobs), pollJobs() (2.5s while any
  job is queued/started; finished → load(); failed → inline error)
- style.css: mini-btn / upload form / job rows + queued/started badges

Verified: node --check clean; uvicorn serves /ui assets 200 and GET /jobs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 12:28:35 +02:00
Patryk GenschandClaude Opus 4.8 f4aa7caaa9 Containerise: Postgres + Redis/RQ + API + Ghidra worker
Brings up the documented target architecture as a docker-compose stack — a
modular monolith with the Ghidra step split into its own async worker.

- worker/: RQ queue (lazy redis import) + run_acquisition task (Job status
  queued→started→finished/failed, drives ams.acquire with sink=db)
- Job model + JobOut schema; Snapshot.data is JSONB on Postgres
- POST/GET /jobs: stream an upload to a shared volume, enqueue, poll status
- docker/api.Dockerfile (slim) + docker/worker.Dockerfile (JDK21 + Ghidra
  fetched at build, overridable via GHIDRA_URL) + docker-compose.yml
- ghidra.py: AMS_GHIDRA_SCRIPTS override for in-container script path
- pyproject: [worker] extra (rq/redis/psycopg), python-multipart in [api]
- tests: 4 new (task success/failure + endpoint enqueue/503) -> 22/22

Verified: API image builds, container serves /health + /ui + /jobs; compose
config validates. Worker image (downloads ~1 GB Ghidra) not built here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 12:24:47 +02:00
Patryk GenschandClaude Opus 4.8 6797ad5ddb Add ISO/ZIP acquisition pipeline (ams.acquire worker)
Closes the chain from a game file to a catalog entry: unpack an ISO/ZIP,
content-identify the engine DLL (CMC_ObjectsContainer marker in RTTI, so a
renamed file is still found), hash it (sha256 + md5 + optional ssdeep via
ppdeep), run Ghidra headless with the extractor, enrich and import the snapshot.

- unpack.py: bsdtar (ISO9660 + ZIP) with a pure-Python zipfile fallback
- identify.py: content-based engine-DLL picker + hashing
- ghidra.py: analyzeHeadless launcher discovery + post-script run
- pipeline.py: orchestration with injectable extract_fn; sink db|http|none
- cli.py: python -m ams.acquire (incl. --identify-only dry run)
- tests: 7 new (forged PE markers + stubbed extractor) -> 18/18

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-31 12:11:56 +02:00
Patryk GenschandClaude Opus 4.8 4542763936 Add Command Center web UI (no build step)
Static HTML/CSS/JS served by FastAPI (mounted at /ui, / redirects there),
talking to the existing JSON API — no node/npm, no bundler.

- games/versions sidebar with A/B version selectors
- visual 4-axis diff (types/methods/events/fields, +/- struct_layout) with
  +/-/~ rows, per-axis counts, class (owner) filter, moved-methods section
- single-snapshot browser (tabs + live filter)
- app.py mounts StaticFiles(html=True) last so API routes win; / -> /ui/

Smoke-tested live on uvicorn: /, /ui/ and assets serve 200; UI wiring drives
the same /games and /diff endpoints verified end-to-end. app.js passes
`node --check`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 23:37:03 +02:00
Patryk GenschandClaude Opus 4.8 bd03c56e98 docs: document diff CLI and FastAPI backend in README
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 22:28:04 +02:00
Patryk GenschandClaude Opus 4.8 8386196653 Add FastAPI catalog backend (games/snapshots/diff) + tests
Modular-monolith backend over SQLAlchemy (SQLite by default, Postgres-ready
via DATABASE_URL). The full snapshot.json is stored verbatim; diffing reads it
back through the ams.diff engine, so the DB never mirrors the snapshot schema.

- ams.api.db/models/schemas/service : Game 1-N Snapshot, sha256-deduped upsert
- routes: POST/GET /games, POST/GET /snapshots (import, deduped), GET /diff
          (?old&new[&owner]) running compute_diff on stored snapshots, /health
- ams.api.importer : bulk CLI loader (python -m ams.api.importer --game ...)
- run: uvicorn ams.api.app:create_app --factory

11 tests pass (6 diff + 5 API via TestClient over the golden pair). Smoke-tested
live on uvicorn: import -> /snapshots -> /diff returns the BlooMoo deltas.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 22:27:24 +02:00
Patryk GenschandClaude Opus 4.8 6885bbee3d Add snapshot diff engine (ams package) + tests
Standalone CLI that diffs two engine-surface snapshots across all four axes,
the foundation the FastAPI/DB layer will sit on.

- ams.snapshot : typed access to a snapshot.json
- ams.diff     : keyed set-diff per axis (added/removed/changed) + cross-owner
                 method-move detection; types keyed by (script_name,
                 via_module_iface) so the dual MULTIARRAY stays stable;
                 filter_by_owner for per-class focus
- ams.render   : human-readable report (+/-/~), owner-grouped
- ams.cli      : python -m ams OLD NEW [--owner C] [--only ...] [--json]

6 tests pass, incl. an integration test over the committed golden pair
(asserts BlooMoo adds GRBUFFER/INTERNET, MOUSE grows 104->128, Animo gains
GETFPS, Animo script fields unchanged).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 22:18:04 +02:00
Patryk GenschandClaude Opus 4.8 91c7a11ba8 Engine-surface extractor for Piklib/BlooMoo + golden-pair snapshots
Ghidra headless post-script (pyghidra/Jython) that extracts the scripting
"surface" of Aidem Media engine DLLs into a versionable snapshot.json, for
diffing engine versions. All four axes validated on the golden pair
(PIKLIB8.dll / MSVC6 vs bloomoodll.dll / MSVC8):

- types   : CMC_ObjectsContainer::resolve factory ladder
            (script name -> C++ class, ctor, object size; + dispatch_addr,
            via_module_iface for the dual MULTIARRAY branch)
- methods : CMC_*_Runner::prepareMthHashSet (name -> id) + inheritance chain
- events  : CMC_*::getBehavioursList (ordered per-class list)
- fields  : CMC_* ctor -> CMElement::getProperty<T>Value (name + type)
            (+ bonus struct_layout: this+offset stores via decompiler P-code)

Extraction rests on semantic anchors (call targets, referenced string
literals, push/immediate operands), never decompiled-C text, so the same
script works across both compilers despite ILT stubs, undefined string
literals, unnamed FUN_ ctors and an MSVC6 inline-strcpy off-by-one.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 22:11:41 +02:00