mirror of
https://github.com/newliver666/apk-reverse.git
synced 2026-09-28 21:56:59 +02:00
inliver233
6cffc3d2ac
Four projects were cloned for comparison (reverse-skill, awesome-game-security,
android-reverse-engineering-claude-skill, MobileRE-Skill) and read for what they do that this
repository does not. None is vendored and none becomes a runtime dependency: what crosses over is a
phase order, a checklist or a command pattern, with the source URL and the retrieval date recorded in
the file that absorbed it.
The rule applied to every candidate: does this make a judgement easier or a verification harder? The
repository is a decision-and-verification layer, and the four things kept are all of that kind.
new scripts/scan_leaks.py target identity scanned before publishing, with an explicit
do-not-anonymize list and four RESULT=/exit states
new references/precedents/ the positive half of the record: route including dead ends, a
grade per assertion, measured pits, and a write-back checklist
new scripts/svc_scan.py does a libc-level hook structurally see this call at all
new scripts/anti_detect_probe.js observer-only: name the check, and report the environment it reads
new references/desensitization-and-leak-scans.md
new references/dart-aot.md sections the 32-bit snapshot's real string-record format
check_repo.py now delegates the identity scan to scripts/scan_leaks.py and runs it over
git ls-files, so the gate reads content instead of layout for the first time. An explicit file list
cannot be narrowed by the default work-area exemptions -- that defect was caught by the gate itself.
Two claims that this repository had already written down turned out to be wrong, and both are
recorded as refuted rather than quietly rewritten:
dart-aot.md the armv7 string table was documented as [len*2 u32le][UTF-16LE]. It is not. The same
extractor keeps 4,980 chained entries on arm64 and zero on armv7, and the byte layer
shows the 32-bit record is [header u32][byte-count u32le][UTF-8 payload]: two adjacent
literals carry 0x46 (70) and 0x48 (72), exactly their byte lengths. The zero was a
format mismatch, not an empty table. The engine-version banner on the same target is
66 bytes of "0.0.1" with a blank build id, so the documented regex returns a
version-shaped lie.
leak scan the pass that added the scanner found 26 strong hits inside its own evidence file on
the day it was written -- the fixture's literal match values, quoted to prove the
rules fire. The corpus may contain values; the published surface may not.
Measured, not asserted. The check that fires was named on a public MASTG target: three arms with one
variable, the control alive at the end and the probe arm dead in about 300 ms, with strstr("frida") at
t=303 ms and no tombstone. Reproducibility is bounded honestly -- one full sequence in three arms, on
a target whose whole window is sub-second. svc_scan.py's two independent decoders agree on an
identical 214-site set, and ptrace-free dumping gained a negative boundary: a target whose dex is
deflated inside its APK has no whole-image dex in memory to export, for reasons that say nothing about
protection.
Also in this pass: the detector-locating pipeline (detection-and-anti-analysis.md Step 3) with its
branch conditions; the layered Java-to-svc descent and the dump's honesty boundary in
advanced-unpacking.md; a sourced capability-gap list in toolchain.md where every row is graded
URL-verified or measured; the RESULT=-token convention; and SKILL.md opening with an immediate-action
block, because the symptom index and the gates were sitting in the part of a long file nobody reads.
Gates: check_repo.py 0 problem, check_refs.py 0 dangling / 0 warning, check_budget.py 0 failure /
0 warning / 1 pre-existing note. The scan reports RESULT=leaks_found_strong_only, exit 0: four
endpoint/weak hits, all of them addresses from the RFC 5737 documentation range in usage examples.
58 lines
15 KiB
Markdown
58 lines
15 KiB
Markdown
# Benchmark set — the regression baseline for this skill
|
||
|
||
This file is a **test matrix**, not a sample collection. It answers one question per row: *when an
|
||
agent meets this shape of target, does the route this repository documents actually work, and what
|
||
is the strength of the evidence?* Every claim in the `Result` column must be reproducible from the
|
||
`Target` column plus the named script — nothing here is a summary of what "should" happen.
|
||
|
||
**Samples are never committed.** Each row names a public source; download it into `tools/_work/bench/`
|
||
(ignored by git) before running the row. Record the sample hash when you run it, because two rows on
|
||
different sample revisions are not comparable. Hashes used in this pass:
|
||
|
||
```
|
||
UnCrackable-Level1.apk 66651 B 1da8bf57d266109f…
|
||
UnCrackable-Level2.apk 901022 B 4c7980ef1f6cc295…
|
||
UnCrackable-Level3.apk 1460555 B 6827f776b7d28443…
|
||
r2pay-v1.0.apk 4293620 B 850d7c98bed8df53…
|
||
```
|
||
|
||
Strength labels follow the repository's own three-tier rule (`references/long-task-discipline.md`
|
||
reserves `observed` for a claim with an exact command and its output behind it):
|
||
|
||
- `observed` — a command was run and its output is quoted in the evidence file.
|
||
- `inferred` — reasoned from documented mechanism or from a neighbouring measurement.
|
||
- `unverified` — assumed, or reported by someone else and not reproduced here.
|
||
|
||
| # | Dimension | Target (public source) | Scripts / documents the row exercises | Result | Strength | Evidence |
|
||
|---|---|---|---|---|---|---|
|
||
| B1 | Equal-length dex patch + repack + re-sign installability (G2/G4) | MASTG `UnCrackable-Level1.apk` | `dex_patch_bytes.py`, `repack.py`, `byte-level-patching.md`, `repack-and-sign.md` | **The route works end to end, and the change is visible on screen.** 2 bytes of the first root-check branch in `MainActivity.onCreate` (`0x9d0: 39000e00` -> `00000000`) patched inside a 5528-byte dex; the whole-file diff is those 4 bytes plus bytes 8..32, and the written file self-verifies (`adler32(d[12:])` and `sha1(d[32:])` both reproduce, proving signature-before-checksum order). `repack.py` produced a STORED + 4-byte-aligned `resources.arsc`, apksigner reported v1+v2+v3 all true, the root install path succeeded with the on-device `base.apk` sha256 equal to the local build, and the blocking `Root detected!` dialog is gone from the screenshots. A zero-change control through the identical pipeline still shows it | observed | `EXTENSION-benchmark-l1-l3.md` §2 |
|
||
| B2 | Anti-debug + self-destruct race (patch before the check fires) | MASTG `UnCrackable-Level3.apk` | `spawn_patch_detach.py`, `hook_patch_only.js`, `dynamic-frida.md` | **Split: the tooling works, the patch does not.** Time-to-death with a probe attached is 5.34 s. After an ordering fix (resume -> wait for `PATCHED` -> detach) the probe wrote `libfoo.so` base+0x2ff0/+0x2ff8/+0x3080 to `mov w0,#0 ; ret` 0.04-0.05 s after resume and the writes survived the detach — the mechanism is real. The patch still does not stop the death: the patched run **and** a read-and-write-back control both die with `SIGABRT` at `libfoo.so (goodbye()+12)` 3.22 s after resume, so neutralising the terminate routine plus both of its branch inputs is insufficient; the region is a dense cluster of `mov w0,#imm ; ret` stubs and locating the real second site needs a full `.text` disassembly. The alternative ordering ("hold paused until `PATCHED`") cannot work — a frozen process has no libraries mapped, so `Process.findModuleByName` never returns (0.11 s after resume it appears). Independently, the pristine original already reports tamper on this device, so the row's assumed clean baseline does not exist here | observed (tooling) / unverified (a surviving target) | `EXTENSION-benchmark-l1-l3.md` §3 |
|
||
| B3 | Extraction-shell skeleton detection (`trivial-body` routing) | self-built 11-variant skeleton set from the `r2pay-v1.0.apk` dex, with a zero-change control | `dex_dump_validate.py`, `advanced-unpacking.md` | **The "stub% threshold" claim this skill used to make is wrong.** `stub%` is *bimodal*, not thresholded: a real skeleton reads ≈100% (`allstub` 100.0%) while a control reads 1.9%, and the shapes real shells actually use are invisible — nop-cleared bodies 0.0% (below the control), throw stubs 1.9% (indistinguishable), partial extraction at 25/50/75% 1.7/1.7/2.1%. `advanced-unpacking.md`'s original "tens of percent means a skeleton" had no support and was deleted. Three script defects were found and fixed (all-nop bodies ranked as real; a legal minimal body mislabelled malformed; the ranking pointing at a 25%-extracted image as "most likely original" — caught by independent review). The new `minimal-form` counter *does* track the extraction ratio monotonically (1299/2524/3736/4946) and now participates in the ranking | observed | `EXTENSION-extraction-shell-bench.md` §1–3 |
|
||
| B4 | Dex-VMP boundary: refuse-to-repack and downgrade the deliverable | DexPatcher `assets/ezAndroid.apk` | `advanced-unpacking.md`, `java2c-and-jni-sinking.md`, `lsposed_scaffold.py`, `frida_rpc_serve.py` | **The sample is not a Dex VMP — it is JNI sinking**: 7 `native` declarations, each with a matching exported `Java_*` symbol, no `JNI_OnLoad` / `RegisterNatives` symbol, 34,566 instructions with zero structural errors under the official `dexdump`. A zero-change repack succeeded (v2 signing, alignment, `classes.dex` CRC verified). Static criteria **can** rule a dex-VMP *out*; they cannot rule runtime-applied hardening out. The first verdict here was the opposite — a hand-written opcode table reported 1393/22424 undecodable bodies and called it a VMP; `dexdump` showed the table was missing an arm64 branch opcode, and that rule (never declare VMP without an official disassembler's failure count) is now in the reference. Installation / launch / refusal remain untested | observed (static + repack) / unverified (install, launch) | `EXTENSION-extraction-shell-bench.md` §4 |
|
||
| B5 | Java2C recognition (do not misdiagnose as an extraction shell) | self-built with `amimo/dcc` (`--no-build`) + real JNI samples | `java2c_probe.py`, `java2c-and-jni-sinking.md` | Native-declaration density separates the shapes by ~2000x: **0.03–0.04%** on three real JNI samples vs **82.76%** on a Java2C-shaped fixture, and `Java_*` symbols matched dex native-method counts 1:1 per ABI (2:2, 3:3). **No Dex-to-C compiler output was ever built** (no NDK, no host clang, no device clang, WSL unavailable), so every Java2C-specific criterion is inferred while the JNI-sinking side is measured. A symbol search fails silently on real Java2C: dcc emits `-fvisibility=hidden`, and C++ `jni.h` inlines `RegisterNatives` so it leaves no symbol | observed (density, JNI sinking) / inferred (Java2C-specific) | `EXTENSION-java2c.md` |
|
||
| B6 | Native tracing: does `Stalker.exclude()` change the outcome? | MASTG `UnCrackable-Level1.apk`, module `libc.so` followed on the main thread for 6 s | `stalker_trace.js`, `native-dbi-and-deobfuscation.md` | **Split result.** A no-follow baseline survives; the same follow with `excludeModules: []` **kills the process**; with 20 system modules excluded it survives — so exclusion is what keeps the target alive (`observed`, with the baseline arm ruling out "frida killed it anyway"). But the treatment arm still reported `blocks=0 blk=0 calls=0`: **exclusion does not fix the zero-event trace**, which partially contradicts the community framing that lists "no events" among its symptoms. The pass also fixed two harness traps that manufacture a false zero-event result (a load-time default follow, and a suspended spawn) and verified a new zero-event `WARN`. No real OLLVM trace was produced | observed (survival, delivery) / unverified (the multiplier, the cause of zero delivery) | `EXTENSION-stalker-exclude.md` |
|
||
| B7 | Native certificate pinning | CyReverse Cronet sample | `protocol-reverse.md` | not run this pass — the CyReverse Cronet sample needs a gradle+NDK build this machine cannot do | unverified | — |
|
||
| B8 | Schemaless protobuf decoding | synthetic fixtures + the official runtime + a real AndroidX DataStore container | `protobuf_decode_raw.py`, `protocol-reverse.md` | 26/26 built-in fixtures, **21/21 against the official protobuf runtime** (re-encode byte-identical to `SerializeToString()`), 8/8 framing checks. A real 81-byte DataStore file round-tripped byte-exact and an edited value re-encoded (81->80 B, length prefix recomputed) was read back by `datastore_inject.py`. Both traps reproduced on real bytes: `0801120178` is a valid nested message *and* a valid packed array; a non-presence `0` is byte-identical to "never assigned". Four script defects were found by these checks and fixed | observed | `EXTENSION-protobuf-raw.md` |
|
||
| B9 | Split APK / App Bundle handling | two real split sets pulled from a device (8 members = base + 7 density splits each carrying its own `resources.arsc`; 9 members = base + ABI split with only `lib/` + 7 density splits with 40-byte placeholder tables) | `repack.py` (split mode), `split-apk.md` | Both branches of the decision ran on real sets: after `resign` every member shares one certificate with v1+v2+v3 all true; `merge` **correctly refused** the set whose members carry their own `resources.arsc` (exit 1, before writing anything) and **succeeded** on the code/native-only set, folding 2 `.so` files into `base`. The single-APK legacy path still passes its regression (`--dexdir` replacement and a zero-change round trip). Signing has no `uber-apk-signer` on this host; the measured call is `zipalign -p -f 4` then `apksigner.jar --v1 --v2 --v3`, and the order cannot be reversed (`[-124]`). `adb install-multiple` and launch confirmation were **not** executed | observed (analyze/resign/merge) / unverified (install, launch) | `EXTENSION-split-apk.md` |
|
||
| B10 | Kernel-side syscall answer forging | KernelSU/APatch module template, host toolchain, device kernel | `kernel-and-environment-hardening.md`, `kernelsu_syscall_mask.py` | The generator produced an 11-file userspace module skeleton plus KPM/LKM/eBPF templates and `verify` reports 0 problems — but **no kernel-side artefact was compiled or loaded**: the `<DEVICE>` kernel is 4.14.186 (eBPF needs 5.10+), the host has no `aarch64-none-elf-gcc` / `make` / `ndk-build`, and there is no kernel source. Documentation correction with teeth: **an ordinary KernelSU module is a userspace module and structurally cannot change a syscall return value** | observed (generation) / unverified (kernel side) | `EXTENSION-kernel-weapons.md` |
|
||
| B11 | JNI dynamic registration discovery (`RegisterNatives` with no `Java_*` symbol) | a real system library + CyReverse `plugin/` | `java2c-and-jni-sinking.md`, `java2c_probe.py` | A real dynamically-registering library contains **zero `RegisterNatives` symbols** (the C++ `jni.h` inlines the call), so symbol-based discovery fails structurally. The combination that did fire on a real library is "**exports `JNI_OnLoad` and zero `Java_*`**" | observed | `EXTENSION-java2c.md` |
|
||
| B12 | Dart AOT string-table format across ABIs (the premise any string patch rests on) | a locally supplied hardened Flutter APK (360 shell + AOT payload, both ABIs; uncommitted, `tools/_work/dart/`) | `dart-aot.md` §7/§10, `scripts/dart_pool_strings.py` | **Two claims tested, both were wrong before this row.** (a) arm64 packed `[0x80\|(len<<1)][payload]` is confirmed — `dart_pool_strings.py` keeps 4,980 chained entries and the tag byte equals `0x80\|(len<<1)` at every literal checked (`AdmobBanner` 11 chars → `0x96`, `showSplashAd` and `AdPgl_Splash` 12 → `0x98`). (b) The armv7 form "`[len*2 u32le][UTF-16LE]`" is **refuted**: the same extractor keeps **0** entries on the 32-bit snapshot, and the byte layer shows why — each 4-byte-aligned record reads `[header u32][byte-count u32le][UTF-8 payload]`, with the two adjacent literals `SystemChrome.setEnabledSystemUIMode` (36) + `SystemChrome.setSystemUIOverlayStyle` (34) carrying lengths `0x46` (70) and `0x48` (72) that exactly equal their byte lengths. The extractor's zero is a format mismatch, not an empty string table. A third claim, "the engine banner gives `x.y.z (stable)`", is also refuted on this target: the banner is 66 bytes of `0.0.1` with a blank build id. Full detail and the raw probes: `EXTENSION-dart-aot-formats.md` | observed (both formats, banner) / unverified (an end-to-end device patch — no repack+install was run) | `EXTENSION-dart-aot-formats.md` |
|
||
| B13 | Naming the check that fires, and the environment it reads (anti-instrumentation triage) | MASTG `owasp.mstg.uncrackable3` (public target) for the three arms; device `libc.so` and a hardened shell library for `svc_scan.py` | `anti_detect_probe.js`, `detection-and-anti-analysis.md` Step 3, `svc_scan.py` | **The check was named, and the asymmetry is the finding.** Three arms on one target with a single variable: control (`spawn` + `resume`, no script) ends `alive_at_end=True`; the arm that also loads the observer probe ends `alive_at_end=False`, with the stream showing `strstr("frida")` at t=303 ms and, in an earlier same-shape arm, the kill itself at t=352 ms — process gone with **no tombstone, no crash and no ANR record**. The probe's own self-report on the same arm: `TracerPid=0` while the target's `/proc/self/maps` carries **4 frida-named mappings**, so a `TracerPid`-only detector passes and a maps/thread-name detector does not. Reproducibility is honestly bounded: 3 same-shape arms, 1 delivered a full sequence (a ≈300 ms self-destruct leaves a sub-second window). Separately, `svc_scan.py`'s two independent decoders — a hand-written word scan and capstone — produced an **identical 214-site set** (set difference empty), and a structural criterion fell out: the device `libc.so`'s 4 termination sites are libc's own exports (so a libc-level hook is structurally able to see them), while the hardened shell library's 21 `svc` sites are **all data** (neighbours are `udf`/NEON filler) with no termination site | observed (the arms, the named check, decode agreement) / unverified (dual-dumper byte agreement — no second producer existed; a dump from the hardened sample) | `EXTENSION-detection-pipeline.md` §1, §4, §5 |
|
||
|
||
## How to read a row
|
||
|
||
`Result` states what actually happened, including negative outcomes — "the route worked", "the route
|
||
worked only with `Stalker.exclude()`", "the tool refused to run at all". A negative result that is
|
||
recorded is worth more than a positive one that is assumed, because the next agent can route around
|
||
it without re-paying the cost.
|
||
|
||
A row whose `Strength` is `unverified` is a statement about *this repository's evidence*, not about
|
||
the mechanism's truth. It means: nobody has yet put a command and its output behind it here.
|
||
|
||
**Rows that changed a conclusion this repository had already recorded.** This is the outcome the
|
||
matrix exists to produce, so it is listed rather than buried: B3 (the trivial-body "threshold" became
|
||
a measured bimodal result), B4 (a VMP verdict was withdrawn after an official disassembler disagreed
|
||
with a hand-written opcode table), B6 (exclusion turns out to keep the process alive but does not fix
|
||
zero-event delivery, contradicting the community framing), and B1 (an earlier "`repack.py` has no
|
||
apksigner signing path" conclusion expired mid-pass when a second writer added one).
|