mirror of
https://github.com/newliver666/apk-reverse.git
synced 2026-09-28 21:56:59 +02:00
inliver233
2beade67f2
An external audit of this repository landed on four structural findings, and this pass is the
response to them. Everything below was re-measured on this tree before being acted on; two of the
audit's claims needed a correction of my own, and both corrections are noted in place.
**The gates were optimistic, and one of them did not read content at all.**
doctor.py the capability table was hard-coded (`repack + sign = bool(java) and python`), so a
machine with a JDK and no zipalign/apksigner was told it could re-sign; the script
inventory was a hand-written list of 28 against 51 files on disk, omitting the nine
most important scripts. It now scans the directory (`registered/checked = 52 / 52`,
unjudgeable entries counted as `unknown`, never skipped) and resolves capability as a
real requires-closure, reporting BLOCKED with the missing piece and an installable
next step. Measured: the old doctor reports `[OK] repack + sign` on a JDK-only PATH
where keytool, jarsigner, zipalign, apksigner and all eight smali jars are absent.
The report is allowed to look bad; that is the point.
check_commands.py (new)
anchor checks prove a section exists, never that a command runs. This parses each
script's argparse table with `ast` and checks every documented invocation against it.
146 commands / 41 interfaces, 0 drift. It caught `sig_probe.py --live --pkg`, which
does not exist -- the reference had been wrong since it was written.
check_routing.py (new)
the symptom index now has one home (`SKILL.md`, because symptom -> file must stay one
hop) and the inventory has another (`references/routing.md`). This gate proves the
mirror agrees, that every reference file is named, and that every script is named.
**so_constpatch.py was corrupting APKs it rebuilt.** The writer copied filename and timestamp onto a
fresh ZipInfo and recompressed the whole archive: 435-entry real package, `extra` lost on 205 entries,
`external_attr` rewritten on all 435, 132 untouched entries recompressed, and `resources.arsc` pushed
to an unaligned offset. Android's own `zipalign -c -v 4` fails the old output and passes the new one;
a 26-case fixture suite pins it, and `extractNativeLibs="false"` is now refused by default with a
pointer at `repack.py`. Fixing it exposed the same defect in `repack.py`, where `^lib/[^/]+\.so$`
cannot match `lib/arm64-v8a/*.so` -- so its alignment handling had never once fired on a real APK.
**The package was not self-contained.** 59 references pointed at the repository root, which `npx
skills add` does not install. They now converge on `references/evidence-summary.md` (capability,
conclusion, strength, evidence that ships) plus `evidence/` with the machine-readable companions: 8
references remain and each is explicitly labelled "repository root, not shipped". A drill in which
the new files were copied to an isolated directory and asked three questions answered all three.
**SKILL.md: 572 lines / 15,733 tokens -> 398 lines / ~10,900 tokens**, without losing the two things
that had to stay. The symptom index remains inline; the reference and script indexes moved to the
on-demand routing file. The freed budget went into making the rules *stronger*, not shorter: the
constraints section is now a table of thirteen non-negotiable constraints each paired with the check
that catches a violation, and the tooling obligations are instructions whose violation is a defect
(hedging language in SKILL.md: 1 line, down from a paragraph of hedged advice). Long references gained
a navigation head so a reader can decide from the top whether to load one.
**New: an executable test layer.** `tests/` (unit, CLI contract, no-device integration) plus
`pytest.ini`, `tests/run_tests.py`, a minimal CI job, and `evals/evals.json` written but never run --
stated as such in the file. 608 passed / 158 skipped / 25 xfailed. The 25 xfails are a work order, not
a pass: thirteen scripts report a missing input by exiting 1 instead of 2, seven let a traceback
escape, four treat `--help` as an error, and `preflight --json` lacks a `status` field. A regression
row for `dex_patch_bytes.py` reproduces a previous artifact byte-for-byte
(`67ae1e58e7a44777e5ac8f9f823ab21e…`) and additionally pins the four bytes that may move.
**Two real defects the new tests found in the tooling itself.** `check_repo.py` decoded subprocess
output with the locale encoding, so on a GBK console a script printing an em dash in its usage line
was reported as crashed; it now decodes UTF-8 explicitly. And a CLI contract test drove a *real
attached device* through `coldstart.py` -- logcat, screenshots, five orphan `adb logcat` processes --
which is fixed twice over: child processes run in a private temporary cwd, and the fifteen
device-bound scripts are never executed beyond `--help`/argument parsing.
**Quoting on the device side is now one implementation.** `devsh.su` escaped only double quotes, so
`;`, `$( )` and backticks in a caller-supplied value still reached the device shell while
`coldstart.Device.shell` quoted correctly. Both use `scripts/device_shell.py` now: POSIX single-quote
escaping, argv arrays by default, and identifier validation that refuses a malformed package, path or
component with exit 2 instead of a traceback. The integration test that used to pin the *defect* (it
asserted `su -c "…"` with `\"` escapes) now asserts the property, across five metacharacter payloads.
Gates: check_repo 0 problems (including the content-reading identity scan over git ls-files),
check_refs 0 dangling / 0 warning, check_routing coherent (53 symptom / 45 reference / 54 script rows),
check_budget 0 failure / 0 warning with the always-loaded body now measured in tokens rather than in
"narrative lines with the index excluded", check_commands 146 checked / 0 drift, pytest 608 passed.
Not done, and stated rather than implied: CI has never run on a runner; the evals have never been
executed; the 25 xfails are open; no device-side install/launch was performed in this pass.
76 lines
4.2 KiB
Markdown
76 lines
4.2 KiB
Markdown
# tests/ - the executable half of this repository's evidence
|
|
|
|
`tests/benchmark.md` is a **record**: what happened when a route was run once, by hand,
|
|
against a real sample and a real device. This directory is the other half: checks that run
|
|
on every commit with nothing but the standard library and pytest.
|
|
|
|
The two are not substitutes. A benchmark row can establish that a route works on a hardened
|
|
target; it cannot tell you that yesterday's refactor broke the dex width table. A test in
|
|
here can tell you that every time, and it can never tell you whether the route achieves the
|
|
goal on a real app.
|
|
|
|
```
|
|
python -m pytest -q # whole suite
|
|
python tests/run_tests.py # same thing, with a clear message if pytest is missing
|
|
python -m pytest -q -m unit # one marker
|
|
python -m pytest -q -m cli # exit codes, flags, --json
|
|
python -m pytest -q -m integration # no-device flows
|
|
python -m pytest -q -ra # list every skip and xfail reason
|
|
```
|
|
|
|
## What the markers mean
|
|
|
|
- `unit` - pure functions and parsers of real shipped scripts: the dex width table and branch
|
|
arithmetic, the scheme-less protobuf decoder, the blob rotation search, the leak rules, and
|
|
the established `dex_patch_bytes` regression hash.
|
|
- `cli` - one subprocess run per script per invocation shape: `--help`, an unknown flag, no
|
|
arguments, and every input pointed at a file that does not exist.
|
|
- `integration` - flows that were made of several parts: adb argv construction under a fake
|
|
runner, logcat signal counting, tombstone parsing, the two-sample launch-health verdict.
|
|
|
|
## Nothing here needs a device
|
|
|
|
No test in this suite opens a frida session or touches the network. The device-dependent
|
|
behaviour is tested by intercepting the command that *would* have been run and asserting on the
|
|
argv list.
|
|
|
|
The exception is deliberate and enforced: `contract.DEVICE_OR_NETWORK_BOUND` names the 15
|
|
scripts that reach a device once their arguments parse, and those are **never executed** with
|
|
complete arguments — only `--help` and an unknown flag, which argparse answers before any
|
|
device call. That blocklist exists because a development host may well have a phone attached
|
|
(the machine this harness was written on did), and an earlier version of the CLI contract test
|
|
drove it for real. General safety net: every child runs in a private temporary working
|
|
directory, so a script that defaults an output path relative to cwd cannot write into the
|
|
shipped tree.
|
|
|
|
Anything that genuinely cannot be reached this way is not faked into a pass — it is listed as a
|
|
coverage gap in `docs/tool-verification/EXTENSION-test-harness.md`.
|
|
|
|
## Skipped is not passed
|
|
|
|
Two kinds of non-passing outcome are kept separate on purpose:
|
|
|
|
- **`skip`** - the check does not apply here. `tools/_work/bench/**` is git-ignored, so the
|
|
`dex_patch_bytes` regression hash self-skips on a fresh checkout and in CI, printing the
|
|
reason. A --json script that needs a sample to reach its JSON path skips too.
|
|
- **`xfail`** - the script is present and does **not** meet the repository contract. The
|
|
reason carries the measured value, for example
|
|
`contract says exit 2, measured 1`. These are work items, not flaky tests.
|
|
|
|
`pytest -q -ra` prints both lists. The xfail list for the CLI layer is the exit-code
|
|
unification work order; it is reproduced in `EXTENSION-test-harness.md` with owners.
|
|
|
|
## Ground rules
|
|
|
|
- **This suite never edits anything under `skills/`.** A defect found here is registered in
|
|
the harness report. Fixing a script's exit code or quoting strategy is its owner's change,
|
|
and doing it from a test would hide the drift the test exists to expose.
|
|
- **The bench tree is evidence and is read-only here.** The regression test copies its input
|
|
into `tmp_path` before patching, so `tools/_work/bench/l1l3/dex/classes.patched.dex` cannot
|
|
be overwritten by a test run.
|
|
- **No real target identity is committed.** The leak-scanner decoys are synthesised in
|
|
`tmp_path` at run time, and token-shaped decoys are assembled from fragments so that this
|
|
directory cannot itself become a finding.
|
|
- Fixtures in `tests/fixtures/` are small, synthetic sample files. `tests/fixtures/apk/**` is
|
|
owned by a different work item and is not touched from here.
|