mirror of
https://github.com/newliver666/apk-reverse.git
synced 2026-09-28 21:56:59 +02:00
Harden the path audit, generalise three absolute paths, and gate the corpus size
An external review found the evidence record honest and the gates real, but raised three
concrete defects. Each is fixed at its cause rather than at its symptom, and the reasoning
is recorded in docs/tool-verification/REPO-DECISIONS.md.
build_scripts.py -- the audit reported 37 findings, most of them noise
It matched any `X:\...` literal, so `C:\Program Files\...`, `C:\Python314\...` and literal
placeholders all counted as machine-specific leftovers. A gate whose output is mostly noise
stops being read -- the same failure scan_leaks.py names when it refuses to exempt the
RFC 5737 ranges. Two explicit exemption families are added instead of a looser regex:
GENERIC_PATH_PREFIXES (install locations that exist on every host of that platform) and
`_leaks_nothing()` (also exempts an already-redacted path, since a placeholder or an
ellipsis means a normalisation pass already ran).
Two traps surfaced only because the change was tested rather than merely run. BARE_ABS stops
at whitespace, so `C:\Program Files\...` arrives truncated as `C:\Program` and a one-way
`startswith` misses the exemption; the comparison is now bidirectional under a minimum
length. And with a lookahead of only `[a-z0-9]` the optional `\\?` backtracks to empty, the
next character is the separator, it passes, and a real personal path is swallowed -- the
lookahead has to exclude the separator as well. Both were caught by the case table, not by
the audit run.
Result: 37 -> 0 findings, exit 2 -> exit 0. The real hits (`E:\apk-reverse\...`,
`E:\agb_apktool.jar`) were fixed in the documents, not by widening the list.
Three absolute paths on the published surface
EXTENSION-desensitization.md, EXTENSION-extraction-shell-bench.md and EXTENSION-vmp-diff.md
quoted a workbench root, a tool jar and an SDK location absolutely.
docs/tool-verification/ already states the rule -- a record may say a run happened at a
`tools/...` path, written relatively -- so the three files were made to match the rule they
were already under. The tool-jar and SDK cases are rephrased rather than placeholder-ised,
so no reader has to learn a new token.
check_budget.py -- blind to the half of the problem it was not measuring
The gate budgets the always-loaded part and could not see the corpus, where a new reference
costs exactly one index line: the corpus can double while the narrative count stays green.
Two note-level thresholds now report it. Notes, not failures -- a large corpus is a choice,
a growth nobody measured is the defect. The first run reports 47 references + 55 scripts =
102 files, 31,423 lines.
A cross-tool inconsistency this pass ran into, recorded rather than papered over
build_scripts.py now exempts any `<...>` placeholder; scan_leaks.py recognises a fixed token
list. An invented placeholder in a source comment passed the first and was read by the
second as the very personal name its `user_path_windows` rule exists to catch, turning
check_repo.py red. That is by design -- a strict list is what stops a real name hiding
behind a plausible-looking placeholder -- but the two tools disagree about what "already
redacted" means, and the disagreement stays invisible until someone writes a new token.
Rephrased here; a future pass may want one shared list.
Verification
- build_scripts.py: 37 -> 0 findings on a clean clone, exit 0.
- An 18-case table pins both directions, 0 failures: generic and already-redacted paths are
exempt, while a real workbench path, a real personal name after `C:\Users\`, a real POSIX
home and a real project path still report. It found two defects in the first
implementation.
- All four gates pass: check_repo 0 problem, check_refs 0 dangling / 0 warning,
check_budget 0 failure / 0 warning / 3 notes, build_scripts 0 findings.
This commit is contained in:
@@ -8,7 +8,7 @@ Strength labels follow `docs/tool-verification/README.md`: **observed** (the exa
|
||||
its output are reproduced here), **inferred** (follows from observed facts; the step itself was
|
||||
not executed), **unverified** (assumed or reported elsewhere and not independently confirmed).
|
||||
|
||||
Reference environment: Windows 11 host, Python 3.14.0, repository at `E:\apk-reverse` on branch
|
||||
Reference environment: Windows 11 host, Python 3.14.0, repository root on branch
|
||||
`main`. **No device, no network and no target sample were involved** — this pass is entirely
|
||||
machine-local, because the thing worth measuring is what a scanner reports against a tree whose
|
||||
contents are already known. Every identifier in the fixture and in every transcript below is
|
||||
@@ -44,13 +44,13 @@ a transcript pasted once:
|
||||
|
||||
```
|
||||
$ python tools/_work/leak-precedent/gen_fixtures.py
|
||||
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\package.txt (411 bytes)
|
||||
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\device.txt (218 bytes)
|
||||
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\token.txt (261 bytes)
|
||||
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\appkey.txt (212 bytes)
|
||||
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\endpoint.txt (146 bytes)
|
||||
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\path.txt (175 bytes)
|
||||
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\exempt\benign.txt (1396 bytes)
|
||||
wrote tools/_work/leak-precedent\fixture\planted\package.txt (411 bytes)
|
||||
wrote tools/_work/leak-precedent\fixture\planted\device.txt (218 bytes)
|
||||
wrote tools/_work/leak-precedent\fixture\planted\token.txt (261 bytes)
|
||||
wrote tools/_work/leak-precedent\fixture\planted\appkey.txt (212 bytes)
|
||||
wrote tools/_work/leak-precedent\fixture\planted\endpoint.txt (146 bytes)
|
||||
wrote tools/_work/leak-precedent\fixture\planted\path.txt (175 bytes)
|
||||
wrote tools/_work/leak-precedent\fixture\exempt\benign.txt (1396 bytes)
|
||||
== 7 fixture file(s) ==
|
||||
```
|
||||
|
||||
@@ -269,7 +269,7 @@ found by running the scanner, not by reasoning about it.
|
||||
|
||||
```
|
||||
$ python skills/apk-reverse/scripts/scan_leaks.py # first run
|
||||
== leak scan: 118 file(s) scanned under E:\apk-reverse ==
|
||||
== leak scan: 118 file(s) scanned under the repository root ==
|
||||
findings by category: device=4, endpoint=20, package=4
|
||||
== result: 28 finding(s) == RESULT=leaks_found (exit 1)
|
||||
```
|
||||
@@ -297,7 +297,7 @@ Inherited state, reproduced at the start of this pass (same tree, before the §3
|
||||
|
||||
```
|
||||
$ python skills/apk-reverse/scripts/scan_leaks.py
|
||||
== leak scan: 128 file(s) scanned under E:\apk-reverse ==
|
||||
== leak scan: 128 file(s) scanned under the repository root ==
|
||||
findings by category: appkey=2, device=12, endpoint=14, package=5, path=3, token=4
|
||||
== result: 40 finding(s) -- each one is a line to look at, not a verdict ==
|
||||
RESULT=leaks_found (exit 1)
|
||||
@@ -317,7 +317,7 @@ $ python skills/apk-reverse/scripts/scan_leaks.py --quiet
|
||||
RESULT=leaks_found_strong_only (exit 0)
|
||||
|
||||
$ python skills/apk-reverse/scripts/scan_leaks.py
|
||||
== leak scan: 129 file(s) scanned under E:\apk-reverse ==
|
||||
== leak scan: 129 file(s) scanned under the repository root ==
|
||||
findings by category: endpoint=4
|
||||
|
||||
skills/apk-reverse/references/desensitization-and-leak-scans.md:81:10 [endpoint/weak] plain_addr_ipliteral
|
||||
@@ -383,7 +383,7 @@ $ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/leak-preced
|
||||
RESULT=leaks_found (exit 1)
|
||||
|
||||
$ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/nope-not-here
|
||||
error: root does not exist: E:\apk-reverse\tools\_work\nope-not-here [stderr]
|
||||
error: root does not exist: tools/_work/nope-not-here [stderr]
|
||||
RESULT=error (exit 2)
|
||||
|
||||
$ python skills/apk-reverse/scripts/scan_leaks.py --only nope
|
||||
|
||||
@@ -406,7 +406,7 @@ $ python skills/apk-reverse/scripts/repack.py \
|
||||
--apksigner E:\tools\android-14\apksigner.bat \
|
||||
--zipalign E:\tools\android-14\zipalign.exe \
|
||||
--keytool "C:\Program Files\Java\jdk-17.0.4.1\bin\keytool.exe"
|
||||
[in ] E:\apk-reverse\tools\_work\bench\unpack\b4\ezAndroid.apk (2984841 bytes)
|
||||
[in ] tools/_work/bench\unpack\b4\ezAndroid.apk (2984841 bytes)
|
||||
[zip] - META-INF/1N0R.SF / 1N0R.RSA / MANIFEST.MF (signature artifact)
|
||||
[zip] unsigned written: 2828919 bytes
|
||||
[zip] AndroidManifest.xml STORED offset=0x31
|
||||
@@ -506,9 +506,10 @@ attributable to memory rather than to disk.
|
||||
- **No end-to-end FART splice.** The reference file's FART loop needs a live
|
||||
extraction-shell target; nothing in `repos/` is one (§1), and building an actively
|
||||
resisting shell is out of scope for this pass.
|
||||
- **A real APK build on this host.** `E:\tools\android-14` ships `d8`, `aapt2`,
|
||||
- **A real APK build on this host.** The build-tools directory ships `d8`, `aapt2`,
|
||||
`apksigner` and `zipalign` but **no `android.jar`**, and no `android.jar` exists
|
||||
anywhere on the machine (searched `E:\tools`, `C:\Program Files`, `E:\Android`). Any
|
||||
anywhere on the machine (searched the tool directories, `C:\Program Files`, and
|
||||
the SDK path). Any
|
||||
`javac` against `android.*` fails with "package android.os does not exist". APKs can
|
||||
still be *repacked* here (`repack.py` does not recompile), which is why §4.4 works
|
||||
and the self-built-APK route in the task brief does not.
|
||||
|
||||
@@ -53,8 +53,8 @@ D8 8.2.2-dev (build facedf41bbd28b563d1e9e09c5f72d7c5ca598d8 from go/r8bot (luci
|
||||
|
||||
Two further tool facts worth recording, both **observed**:
|
||||
|
||||
- **`E:\agb_apktool.jar` does not exist**, so `E:\tools\bin\apktool.bat` (which is
|
||||
`java -jar "E:\agb_apktool.jar" %*`) resolves to nothing. apktool is unavailable on this
|
||||
- **The apktool jar the wrapper points at does not exist**, so `<tools>\bin\apktool.bat` (which is
|
||||
`java -jar "<tools>\agb_apktool.jar" %*`) resolves to nothing. apktool is unavailable on this
|
||||
machine, and a bare `apktool d` fails with a jar-not-found error rather than a
|
||||
recognisable "tool missing" message.
|
||||
- `javac` is on PATH only through the Oracle `javapath` forwarder; the real JDK is at
|
||||
|
||||
@@ -233,3 +233,73 @@ evidentiary gap F5 describes, and the repository's rule that nothing ships unrun
|
||||
much as to code. It is dependency-free (stdlib only) so it reproduces on a machine with no Android
|
||||
toolchain, which is the machine the defect was found on. It carries no target data: the dex is
|
||||
synthesised from constants in the file.
|
||||
|
||||
## Maintenance pass: the path audit's signal, three path families, and a corpus note
|
||||
|
||||
An external review found the repository in good shape on evidence and honesty but raised three
|
||||
concrete defects. All three are fixed here; the fourth item is a measurement added so the next
|
||||
pass can see what this one could not.
|
||||
|
||||
### 1. `build_scripts.py` reported 37 findings, most of them noise
|
||||
|
||||
The audit matched **any** `X:\...` literal, so `C:\Program Files\...`, `C:\Python314\...` and
|
||||
literal placeholders like `<user>` all counted as "machine-specific leftovers". A gate whose output
|
||||
is mostly noise stops being read, which is the same failure `scan_leaks.py` names when it refuses to
|
||||
exempt the RFC 5737 documentation ranges.
|
||||
|
||||
Added two explicit, auditable exemption families rather than a looser regex:
|
||||
|
||||
- `GENERIC_PATH_PREFIXES` — install locations that exist on every host of that platform
|
||||
(`Program Files`, `Python*`, `MinGW`, `/usr/`, `/opt/`, ...).
|
||||
- `_leaks_nothing()` — also exempts an already-redacted path, i.e. one carrying a placeholder or an
|
||||
ellipsis, because that mark means a normalisation pass already ran.
|
||||
|
||||
**One trap had to be handled explicitly.** `BARE_ABS` stops at whitespace, so a real
|
||||
`C:\Program Files\...` arrives as the truncated `C:\Program`; a one-directional `startswith` misses
|
||||
the exemption. The comparison is therefore bidirectional, gated by a minimum length so a short drive
|
||||
prefix cannot become a blanket pass. The same function needed a second correction: with a lookahead
|
||||
of only `[a-z0-9]`, the optional `\\?` backtracks to the empty string, the next character is the
|
||||
separator, it passes, and a real personal path is swallowed — the lookahead must exclude the
|
||||
separator too.
|
||||
|
||||
### 2. Three absolute paths left on the published surface
|
||||
|
||||
`EXTENSION-desensitization.md`, `EXTENSION-extraction-shell-bench.md` and `EXTENSION-vmp-diff.md`
|
||||
quoted a workbench root, a tool jar and an SDK location as absolute paths. `docs/tool-verification/`
|
||||
already states the rule (§**Workbench paths are an operation record, not a pointer**): the record
|
||||
may say a run happened at a `tools/...` path, written relatively. The three files were made to match
|
||||
the rule they were already under; the tool-jar and SDK cases were rephrased rather than
|
||||
placeholder-ised, so no reader has to learn a new token.
|
||||
|
||||
### 3. `check_budget.py` could not see corpus growth
|
||||
|
||||
The gate budgets the always-loaded part and was silently blind to the other half of the problem: a
|
||||
new reference costs exactly **one index line**, so the corpus can double while the narrative count
|
||||
stays green. Two note-level thresholds now report it. Notes, not failures — a large corpus is a
|
||||
choice; a growth nobody measured is the defect.
|
||||
|
||||
### A cross-tool inconsistency this pass ran into
|
||||
|
||||
`build_scripts.py` now exempts any `<...>` placeholder; `scan_leaks.py` recognises a **fixed token
|
||||
list** (`<user>`, `<name>`, `<path>`, ...). An invented placeholder in a source comment —
|
||||
`<someone>` — therefore passed the first and was read by the second as the very personal name its
|
||||
`user_path_windows` rule exists to catch, turning `check_repo.py` red. That is by design rather than
|
||||
a bug (a strict list is what stops `<realname>` hiding a real one), but the two tools disagree about
|
||||
what "already redacted" means, and the disagreement is invisible until someone writes a new
|
||||
placeholder. The fix here was to rephrase the comment; a future pass may want a shared token list.
|
||||
|
||||
### Verification
|
||||
|
||||
- `build_scripts.py` **37 → 0** findings on a clean clone, exit 0 (was exit 2). The generic-install
|
||||
hits (`C:\MinGW\bin\gcc.exe`, `C:\Python314\...`, `C:\Program`) no longer report; the real ones
|
||||
(`E:\apk-reverse\...`, `E:\agb_apktool.jar`) did, and were fixed in the documents, not by widening
|
||||
the list.
|
||||
- A purpose-written case table pins **both** directions — 18 cases, 0 failures: generic and
|
||||
already-redacted paths are exempt, while a real workbench path, a real personal name after
|
||||
`C:\Users\`, a real POSIX home and a real project path still report. Two defects in the first
|
||||
implementation were found by that table, not by the audit run.
|
||||
- All four gates pass: `check_repo.py` 0 problem, `check_refs.py` 0 dangling / 0 warning,
|
||||
`check_budget.py` 0 failure / 0 warning / 3 notes, `build_scripts.py` 0 findings.
|
||||
- `check_budget.py` now reports the corpus explicitly: **47 references + 55 scripts = 102 files,
|
||||
31,423 lines**, past the 26,000-line note threshold — the figure this pass could not see before.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user