PR #110 fixed this incorrectly. I misinterpreted "set to zero in the
local header" from §4.4.4 in the spec as referring to the two 32-bit
size fields in the local header. However, the spec considers the zip64
extra record as part of the local header. The correct behavior for
writing zip64 entries to unseekable files is 0xffffffff in the 32-bit
local header size fields (to enable zip64), 0 in the zip64 extra record
(due to streaming writes), and the actual sizes in the data descriptor.
Fixes: #109
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This commit adds support for replacing any partition image within
the payload with a custom image. This is useful, for example, to add a
custom kernel to an OTA, which may involve partitions that wouldn't
normally be touched (eg. `vendor_dlkm`).
Any image specified via `--replace` will have its corresponding
descriptor in the vbmeta image updated. This is handled recursively. For
example, replacing `vendor_dlkm` would update both `vbmeta_vendor` and
`vbmeta`. This requires all vbmeta images to be extracted during the
patching process so that a complete dependency graph can be computed.
The performance hit in doing so is negligible, but does require the
checksums of the stripped images to be updated for the tests.
Fixes: #102
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This commit fixes two issues:
* Python 3.11.4 has a regression where it sets the local file header's
compressed and uncompressed size fields to 0xffffffff when data
descriptors are used. These should be set to 0 (along with the CRC)
according to §4.4.4 of the spec. We work around this by monkey
patching zipfile's local file header serialization function to correct
the fields.
* avbroot tries to preserve as much metadata from the original zip as
possible, including the `extra` records. This caused zip64 records
(0x0001) to be duplicated. The first instance was from the original
zip (containing invalid size fields) and the second instance was newly
created by zipfile. We fix this by stripping out all 0x0001 records,
similar to what we already do for 0xd935.
Fixes: #109
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Newer Magisk versions no longer try to autodetect a writable ext4
partition for storing SELinux rules during boot. Instead, the block
device is detected during boot image patching and is stored in the
ramdisk's `.backup/.magisk` as `makedev(rdev_maj, rdev_min)`.
This unfortunately complicates the patching process for avbroot. Even if
we replicate Magisk's algorithm for finding a suitable partition,
there's no way to find the block device's rdev for it with the
information contained in the OTA package. This is the first change that
adds a hard dependency on information only attainable from a running
device.
For these newer Magisk versions, the user will have to patch the boot
image once in the Magisk app (must be on the target device) and then run
`avbroot magisk-info` to show the computed device ID. Then, avbroot can
use this during patching via `--magisk-rules-device`. If the user's
device is unable to run the Magisk app prior to patching (eg.
unbootable), they'll have to go through the patching process twice, once
with `--ignore-magisk-warnings` and a second time with the proper device
ID specified.
When using `--ignore-magisk-warnings`, root still works, but there will
be subtle issues, like certain modules failing to load.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This way, the module path is automatically updated when avbroot is
imported. This also fixes the lint warning about `external` being
imported, but not used.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This commit merges the CI tests from `tests_ci/` into `tests/` so we
have a single way to test against both full OTAs and stripped/dummy
OTAs.
Features kept from tests_ci:
* Support for dummy OTAs. They're now called stripped OTAs to hopefully
better signify that they're a stripped down version of the original
file instead of a test file created from scratch.
* Extracting AVB partitions and verifying their hashes.
* Writing sparse files where possible.
* Ability to change the working directory for downloads and output
files.
Test script changes:
* There are now 4 subcommands:
strip -i <input> -o <output>
add -u <url> -d <device> [-H <expected hash>]
download [--magisk | --no-magisk] [[-d <device>] ... | --no-devices]
test [[-d <device>] ...]
* Downloads now use the parallel downloader for both full and stripped
OTAs.
Config file changes:
* It now uses strictyaml instead of configparser/TOML since it's a bit
more readable now that we have more nesting in the data structure. The
config loader now also makes use of strictyaml's schema feature.
* All hashes are now SHA-256 for consistency.
* For stripped OTAs, the list of byte ranges now uses half-open
intervals for easier calculations.
* Use device IDs as the key instead of the marketing model name.
CI changes:
* Compute all cache keys (and prefixes) in the main workflow and pass
them to the preload scripts.
* Only run on `push` for the `master` branch to avoid double workflow
runs on PRs from internal branches.
* Increase timeout for patching tests because `tests.py` patches twice
(once with `--magisk` and once with `--prepatched`).
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
No fancy algorithm. Just a simple moving average over 5 seconds, updated
at 100ms intervals.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This is useful for folks who want to apply the Magisk root patch
themselves via the Magisk app or use KernelSU. When `--prepatched` is
used, avbroot will skip the root patch, but still apply the otacerts
patch.
To help protect against accidental use of the wrong boot image, avbroot
will only accept a prepatched image if all the header fields and section
types match the original image. The only exception is the number of
ramdisks, which is allowed to increase (Magisk may create a ramdisk in
an image that didn't originally have one).
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Yes, this is installing msys2 packages directory on the Arch Linux root
filesystem. Yes, it works!
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
The officially supported distros are now Alpine, Arch, Debian, Fedora,
Fedora, OpenSUSE, and Ubuntu. The new script will build images for these
distros and concurrently run tests.py inside containers.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This just adds a simple CLI for avbroot.formats.bootimage library. It's
not strictly related to avbroot, but is useful for troubleshooting since
the library supports all AOSP-compatible boot images.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Andrew Gunnerson