217 Commits

Author SHA1 Message Date
Pascal Roeleven 5e0a0be981 Import external in ota 2023-02-27 16:38:03 +01:00
Andrew Gunnerson d911945a05 Merge pull request #67 from chenxiaolong/kmi
avbroot/boot.py: Check KMI version when using prepatched image
2023-02-27 01:19:20 -05:00
Andrew Gunnerson 8ea3d185ca avbroot/boot.py: Check KMI version when using prepatched image
This ensures that the user isn't replacing the kernel image with another
one that's not binary compatible (based on the GKI kernel module
interface version).

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-27 01:10:43 -05:00
Andrew Gunnerson 794755be0c Merge pull request #66 from chenxiaolong/readme
tests: README.md: Document that --prepatched is now tested
2023-02-26 22:42:29 -05:00
Andrew Gunnerson d74f7c77ff tests: README.md: Document that --prepatched is now tested
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-26 22:41:54 -05:00
Andrew Gunnerson 0449b606ff Merge pull request #65 from chenxiaolong/extract
README.md: Add section about extracting the entire OTA
2023-02-26 22:39:17 -05:00
Andrew Gunnerson ed1310d243 README.md: Add section about extracting the entire OTA
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-26 22:38:32 -05:00
Andrew Gunnerson 56013dd027 Merge pull request #64 from chenxiaolong/parallel
Extract images from payload in parallel
2023-02-26 22:37:30 -05:00
Andrew Gunnerson 282adf36cb Extract images from payload in parallel
Due to zipfile's internal locking (no pread support), the actual I/O
isn't parallelized. However, this still speeds up `extract --all` very
significantly because the extraction process is mostly CPU bound by
decompression.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-26 21:25:10 -05:00
Andrew Gunnerson b326b6f4a0 Merge pull request #63 from chenxiaolong/all
avbroot/main.py: Add --all option for extract subcommand
2023-02-26 20:26:47 -05:00
Andrew Gunnerson 3f85a441e4 avbroot/main.py: Add --all option for extract subcommand
Not really needed for normal usage, but nice to have a way to extract
everything without needing other tools.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-26 20:25:10 -05:00
Andrew Gunnerson 02f8942642 Merge pull request #62 from chenxiaolong/clear_vbmeta_flags
Add option to clear vbmeta flags
2023-02-26 20:22:33 -05:00
Andrew Gunnerson a927d09936 Add option to clear vbmeta flags
Some Android builds ship with a root vbmeta image with flags that
disable AVB completely. This commit adds a new check for these flags so
that the patching process will fail and also adds a new
`--clear-vbmeta-flags` option for forcibly setting the flags to 0.

Issue: #60

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-26 19:01:34 -05:00
Andrew Gunnerson 1fb6d6c65c Merge pull request #61 from chenxiaolong/prepatched
Add support for using prepatched boot images
2023-02-26 18:08:44 -05:00
Andrew Gunnerson d80ce411c7 Add support for using prepatched boot images
This is useful for folks who want to apply the Magisk root patch
themselves via the Magisk app or use KernelSU. When `--prepatched` is
used, avbroot will skip the root patch, but still apply the otacerts
patch.

To help protect against accidental use of the wrong boot image, avbroot
will only accept a prepatched image if all the header fields and section
types match the original image. The only exception is the number of
ramdisks, which is allowed to increase (Magisk may create a ramdisk in
an image that didn't originally have one).

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-26 18:01:03 -05:00
Andrew Gunnerson 26a4c054a6 Merge pull request #58 from chenxiaolong/clearotacerts_build
clearotacerts/build.py: Make executable and fix lint warnings
2023-02-17 19:33:20 -05:00
Andrew Gunnerson 48ce06ca41 clearotacerts/build.py: Make executable and fix lint warnings
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-17 19:32:57 -05:00
Andrew Gunnerson 67d2f171f2 Merge pull request #57 from chenxiaolong/arch-wine
tests: Add new image for running tests for Windows in Wine
2023-02-16 22:44:33 -05:00
Andrew Gunnerson 545a2a9587 tests/tests_containerized.py: Include elapsed time in result table
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-16 22:32:18 -05:00
Andrew Gunnerson 37ef00d7b4 tests/tests_containerized.py: Fix swapped if condition for network access
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-16 21:57:36 -05:00
Andrew Gunnerson 2238c37556 tests: Add new image for running tests for Windows in Wine
Yes, this is installing msys2 packages directory on the Arch Linux root
filesystem. Yes, it works!

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-16 21:57:15 -05:00
Andrew Gunnerson d14549a49d Merge pull request #56 from chenxiaolong/unlink
avbroot/util.py: open_output_file: Close file before unlinking on Windows
2023-02-16 20:20:20 -05:00
Andrew Gunnerson d797623acd avbroot/util.py: open_output_file: Close file before unlinking on Windows
This fixes temporary files being left behind when interrupted on
Windows.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-16 20:19:30 -05:00
Andrew Gunnerson 8a01ada343 Merge pull request #55 from chenxiaolong/zip_cd_reproducibility
avbroot/boot.py: Mark otacerts.zip entries as created on Unix
2023-02-16 20:16:51 -05:00
Andrew Gunnerson 4d091a0f83 avbroot/boot.py: Mark otacerts.zip entries as created on Unix
Otherwise, the "version made by" field in the central directory (2 bytes
immediately after CD magic) are \x14\x00 on Windows and \x14\x03
everywhere else.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-16 20:14:33 -05:00
Andrew Gunnerson 93eb5a493e Merge pull request #54 from chenxiaolong/epipe
avbroot/ota.py: Fix EPIPE when interrupted by controlling TTY
2023-02-15 18:48:44 -05:00
Andrew Gunnerson f62c129198 avbroot/ota.py: Fix EPIPE when interrupted by controlling TTY
This fixes an issue where BrokenPipeError was being raised while
handling a KeyboardInterrupt on Linux. This was happening because ^C and
^\ cause SIGINT/SIGQUIT to be sent to the entire process group,
including `openssl cms`. openssl would die and ZipFile's __exit__ would
try to finalize the zip during stack unwinding, causing a write to a
dead pipe.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-15 18:47:36 -05:00
Andrew Gunnerson a3ab7c5b56 Merge pull request #53 from chenxiaolong/exceptions
Fix skipped cleanup logic caused by fixing bare except lint warnings
2023-02-15 00:21:49 -05:00
Andrew Gunnerson b758174236 Fix skipped cleanup logic caused by fixing bare except lint warnings
This fixes temporary output files sometimes not being cleaned up and
openssl cms sometimes not being killed when the user interrupts the
process.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-15 00:20:18 -05:00
Andrew Gunnerson 65bf1e0065 Merge pull request #52 from chenxiaolong/python_3.9
Add support for Python 3.9
2023-02-15 00:05:04 -05:00
Andrew Gunnerson 7435c49f50 Add support for Python 3.9
This is needed for Debian 11 compatibility.

Fixes: #50

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-15 00:04:12 -05:00
Andrew Gunnerson 1dc6f1a088 Merge pull request #51 from chenxiaolong/podman_tests
tests: Add infrastructure for running tests in podman
2023-02-15 00:03:04 -05:00
Andrew Gunnerson 7557d30183 tests: Add infrastructure for running tests in podman
The officially supported distros are now Alpine, Arch, Debian, Fedora,
Fedora, OpenSUSE, and Ubuntu. The new script will build images for these
distros and concurrently run tests.py inside containers.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-14 23:55:34 -05:00
Andrew Gunnerson faaedf3f88 Merge pull request #49 from chenxiaolong/bootimagecli
Add new bootimagetool and cpiotool utilities
2023-02-14 02:02:30 -05:00
Andrew Gunnerson 4c81c5180d Add new cpiotool utility for dumping/repacking cpio archives
Like bootimagetool, this is another CLI interface to the
avbroot.formats.compression and avbroot.formats.cpio libraries to help
with troubleshooting.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-14 02:01:15 -05:00
Andrew Gunnerson 98bd336c91 Add new bootimagetool utility for working with boot images
This just adds a simple CLI for avbroot.formats.bootimage library. It's
not strictly related to avbroot, but is useful for troubleshooting since
the library supports all AOSP-compatible boot images.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-14 01:47:10 -05:00
Andrew Gunnerson 1a416a5c68 Merge pull request #48 from chenxiaolong/lint
avbroot/util.py: Fix f-string with no placeholders
2023-02-13 22:24:04 -05:00
Andrew Gunnerson dc1ec187a5 avbroot/util.py: Fix f-string with no placeholders
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-13 22:23:44 -05:00
Andrew Gunnerson 81c20b1467 Merge pull request #47 from chenxiaolong/gzip
avbroot/formats/compression.py: Hardcode timestamp in gzip header
2023-02-13 22:20:19 -05:00
Andrew Gunnerson 75ea4331d3 avbroot/formats/compression.py: Hardcode timestamp in gzip header
For v2 boot images that use gzip headers, the current system time was
begin added to the gzip header while recompressing the ramdisk, which
broke reproducibility. This commit hardcodes the timestamp to 0 (unix
epoch time).

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-13 22:19:32 -05:00
Andrew Gunnerson 558fef4232 Merge pull request #46 from chenxiaolong/tests
Add end-to-end patching tests
2023-02-13 22:16:59 -05:00
Andrew Gunnerson 904a9f666a Add end-to-end patching tests
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-13 22:14:27 -05:00
Andrew Gunnerson c3f5235d11 Merge pull request #45 from chenxiaolong/magisk_version
avbroot.py: Validate Magisk apk version
2023-02-13 17:34:18 -05:00
Andrew Gunnerson 0f3bb92ab3 avbroot.py: Validate Magisk apk version
This helps guard against future directory structure changes. So far,
this has remained the same across Magisk versions 22 through 25.2.

The check can be overridden with `--ignore-magisk-version`.

Issue: #37

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-13 17:25:27 -05:00
Andrew Gunnerson 3278e01f4d Merge pull request #44 from chenxiaolong/ramdisk_from_scratch
avbroot/boot.py: Construct Magisk ramdisk from scratch if no ramdisk exists
2023-02-13 16:50:38 -05:00
Andrew Gunnerson 248b78044c avbroot/boot.py: Construct Magisk ramdisk from scratch if no ramdisk exists
On bluejay, the boot image does not contain a ramdisk. In this scenario,
magiskboot creates a new lz4_legacy-compressed ramdisk, so we'll do the
same.

This commit also changes the .backup/ directory structure generation so
that it is done dynamically.

Fixes: #42

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-13 16:40:33 -05:00
Andrew Gunnerson 4f814a9971 Merge pull request #41 from chenxiaolong/threads
avbroot.py: Run boot image patches in parallel
2023-02-12 18:02:16 -05:00
Andrew Gunnerson 51a3742837 avbroot.py: Run boot image patches in parallel
There are no interdependencies across different boot images, so run the
patches in parallel. It only saves a few seconds, but it's a free
optimization.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-12 18:01:48 -05:00
Andrew Gunnerson 8ab265e08f Merge pull request #40 from chenxiaolong/patch
Use unittest.mock.patch for replacing classes and variables
2023-02-11 20:39:14 -05:00
Andrew Gunnerson 505ef91bc9 Use unittest.mock.patch for replacing classes and variables
The mechanism is still the same (modifies global state), but the code is
a bit cleaner.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2023-02-11 20:38:01 -05:00