In light of the recently discovered backdoor in the xz project where a
part of the malicious code was distributed in the test files, let's
remove all of our test files and generate them at runtime. While our
test files are very simple and consist mostly of zeros, someone who is
not very familiar with these binary formats would have a harder time
examining them and making sure they aren't malicious. With this change,
the data structures are now plainly visible in the test code.
The files generated at runtime are byte-for-byte identical to the test
files being removed. One can verify by comparing the sha512 checksums of
the files with the sha512 checksums hardcoded in the new test code.
Closes: #265
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
`fastboot flashall` is identical to the `fastboot update` command used
by the Pixel factory images, except it reads from a directory instead of
a zip file. It knows how to flip between the fastboot and fastbootd
modes without user intervention.
Fixes: #252
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Patching the image was never implemented because it's specific to
OnePlus devices where the bootloader does not respect the custom root of
trust.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
x509_cert is unable to parse files that contains non-empty lines outside
of the BEGIN CERTIFICATE and END CERTIFICATE markers.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
With the dm-verity hash tree format, each digest is salted, meaning it
first hashes the salt byte string and then the actual data. Previously,
the salt was being rehashed for each digest operation. Instead, the
salted SHA-256 context can be computed once and then cloned when needed.
The performance improvement is pretty minor, but it's worth doing anyway
since it's not any more complex.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Instead of println'ing everything, this commit switches the code base to
using the tracing library. There are now proper log levels and multiple
logging output formats. A bunch of new debug and trace-level messages
have also been added to help with future troubleshooting.
By default, the output is kept nice and short. Spans won't be shown
unless the log level is set to debug or lower.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Upstream Magisk now xz-compresses files in modifies in the ramdisk. This
commit also implements the same in avbroot's MagiskRootPatcher.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
We'll switch to using `fastboot flashall` in the future once that has
been implemented and tested.
Fixes: #252
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This commit replaces the previous approach of patching real OTAs with
patching mock OTAs. The motivation for this change is to make it
possible to test the system partition otacerts.zip patching without
needing to download huge files. Adding the system image to the stripped
OTAs would increase the file size by an order of magnitude.
The mock OTAs are generated from a set of profiles defined in e2e.toml.
The four included profiles are meant to mimic the OTAs used for testing
before:
* pixel_v4_gki ~= cheetah
* pixel_v4_non_gki ~= bluejay
* pixel_v3 ~= bramble
* pixel_v2 ~= sunfish
There is no equivalent profile for ossi because newer OnePlus devices no
longer support custom signing keys properly.
The mock OTAs are perfectly valid, structure and signature-wise. They
just don't include any real partition data where possible. They are
initially signed with a different set of keys to ensure that the changes
made by the patching process are actually visible.
With how small the mock OTAs are, testing every profile only takes about
two seconds. Thus, the Github Actions workflow was adjusted to just run
e2e in the same job as the build.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
When the original payload extents are all in order and have no gaps, we
can efficiently copy unmodified chunks of the system image from the
original payload into the new payload. Only the chunks containing the
modified regions (`otacerts.zip`, hash tree, FEC data, and AVB metadata)
need to be recompressed. This massively reduces the CPU usage since
usually only <20 MiB need to be recompressed.
If the conditions for the optimized path aren't satisfied (eg. extents
aren't in order or `--replace` is used), then it falls back to splitting
and compressing the entire system image.
This fixes the performance issues introduced in #240. This is probably
the best that we can do given that we now always patch the system image.
Issue: #225
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Andrew Gunnerson