Our previous limit was 15 bytes for the <offset>:<size> placeholder,
matching AOSP's ota_utils.py. Since the size of metadata.pb is almost
always 4 digits, this leaves 10 digits for the offset, which isn't
enough for large OTAs. AOSP never actually hits the limit because it
puts metadata and metadata.pb at the beginning of the output zip file.
We put the files at the end of the zip since we do streaming writes.
Fixes: #451
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This also fixes a number of disabled-by-default clippy warnings and
updates the Rust edition to 2024.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
It was hardcoded to set the CoW version to v2 in the payload manifest.
This commit also updates the pixel_v2 profile to disable VABC so that
scenario gets tested. The pixel_v3 profile now uses CoW v2 with gz.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
No known device uses this, but it's very useful for testing that our
overhead calculation is identical to AOSP's. A few more bugs were found
in our overhead calculation logic:
* The CowHeaderV3 size was missing the original CowHeader (v2) fields
that are supposed to be included due to inheritance in the C++ class.
* The additional 1% overhead was incorrectly calculated against the
initial CoW estimate before static overhead for CoW headers was added.
* The V3 num_ops estimation did not set a minimum of 25 to match
delta_generator.
* The V2 size estimation did not take into account that a cluster of CoW
operations cannot be truncated. It must be a multiple of cluster_ops
(200 for avbroot).
With these fixes, the CoW estimation when compression is disabled
matches AOSP exactly. This means all the overhead calculation is now
correct and the only difference when compression is enabled is in the
compression ratios of the various lz4/gz implementations.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
AOSP has long supported CoW version 3, but it wasn't used by the stock
OS on any Pixel devices until the new Pixel 9a.
CoW version 3 is fundamentally similar to version 2, though with
differences in the main header and how the operation headers are stored.
The compression is no longer done in fixed-size chunks equal to the
block size. Instead, the payload specifies a "compression factor", which
is the maximum chunk size to pass to the compressor. The actual chunk
size is the largest power of 2 <= the compression factor and the
remaining input size. Additionally, for version 3, the payload stores an
additional estimate_op_count_max field containing the number of CoW
operations.
While working on support for version 3, a few bugs in the version 2
estimation logic were found and fixed:
* The cluster_ops * sizeof(CowOperationV2) overhead incorrectly
assumed that cluster_ops was a constant 200 instead of the actual
number of CoW operations.
* The overhead did not account for kCowLabelOp headers, which
delta_generator emits once for every InstallOperation in the payload.
* The overhead did not account for kCowClusterOp headers, which batch
CoW operations into groups of 200.
* The overhead did not account for the CowFooter.
The version 3 overhead is much simpler and easier to calculate compared
to version 2.
Fixes: #441
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
While avbroot initially started as a way to allow a rooted boot image to
be used with a locked bootloader, it has evolved much since then.
Nowadays, many folks use it to make modifications to their OTAs that
don't involve enabling root access. avbroot also has many subcommands
for packing and unpacking various Android image formats that people use
without ever using avbroot's main OTA patching functionality.
This commit updates the project description to reflect this and
simplifies the wording a bit.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This reverts commit e929ecbe44.
Ring is back to being maintained again, so let's switch back to it since
it has fewer build dependencies and is much faster to compile.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Devices that launch with Android <14 generally use gzip as the CoW
compression algorithm. This never changes because future full OTAs
always need to be installable from the version of Android the device
launched with.
However, for users that don't care about the upgrade path from old
versions of Android, a new --vabc-algo option can be used to switch from
gz to lz4 compression. This can cut down the OTA installation time by
more than 2/3rds when installing via a custom OTA updater app. On my
Pixel Tablet, the installation time for the update_engine DOWNLOADING
phase decreased from 32:05 to 9:41. Note that this has absolutely no
effect on the performance when sideloading from recovery mode because
that does not use CoW.
When this new option is used, all dynamic partitions need to be
extracted from the OTA during patching so that the CoW estimates can be
recomputed. This will slow down the patching process and use up more
temporary disk space.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Instead of compressing the 64 MiB input in 2 MiB chunks, each loop
iteration was compressing the full 64 MiB. This massively slowed down
the patching process from seconds to potentially hours and would
temporarily waste a bunch of space during OTA installation.
Fixes: #433
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This way, we can fail hard for parsing errors, but not for verification
errors in `avbroot ota verify`.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
* Try to avoid fail-fast behavior to show as many errors as possible.
Parsing errors always fail immediately, but verification errors don't.
* Move the recovery otacerts.zip check to the end to let more important
checks run first.
* Improve error message when otacerts.zip does not contain the signing
certificate for the OTA to make it clear the issue is not that the zip
contains no certificates at all.
* Always run the recovery otacerts.zip check, but just log the error as
a warning when running with --skip-recovery-ota-cert.
* Fix unformatted error context string when parsing a boot image's
otacerts.zip file fails.
Discussion: #426
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
There are two documented behaviors in liblp that are violated with the
on-disk super partition layout after a virtual A/B CoW merge:
1. The partition name includes a `-` due to the `-cow` suffix. This is
not meant to be a valid character.
2. The extent list is likely to have many gaps and not be sorted. The
format documentation says that gaps are not allowed.
This commit updates avbroot's LP parser to be less strict so that it can
load real on-device super partitions. The extent allocator for the
`pack` subcommand remains unchanged though, so avbroot will always
produce LP images with sorted, gapless extents.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This is the same optimization as is currently done for boot images.
There's no reason to keep the temp file around for the entire patching
process if it's unmodified and we're not going to be copying its data
into the payload.
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
Andrew Gunnerson