Compare commits

..

36 Commits

Author SHA1 Message Date
Andrew Gunnerson 8ca4eb5ad9 Version 3.17.1
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-07-01 19:07:13 -04:00
Andrew Gunnerson e2b1ccb7a1 CHANGELOG.md: Add entry for PR #467
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-07-01 19:06:37 -04:00
Andrew Gunnerson 71a31ae01b Bump Magisk version upper bound to 30100
There are no breaking changes.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-07-01 18:55:14 -04:00
Andrew Gunnerson 83d7ffbc5e CHANGELOG.md: Add entry for PR #464
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-06-19 20:20:54 -04:00
Andrew Gunnerson e397998d9e Update dependencies
The zip crate gained support for streaming writes in its master branch,
so we can finally upgrade from our ancient fork of it. The new
implementation is done a bit differently, with seekable writers having
the ZipWriter<W> type and streaming writers having the
ZipWriter<StreamWriter<W>> type. This forces us to add a new wrapper
type since we have to switch between them at runtime.

We still need to maintain a (hopefully temporary) fork of the crate due
to a few issues:

1. There's no way to get the original underlying writer instance back
   after finalizing a streaming zip. A fix for this has been submitted
   upstream:
   https://github.com/zip-rs/zip2/pull/367

2. The streaming writes implementation does not include the magic
   signature for data descriptors. While the zip spec says the magic
   value is optional and parsers should not require it, older versions
   of Android's libziparchive do. A fix for this has been submitted
   upstream:
   https://github.com/zip-rs/zip2/pull/368

3. There is currently no way to get the data offset of zip entries.
   avbroot requires this to fill in the OTA metadata's "property files"
   entries, which Android uses to read file data without parsing the zip
   file structures.

This new zip update produces files that are slightly different to
before. The "version made by" and "version needed to extract" fields are
now set to their minimum possible values. Previously, the zip crate was
hardcoded to use versions 4.6 and 2.0, respectively.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-06-19 18:55:08 -04:00
Andrew Gunnerson 8ef22508f5 CHANGELOG.md: Add entry for PR #463
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-06-19 18:49:32 -04:00
Andrew Gunnerson bf42a6a75c e2e: Split streaming and seekable work directories
Makes troubleshooting easier when files aren't being overwritten.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-06-19 17:49:00 -04:00
lefuglyduck a2fe6fc9d8 Update README.md
Clarify the updates section so it doesn't "loop back" to the usage section. 

Signed-off-by: lefuglyduck <31975903+lefuglyduck@users.noreply.github.com>
2025-06-12 16:12:13 -07:00
lefuglyduck faeb1fe988 Update README.md
Minor grammatical change.

Signed-off-by: lefuglyduck <31975903+lefuglyduck@users.noreply.github.com>
2025-06-12 17:53:52 -04:00
Andrew Gunnerson f1b2c6f468 Merge PR #458
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-06-12 17:51:40 -04:00
lefuglyduck 2f964bf113 Update README.md
Clarify step 3 of usage section.

Signed-off-by: lefuglyduck <31975903+lefuglyduck@users.noreply.github.com>
2025-06-11 22:36:24 -07:00
Andrew Gunnerson f393d7adc4 README.md: Add warning about post-installation snapshot merge operation
Issue: #454

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-06-08 17:00:21 -04:00
Andrew Gunnerson 72a1c3f216 Version 3.17.0
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-31 12:00:31 -04:00
Andrew Gunnerson 2db8d3826e CHANGELOG.md: Add entry for PR #453
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-31 10:26:46 -04:00
Andrew Gunnerson 1448e55205 Update dependencies
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-31 10:21:14 -04:00
Andrew Gunnerson b3862a9c4a CHANGELOG.md: Add entry for PR #452
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-31 10:18:11 -04:00
Andrew Gunnerson 088db04673 ota: Reserve 16 bytes for OTA metadata property files
Our previous limit was 15 bytes for the <offset>:<size> placeholder,
matching AOSP's ota_utils.py. Since the size of metadata.pb is almost
always 4 digits, this leaves 10 digits for the offset, which isn't
enough for large OTAs. AOSP never actually hits the limit because it
puts metadata and metadata.pb at the beginning of the output zip file.
We put the files at the end of the zip since we do streaming writes.

Fixes: #451

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-31 10:16:45 -04:00
Andrew Gunnerson b1410c869d Version 3.16.1
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-19 22:54:52 -04:00
Andrew Gunnerson dd6eaf8e78 CHANGELOG.md: Add entry for PR #449
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-19 22:50:59 -04:00
Andrew Gunnerson 0eac8e6614 Update dependencies
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-19 22:37:32 -04:00
Andrew Gunnerson 8a4f90176e CHANGELOG.md: Add entry for PR #448
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-19 22:33:01 -04:00
Andrew Gunnerson bd166594e2 Bump Magisk version upper bound to 29100
There are no breaking changes.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-19 22:31:34 -04:00
Andrew Gunnerson 19129ae927 Version 3.16.0
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-04 18:15:02 -04:00
Andrew Gunnerson fb2aaed042 CHANGELOG.md: Add entry for PR #446
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-04 18:14:25 -04:00
Andrew Gunnerson 182d937d34 Update all dependencies
This also fixes a number of disabled-by-default clippy warnings and
updates the Rust edition to 2024.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-04 18:06:59 -04:00
Andrew Gunnerson 113bdec6dc CHANGELOG.md: Add entry for PR #445
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-04 16:57:03 -04:00
Andrew Gunnerson 1f0d012ac0 e2e: Fix incorrect metadata when generating a non-CoW-V2 payload
It was hardcoded to set the CoW version to v2 in the payload manifest.
This commit also updates the pixel_v2 profile to disable VABC so that
scenario gets tested. The pixel_v3 profile now uses CoW v2 with gz.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-04 16:55:38 -04:00
Andrew Gunnerson e4fbe00ea2 CHANGELOG.md: Add entry for PR #444
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-04 16:29:40 -04:00
Andrew Gunnerson ad0b3d5aa8 payload: Add support for custom CoW compression levels
No known device uses this functionality, but AOSP supports it, so we
should too.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-03 23:34:25 -04:00
Andrew Gunnerson f4394b0c21 CHANGELOG.md: Add entry for PR #443
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-03 23:03:44 -04:00
Andrew Gunnerson 9dd98fd715 payload: Add support for uncompressed CoW and fix more estimation bugs
No known device uses this, but it's very useful for testing that our
overhead calculation is identical to AOSP's. A few more bugs were found
in our overhead calculation logic:

* The CowHeaderV3 size was missing the original CowHeader (v2) fields
  that are supposed to be included due to inheritance in the C++ class.
* The additional 1% overhead was incorrectly calculated against the
  initial CoW estimate before static overhead for CoW headers was added.
* The V3 num_ops estimation did not set a minimum of 25 to match
  delta_generator.
* The V2 size estimation did not take into account that a cluster of CoW
  operations cannot be truncated. It must be a multiple of cluster_ops
  (200 for avbroot).

With these fixes, the CoW estimation when compression is disabled
matches AOSP exactly. This means all the overhead calculation is now
correct and the only difference when compression is enabled is in the
compression ratios of the various lz4/gz implementations.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-03 22:59:15 -04:00
Andrew Gunnerson b6e3c68241 CHANGELOG.md: Add entry for PR #442
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-03 20:39:57 -04:00
Andrew Gunnerson 6fee5346bb payload: Add support for CoW version 3
AOSP has long supported CoW version 3, but it wasn't used by the stock
OS on any Pixel devices until the new Pixel 9a.

CoW version 3 is fundamentally similar to version 2, though with
differences in the main header and how the operation headers are stored.
The compression is no longer done in fixed-size chunks equal to the
block size. Instead, the payload specifies a "compression factor", which
is the maximum chunk size to pass to the compressor. The actual chunk
size is the largest power of 2 <= the compression factor and the
remaining input size. Additionally, for version 3, the payload stores an
additional estimate_op_count_max field containing the number of CoW
operations.

While working on support for version 3, a few bugs in the version 2
estimation logic were found and fixed:

* The cluster_ops * sizeof(CowOperationV2) overhead incorrectly
  assumed that cluster_ops was a constant 200 instead of the actual
  number of CoW operations.
* The overhead did not account for kCowLabelOp headers, which
  delta_generator emits once for every InstallOperation in the payload.
* The overhead did not account for kCowClusterOp headers, which batch
  CoW operations into groups of 200.
* The overhead did not account for the CowFooter.

The version 3 overhead is much simpler and easier to calculate compared
to version 2.

Fixes: #441

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-05-03 20:28:50 -04:00
Ivan 7d786150cf README.ru.md: update translation
* https://github.com/chenxiaolong/avbroot/commit/de433a27245adb26e578bdeb34a404552c79ea85
* https://github.com/chenxiaolong/avbroot/commit/5d66774d131505928b31e1968da0498732ac28ab
* https://github.com/chenxiaolong/avbroot/commit/b922f0d23ce841ac0bd14bfd33da46273f183f68
2025-04-21 19:51:43 +03:00
Andrew Gunnerson b922f0d23c README.md: Deemphasize rooting in project description
While avbroot initially started as a way to allow a rooted boot image to
be used with a locked bootloader, it has evolved much since then.
Nowadays, many folks use it to make modifications to their OTAs that
don't involve enabling root access. avbroot also has many subcommands
for packing and unpacking various Android image formats that people use
without ever using avbroot's main OTA patching functionality.

This commit updates the project description to reflect this and
simplifies the wording a bit.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-04-18 20:51:54 -04:00
Andrew Gunnerson 1f2b2170b3 CHANGELOG.md: Fix incorrect link to VABC compression algorithm documentation
Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
2025-04-11 21:13:17 -04:00
43 changed files with 1304 additions and 643 deletions
+1 -1
View File
@@ -27,7 +27,7 @@ jobs:
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Create release
uses: softprops/action-gh-release@c95fe1489396fe8a9eb87c0abf8aa5b2ef267fda # v2.2.1
uses: softprops/action-gh-release@da05d552573ad5aba039eaac05058a918a7bf631 # v2.2.2
with:
tag_name: v${{ steps.get_version.outputs.version }}
name: Version ${{ steps.get_version.outputs.version }}
+42 -1
View File
@@ -7,11 +7,38 @@
to update the actual links at the bottom of the file.
-->
### Version 3.17.1
* Update end-to-end tests to place streaming and seekable OTAs in separate directories for easier troubleshooting ([PR #463])
* Update dependencies ([PR #464])
* Add support for Magisk 30000 ([PR #467])
### Version 3.17.0
* Fix reserved space error when patching OTA zips larger than ~10 GB ([Issue #451], [PR #452])
* Update dependencies ([PR #453])
### Version 3.16.1
* Add support for Magisk 29000 ([PR #448])
* Update dependencies ([PR #449])
### Version 3.16.0
* Add support for CoW version 3 for virtual A/B ([Issue #441], [PR #442], [PR #445])
* This was recently introduced with the Pixel 9a. Previous devices all used CoW version 2.
* Add support for uncompressed CoW for virtual A/B ([PR #443])
* This is not used on actual devices, but is very useful for testing the CoW estimation logic.
* All differences between avbroot's and AOSP delta_generator's estimation logic are now fixed.
* Add support for custom CoW compression levels for virtual A/B ([PR #444])
* This is also not used on actual devices, but is supported by AOSP, so avbroot should support it too.
* Update dependencies ([PR #446])
### Version 3.15.0
* Add support for changing the virtual A/B compression algorithm ([PR #437])
* For devices that launched with Android <14, `--vabc-algo lz4` can significantly increase OTA installation speed when using a custom OTA updater app (with caveats). There is no difference when sideloading from recovery mode.
* See [the documentation](#changing-virtual-ab-cow-compression-algorithm) for more details.
* See [the documentation](./README.md#changing-virtual-ab-cow-compression-algorithm) for more details.
* Switch back to the ring library now that it is maintained again ([PR #438])
* Update dependencies ([PR #439])
@@ -337,6 +364,8 @@ Behind-the-scenes changes:
[Issue #366]: https://github.com/chenxiaolong/avbroot/issues/366
[Issue #393]: https://github.com/chenxiaolong/avbroot/issues/393
[Issue #433]: https://github.com/chenxiaolong/avbroot/issues/433
[Issue #441]: https://github.com/chenxiaolong/avbroot/issues/441
[Issue #451]: https://github.com/chenxiaolong/avbroot/issues/451
[PR #130]: https://github.com/chenxiaolong/avbroot/pull/130
[PR #132]: https://github.com/chenxiaolong/avbroot/pull/132
[PR #133]: https://github.com/chenxiaolong/avbroot/pull/133
@@ -491,3 +520,15 @@ Behind-the-scenes changes:
[PR #437]: https://github.com/chenxiaolong/avbroot/pull/437
[PR #438]: https://github.com/chenxiaolong/avbroot/pull/438
[PR #439]: https://github.com/chenxiaolong/avbroot/pull/439
[PR #442]: https://github.com/chenxiaolong/avbroot/pull/442
[PR #443]: https://github.com/chenxiaolong/avbroot/pull/443
[PR #444]: https://github.com/chenxiaolong/avbroot/pull/444
[PR #445]: https://github.com/chenxiaolong/avbroot/pull/445
[PR #446]: https://github.com/chenxiaolong/avbroot/pull/446
[PR #448]: https://github.com/chenxiaolong/avbroot/pull/448
[PR #449]: https://github.com/chenxiaolong/avbroot/pull/449
[PR #452]: https://github.com/chenxiaolong/avbroot/pull/452
[PR #453]: https://github.com/chenxiaolong/avbroot/pull/453
[PR #463]: https://github.com/chenxiaolong/avbroot/pull/463
[PR #464]: https://github.com/chenxiaolong/avbroot/pull/464
[PR #467]: https://github.com/chenxiaolong/avbroot/pull/467
Generated
+426 -281
View File
File diff suppressed because it is too large Load Diff
+2 -2
View File
@@ -4,9 +4,9 @@ members = ["avbroot", "e2e", "fuzz", "xtask"]
resolver = "2"
[workspace.package]
version = "3.15.0"
version = "3.17.1"
license = "GPL-3.0-only"
edition = "2021"
edition = "2024"
repository = "https://github.com/chenxiaolong/avbroot"
[workspace.lints.clippy]
+14 -10
View File
@@ -2,12 +2,10 @@
(This page is also available in: [Russian (Русский)](./README.ru.md).)
avbroot is a program for patching Android A/B-style OTA images for root access while preserving AVB (Android Verified Boot) using custom signing keys. It is compatible with both Magisk and KernelSU. If desired, it can also just re-sign an OTA without enabling root access.
avbroot is a tool for modifying Android A/B OTA images reproducibly and re-signing them with custom keys. It also includes a [collection of subcommands](./README.extra.md) for packing and unpacking numerous Android image formats.
Having a good understanding of how AVB and A/B OTAs work is recommended prior to using avbroot. At the very least, please make sure the [warnings and caveats](#warnings-and-caveats) are well-understood to avoid the risk of hard bricking.
**NOTE:** avbroot 2.0 has been rewritten in Rust and no longer relies on any AOSP code. The CLI is fully backwards compatible, but the old Python implementation can be found in the `python` branch if needed.
## Requirements
* Only devices that use modern A/B partitioning are supported. This is the case for most non-Samsung devices launched with Android 10 or newer. To check if a device uses this partitioning scheme, open the OTA zip file and check that:
@@ -23,7 +21,7 @@ Having a good understanding of how AVB and A/B OTAs work is recommended prior to
avbroot applies the following patches to the partition images:
* The `boot` or `init_boot` image, depending on device, is patched to enable root access. For Magisk, the patch is equivalent to what would be normally done by the Magisk app.
* The `boot` or `init_boot` image, depending on device, is patched to enable root access if requested.
* The `boot`, `recovery`, or `vendor_boot` image, depending on device, is patched to replace the OTA signature verification certificates with the custom OTA signing certificate. This allows future patched OTAs to be sideloaded from recovery mode after the bootloader has been locked. It also prevents accidental flashing of the original unpatched OTA.
@@ -31,7 +29,7 @@ avbroot applies the following patches to the partition images:
## Warnings and Caveats
* **Always leave the `OEM unlocking` checkbox enabled when using a locked bootloader with root.** This is critically important. Root access allows the boot partition to potentially be overwritten, either accidentally or intentionally, with an image that is not properly signed. In this scenario, if the checkbox is turned off, both the OS and recovery mode will be made unbootable and `fastboot flashing unlock` will not be allowed. This effectively renders the device **_hard bricked_**.
* **Always leave the `OEM unlocking` checkbox enabled when using a locked bootloader while rooted.** This is critically important. Root access allows the boot partition to potentially be overwritten, either accidentally or intentionally, with an image that is not properly signed. In this scenario, if the checkbox is turned off, both the OS and recovery mode will be made unbootable and `fastboot flashing unlock` will not be allowed. This effectively renders the device **_hard bricked_**.
Repeat: **_ALWAYS leave `OEM unlocking` enabled if rooted._**
@@ -53,6 +51,8 @@ avbroot applies the following patches to the partition images:
3. Follow the steps to [generate signing keys](#generating-keys).
Skip this step if you're updating Android, Magisk, or KernelSU after you've performed an [initial setup](#initial-setup). [Updates](#updates) do not require signing keys since you have already generated them in the initial setup.
4. Patch the OTA zip. The base command is:
```bash
@@ -225,21 +225,25 @@ If you lose your AVB or OTA signing key, you will no longer be able to sign new
**WARNING**: If you are flashing CalyxOS, the setup wizard will [automatically turn off the `OEM unlocking` switch](https://github.com/CalyxOS/platform_packages_apps_SetupWizard/blob/7d2df25cedcbff83ddb608e628f9d97b38259c26/src/org/lineageos/setupwizard/SetupWizardApp.java#L135-L140). Make sure to manually reenable it again from Android's developer settings. Consider using the [`OEMUnlockOnBoot` module](https://github.com/chenxiaolong/OEMUnlockOnBoot) to automatically ensure OEM unlocking is enabled on every boot.
10. That's it! To install future OS, Magisk, or KernelSU updates, see the [next section](#updates).
10. That's it! To update the OS, Magisk, or KernelSU see the [next section](#updates).
## Updates
Updates to Android, Magisk, and KernelSU are all done the same way by patching (or repatching) the OTA.
Updates to Android, Magisk, and KernelSU are all done the same way: by patching (or repatching) the OTA.
1. If Magisk or KernelSU is being updated, first install their new `.apk`. If you happen to open the app, make sure it **does not** flash the boot image. Cancel the boot image update prompts if needed.
1. Generate a new patched OTA by following the steps in the [usage section](#usage).
2. Follow the step in the [usage section](#usage) to patch the new OTA.
2. If Magisk or KernelSU is being updated, first install their new `.apk`. If you happen to open the app, make sure it **does not** flash the boot image. Cancel the boot image update prompts if needed.
3. Reboot to recovery mode. If the screen is stuck at a `No command` message, press the volume up button once while holding down the power button.
4. Sideload the patched OTA with `adb sideload`.
5. That's it!
5. Restart your phone. Note: the phone will likely take a long time to startup after an OS update (a few minutes in some cases).
**Warning**: Due to how virtual A/B works, there is a snapshot merge operation that Android runs invisibly in the background after installing an OTA and rebooting. During the snapshot merge process, it's not possible to sideload another OTA from recovery mode. Avoid doing anything that could result in a boot loop (eg. installing modules) until this process is complete because there is no way to recover, aside from unlocking the bootloader (and wiping) again.
The status can be found by running `adb logcat -v color -s update_engine`. Alternatively, if [Custota](https://github.com/chenxiaolong/Custota) is installed (even if it's not configured to point to a custom OTA server), it will show a notification until the snapshot merge operation completes.
## Reverting to stock firmware
+39 -17
View File
@@ -1,11 +1,9 @@
# avbroot
avbroot это программа для модификации OTA-образов Android A/B-формата с целью получения root-прав при сохранении прохождения AVB (Android Verified Boot) с использованием кастомных (пользовательских) ключей подписи. Она совместима как с Magisk, так и с KernelSU. При необходимости можно просто переподписать OTA, без получения root-доступа.
avbroot это утилита для воспроизводимой модификации OTA-образов Android A/B-формата и их переподписания пользовательскими ключами. Она также включает в себя [набор подкоманд](./README.extra.md) для упаковки и распаковки образов Android различных форматов.
Прежде чем использовать avbroot, рекомендуется иметь хорошее понимание того, как работают AVB и OTA в формате A/B. Как минимум, следует ознакомиться с [разделом предостережений,](#предостережения) чтобы избежать хардбрика устройства.
**ПРИМЕЧАНИЕ:** avbroot 2.0 была переписана на Rust и больше не имеет в основе никакого кода AOSP, а CLI полностью обратно совместим. Тем не менее, старую реализацию на Python можно найти в одноименной ветке `python`.
## Требования
* Поддерживаются только устройства, использующие современную A/B-разметку. Это большинство девайсов, выпускаемых с Android 10 и новее (за исключением устройств от Samsung). Чтобы проверить, использует ли ваш телефон необходимую схему разметки, откройте zip-архив OTA и проверьте:
@@ -21,7 +19,7 @@ avbroot – это программа для модификации OTA-обра
avbroot модифицирует следующие образы:
* `boot` или `init_boot`, в зависимости от устройства, модифицируется для получения root-доступа. В случае с Magisk, патч будет эквивалентен тому, что производится в самом приложении Magisk.
* `boot` или `init_boot`, в зависимости от устройства, модифицируется для получения root-доступа, если это запрашивается.
* `boot`, `recovery` или `vendor_boot`, в зависимости от устройства, модифицируется для замены сертификата проверки подписи OTA на пользовательский. Это позволяет устанавливать будущие пропатченные OTA через режим Recovery уже после блокировки загрузчика, то есть в качестве обновления. Также это предотвращает случайную установку оригинального непропатченного OTA.
@@ -122,7 +120,7 @@ avbroot модифицирует следующие образы:
avbroot key encode-avb -k avb.key -o avb_pkmd.bin
```
3. Сгенерируйте самоподписанный сертификат для ключа подписи OTA. Он используется режимом Recovery для проверки подписи OTA при сайдлоадинге обновления.
3. Сгенерируйте самоподписанный сертификат для ключа подписи OTA. Он используется режимом Recovery для проверки подписи OTA при установке обновления.
```bash
avbroot key generate-cert -k ota.key -o ota.crt
@@ -185,7 +183,7 @@ avbroot совместим с любым стандартным 4096-битны
fastboot flashall --skip-reboot
```
Обратите внимание, что так прошиваются лишь те образы, что относятся к системе. Разделы загрузчика и модема же остаются нетронутыми из-за ограничений fastboot. Если они не обновлены до необходимой версии, или вы не уверены в этом, после прошивки перейдите к пункту [обновлений](#обновления) и установите пропатченный OTA-архив сайдлоадом в режиме Recovery. Прошивка полного OTA гарантирует, что абсолютно все разделы будут обновлены.
Обратите внимание, что так прошиваются лишь те образы, что относятся к системе. Разделы загрузчика и модема же остаются нетронутыми из-за ограничений fastboot. Если они не обновлены до необходимой версии, или вы не уверены в этом, после прошивки перейдите к пункту [обновлений](#обновления) и установите пропатченный OTA в режиме Recovery. Прошивка полного OTA гарантирует, что абсолютно все разделы будут обновлены.
Для устройств Pixel есть ещё один вариант: запуск скрипта `flash-base.sh` из папки заводских образов (factory images) обновит загрузчик и модем.
@@ -255,11 +253,11 @@ avbroot совместим с любым стандартным 4096-битны
## OTA-обновления
avbroot заменяет `/system/etc/security/otacerts.zip` в разделах системы и Recovery на новый архив, содержащий пользовательский сертификат подписи OTA. Это предотвращает случайную установку непропатченных OTA как при загрузке в Android, так и при сайдлоадинге через Recovery.
avbroot заменяет `/system/etc/security/otacerts.zip` в разделах системы и Recovery на новый архив, содержащий пользовательский сертификат подписи OTA. Это предотвращает случайную установку непропатченных OTA как из-под загруженной системы, так и при прошивке через Recovery.
Рекомендуется отключить приложение обновлений системы, чтобы оно не пыталось установить непропатченные OTA:
Рекомендуется отключить системное приложение для обновлений, чтобы оно не пыталось установить непропатченные OTA:
* Стоковая прошивка: Отключите `Автоматические обновления системы` (Automatic system updates в англ.) в настройках для разработчиков.
* Стоковая (заводская) прошивка: Отключите `Автоматические обновления системы` (Automatic system updates в англ.) в настройках для разработчиков.
* Кастомная прошивка: Отключите приложение обновлений системы (или запретите ему доступ к Интернету) через Настройки -> Приложения -> Все приложения -> (меню/три точки) -> Показать системные -> (найдите приложение обновлений, например Обновления системы/Updater).
Это особенно важно для некоторых кастомных прошивок, поскольку их фирменное приложение для обновления системы может уйти в бесконечный цикл, загружая OTA-обновление, а затем повторяя попытку загрузки и установки при неудачной проверке подписи.
@@ -384,20 +382,32 @@ avbroot можно использовать для простого перепо
### Пропуск патчинга сертификата OTA
avbroot может пропускать изменение `otacerts.zip` с помощью аргументов `--skip-system-ota-cert` и `--skip-recovery-ota-cert`. **Не используйте их без веской причины.** (Например, если вы уже самостоятельно встроили сертификат OTA в загрузочный (boot) образ и передаете его программе через опции `--prepatched` или `--replace`.)
В противном случае, на устройстве может не остаться возможности устанавливать дальнейшие обновления.
Вы можете пропустить изменение otacerts.zip, используя аргументы `--skip-system-ota-cert` и `--skip-recovery-ota-cert`. **Не используйте их без веской причины.**
При использовании `--skip-system-ota-cert`, никаких изменений в образ `system` не вносится.
При использовании `--skip-system-ota-cert`, сертификаты OTA в образе `system` изменены не будут. Это не позволит сторонним приложениям для OTA-обновлений устанавливать будущие пропатченные OTA из-под загруженной системы.
При использовании `--skip-recovery-ota-cert` совместно с `--rootless` и без указания `--dsu`, не вносится никаких изменений в загрузочные образы, кроме обеспечения их корректной подписи.
При использовании `--skip-recovery-ota-cert`, сертификаты OTA в образах `vendor_boot` или `recovery` изменены не будут. **Это не позволит устанавливать будущие пропатченные OTA в режиме Recovery.**
Если вы вручную добавили сертификат OTA в загрузочный (boot) образ, рекомендуем [предварительно проверить пропатченный OTA.](#проверка-ota)
Если вы используете аргумент `--skip-recovery-ota-cert`, потому что уже добавили сертификат OTA в загрузочный образ вручную, рекомендуетcя [проверить пропатченный OTA](#проверка-ota), дабы удостовериться, что замена произведена корректно. Процесс верификации проверяет только копию сертификатов OTA в загрузочном образе, не проверяя копию в образе системы.
### Подмена разделов
### Пропуск всех патчей
Чтобы внести самый минимум изменений, укажите аргументы:
* `--skip-system-ota-cert`
* `--skip-recovery-ota-cert`
* `--rootless`
* не используйте аргумент `--dsu`.
Так, пользовательскими ключами будут переподписаны лишь образ `vbmeta` и OTA, остальные разделы останутся нетронутыми.
**Это следует использовать только для устранения неполадок.** Без патчей сертификатов, поверх полученного OTA не получится установить никакие обновления.
avbroot поддерживает подмену целых образов в OTA, даже тех, что не являются загрузочными (например, `vendor_dlkm`). Образ можно заменить, указав аргумент `--replace <имя раздела> /путь/к/образу.img`.
### Подмена образов
Единственное, что меняется – это то, откуда считывается раздел. При использовании `--replace` вместо использования образа раздела из оригинального `payload.bin` в OTA, он берется напрямую по указанному вами пути. Таким образом, заменяющие образы разделов должны иметь правильные колонтитулы vbmeta, соответствующие оригинальным.
avbroot поддерживает подмену целых образов в OTA, даже тех, что не являются загрузочными (например, `vendor_dlkm`). Образ можно заменить, используя аргумент `--replace <имя раздела> /путь/к/образу.img`.
Единственное, что меняется – это то, откуда считывается файл. При использовании `--replace` вместо образа раздела из оригинального `payload.bin` в OTA, он берется напрямую по указанному вами пути. Заменяющие образы разделов должны иметь правильные колонтитулы vbmeta, соответствующие оригинальным.
Это не влияет на ход применения пачтей. Например, при использовании Magisk, патч получения root-прав применяется к загрузочному образу одинаково, независимо от того, был ли он получен из оригинального `payload.bin` или это файл, указанный через `--replace`.
@@ -411,6 +421,18 @@ Verified boot is disabled by vbmeta's header flags: 0x3
Чтобы принудительно включить AVB (очистив флаги), укажите аргумент `--clear-vbmeta-flags`.
### Изменение алгоритма CoW сжатия для вирутального A/B
Алгоритм CoW (copy-on-write) сжатия для виртуального A/B можно изменить, используя аргумент `--vabc-algo <алгоритм>`, указав `gz` или `lz4`. Как правило, по умолчанию OTA использует алгоритм, который совместим с изначальной версией Android, на которой поставлялось устройство.
* Девайсы, поставляемые с Android 12, поддерживают `gz` и `brotli` (последний не поддерживается avbroot)
* Девайсы, поставляемые с Android 14, поддерживают `lz4`
* Девайсы, поставляемые с Android 15, поддерживают `zstd` (не поддерживается avbroot)
Выбор быстрого алгоритма, такого как lz4, может значительно ускорить установку OTA из-под системы (при использованием стороннего приложения для OTA-обновлений). Однако, при установке OTA в режиме Recovery, разницы в скорости не будет.
Обратите внимание, что текущая используемая версия Android должна поддерживать выбранный алгоритм сжатия. В противном случае установка завершится ошибкой. Например, попытка установить OTA-обновление с Android 14, использующее алгоритм lz4, приведет к ошибке, если установка производится из-под Android 13.
### Использование в неинтерактивном режиме
По умолчанию avbroot интерактивно запрашивает пароли к приватным ключам. Чтобы запустить avbroot в неинтерактивном режиме, можно:
+11 -9
View File
@@ -13,7 +13,7 @@ anyhow = "1.0.75"
base64 = "0.22.1"
bitflags = { version = "2.4.1", features = ["serde"] }
bstr = "1.6.2"
bzip2 = { version = "0.5.1", default-features = false, features = ["libbz2-rs-sys"] }
bzip2 = "0.6.0"
cap-std = "3.0.0"
cap-tempfile = "3.0.0"
clap = { version = "4.4.1", features = ["derive"] }
@@ -28,15 +28,15 @@ dlv-list = "0.6.0"
flate2 = { version = "1.0.29", features = ["zlib-rs"] }
gf256 = { version = "0.3.0", features = ["rs"] }
hex = { version = "0.4.3", features = ["serde"] }
liblzma = "0.3.0"
liblzma = "0.4.1"
lz4_flex = "0.11.1"
memchr = "2.6.0"
num-bigint-dig = "0.8.4"
num-traits = "0.2.16"
passterm = "2.0.3"
phf = { version = "0.11.2", features = ["macros"] }
phf = { version = "0.12.1", features = ["macros"] }
pkcs8 = { version = "0.10.2", features = ["encryption", "pem"] }
prost = "0.13.1"
prost = "0.14.1"
# We can't upgrade to 0.9.0 until rsa updates its rand_core dependency.
rand = "0.8.5"
rayon = "1.7.0"
@@ -60,10 +60,12 @@ x509-cert = { version = "0.2.4", features = ["builder"] }
zerocopy = { version = "0.8.10", features = ["std"] }
zerocopy-derive = "0.8.5"
# https://github.com/zip-rs/zip/pull/383
# https://github.com/zip-rs/zip2/pull/367
# https://github.com/zip-rs/zip2/pull/368
# For getting the data offset when writing new zip entries.
[dependencies.zip]
git = "https://github.com/chenxiaolong/zip"
rev = "989101f9384b9e94e36e6e9e0f51908fdf98bde6"
git = "https://github.com/chenxiaolong/zip2"
rev = "59685f4dadbfee8cb3ea74c8fbb402b60d8137e8"
default-features = false
features = ["deflate"]
@@ -73,8 +75,8 @@ rustix = { version = "1.0.3", default-features = false, features = ["process"] }
[build-dependencies]
constcat = "0.6.0"
prost-build = "0.13.1"
protox = "0.7.0"
prost-build = "0.14.1"
protox = "0.9.0"
[dev-dependencies]
assert_matches = "1.5.0"
+1 -1
View File
@@ -10,7 +10,7 @@ use std::{
use anyhow::Result;
use clap::{Parser, Subcommand, ValueEnum};
use tracing::{debug, Level};
use tracing::{Level, debug};
use tracing_subscriber::fmt::{format::Writer, time::FormatTime};
use crate::cli::{avb, boot, completion, cpio, fec, hashtree, key, lp, ota, payload, sparse};
+3 -3
View File
@@ -10,7 +10,7 @@ use std::{
sync::atomic::AtomicBool,
};
use anyhow::{anyhow, bail, Context, Result};
use anyhow::{Context, Result, anyhow, bail};
use cap_std::{
ambient_authority,
fs::{Dir, OpenOptions},
@@ -19,7 +19,7 @@ use clap::{Args, Parser, Subcommand};
use rayon::prelude::{IntoParallelRefIterator, ParallelIterator};
use rsa::RsaPublicKey;
use serde::{Deserialize, Serialize};
use tracing::{debug_span, info, warn, Span};
use tracing::{Span, debug_span, info, warn};
use crate::{
crypto::{self, PassphraseSource, RsaSigningKey},
@@ -27,7 +27,7 @@ use crate::{
self, AlgorithmType, AppendedDescriptorMut, AppendedDescriptorRef, Descriptor, Footer,
HashTreeDescriptor, Header, KernelCmdlineDescriptor,
},
stream::{self, check_cancel, PSeekFile, ReadFixedSizeExt, Reopen, ToWriter},
stream::{self, PSeekFile, ReadFixedSizeExt, Reopen, ToWriter, check_cancel},
util,
};
+1 -1
View File
@@ -7,7 +7,7 @@ use std::{
path::{Path, PathBuf},
};
use anyhow::{bail, Context, Result};
use anyhow::{Context, Result, bail};
use clap::{Parser, Subcommand};
use crate::{
+1 -1
View File
@@ -9,7 +9,7 @@ use std::{
sync::atomic::AtomicBool,
};
use anyhow::{anyhow, Context, Result};
use anyhow::{Context, Result, anyhow};
use bstr::ByteSlice;
use cap_std::{ambient_authority, fs::Dir};
use clap::{Parser, Subcommand};
+1 -1
View File
@@ -9,7 +9,7 @@ use std::{
sync::atomic::AtomicBool,
};
use anyhow::{bail, Context, Result};
use anyhow::{Context, Result, bail};
use cap_std::{ambient_authority, fs::Dir};
use clap::{CommandFactory, Parser, Subcommand};
use rayon::iter::{
+88 -46
View File
@@ -10,20 +10,21 @@ use std::{
io::{self, BufReader, BufWriter, Read, Seek, SeekFrom, Write},
ops::Range,
path::{Path, PathBuf},
sync::{atomic::AtomicBool, Mutex},
str::FromStr,
sync::{Mutex, atomic::AtomicBool},
};
use anyhow::{anyhow, bail, Context, Result};
use anyhow::{Context, Result, anyhow, bail};
use bitflags::bitflags;
use cap_std::{ambient_authority, fs::Dir};
use cap_tempfile::TempDir;
use clap::{value_parser, ArgAction, Args, Parser, Subcommand, ValueEnum};
use clap::{ArgAction, Args, Parser, Subcommand, value_parser};
use rayon::{iter::IntoParallelRefIterator, prelude::ParallelIterator};
use tempfile::NamedTempFile;
use topological_sort::TopologicalSort;
use tracing::{debug_span, error, info, warn};
use x509_cert::Certificate;
use zip::{write::FileOptions, CompressionMethod, ZipArchive, ZipWriter};
use zip::{CompressionMethod, DateTime, ZipArchive, write::SimpleFileOptions};
use crate::{
cli,
@@ -32,7 +33,8 @@ use crate::{
avb::{self, Descriptor, Header},
ota::{self, SigningWriter, ZipEntry, ZipMode},
padding,
payload::{self, PayloadHeader, PayloadWriter, VabcAlgo},
payload::{self, CowVersion, PayloadHeader, PayloadWriter, VabcAlgo, VabcParams},
zip::ZipWriterWrapper,
},
patch::{
boot::{
@@ -272,7 +274,7 @@ fn patch_system_image<'a>(
};
if system_iter.next().is_some() {
bail!("Multiple system partitions found");
};
}
let _span = debug_span!("image", name = target).entered();
@@ -494,7 +496,9 @@ fn update_security_descriptors(
*pd = cd.clone();
}
_ => {
bail!("{child_name} descriptor ({child_type}) does not match entry in {parent_name} ({parent_type})");
bail!(
"{child_name} descriptor ({child_type}) does not match entry in {parent_name} ({parent_type})"
);
}
}
} else {
@@ -504,7 +508,9 @@ fn update_security_descriptors(
child_header.public_key.clone_into(&mut pd.public_key);
}
_ => {
bail!("{child_name} descriptor ({parent_type}) in {parent_name} must be a chain descriptor");
bail!(
"{child_name} descriptor ({parent_type}) in {parent_name} must be a chain descriptor"
);
}
}
}
@@ -576,9 +582,10 @@ fn update_metadata_descriptors(parent_header: &mut Header, child_header: &Header
}
}
/// Get the VABC algorithm from the payload header. This will fail if an
/// unsupported VABC algorithm is specified, but not if VABC is disabled.
fn get_vabc_algo(header: &PayloadHeader) -> Result<Option<VabcAlgo>> {
/// Get the VABC parameters from the payload header. This will fail if an
/// unsupported VABC algorithm or CoW version is specified, but not if VABC is
/// disabled.
fn get_vabc_params(header: &PayloadHeader) -> Result<Option<VabcParams>> {
// Only CoW v2 seems to exist in the wild currently, so that is all we
// support.
let Some(dpm) = &header.manifest.dynamic_partition_metadata else {
@@ -589,17 +596,32 @@ fn get_vabc_algo(header: &PayloadHeader) -> Result<Option<VabcAlgo>> {
return Ok(None);
}
let cow_version = dpm.cow_version();
if dpm.cow_version() != 2 {
bail!("Unsupported CoW version: {cow_version}");
}
let cow_version = match dpm.cow_version() {
2 => CowVersion::V2,
3 => CowVersion::V3,
v => bail!("Unsupported CoW version: {v}"),
};
let compression = dpm.vabc_compression_param();
let Ok(vabc_algo) = VabcAlgo::from_str(compression, false) else {
let Ok(vabc_algo) = VabcAlgo::from_str(compression) else {
bail!("Unsupported VABC compression: {compression}");
};
Ok(Some(vabc_algo))
// This is unused by v2, but delta_generator sets it anyway.
let Some(compression_factor) = dpm.compression_factor else {
bail!("No CoW compression factor specified");
};
let Ok(compression_factor) = u32::try_from(compression_factor) else {
bail!("CoW compression factor is too large: {compression_factor}");
};
let vabc_params = VabcParams {
version: cow_version,
algo: vabc_algo,
compression_factor,
};
Ok(Some(vabc_params))
}
/// Set the VABC algorithm in the payload header and return whether it was
@@ -615,7 +637,7 @@ fn set_vabc_algo(header: &mut PayloadHeader, vabc_algo: VabcAlgo) -> Result<bool
}
let compression = dpm.vabc_compression_param();
let Ok(old_vabc_algo) = VabcAlgo::from_str(compression, false) else {
let Ok(old_vabc_algo) = VabcAlgo::from_str(compression) else {
bail!("Unsupported VABC compression: {compression}");
};
@@ -731,7 +753,7 @@ pub fn compress_image(
.map(PSeekFile::new)
.with_context(|| format!("Failed to create temp file for: {name}"))?;
let vabc_algo = get_vabc_algo(header)?;
let vabc_params = get_vabc_params(header)?;
let block_size = header.manifest.block_size();
let partition = header
.manifest
@@ -742,14 +764,17 @@ pub fn compress_image(
// If VABC is enabled, we need to update the CoW size estimate or else the
// CoW block device may run out of space during flashing.
let vabc_algo = if partition.estimate_cow_size.is_some() {
let Some(vabc_algo) = vabc_algo else {
let vabc_params = if partition.estimate_cow_size.is_some() {
let Some(vabc_params) = vabc_params else {
bail!("Partition has CoW estimate, but VABC is disabled: {name}");
};
info!("Needs updated {vabc_algo} CoW size estimate: {name}");
info!(
"Needs updated {} CoW size estimate: {name}",
vabc_params.algo,
);
Some(vabc_algo)
Some(vabc_params)
} else {
None
};
@@ -770,16 +795,19 @@ pub fn compress_image(
// The changes we make usually aren't any less compressible, but
// we'll still recompute the CoW size estimate to handle the
// case where the user requested a different algorithm.
if let Some(vabc_algo) = vabc_algo {
if let Some(vabc_params) = vabc_params {
let cow_estimate = payload::compute_cow_estimate(
&*file,
partition.operations.len() as u64,
name,
block_size,
vabc_algo,
vabc_params,
cancel_signal,
)?;
partition.estimate_cow_size = Some(cow_estimate);
partition.estimate_cow_size = Some(cow_estimate.size);
partition.estimate_op_count_max =
(vabc_params.version == CowVersion::V3).then_some(cow_estimate.num_ops);
}
*file = writer;
@@ -797,12 +825,20 @@ pub fn compress_image(
info!("Compressing full image: {name}");
let (partition_info, operations, cow_estimate) =
payload::compress_image(&*file, &writer, name, block_size, vabc_algo, cancel_signal)?;
let (partition_info, operations, cow_estimate) = payload::compress_image(
&*file,
&writer,
name,
block_size,
vabc_params,
cancel_signal,
)?;
partition.new_partition_info = Some(partition_info);
partition.operations = operations;
partition.estimate_cow_size = cow_estimate;
partition.estimate_cow_size = cow_estimate.map(|e| e.size);
let is_v3 = vabc_params.is_some_and(|p| p.version == CowVersion::V3);
partition.estimate_op_count_max = cow_estimate.and_then(|e| is_v3.then_some(e.num_ops));
*file = writer;
@@ -822,7 +858,7 @@ fn recow_image(
file.rewind()?;
let vabc_algo = get_vabc_algo(header)?;
let vabc_params = get_vabc_params(header)?;
let block_size = header.manifest.block_size();
let partition = header
.manifest
@@ -833,18 +869,26 @@ fn recow_image(
if partition.estimate_cow_size.is_none() {
bail!("Partition has no original CoW estimate: {name}");
};
}
let Some(vabc_algo) = vabc_algo else {
let Some(vabc_params) = vabc_params else {
bail!("Partition has CoW estimate, but VABC is disabled: {name}");
};
info!("Recomputing {vabc_algo} CoW size estimate: {name}");
info!("Recomputing {} CoW size estimate: {name}", vabc_params.algo);
let cow_estimate =
payload::compute_cow_estimate(&*file, name, block_size, vabc_algo, cancel_signal)?;
let cow_estimate = payload::compute_cow_estimate(
&*file,
partition.operations.len() as u64,
name,
block_size,
vabc_params,
cancel_signal,
)?;
partition.estimate_cow_size = Some(cow_estimate);
partition.estimate_cow_size = Some(cow_estimate.size);
partition.estimate_op_count_max =
(vabc_params.version == CowVersion::V3).then_some(cow_estimate.num_ops);
Ok(())
}
@@ -1072,7 +1116,7 @@ fn patch_ota_payload(
fn patch_ota_zip(
raw_reader: &PSeekFile,
zip_reader: &mut ZipArchive<impl Read + Seek>,
mut zip_writer: &mut ZipWriter<impl Write>,
mut zip_writer: &mut ZipWriterWrapper<impl Write>,
external_images: &HashMap<String, PathBuf>,
boot_patchers: &[Box<dyn BootImagePatch + Sync>],
skip_system_ota_cert: bool,
@@ -1126,7 +1170,8 @@ fn patch_ota_zip(
// threshold. This should be sufficient since the output file is likely
// to be larger.
let use_zip64 = reader.size() >= 0xffffffff;
let options = FileOptions::default()
let options = SimpleFileOptions::default()
.last_modified_time(DateTime::default())
.compression_method(CompressionMethod::Stored)
.large_file(use_zip64);
@@ -1163,12 +1208,9 @@ fn patch_ota_zip(
}
// All remaining entries are written immediately.
zip_writer
.start_file_with_extra_data(path, options)
.with_context(|| format!("Failed to begin new zip entry: {path}"))?;
let offset = zip_writer
.end_extra_data()
.with_context(|| format!("Failed to end new zip entry: {path}"))?;
.start_file(path, options)
.with_context(|| format!("Failed to begin new zip entry: {path}"))?;
let mut writer = CountingWriter::new(&mut zip_writer);
match path.as_str() {
@@ -1467,7 +1509,7 @@ pub fn patch_subcommand(cli: &PatchCli, cancel_signal: &AtomicBool) -> Result<()
)));
} else {
assert!(cli.root.rootless);
};
}
if cli.skip_system_ota_cert {
warn!("Not inserting OTA cert into system image; sideloading further updates may fail");
@@ -1501,11 +1543,11 @@ pub fn patch_subcommand(cli: &PatchCli, cancel_signal: &AtomicBool) -> Result<()
let mut zip_writer = match cli.zip_mode {
ZipMode::Streaming => {
let signing_writer = SigningWriter::new_streaming(temp_writer);
ZipWriter::new_streaming(signing_writer)
ZipWriterWrapper::new_streaming(signing_writer)
}
ZipMode::Seekable => {
let signing_writer = SigningWriter::new_seekable(temp_writer);
ZipWriter::new(signing_writer)
ZipWriterWrapper::new_seekable(signing_writer)
}
};
+1 -1
View File
@@ -10,7 +10,7 @@ use std::{
sync::atomic::AtomicBool,
};
use anyhow::{anyhow, bail, Context, Result};
use anyhow::{Context, Result, anyhow, bail};
use cap_std::{ambient_authority, fs::Dir};
use clap::{Args, Parser, Subcommand};
use tracing::info;
+2 -2
View File
@@ -10,10 +10,10 @@ use std::{
sync::atomic::AtomicBool,
};
use anyhow::{anyhow, bail, Context, Result};
use anyhow::{Context, Result, anyhow, bail};
use clap::{Parser, Subcommand};
use crc32fast::Hasher;
use zerocopy::{little_endian, IntoBytes};
use zerocopy::{IntoBytes, little_endian};
use crate::{
format::{
+5 -4
View File
@@ -21,25 +21,25 @@ use cms::{
};
use passterm::PromptError;
use pkcs8::{
pkcs5::{pbes2, scrypt},
DecodePrivateKey, DecodePublicKey, EncodePrivateKey, EncodePublicKey, EncryptedPrivateKeyInfo,
LineEnding, PrivateKeyInfo,
pkcs5::{pbes2, scrypt},
};
use rand::RngCore;
use rsa::{
pkcs1v15::SigningKey, traits::PublicKeyParts, Pkcs1v15Sign, RsaPrivateKey, RsaPublicKey,
Pkcs1v15Sign, RsaPrivateKey, RsaPublicKey, pkcs1v15::SigningKey, traits::PublicKeyParts,
};
use serde::{Deserialize, Serialize};
use sha1::Sha1;
use sha2::{Digest, Sha256, Sha512};
use thiserror::Error;
use x509_cert::{
Certificate,
builder::{Builder, CertificateBuilder, Profile},
der::{pem::PemLabel, referenced::OwnedToRef, Any, Decode, DecodePem, EncodePem},
der::{Any, Decode, DecodePem, EncodePem, pem::PemLabel, referenced::OwnedToRef},
serial_number::SerialNumber,
spki::{AlgorithmIdentifierOwned, SubjectPublicKeyInfoOwned},
time::Validity,
Certificate,
};
use crate::util::DebugString;
@@ -144,6 +144,7 @@ pub enum PassphraseSource {
impl PassphraseSource {
pub fn new(key_file: &Path, pass_file: Option<&Path>, env_var: Option<&OsStr>) -> Self {
#[allow(clippy::option_if_let_else)]
if let Some(v) = env_var {
Self::EnvVar(v.to_owned())
} else if let Some(p) = pass_file {
+1 -1
View File
@@ -4,7 +4,7 @@
use std::{fmt, marker::PhantomData};
use bstr::{ByteSlice, ByteVec};
use serde::{de::Visitor, Deserializer, Serializer};
use serde::{Deserializer, Serializer, de::Visitor};
use thiserror::Error;
#[derive(Clone, Debug, Error)]
+2 -2
View File
@@ -14,10 +14,10 @@ use bstr::ByteSlice;
use num_bigint_dig::{ModInverse, ToBigInt};
use num_traits::{Pow, ToPrimitive};
use ring::digest::{Algorithm, Context};
use rsa::{traits::PublicKeyParts, BigUint, RsaPublicKey};
use rsa::{BigUint, RsaPublicKey, traits::PublicKeyParts};
use serde::{Deserialize, Serialize};
use thiserror::Error;
use zerocopy::{big_endian, FromBytes, IntoBytes};
use zerocopy::{FromBytes, IntoBytes, big_endian};
use zerocopy_derive::{FromBytes, Immutable, IntoBytes, KnownLayout, Unaligned};
use crate::{
+4 -11
View File
@@ -13,7 +13,7 @@ use num_traits::ToPrimitive;
use ring::digest::Context;
use serde::{Deserialize, Serialize};
use thiserror::Error;
use zerocopy::{little_endian, FromBytes, IntoBytes};
use zerocopy::{FromBytes, IntoBytes, little_endian};
use zerocopy_derive::{FromBytes, Immutable, IntoBytes, KnownLayout, Unaligned};
use crate::{
@@ -269,6 +269,7 @@ impl fmt::Display for BootImageV0Through2 {
impl BootImageExt for BootImageV0Through2 {
fn header_version(&self) -> u32 {
#[allow(clippy::bool_to_int_with_if)]
if self.v2_extra.is_some() {
2
} else if self.v1_extra.is_some() {
@@ -668,11 +669,7 @@ impl fmt::Display for BootImageV3Through4 {
impl BootImageExt for BootImageV3Through4 {
fn header_version(&self) -> u32 {
if self.v4_extra.is_some() {
4
} else {
3
}
if self.v4_extra.is_some() { 4 } else { 3 }
}
fn header_size(&self) -> u32 {
@@ -1093,11 +1090,7 @@ impl fmt::Display for VendorBootImageV3Through4 {
impl BootImageExt for VendorBootImageV3Through4 {
fn header_version(&self) -> u32 {
if self.v4_extra.is_some() {
4
} else {
3
}
if self.v4_extra.is_some() { 4 } else { 3 }
}
fn header_size(&self) -> u32 {
+1 -1
View File
@@ -3,7 +3,7 @@
use std::io::{self, Read, Seek, Write};
use flate2::{read::GzDecoder, write::GzEncoder, Compression};
use flate2::{Compression, read::GzDecoder, write::GzEncoder};
use liblzma::{
read::XzDecoder,
stream::{Check, Stream},
+1 -5
View File
@@ -771,11 +771,7 @@ pub fn sort(entries: &mut [CpioEntry]) {
/// 300000.
pub fn assign_inodes(entries: &mut [CpioEntry], missing_only: bool) -> Result<()> {
fn next_non_zero(i: u32) -> u32 {
if i == u32::MAX {
1
} else {
i.wrapping_add(1)
}
if i == u32::MAX { 1 } else { i.wrapping_add(1) }
}
// (dev maj, dev min) -> (inode set, last assigned inode)
+2 -2
View File
@@ -16,7 +16,7 @@ use rayon::{
slice::{ParallelSlice, ParallelSliceMut},
};
use thiserror::Error;
use zerocopy::{little_endian, FromBytes, IntoBytes};
use zerocopy::{FromBytes, IntoBytes, little_endian};
use zerocopy_derive::{FromBytes, Immutable, IntoBytes, KnownLayout, Unaligned};
use crate::{
@@ -827,7 +827,7 @@ impl<W: Write> ToWriter<W> for FecImage {
mod tests {
use std::{
io::{Cursor, Seek},
sync::{atomic::AtomicBool, Arc},
sync::{Arc, atomic::AtomicBool},
};
use assert_matches::assert_matches;
+1 -1
View File
@@ -16,7 +16,7 @@ use rayon::{
};
use ring::digest::{Algorithm, Context};
use thiserror::Error;
use zerocopy::{little_endian, FromBytes, IntoBytes};
use zerocopy::{FromBytes, IntoBytes, little_endian};
use zerocopy_derive::{FromBytes, Immutable, IntoBytes, KnownLayout, Unaligned};
use crate::{
+6 -6
View File
@@ -13,7 +13,7 @@ use bitflags::bitflags;
use bstr::ByteSlice;
use serde::{Deserialize, Serialize};
use thiserror::Error;
use zerocopy::{byteorder::little_endian, FromBytes, FromZeros, Immutable, IntoBytes};
use zerocopy::{FromBytes, FromZeros, Immutable, IntoBytes, byteorder::little_endian};
use zerocopy_derive::{FromBytes, Immutable, IntoBytes, KnownLayout, Unaligned};
use crate::{
@@ -21,7 +21,7 @@ use crate::{
stream::{
CountingReader, FromReader, ReadDiscardExt, ReadFixedSizeExt, ToWriter, WriteZerosExt,
},
util::{self, is_zero, DebugString},
util::{self, DebugString, is_zero},
};
/// Magic value for [`RawGeometry::magic`].
@@ -812,7 +812,7 @@ impl RawExtent {
return Err(Error::ExtentInvalidType {
index,
extent_type: n,
})
});
}
}
@@ -1640,7 +1640,7 @@ impl TryFrom<&RawMetadataSlot> for MetadataSlot {
type Error = Error;
fn try_from(raw_slot: &RawMetadataSlot) -> Result<Self> {
let mut slot = MetadataSlot {
let mut slot = Self {
major_version: raw_slot.header.major_version.get(),
minor_version: raw_slot.header.minor_version.get(),
groups: Vec::with_capacity(raw_slot.groups.len()),
@@ -1720,7 +1720,7 @@ impl TryFrom<&MetadataSlot> for RawMetadataSlot {
fn try_from(slot: &MetadataSlot) -> Result<Self> {
let header_size = RawHeader::size_for_version(slot.major_version, slot.minor_version);
let mut raw_slot = RawMetadataSlot {
let mut raw_slot = Self {
header: RawHeader {
magic: HEADER_MAGIC.into(),
major_version: slot.major_version.into(),
@@ -1937,7 +1937,7 @@ impl TryFrom<&Metadata> for RawMetadata {
// We only do the bare minimum calculations needed here to fill out the
// raw fields. There is no semantic validation.
let mut raw_metadata = RawMetadata {
let mut raw_metadata = Self {
image_type: metadata.image_type,
geometry: RawGeometry {
magic: GEOMETRY_MAGIC.into(),
+2 -1
View File
@@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: 2023 Andrew Gunnerson
// SPDX-FileCopyrightText: 2023-2025 Andrew Gunnerson
// SPDX-License-Identifier: GPL-3.0-only
pub mod avb;
@@ -13,3 +13,4 @@ pub mod padding;
pub mod payload;
pub mod sparse;
pub mod verityrs;
pub mod zip;
+30 -28
View File
@@ -12,18 +12,21 @@ use std::{
use clap::ValueEnum;
use cms::signed_data::SignedData;
use const_oid::{db::rfc5912, ObjectIdentifier};
use const_oid::{ObjectIdentifier, db::rfc5912};
use memchr::memmem;
use prost::Message;
use ring::digest::{Algorithm, Context};
use thiserror::Error;
use x509_cert::{der::Encode, Certificate};
use zip::{result::ZipError, write::FileOptions, CompressionMethod, ZipArchive, ZipWriter};
use x509_cert::{Certificate, der::Encode};
use zip::{CompressionMethod, DateTime, ZipArchive, result::ZipError, write::SimpleFileOptions};
use crate::{
crypto::{self, RsaPublicKeyExt, RsaSigningKey, SignatureAlgorithm},
format::payload::{self, PayloadHeader},
protobuf::build::tools::releasetools::{ota_metadata::OtaType, OtaMetadata},
format::{
payload::{self, PayloadHeader},
zip::ZipWriterWrapper,
},
protobuf::build::tools::releasetools::{OtaMetadata, ota_metadata::OtaType},
stream::{self, FromReader, HashingReader, HashingWriter, ReadFixedSizeExt},
};
@@ -323,6 +326,15 @@ fn compute_property_files(
max_length: Option<usize>,
want_pb: bool,
) -> Result<String> {
// AOSP's ota_utils.py reserves 15 bytes for the `<offset>:<size>`
// placeholder. Since the size of `metadata.pb` is almost always 4 digits,
// this prevents the offset from exceeding 10 digits. In the wild, there are
// OTA files larger than 10 GB. With ota_utils.py, this limit is never
// reached because it puts the OTA metadata files at the beginning of the
// zip. However, avbroot needs to put them at the end due to streaming
// writes, so we reserve an additional byte to allow offsets <100 GB.
const RESERVATION_SIZE: usize = 16;
let compute = |path: &'static str| -> Result<String> {
let entry = entries
.iter()
@@ -355,9 +367,9 @@ fn compute_property_files(
}
if max_length.is_none() {
tokens.push(format!("metadata:{}", " ".repeat(15)));
tokens.push(format!("metadata:{}", " ".repeat(RESERVATION_SIZE)));
if want_pb {
tokens.push(format!("metadata.pb:{}", " ".repeat(15)));
tokens.push(format!("metadata.pb:{}", " ".repeat(RESERVATION_SIZE)));
}
} else {
tokens.push(compute(PATH_METADATA)?);
@@ -377,7 +389,7 @@ fn compute_property_files(
}
let remain = l - joined.len();
joined.extend(iter::repeat(' ').take(remain));
joined.extend(iter::repeat_n(' ', remain));
}
Ok(joined)
@@ -426,14 +438,16 @@ impl fmt::Display for ZipMode {
/// directory would start.
pub fn add_metadata(
zip_entries: &[ZipEntry],
zip_writer: &mut ZipWriter<impl Write>,
zip_writer: &mut ZipWriterWrapper<impl Write>,
next_offset: u64,
metadata: &OtaMetadata,
payload_metadata_size: u64,
zip_mode: ZipMode,
) -> Result<OtaMetadata> {
let mut metadata = metadata.clone();
let options = FileOptions::default().compression_method(CompressionMethod::Stored);
let options = SimpleFileOptions::default()
.last_modified_time(DateTime::default())
.compression_method(CompressionMethod::Stored);
let mut zip_entries = zip_entries.to_owned();
add_payload_metadata_entry(&mut zip_entries, payload_metadata_size)?;
@@ -453,25 +467,19 @@ pub fn add_metadata(
let (legacy_raw, modern_raw) = serialize_metadata(&metadata);
let raw_writer = Cursor::new(Vec::new());
let mut writer = match zip_mode {
ZipMode::Streaming => ZipWriter::new_streaming(raw_writer),
ZipMode::Seekable => ZipWriter::new(raw_writer),
ZipMode::Streaming => ZipWriterWrapper::new_streaming(raw_writer),
ZipMode::Seekable => ZipWriterWrapper::new_seekable(raw_writer),
};
writer
.start_file_with_extra_data(PATH_METADATA, options)
.map_err(|e| Error::ZipEntryStart(PATH_METADATA, e))?;
let legacy_offset = writer
.end_extra_data()
.start_file(PATH_METADATA, options)
.map_err(|e| Error::ZipEntryStart(PATH_METADATA, e))?;
writer
.write_all(legacy_raw.as_bytes())
.map_err(|e| Error::ZipEntryWrite(PATH_METADATA, e))?;
writer
.start_file_with_extra_data(PATH_METADATA_PB, options)
.map_err(|e| Error::ZipEntryStart(PATH_METADATA_PB, e))?;
let modern_offset = writer
.end_extra_data()
.start_file(PATH_METADATA_PB, options)
.map_err(|e| Error::ZipEntryStart(PATH_METADATA_PB, e))?;
writer
.write_all(&modern_raw)
@@ -500,21 +508,15 @@ pub fn add_metadata(
{
let (legacy_raw, modern_raw) = serialize_metadata(&metadata);
zip_writer
.start_file_with_extra_data(PATH_METADATA, options)
.map_err(|e| Error::ZipEntryStart(PATH_METADATA, e))?;
let legacy_offset = zip_writer
.end_extra_data()
.start_file(PATH_METADATA, options)
.map_err(|e| Error::ZipEntryStart(PATH_METADATA, e))?;
zip_writer
.write_all(legacy_raw.as_bytes())
.map_err(|e| Error::ZipEntryWrite(PATH_METADATA, e))?;
zip_writer
.start_file_with_extra_data(PATH_METADATA_PB, options)
.map_err(|e| Error::ZipEntryStart(PATH_METADATA_PB, e))?;
let modern_offset = zip_writer
.end_extra_data()
.start_file(PATH_METADATA_PB, options)
.map_err(|e| Error::ZipEntryStart(PATH_METADATA_PB, e))?;
zip_writer
.write_all(&modern_raw)
+374 -96
View File
@@ -5,20 +5,22 @@ use std::{
collections::{HashMap, HashSet},
fmt,
io::{self, Cursor, Read, Seek, SeekFrom, Write},
ops::Range,
num::NonZeroU32,
ops::{Add, Range},
str::FromStr,
sync::atomic::AtomicBool,
};
use base64::engine::general_purpose::STANDARD;
use base64::Engine;
use base64::engine::general_purpose::STANDARD;
use bzip2::write::BzDecoder;
use clap::ValueEnum;
use flate2::{write::GzEncoder, Compression};
use flate2::{Compression, write::GzEncoder};
use liblzma::{
stream::{Check, Stream},
write::XzDecoder,
write::XzEncoder,
};
use num_traits::CheckedAdd;
use prost::Message;
use rayon::{
iter::{IndexedParallelIterator, IntoParallelRefMutIterator},
@@ -28,14 +30,14 @@ use ring::digest::{Context, Digest};
use serde::{Deserialize, Serialize};
use thiserror::Error;
use x509_cert::Certificate;
use zerocopy::{big_endian, FromBytes, IntoBytes};
use zerocopy::{FromBytes, IntoBytes, big_endian};
use zerocopy_derive::{FromBytes, Immutable, IntoBytes, KnownLayout, Unaligned};
use crate::{
crypto::{self, RsaPublicKeyExt, RsaSigningKey, SignatureAlgorithm},
protobuf::chromeos_update_engine::{
install_operation::Type, signatures::Signature, DeltaArchiveManifest, Extent,
InstallOperation, PartitionInfo, PartitionUpdate, Signatures,
DeltaArchiveManifest, Extent, InstallOperation, PartitionInfo, PartitionUpdate, Signatures,
install_operation::Type, signatures::Signature,
},
stream::{
self, CountingReader, FromReader, HashingWriter, ReadDiscardExt, ReadFixedSizeExt,
@@ -49,7 +51,9 @@ const PAYLOAD_VERSION: u64 = 2;
const MANIFEST_MAX_SIZE: usize = 4 * 1024 * 1024;
/// Size of each extent. This matches what AOSP's delta_generator does.
/// Size of each extent. This matches what AOSP's delta_generator does. We also
/// require this to be a multiple of the block size and a multiple of the
/// maximum CoW compression chunk size.
const CHUNK_SIZE: u64 = 2 * 1024 * 1024;
#[derive(Debug, Error)]
@@ -81,6 +85,10 @@ pub enum Error {
expected: Option<String>,
actual: String,
},
#[error("Invalid block size: {0}")]
InvalidBlockSize(u32),
#[error("Invalid maximum CoW compression chunk size: {0}")]
InvalidMaxCompressionChunkSize(u32),
#[error("Size of {name} ({size}) is not aligned to the block size ({block_size})")]
InvalidPartitionSize {
name: String,
@@ -113,7 +121,9 @@ pub enum Error {
DataWrite(&'static str, #[source] io::Error),
#[error("Expected {expected} bytes, but only wrote {actual} bytes")]
UnwrittenData { actual: u64, expected: u64 },
#[error("I/O error when applying {op_type:?} operation for {num_blocks} blocks starting at {start_block}")]
#[error(
"I/O error when applying {op_type:?} operation for {num_blocks} blocks starting at {start_block}"
)]
OperationApply {
op_type: Type,
start_block: u64,
@@ -926,6 +936,8 @@ pub fn extract_images<'a>(
.collect()
}
/// Compress raw data into a chunk to be used with a [`Type::ReplaceXz`]
/// [`InstallOperation`].
fn compress_chunk(raw_data: &[u8], cancel_signal: &AtomicBool) -> Result<(Vec<u8>, Digest)> {
let reader = Cursor::new(raw_data);
let writer = Cursor::new(Vec::new());
@@ -949,87 +961,303 @@ fn compress_chunk(raw_data: &[u8], cancel_signal: &AtomicBool) -> Result<(Vec<u8
Ok((data, digest_compressed))
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Deserialize, Serialize, ValueEnum)]
pub enum VabcAlgo {
#[derive(Clone, Copy, Debug, PartialEq, Eq, Deserialize, Serialize)]
pub enum CowVersion {
V2,
V3,
}
impl CowVersion {
/// Compute the size overhead required to store the headers and footers
/// needed for this version of the on-disk CoW format.
fn size_overhead(self, cow_replace_ops: u64, payload_install_ops: u64) -> u64 {
const BUFFER_REGION_DEFAULT_SIZE: u64 = 2 * 1024 * 1024;
const CLUSTER_OPS: u64 = 200;
const NUM_RESUME_POINTS: u64 = 4;
const SIZEOF_COW_FOOTER_V2: u64 = 84;
const SIZEOF_COW_HEADER_V2: u64 = 38;
const SIZEOF_COW_HEADER_V3: u64 = SIZEOF_COW_HEADER_V2 + 40;
const SIZEOF_COW_OPERATION_V2: u64 = 20;
const SIZEOF_COW_OPERATION_V3: u64 = 16;
const SIZEOF_RESUME_POINT_V3: u64 = 16;
let mut overhead = 0;
match self {
Self::V2 => {
// sizeof(CowHeader).
// AOSP: CowWriterV2::InitPos()
overhead += SIZEOF_COW_HEADER_V2;
// header_.buffer_size. update_engine uses the default value.
// AOSP: CowWriterV2::InitPos()
overhead += BUFFER_REGION_DEFAULT_SIZE;
// Add all the CoW operation headers:
//
// - There is a kCowReplaceOp for each compressed chunk.
// - There is a kCowLabelOp for each InstallOperation in the
// payload. This is added by delta_generator in CowDryRun().
// - There is a kCowClusterOp at the end of each cluster of
// operations (which includes the kCowClusterOp itself). The
// cluster size used to be 200, but was changed to 1024 in
// 5e8e488c13cbff9e0a305ce7c22fd6a13aabb886. We'll use the
// smaller value because it's better to overestimate.
//
// AOSP: CowWriterV2::EmitClusterIfNeeded()
let cow_label_ops = payload_install_ops;
let cow_cluster_ops = (cow_replace_ops + cow_label_ops).div_ceil(CLUSTER_OPS - 1);
// A cluster cannot be truncated, so round up to the nearest
// cluster boundary.
// AOSP: CowWriterV2::AddOperation()
overhead += cow_cluster_ops * CLUSTER_OPS * SIZEOF_COW_OPERATION_V2;
// sizeof(CowFooter).
// AOSP: CowWriterV2::GetCowSizeInfo()
overhead += SIZEOF_COW_FOOTER_V2;
}
Self::V3 => {
// AOSP: CowWriterV3::OpenForWrite() -> GetDataOffset()
overhead += SIZEOF_COW_HEADER_V3;
overhead += BUFFER_REGION_DEFAULT_SIZE;
overhead += NUM_RESUME_POINTS * SIZEOF_RESUME_POINT_V3;
// Add an operation header (sizeof(CowOperationV3)) for each
// chunk of compressed data.
// AOSP: CowWriterV3::WriteOperation()
overhead += cow_replace_ops * SIZEOF_COW_OPERATION_V3;
}
}
overhead
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum ChunkingMethod {
/// Compress data in block sized chunks each iteration.
Exact,
/// Compress data in chunks where each chunk is sized at the largest power
/// of 2 that's `<=` the specified size and the remaining input size.
MaxPowerOf2(NonZeroU32),
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
struct ChunkingParams {
block_size: u32,
method: ChunkingMethod,
}
impl ChunkingParams {
fn chunk_size(self, num_blocks: u64) -> u32 {
match self.method {
ChunkingMethod::Exact => self.block_size,
ChunkingMethod::MaxPowerOf2(max_chunk_size) => {
assert!(
max_chunk_size.is_power_of_two() && max_chunk_size.get() % self.block_size == 0
);
let mut chunk_size = max_chunk_size.get();
while chunk_size > self.block_size {
let min_blocks = chunk_size / self.block_size;
if num_blocks >= u64::from(min_blocks) {
return chunk_size;
}
chunk_size >>= 1;
}
self.block_size
}
}
}
}
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)]
pub struct CowEstimate {
/// Size of estimate in bytes.
pub size: u64,
/// Number of CoW operations (v3 only).
pub num_ops: u64,
}
impl CowEstimate {
/// Add fudge factor to account for overhead.
fn fudged(&self, payload_install_ops: u64, cow_version: CowVersion) -> Option<Self> {
let version_overhead = cow_version.size_overhead(self.num_ops, payload_install_ops);
let mut size = self.size.checked_add(version_overhead)?;
// delta_generator adds 1% overhead to the original CoW size estimate,
// even if compression is disabled. We'll do the same too. For the
// compressed scenario, we rely on this more because lz4_flex and
// zlib-rs usually compress better than the lz4 and zlib implementations
// used by libsnapshot_cow.
size += size / 100;
// AOSP: PartitionProcessor::Run()
let num_ops = self.num_ops.max(25);
Some(Self { size, num_ops })
}
}
impl Add for CowEstimate {
type Output = Self;
fn add(self, rhs: Self) -> Self::Output {
Self {
size: self.size + rhs.size,
num_ops: self.num_ops + rhs.num_ops,
}
}
}
impl CheckedAdd for CowEstimate {
fn checked_add(&self, rhs: &Self) -> Option<Self> {
Some(Self {
size: self.size.checked_add(rhs.size)?,
num_ops: self.num_ops.checked_add(rhs.num_ops)?,
})
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Deserialize, Serialize)]
pub enum VabcAlgoKind {
None,
Lz4,
Gz,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Deserialize, Serialize)]
pub struct VabcAlgo {
/// Compression algorithm.
pub kind: VabcAlgoKind,
/// Compression level. AOSP allows this to be present even if the algorithm
/// can't use it.
pub level: Option<u32>,
}
impl VabcAlgo {
fn compressed_size(self, mut raw_data: &[u8], block_size: u32) -> Result<u64> {
let mut total = 0;
/// Compute the compressed size of the raw data when split into chunks based
/// on the specified [`ChunkingParams`]. The length of `raw_data` must be a
/// multiple of the block size or else this will panic. The compressed data
/// for each chunk is temporarily stored in memory, but discarded after each
/// loop iteration.
fn compressed_size(self, mut raw_data: &[u8], chunking: ChunkingParams) -> Result<CowEstimate> {
assert!(raw_data.len() as u64 % u64::from(chunking.block_size) == 0);
let mut size = 0;
let mut num_ops = 0;
while !raw_data.is_empty() {
let n = raw_data.len().min(block_size as usize);
let (chunk, remaining) = raw_data.split_at(n);
let num_blocks = raw_data.len() as u64 / u64::from(chunking.block_size);
let chunk_size = chunking.chunk_size(num_blocks) as usize;
let (chunk, remaining) = raw_data.split_at(chunk_size);
// This should match CompressWorker::GetDefaultCompressionLevel() in
// AOSP's libsnapshot.
let compressed = match self {
Self::Lz4 => lz4_flex::block::compress(chunk),
Self::Gz => {
let mut encoder = GzEncoder::new(Vec::new(), Compression::best());
//
// CoW v3 uses the raw data instead of the compressed data if the
// raw data is smaller. Because we use a different implementation of
// the compression algorithms, we don't implement this. It's safer
// to just overestimate and use the (larger) compressed size.
size += match self.kind {
VabcAlgoKind::None => chunk_size as u64,
VabcAlgoKind::Lz4 => lz4_flex::block::compress(chunk).len() as u64,
VabcAlgoKind::Gz => {
let level = self.level.map_or(Compression::best(), Compression::new);
let mut encoder = GzEncoder::new(Vec::new(), level);
encoder.write_all(chunk).map_err(Error::GzCompress)?;
encoder.finish().map_err(Error::GzCompress)?
encoder.finish().map_err(Error::GzCompress)?.len() as u64
}
};
total += compressed.len().min(n) as u64;
num_ops += 1;
raw_data = remaining;
}
Ok(total)
Ok(CowEstimate { size, num_ops })
}
}
impl fmt::Display for VabcAlgo {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.write_str(self.to_possible_value().ok_or(fmt::Error)?.get_name())
let name = match self.kind {
VabcAlgoKind::None => "none",
VabcAlgoKind::Lz4 => "lz4",
VabcAlgoKind::Gz => "gz",
};
f.write_str(name)?;
if let Some(level) = self.level {
write!(f, ",{level}")?;
}
Ok(())
}
}
/// Add fudge factor to CoW estimate to account for overhead.
fn fudge_cow_estimate(mut estimate: u64) -> Option<u64> {
// lz4_flex and zlib-rs usually compress better than the lz4 and zlib
// implementations used by libsnapshot_cow. Make up for this by adding
// percentage-based overhead.
estimate = estimate.checked_add(estimate / 100)?;
#[derive(Clone, Debug, Error)]
#[error("Invalid VABC algorithm: {0:?} (must be {{none|lz4|gz}}[,<level>])")]
pub struct InvalidVabcAlgo(String);
// We also need to account for constant overhead, especially with smaller
// partitions. We can match what delta_generator normally adds in
// CowWriterV2::InitPos() exactly. Since we only ever create full OTAs, we
// can assume that all CoW operations are kCowReplaceOp.
impl FromStr for VabcAlgo {
type Err = InvalidVabcAlgo;
// sizeof(CowHeader).
estimate = estimate.checked_add(38)?;
// header_.buffer_size (equal to BUFFER_REGION_DEFAULT_SIZE).
estimate = estimate.checked_add(2 * 1024 * 1024)?;
// CowOptions::cluster_ops * sizeof(CowOperationV2).
estimate = estimate.checked_add(200 * 20)?;
fn from_str(s: &str) -> std::result::Result<Self, Self::Err> {
let (prefix, suffix) = s.split_once(',').unwrap_or((s, ""));
Some(estimate)
// AOSP allows any algorithm to accept a level, even if it's unused.
let level = if !suffix.is_empty() {
Some(suffix.parse().map_err(|_| InvalidVabcAlgo(s.to_owned()))?)
} else {
None
};
let kind = match prefix {
"" | "none" => VabcAlgoKind::None,
"lz4" => VabcAlgoKind::Lz4,
"gz" => VabcAlgoKind::Gz,
_ => return Err(InvalidVabcAlgo(s.to_owned())),
};
Ok(Self { kind, level })
}
}
/// Compute the VABC CoW v2 size estimate. The caller must update
/// [`PartitionUpdate::estimate_cow_size`] with this value or else update_engine
/// may fail to flash the partition due to running out of space on the CoW block
/// device. CoW v2 + other algorithms and also CoW v3 are currently unsupported
/// because there currently are no known OTAs that use those configurations.
pub fn compute_cow_estimate(
input: &(dyn ReadSeekReopen + Sync),
partition_name: &str,
block_size: u32,
vabc_algo: VabcAlgo,
cancel_signal: &AtomicBool,
) -> Result<u64> {
let file_size = input
.reopen_boxed()
.and_then(|mut r| r.seek(SeekFrom::End(0)))
.map_err(|e| Error::InputOpen(partition_name.to_owned(), e))?;
let final_chunk_different = file_size % CHUNK_SIZE != 0;
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub struct VabcParams {
/// CoW on-disk format version.
pub version: CowVersion,
/// CoW compression algorithm.
pub algo: VabcAlgo,
/// The maximum number of bytes to compress at a time.
pub compression_factor: u32,
}
if file_size % u64::from(block_size) != 0 || CHUNK_SIZE % u64::from(block_size) != 0 {
/// Ensure that the partition size is aligned to the block size and that the
/// block size and compression factor are factors of our [`CHUNK_SIZE`].
fn validate_partition_size(
partition_name: &str,
file_size: u64,
block_size: u32,
compression_factor: u32,
) -> Result<()> {
if block_size == 0 || !block_size.is_power_of_two() || CHUNK_SIZE % u64::from(block_size) != 0 {
return Err(Error::InvalidBlockSize(block_size));
}
if compression_factor == 0
|| !compression_factor.is_power_of_two()
|| CHUNK_SIZE % u64::from(compression_factor) != 0
{
return Err(Error::InvalidMaxCompressionChunkSize(compression_factor));
}
if file_size % u64::from(block_size) != 0 {
return Err(Error::InvalidPartitionSize {
name: partition_name.to_owned(),
size: file_size,
@@ -1037,11 +1265,51 @@ pub fn compute_cow_estimate(
});
}
Ok(())
}
/// Compute the VABC CoW size estimate. For a more accurate size estimate with
/// CoW version 2, `payload_install_ops` must be equal to the number of
/// [`InstallOperation`]s in the payload. The caller must update
/// [`PartitionUpdate::estimate_cow_size`] and
/// [`PartitionUpdate::estimate_op_count_max`] or else update_engine may fail to
/// flash the partition due to running out of space on the CoW block device.
pub fn compute_cow_estimate(
input: &(dyn ReadSeekReopen + Sync),
payload_install_ops: u64,
partition_name: &str,
block_size: u32,
vabc_params: VabcParams,
cancel_signal: &AtomicBool,
) -> Result<CowEstimate> {
let file_size = input
.reopen_boxed()
.and_then(|mut r| r.seek(SeekFrom::End(0)))
.map_err(|e| Error::InputOpen(partition_name.to_owned(), e))?;
let final_chunk_different = file_size % CHUNK_SIZE != 0;
validate_partition_size(
partition_name,
file_size,
block_size,
vabc_params.compression_factor,
)?;
let chunking = ChunkingParams {
block_size,
method: match vabc_params.version {
CowVersion::V2 => ChunkingMethod::Exact,
CowVersion::V3 => {
ChunkingMethod::MaxPowerOf2(vabc_params.compression_factor.try_into().unwrap())
}
},
};
let chunks_total = file_size.div_ceil(CHUNK_SIZE);
let cow_estimate = (0..chunks_total)
let initial_estimate = (0..chunks_total)
.into_par_iter()
.map(|chunk| -> Result<u64> {
.map(|chunk| -> Result<CowEstimate> {
let data = (|| {
let mut reader = input.reopen_boxed()?;
reader.seek(SeekFrom::Start(chunk * CHUNK_SIZE))?;
@@ -1057,26 +1325,25 @@ pub fn compute_cow_estimate(
})()
.map_err(Error::ChunkRead)?;
vabc_algo.compressed_size(&data, block_size)
vabc_params.algo.compressed_size(&data, chunking)
})
.try_fold(
|| 0u64,
|total, chunk_estimate| -> Result<u64> {
CowEstimate::default,
|total, chunk_estimate| -> Result<CowEstimate> {
total
.checked_add(chunk_estimate?)
.ok_or(Error::IntOverflow("cow_estimate"))
.checked_add(&chunk_estimate?)
.ok_or(Error::IntOverflow("initial_estimate"))
},
)
.try_reduce(
|| 0u64,
|total, partial| {
total
.checked_add(partial)
.ok_or(Error::IntOverflow("cow_estimate"))
},
)?;
.try_reduce(CowEstimate::default, |total, partial| {
total
.checked_add(&partial)
.ok_or(Error::IntOverflow("initial_estimate"))
})?;
fudge_cow_estimate(cow_estimate).ok_or(Error::IntOverflow("cow_estimate_fudged"))
initial_estimate
.fudged(payload_install_ops, vabc_params.version)
.ok_or(Error::IntOverflow("fudged_estimate"))
}
/// Compress the image and return the corresponding information to insert into
@@ -1095,9 +1362,9 @@ pub fn compress_image(
output: &(dyn WriteSeekReopen + Sync),
partition_name: &str,
block_size: u32,
vabc_algo: Option<VabcAlgo>,
vabc_params: Option<VabcParams>,
cancel_signal: &AtomicBool,
) -> Result<(PartitionInfo, Vec<InstallOperation>, Option<u64>)> {
) -> Result<(PartitionInfo, Vec<InstallOperation>, Option<CowEstimate>)> {
const CHUNK_GROUP: u64 = 32;
let file_size = input
@@ -1106,18 +1373,23 @@ pub fn compress_image(
.map_err(|e| Error::InputOpen(partition_name.to_owned(), e))?;
let final_chunk_different = file_size % CHUNK_SIZE != 0;
if file_size % u64::from(block_size) != 0 || CHUNK_SIZE % u64::from(block_size) != 0 {
return Err(Error::InvalidPartitionSize {
name: partition_name.to_owned(),
size: file_size,
block_size,
});
}
let compression_factor = vabc_params.map_or(block_size, |p| p.compression_factor);
validate_partition_size(partition_name, file_size, block_size, compression_factor)?;
let chunking = ChunkingParams {
block_size,
method: match vabc_params.map(|p| p.version) {
Some(CowVersion::V3) => {
ChunkingMethod::MaxPowerOf2(compression_factor.try_into().unwrap())
}
_ => ChunkingMethod::Exact,
},
};
let chunks_total = file_size.div_ceil(CHUNK_SIZE);
let mut bytes_compressed = 0u64;
let mut context_uncompressed = Context::new(&ring::digest::SHA256);
let mut cow_estimate = 0u64;
let mut initial_estimate = CowEstimate::default();
let mut operations = vec![];
// Read the file one group at a time. This allows for some parallelization
@@ -1154,12 +1426,12 @@ pub fn compress_image(
let mut compressed_data_group = uncompressed_data_group
.into_par_iter()
.map(
|(raw_offset, raw_data)| -> Result<(Vec<u8>, InstallOperation, u64)> {
|(raw_offset, raw_data)| -> Result<(Vec<u8>, InstallOperation, CowEstimate)> {
let (data, digest_compressed) = compress_chunk(&raw_data, cancel_signal)?;
let cow_size = vabc_algo
.map(|a| a.compressed_size(&raw_data, block_size))
let cow_estimate = vabc_params
.map(|p| p.algo.compressed_size(&raw_data, chunking))
.transpose()?
.unwrap_or(0);
.unwrap_or_default();
let extent = Extent {
start_block: Some(raw_offset / u64::from(block_size)),
@@ -1172,19 +1444,19 @@ pub fn compress_image(
operation.dst_extents.push(extent);
operation.data_sha256_hash = Some(digest_compressed.as_ref().to_vec());
Ok((data, operation, cow_size))
Ok((data, operation, cow_estimate))
},
)
.collect::<Result<Vec<_>>>()?;
for (data, operation, cow_size) in &mut compressed_data_group {
for (data, operation, cow_estimate) in &mut compressed_data_group {
operation.data_offset = Some(bytes_compressed);
bytes_compressed = bytes_compressed
.checked_add(data.len() as u64)
.ok_or(Error::IntOverflow("bytes_compressed"))?;
cow_estimate = cow_estimate
.checked_add(*cow_size)
.ok_or(Error::IntOverflow("cow_estimate"))?;
initial_estimate = initial_estimate
.checked_add(cow_estimate)
.ok_or(Error::IntOverflow("initial_estimate"))?;
}
let group_operations = compressed_data_group
@@ -1208,8 +1480,12 @@ pub fn compress_image(
hash: Some(digest_uncompressed.as_ref().to_vec()),
};
let cow_estimate = if vabc_algo.is_some() {
Some(fudge_cow_estimate(cow_estimate).ok_or(Error::IntOverflow("cow_estimate_fudged"))?)
let cow_estimate = if let Some(p) = vabc_params {
Some(
initial_estimate
.fudged(operations.len() as u64, p.version)
.ok_or(Error::IntOverflow("fudged_estimate"))?,
)
} else {
None
};
@@ -1356,6 +1632,8 @@ pub fn compress_modified_image(
writer.seek(SeekFrom::Start(operation.data_offset.unwrap()))?;
writer.write_all(&data)?;
// Clippy doesn't know we're returning a Range.
#[allow(clippy::range_plus_one)]
Ok(i..i + 1)
})
.collect::<io::Result<Vec<_>>>()
+5 -5
View File
@@ -11,7 +11,7 @@ use std::{
use crc32fast::Hasher;
use dlv_list::{Index, VecList};
use thiserror::Error;
use zerocopy::{byteorder::little_endian, FromBytes, IntoBytes};
use zerocopy::{FromBytes, IntoBytes, byteorder::little_endian};
use zerocopy_derive::{FromBytes, Immutable, IntoBytes, KnownLayout, Unaligned};
use crate::stream::ReadDiscardExt;
@@ -159,7 +159,7 @@ impl RawHeader {
return Err(Error::UnsupportedMajorVersion(self.major_version.get()));
}
if self.file_hdr_sz.get() < mem::size_of::<RawHeader>() as u16 {
if self.file_hdr_sz.get() < mem::size_of::<Self>() as u16 {
return Err(Error::InvalidFileHeaderSize(self.file_hdr_sz.get()));
} else if self.chunk_hdr_sz.get() < mem::size_of::<RawChunk>() as u16 {
return Err(Error::InvalidChunkHeaderSize(self.chunk_hdr_sz.get()));
@@ -173,7 +173,7 @@ impl RawHeader {
}
fn excess_raw_header_bytes(&self) -> u16 {
self.file_hdr_sz.get() - mem::size_of::<RawHeader>() as u16
self.file_hdr_sz.get() - mem::size_of::<Self>() as u16
}
fn excess_raw_chunk_bytes(&self) -> u16 {
@@ -225,7 +225,7 @@ impl RawChunk {
return Err(Error::InvalidChunkType {
index,
chunk_type: t,
})
});
}
};
@@ -811,7 +811,7 @@ impl<R: Read> SparseReader<R> {
data = ChunkData::Crc32(expected.get());
}
_ => unreachable!(),
};
}
let chunk = Chunk {
bounds: ChunkBounds {
+103
View File
@@ -0,0 +1,103 @@
// SPDX-FileCopyrightText: 2025 Andrew Gunnerson
// SPDX-License-Identifier: GPL-3.0-only
use std::io::{self, Seek, SeekFrom, Write};
use zip::{
ZipWriter,
result::ZipResult,
write::{FileOptionExtension, FileOptions, StreamWriter},
};
/// A wrapper around a seekable writer. `W` must implement [`Seek`], but only
/// during the creation of a new instance. The resulting type can be stored in a
/// parent container where the generic type does not implement [`Seek`].
pub struct SeekWriter<W: Write> {
inner: W,
seek_fn: fn(&mut W, SeekFrom) -> io::Result<u64>,
}
impl<W: Write> SeekWriter<W> {
pub fn into_inner(self) -> W {
self.inner
}
}
impl<W: Write + Seek> SeekWriter<W> {
pub fn new(inner: W) -> Self {
Self {
inner,
seek_fn: W::seek,
}
}
}
impl<W: Write> Write for SeekWriter<W> {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.inner.write(buf)
}
fn flush(&mut self) -> io::Result<()> {
self.inner.flush()
}
}
impl<W: Write> Seek for SeekWriter<W> {
fn seek(&mut self, pos: SeekFrom) -> io::Result<u64> {
(self.seek_fn)(&mut self.inner, pos)
}
}
/// This is an ugly hack to have a single type represent both seekable and
/// streaming [`ZipWriter`]s. `W` only needs to implement [`Seek`] when creating
/// a seekable instance via [`Self::new_seekable`].
pub enum ZipWriterWrapper<W: Write> {
Streaming(ZipWriter<StreamWriter<W>>),
Seekable(ZipWriter<SeekWriter<W>>),
}
impl<W: Write + Seek> ZipWriterWrapper<W> {
pub fn new_seekable(inner: W) -> Self {
Self::Seekable(ZipWriter::new(SeekWriter::new(inner)))
}
}
impl<W: Write> ZipWriterWrapper<W> {
pub fn new_streaming(inner: W) -> Self {
Self::Streaming(ZipWriter::new_stream(inner))
}
pub fn start_file(
&mut self,
name: impl ToString,
options: FileOptions<impl FileOptionExtension>,
) -> ZipResult<u64> {
match self {
Self::Streaming(z) => z.start_file(name, options),
Self::Seekable(z) => z.start_file(name, options),
}
}
pub fn finish(self) -> ZipResult<W> {
match self {
Self::Streaming(z) => Ok(z.finish()?.into_inner()),
Self::Seekable(z) => Ok(z.finish()?.into_inner()),
}
}
}
impl<W: Write> Write for ZipWriterWrapper<W> {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
match self {
Self::Streaming(z) => z.write(buf),
Self::Seekable(z) => z.write(buf),
}
}
fn flush(&mut self) -> io::Result<()> {
match self {
Self::Streaming(z) => z.flush(),
Self::Seekable(z) => z.flush(),
}
}
}
+1 -1
View File
@@ -4,8 +4,8 @@
use std::{
process::ExitCode,
sync::{
atomic::{AtomicBool, Ordering},
Arc,
atomic::{AtomicBool, Ordering},
},
};
+1 -1
View File
@@ -10,7 +10,7 @@ use std::{
};
use num_traits::{Num, PrimInt};
use serde::{de::Visitor, Deserializer, Serializer};
use serde::{Deserializer, Serializer, de::Visitor};
pub fn serialize<S, T>(data: &T, serializer: S) -> Result<S::Ok, S::Error>
where
+6 -6
View File
@@ -24,9 +24,9 @@ use regex::bytes::Regex;
use ring::digest::Context;
use rsa::RsaPublicKey;
use thiserror::Error;
use tracing::{debug, debug_span, trace, warn, Span};
use tracing::{Span, debug, debug_span, trace, warn};
use x509_cert::Certificate;
use zip::{result::ZipError, ZipArchive};
use zip::{ZipArchive, result::ZipError};
use crate::{
crypto::{self, RsaSigningKey},
@@ -194,7 +194,7 @@ impl MagiskRootPatcher {
// replaced by PREINITDEVICE
// - Versions newer than the latest supported version are assumed to support
// the same features as the latest version
const VERS_SUPPORTED: &'static [Range<u32>] = &[25102..25207, 25211..28200];
const VERS_SUPPORTED: &'static [Range<u32>] = &[25102..25207, 25211..30100];
const VER_PREINIT_DEVICE: RangeFrom<u32> = 25211..;
const VER_RANDOM_SEED: Range<u32> = 25211..26103;
const VER_PATCH_VBMETA: Range<u32> = Self::VERS_SUPPORTED[0].start..26202;
@@ -432,7 +432,7 @@ impl BootImagePatch for MagiskRootPatcher {
targets.push("init_boot");
} else if boot_images.contains_key("boot") {
targets.push("boot");
};
}
Ok(targets)
}
@@ -781,7 +781,7 @@ impl DsuPubKeyPatcher {
e.data = data;
} else {
entries.push(CpioEntry::new_file(Self::AVBROOT_KEY_PATH, 0o644, data));
};
}
*ramdisk = save_ramdisk(&entries, ramdisk_format, cancel_signal)?;
@@ -971,7 +971,7 @@ impl BootImagePatch for PrepatchedImagePatcher {
targets.push("init_boot");
} else if boot_images.contains_key("boot") {
targets.push("boot");
};
}
Ok(targets)
}
+5 -3
View File
@@ -6,8 +6,8 @@ use std::{borrow::Cow, cmp::Ordering, io::Cursor, path::Path};
use bitflags::bitflags;
use thiserror::Error;
use tracing::trace;
use x509_cert::{der::asn1::BitString, Certificate};
use zip::{result::ZipError, write::FileOptions, CompressionMethod, ZipWriter};
use x509_cert::{Certificate, der::asn1::BitString};
use zip::{CompressionMethod, DateTime, ZipWriter, result::ZipError, write::SimpleFileOptions};
use crate::{crypto, format::ota};
@@ -79,7 +79,9 @@ pub fn create_zip(cert: &Certificate, flags: OtaCertBuildFlags) -> Result<Vec<u8
CompressionMethod::Stored
};
let options = FileOptions::default().compression_method(compression_method);
let options = SimpleFileOptions::default()
.last_modified_time(DateTime::default())
.compression_method(compression_method);
let name = "ota.x509.pem";
writer.start_file(name, options).map_err(Error::ZipWrite)?;
+1 -1
View File
@@ -10,7 +10,7 @@ use std::{
use memchr::memmem;
use rayon::iter::{IntoParallelIterator, ParallelIterator};
use thiserror::Error;
use tracing::{debug, debug_span, trace, Span};
use tracing::{Span, debug, debug_span, trace};
use x509_cert::Certificate;
use zip::ZipArchive;
+1
View File
@@ -1,3 +1,4 @@
#![allow(clippy::all)]
#![allow(clippy::nursery)]
#![allow(clippy::pedantic)]
+1 -1
View File
@@ -5,8 +5,8 @@ use std::{
fs::File,
io::{self, BufReader, BufWriter, Cursor, Read, Seek, SeekFrom, Write},
sync::{
atomic::{AtomicBool, Ordering},
Arc, Mutex, RwLock,
atomic::{AtomicBool, Ordering},
},
};
+1 -1
View File
@@ -69,5 +69,5 @@ bypass = [
unknown-registry = "deny"
unknown-git = "deny"
allow-git = [
"https://github.com/chenxiaolong/zip",
"https://github.com/chenxiaolong/zip2",
]
+5 -3
View File
@@ -24,10 +24,12 @@ tracing = "0.1.40"
tracing-subscriber = "0.3.18"
x509-cert = "0.2.5"
# https://github.com/zip-rs/zip/pull/383
# https://github.com/zip-rs/zip2/pull/367
# https://github.com/zip-rs/zip2/pull/368
# For getting the data offset when writing new zip entries.
[dependencies.zip]
git = "https://github.com/chenxiaolong/zip"
rev = "989101f9384b9e94e36e6e9e0f51908fdf98bde6"
git = "https://github.com/chenxiaolong/zip2"
rev = "59685f4dadbfee8cb3ea74c8fbb402b60d8137e8"
default-features = false
[features]
+26 -25
View File
@@ -12,8 +12,10 @@ security_patch_level = "2024-01-01"
# Google Pixel 7 Pro
# What's unique: init_boot (boot v4) + vendor_boot (vendor v4)
[profile.pixel_v4_gki]
vabc_algo = "Lz4"
[profile.pixel_v4_gki.vabc]
# CoW v3 is used starting with the Google Pixel 9a.
version = "V3"
algo = { kind = "Lz4" }
[profile.pixel_v4_gki.partitions.boot]
avb.signed = true
@@ -49,18 +51,19 @@ data.version = "vendor_v4"
data.ramdisks = [["otacerts", "first_stage", "dsu_key_dir"]]
[profile.pixel_v4_gki.hashes_streaming]
original = "c00f891f941f3dddb28966f7b07f3acea773bee104dace82b37c2d1341f09422"
patched = "6c27ffb07f4497af8539f8283e506066af9417580230c3209c9875fc15d5069d"
original = "ef6261cd9ebea90f036e52a46160a400c5b8f6ef24ed2469c4a1e9689987aa06"
patched = "37fd353a766a7b9a339fbf51fa79c703e94640dc6a2c6310d79357aaefcc7ca1"
[profile.pixel_v4_gki.hashes_seekable]
original = "96a6c366b5de1c3b10d4d6cb4ca503c83ac4cd9ca952a965cceb041990ba7022"
patched = "e4fc12523ffc312796b92210bc1e3bbb70dd60797a47daae76c8b5852e48b382"
original = "8a2c717607c10dfa5483d6f9a9f37b3d978acaf8d2ea18e36544af267943e750"
patched = "2c4734c9e1d028ee6aaf02bb416e5e173857faffd2ca067366790655147b3afa"
# Google Pixel 6a
# What's unique: boot (boot v4, no ramdisk) + vendor_boot (vendor v4, 2 ramdisks)
[profile.pixel_v4_non_gki]
vabc_algo = "Lz4"
[profile.pixel_v4_non_gki.vabc]
version = "V2"
algo = { kind = "Lz4" }
[profile.pixel_v4_non_gki.partitions.boot]
avb.signed = true
@@ -90,18 +93,19 @@ data.version = "vendor_v4"
data.ramdisks = [["init", "otacerts", "first_stage", "dsu_key_dir"], ["dlkm"]]
[profile.pixel_v4_non_gki.hashes_streaming]
original = "4d692bc777b568b0626d3c08d2e6f83f1b472db5ad903486daaec6a78d0cc26e"
patched = "6832ded3e98a14edc8c5ea7284fcea0b958fa710ebf222c27116faec8dfefe2e"
original = "630220ef813a2b4743d1941179cc9705da86ad4805f1c52341dcb38fbce3d29e"
patched = "b725e91751fe58aed20495aecbf9b4bdc14d2799cd88dcbd58f3a3b02b3af15b"
[profile.pixel_v4_non_gki.hashes_seekable]
original = "ea27ecd9718c17b63400b2548680bb3cee93ce63b4fc44ff9654ca0d9c5372a8"
patched = "114f8936e917d7e4a71bc1521adb3c8e676de3a738f8c7c505b86464d20bd95c"
original = "1afbe6867ded345d941098ee7c7fcf94a3df52c50ff96ab8f3a67b2ab957259a"
patched = "4357b977249006b101002c961916f962787315a80b8608494c6a1f0cf09cecd1"
# Google Pixel 4a 5G
# What's unique: boot (boot v3) + vendor_boot (vendor v3)
[profile.pixel_v3]
vabc_algo = "Lz4"
[profile.pixel_v3.vabc]
version = "V2"
algo = { kind = "Gz" }
[profile.pixel_v3.partitions.boot]
avb.signed = true
@@ -132,19 +136,16 @@ data.version = "vendor_v3"
data.ramdisks = [["otacerts", "first_stage", "dsu_key_dir"]]
[profile.pixel_v3.hashes_streaming]
original = "f432dc7931520feb238474aa707dd5299747562ffe6129f3f763b5f11ac473ab"
patched = "1f28d9210a17e233cd5da4af55b07db764b19eeab991394514170b405240464f"
original = "9b65037343d45211e0f9706929cba34643a9c54274d1b39740c43f45974984e0"
patched = "fb23ab9616968b38b96d1e5e6a503154f89aebc1741e89a9e9dfd2c4d9946b05"
[profile.pixel_v3.hashes_seekable]
original = "7d29ecc6780953c22052a576b8dc85066c8667a875e918a786a08ff4545b47d1"
patched = "27b80c7be9c1e527ea26abe3dabde245c580e6f26ec084204278fbfd81a39f83"
original = "e581934887dd93b8a9d9c3aa5dec1d48aa7e01bf01ac507e8c5fb256b59cbe7d"
patched = "669a826abc6d67e7e0b1def724aa7b470461087255c65663d195df1426a355f0"
# Google Pixel 4a
# What's unique: boot (boot v2)
[profile.pixel_v2]
vabc_algo = "Gz"
[profile.pixel_v2.partitions.boot]
avb.signed = false
data.type = "boot"
@@ -168,9 +169,9 @@ data.type = "vbmeta"
data.deps = ["system"]
[profile.pixel_v2.hashes_streaming]
original = "bd2f19cf3d2285e35e8b36d44f75ed910e8e0be44c3ebd29f17a812521ba754b"
patched = "cf65d5b90500af54cd1204a646379bb852825061bc7c3f973b7a042f353f75ad"
original = "f10ee15c900a474cc6bbefa705f272cef42636ea096e75563d2d78f6c4327fd1"
patched = "6929f65909037f5550a53982b71e96bdf69ab876bc5e86702c469ed601be8a9a"
[profile.pixel_v2.hashes_seekable]
original = "7f96ebf7366e0b60c91ac1e5f196a2189ffdb0bbc73f77804a736466fcab7315"
patched = "c2d9d60d73c038da39f82073ffadb459c96b901d66db7af11f59da58e0dd53e4"
original = "4e863d251b9ff6eaa1511f9c03e9bdb8919650b2e0eaf23e33892a639edafcaf"
patched = "9a103222e73df70a097281525546d25c850df2ae7a2ba715aa5dfbbba3f7972b"
+21 -14
View File
@@ -1,14 +1,14 @@
// SPDX-FileCopyrightText: 2023-2024 Andrew Gunnerson
// SPDX-FileCopyrightText: 2023-2025 Andrew Gunnerson
// SPDX-License-Identifier: GPL-3.0-only
use std::{collections::BTreeMap, fs, path::Path};
use anyhow::{Context, Result};
use avbroot::format::payload::VabcAlgo;
use avbroot::format::payload::{CowVersion, VabcAlgo};
use serde::{Deserialize, Serialize};
use toml_edit::DocumentMut;
#[derive(Serialize, Deserialize)]
#[derive(Clone, Copy, Serialize, Deserialize)]
pub struct Sha256Hash(
#[serde(
serialize_with = "hex::serialize",
@@ -17,7 +17,7 @@ pub struct Sha256Hash(
pub [u8; 32],
);
#[derive(Serialize, Deserialize)]
#[derive(Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct OtaInfo {
pub device: String,
@@ -29,7 +29,7 @@ pub struct OtaInfo {
pub security_patch_level: String,
}
#[derive(Serialize, Deserialize)]
#[derive(Clone, Copy, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Avb {
pub signed: bool,
@@ -55,7 +55,7 @@ pub enum BootVersion {
VendorV4,
}
#[derive(Serialize, Deserialize)]
#[derive(Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct BootData {
pub version: BootVersion,
@@ -71,19 +71,19 @@ pub enum DmVerityContent {
SystemOtacerts,
}
#[derive(Serialize, Deserialize)]
#[derive(Clone, Copy, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct DmVerityData {
pub content: DmVerityContent,
}
#[derive(Serialize, Deserialize)]
#[derive(Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct VbmetaData {
pub deps: Vec<String>,
}
#[derive(Serialize, Deserialize)]
#[derive(Clone, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum Data {
Boot(BootData),
@@ -91,30 +91,37 @@ pub enum Data {
Vbmeta(VbmetaData),
}
#[derive(Serialize, Deserialize)]
#[derive(Clone, Copy, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Hashes {
pub original: Sha256Hash,
pub patched: Sha256Hash,
}
#[derive(Serialize, Deserialize)]
#[derive(Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Partition {
pub avb: Avb,
pub data: Data,
}
#[derive(Serialize, Deserialize)]
#[derive(Clone, Copy, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct VabcSettings {
pub version: CowVersion,
pub algo: VabcAlgo,
}
#[derive(Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Profile {
pub vabc_algo: Option<VabcAlgo>,
pub vabc: Option<VabcSettings>,
pub partitions: BTreeMap<String, Partition>,
pub hashes_streaming: Hashes,
pub hashes_seekable: Hashes,
}
#[derive(Serialize, Deserialize)]
#[derive(Clone, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Config {
pub ota_info: OtaInfo,
+59 -41
View File
@@ -1,4 +1,4 @@
// SPDX-FileCopyrightText: 2023-2024 Andrew Gunnerson
// SPDX-FileCopyrightText: 2023-2025 Andrew Gunnerson
// SPDX-FileCopyrightText: 2023 Pascal Roeleven
// SPDX-License-Identifier: GPL-3.0-only
@@ -14,12 +14,12 @@ use std::{
path::{Path, PathBuf},
slice,
sync::{
atomic::{AtomicBool, Ordering},
Arc,
atomic::{AtomicBool, Ordering},
},
};
use anyhow::{anyhow, bail, Context, Result};
use anyhow::{Context, Result, anyhow, bail};
use avbroot::{
cli::ota::{ExtractCli, PatchCli, VerifyCli},
crypto::{self, PassphraseSource, RsaSigningKey},
@@ -36,11 +36,12 @@ use avbroot::{
cpio::{self, CpioEntry, CpioEntryData},
ota::{self, SigningWriter, ZipEntry, ZipMode},
padding,
payload::{self, PayloadHeader, PayloadWriter},
payload::{self, CowVersion, PayloadHeader, PayloadWriter, VabcParams},
zip::ZipWriterWrapper,
},
patch::otacert::{self, OtaCertBuildFlags},
protobuf::{
build::tools::releasetools::{ota_metadata::OtaType, DeviceState, OtaMetadata},
build::tools::releasetools::{DeviceState, OtaMetadata, ota_metadata::OtaType},
chromeos_update_engine::{
DeltaArchiveManifest, DynamicPartitionGroup, DynamicPartitionMetadata, PartitionUpdate,
},
@@ -48,12 +49,12 @@ use avbroot::{
stream::{self, CountingWriter, FromReader, HashingReader, PSeekFile, Reopen, ToWriter},
};
use clap::Parser;
use rsa::{rand_core::OsRng, traits::PublicKeyParts, BigUint};
use rsa::{BigUint, rand_core::OsRng, traits::PublicKeyParts};
use tempfile::TempDir;
use topological_sort::TopologicalSort;
use tracing::{info, info_span};
use x509_cert::Certificate;
use zip::{write::FileOptions, CompressionMethod, ZipWriter};
use zip::{CompressionMethod, DateTime, ZipWriter, write::SimpleFileOptions};
use crate::{
cli::{Cli, Command, HelperCli, ListCli, PassSource, ProfileGroup, TestCli},
@@ -97,7 +98,7 @@ fn verify_hash(path: &Path, sha256: &[u8; 32], cancel_signal: &AtomicBool) -> Re
fn append_avb(
file: &mut PSeekFile,
name: &str,
avb: &Avb,
avb: Avb,
hash_tree: bool,
ota_info: &OtaInfo,
key_avb: &RsaSigningKey,
@@ -295,7 +296,7 @@ fn create_ramdisk(
fn create_boot_image(
file: &mut PSeekFile,
name: &str,
avb: &Avb,
avb: Avb,
boot_data: &BootData,
ota_info: &OtaInfo,
key_avb: &RsaSigningKey,
@@ -374,7 +375,7 @@ fn create_boot_image(
.ramdisks
.iter()
.map(|c_list| {
if c_list.iter().any(|c| *c == RamdiskContent::Dlkm) {
if c_list.contains(&RamdiskContent::Dlkm) {
RamdiskMeta {
ramdisk_type: bootimage::VENDOR_RAMDISK_TYPE_DLKM,
ramdisk_name: "dlkm".to_owned(),
@@ -422,8 +423,8 @@ fn create_boot_image(
fn create_dm_verity_image(
file: &mut PSeekFile,
name: &str,
avb: &Avb,
dm_verity_data: &DmVerityData,
avb: Avb,
dm_verity_data: DmVerityData,
ota_info: &OtaInfo,
key_avb: &RsaSigningKey,
cert_ota: &Certificate,
@@ -451,7 +452,7 @@ fn create_dm_verity_image(
fn create_vbmeta_image(
file: &mut PSeekFile,
name: &str,
avb: &Avb,
avb: Avb,
vbmeta_data: &VbmetaData,
inputs: &BTreeMap<String, PSeekFile>,
key: &RsaSigningKey,
@@ -537,7 +538,7 @@ fn create_partition_images(
create_boot_image(
&mut file,
name,
&partition.avb,
partition.avb,
data,
ota_info,
key_avb,
@@ -550,8 +551,8 @@ fn create_partition_images(
create_dm_verity_image(
&mut file,
name,
&partition.avb,
data,
partition.avb,
*data,
ota_info,
key_avb,
cert_ota,
@@ -560,7 +561,7 @@ fn create_partition_images(
.with_context(|| format!("Failed to create dm-verity image: {name}"))?;
}
Data::Vbmeta(data) => {
create_vbmeta_image(&mut file, name, &partition.avb, data, &files, key_avb)
create_vbmeta_image(&mut file, name, partition.avb, data, &files, key_avb)
.with_context(|| format!("Failed to create vbmeta image: {name}"))?;
}
}
@@ -580,6 +581,8 @@ fn create_payload(
key_ota: &RsaSigningKey,
cancel_signal: &AtomicBool,
) -> Result<(String, u64)> {
const COMPRESSION_FACTOR: u32 = 64 * 1024;
let dynamic_partitions_names = partitions
.iter()
.filter(|(_, p)| matches!(&p.data, Data::DmVerity(_)))
@@ -594,17 +597,23 @@ fn create_payload(
.map(PSeekFile::new)
.with_context(|| format!("Failed to create temp file for: {name}"))?;
let vabc_algo = if dynamic_partitions_names.contains(name) {
profile.vabc_algo
let vabc_params = if dynamic_partitions_names.contains(name) {
profile.vabc.map(|v| VabcParams {
version: v.version,
algo: v.algo,
compression_factor: COMPRESSION_FACTOR,
})
} else {
None
};
let (partition_info, operations, cow_estimate) =
payload::compress_image(file, &writer, name, 4096, vabc_algo, cancel_signal)?;
payload::compress_image(file, &writer, name, 4096, vabc_params, cancel_signal)?;
compressed.insert(name, writer);
let is_v3 = profile.vabc.is_some_and(|e| e.version == CowVersion::V3);
payload_partitions.push(PartitionUpdate {
partition_name: name.clone(),
run_postinstall: None,
@@ -624,8 +633,8 @@ fn create_payload(
fec_roots: None,
version: None,
merge_operations: vec![],
estimate_cow_size: cow_estimate,
estimate_op_count_max: None,
estimate_cow_size: cow_estimate.map(|e| e.size),
estimate_op_count_max: cow_estimate.and_then(|e| is_v3.then_some(e.num_ops)),
});
}
@@ -645,11 +654,16 @@ fn create_payload(
partition_names: dynamic_partitions_names,
}],
snapshot_enabled: Some(true),
vabc_enabled: Some(true),
vabc_compression_param: profile.vabc_algo.map(|a| a.to_string()),
cow_version: Some(2),
// Everything below is meant to be unset if VABC is not
// supported.
vabc_enabled: profile.vabc.map(|_| true),
vabc_compression_param: profile.vabc.map(|v| v.algo.to_string()),
cow_version: profile.vabc.map(|v| match v.version {
CowVersion::V2 => 2,
CowVersion::V3 => 3,
}),
vabc_feature_set: None,
compression_factor: None,
compression_factor: profile.vabc.map(|_| COMPRESSION_FACTOR.into()),
}),
partial_update: None,
apex_info: vec![],
@@ -728,14 +742,15 @@ fn create_ota(
let mut zip_writer = match zip_mode {
ZipMode::Streaming => {
let signing_writer = SigningWriter::new_streaming(raw_writer);
ZipWriter::new_streaming(signing_writer)
ZipWriterWrapper::new_streaming(signing_writer)
}
ZipMode::Seekable => {
let signing_writer = SigningWriter::new_seekable(raw_writer);
ZipWriter::new(signing_writer)
ZipWriterWrapper::new_seekable(signing_writer)
}
};
let options = FileOptions::default()
let options = SimpleFileOptions::default()
.last_modified_time(DateTime::default())
.compression_method(CompressionMethod::Stored)
.large_file(false);
@@ -745,12 +760,9 @@ fn create_ota(
for path in [ota::PATH_OTACERT, ota::PATH_PAYLOAD, ota::PATH_PROPERTIES] {
// All remaining entries are written immediately.
zip_writer
.start_file_with_extra_data(path, options)
.with_context(|| format!("Failed to begin new zip entry: {path}"))?;
let offset = zip_writer
.end_extra_data()
.with_context(|| format!("Failed to end new zip entry: {path}"))?;
.start_file(path, options)
.with_context(|| format!("Failed to begin new zip entry: {path}"))?;
let mut writer = CountingWriter::new(&mut zip_writer);
match path {
@@ -851,6 +863,7 @@ fn create_fake_magisk(output: &Path) -> Result<()> {
let raw_writer =
File::create(output).with_context(|| format!("Failed to open for writing: {output:?}"))?;
let mut zip_writer = ZipWriter::new(raw_writer);
let options = SimpleFileOptions::default().last_modified_time(DateTime::default());
for path in [
"assets/stub.apk",
@@ -867,12 +880,12 @@ fn create_fake_magisk(output: &Path) -> Result<()> {
"lib/x86_64/libmagisk64.so",
"lib/x86_64/libmagiskinit.so",
] {
zip_writer.start_file(path, FileOptions::default())?;
zip_writer.start_file(path, options)?;
write!(zip_writer, "dummy contents for {path}")?;
}
// avbroot looks for the version number in this file.
zip_writer.start_file("assets/util_functions.sh", FileOptions::default())?;
zip_writer.start_file("assets/util_functions.sh", options)?;
zip_writer.write_all(b"MAGISK_VER_CODE=27000\n")?;
Ok(())
@@ -1127,7 +1140,7 @@ fn clean_boot_image_certs(path: &Path, cancel_signal: &AtomicBool) -> Result<()>
.iter_mut()
.find(|e| e.path == b"system/etc/security/otacerts.zip")
{
let mut zip_writer = ZipWriter::new(Cursor::new(Vec::new()));
let zip_writer = ZipWriter::new(Cursor::new(Vec::new()));
let empty_zip = zip_writer.finish()?.into_inner();
entry.data = CpioEntryData::Data(empty_zip);
@@ -1189,6 +1202,7 @@ fn test_subcommand(cli: &TestCli, cancel_signal: &AtomicBool) -> Result<()> {
Some(_) => None,
None => Some(TempDir::new().context("Failed to create temp directory")?),
};
#[allow(clippy::option_if_let_else)]
let work_dir = match &cli.config.work_dir {
Some(w) => w.as_path(),
None => work_temp_dir.as_ref().unwrap().path(),
@@ -1220,8 +1234,9 @@ fn test_subcommand(cli: &TestCli, cancel_signal: &AtomicBool) -> Result<()> {
] {
let _span = info_span!("profile", name, %zip_mode).entered();
// Can't used NamedTempFile because avbroot does atomic replaces.
let profile_dir = work_dir.join(name);
// Can't use NamedTempFile because avbroot does atomic replaces.
let mut profile_dir = work_dir.join(name);
profile_dir.push(zip_mode.to_string());
let out_original = profile_dir.join("ota.zip");
let out_magisk = profile_dir.join("ota_magisk.zip");
let out_prepatched = profile_dir.join("ota_prepatched.zip");
@@ -1324,7 +1339,7 @@ fn helper_mode() -> Result<()> {
let cli = HelperCli::parse();
let private_key_path = {
let parent = cli.public_key.parent().unwrap_or(Path::new("."));
let parent = cli.public_key.parent().unwrap_or_else(|| Path::new("."));
let name = cli
.public_key
.file_name()
@@ -1398,7 +1413,10 @@ fn main() -> Result<()> {
if env::var_os(ENV_HELPER_MODE).is_some() {
return helper_mode();
}
env::set_var(ENV_HELPER_MODE, "true");
// SAFETY: No multithreading at this point.
unsafe {
env::set_var(ENV_HELPER_MODE, "true");
}
// Set up a cancel signal so we can properly clean up any temporary files.
let cancel_signal = Arc::new(AtomicBool::new(false));
+4 -4
View File
@@ -1,15 +1,15 @@
// SPDX-FileCopyrightText: 2023 Andrew Gunnerson
// SPDX-FileCopyrightText: 2023-2025 Andrew Gunnerson
// SPDX-License-Identifier: GPL-3.0-only
use std::{
collections::BTreeMap,
fmt,
fmt::{self, Write as _},
fs::{self, File},
io::{BufRead, BufReader},
path::Path,
};
use anyhow::{anyhow, bail, Result};
use anyhow::{Result, anyhow, bail};
use regex::Regex;
use crate::WORKSPACE_DIR;
@@ -108,7 +108,7 @@ fn update_changelog_links(path: &Path, base_url: &str) -> Result<()> {
}
for (link_ref, link) in links {
result.push_str(&format!("{link_ref}: {link}\n"));
let _ = writeln!(result, "{link_ref}: {link}");
}
fs::write(path, result)?;
+2 -2
View File
@@ -7,9 +7,9 @@ use std::{
path::Path,
};
use anyhow::{bail, Result};
use anyhow::{Result, bail};
use clap::Parser;
use toml_edit::{value, DocumentMut};
use toml_edit::{DocumentMut, value};
use crate::WORKSPACE_DIR;