Add support for packing and unpacking logical partition images

This supports both empty and normal LP images, including those that span
multiple files/devices.

Currently, repacked files are semantically equivalent, but not exactly
identical. avbroot's data structure for the metadata does not preserve
the arbitrary partition ordering of the LP image. Instead, to make the
API a bit nicer, it only preserves the relative partition ordering
within partition groups.

Signed-off-by: Andrew Gunnerson <accounts+github@chiller3.com>
This commit is contained in:
Andrew Gunnerson
2024-08-27 23:28:20 -04:00
parent dd40f64918
commit 13c910274e
11 changed files with 3057 additions and 3 deletions
Generated
+5
View File
@@ -164,6 +164,8 @@ dependencies = [
"tracing",
"tracing-subscriber",
"x509-cert",
"zerocopy",
"zerocopy-derive",
"zip",
]
@@ -190,6 +192,9 @@ name = "bitflags"
version = "2.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b048fb63fd8b5923fc5aa7b340d8e156aec7ec02f0c78fa8a6ddc2613f6f71de"
dependencies = [
"serde",
]
[[package]]
name = "block-buffer"
+52
View File
@@ -253,6 +253,58 @@ avbroot hash-tree verify -i <input data file> -H <input hash tree file>
This will check if the input file has any corrupted blocks. Currently, the command cannot report which specific blocks are corrupted, only whether the file is valid.
## `avbroot lp`
This set of commands is for working with LP (logical partition) images. These are the containers for dynamically-allocated partitions, like `system`. All LP images are supported:
* Empty images: These are the `super_empty.img` images in the factory images for newer Google Pixel devices. They define the layout of the `super` partition, but don't contain any actual data. They also do not contain a backup copy of the metadata. As an optimization, `fastboot` can fill in the actual data during flashing to avoid needing to reboot to fastbootd mode.
* Normal images backed by a single device: These are standalone `super.img` images and are how logical partitions are physically stored on disk in most newer devices. They contain a backup copy of all metadata as well as actual partition data.
* Normal images backed by multiple devices: These are images split across multiple files/partitions and are used on devices where support for LP was retrofitted. For example, the LP setup on newer Android builds for the Google Pixel 3a XL reuse the legacy `system` and `vendor` partitions because there is no `super` partition. These are similar to the single-file LP setups, except that data can be stored across all of the LP images. However, the metadata is only stored on the first LP image.
### Unpacking an LP image
```bash
avbroot lp unpack -i <input LP image> [-i <input LP image>]...
```
This subcommand unpacks the LP metadata to `lp.toml` and the partition images to the `lp_images` directory.
If there are multiple images, they must be specified in order. If the order is not known, run `avbroot lp info` on each of the images. The one that successfully parses is the first image and the `block_devices` field in the output specifies the full ordering.
An LP image can have multiple slots. If the LP image originated from a factory image or OTA, all slots are likely identical. If the LP image was dumped from a real device that installed OTA updates in the past, the slots may differ. If the slots are not identical, then the `--slot` option is required to specify which slot to unpack.
When unpacking an empty image, files are still created in the `lp_images` directory. These files are sparse files that don't contain any actual data, but do have the correct file size. They are necessary when packing a new LP image so that avbroot can determine the size of each partition.
### Packing an LP image
```bash
avbroot lp pack -o <output LP image> [-o <output LP image>]...
```
This subcommand packs a new LP image from the `lp.toml` file and `lp_images` directory. Any `.img` files in the `lp_images` directory that don't have a corresponding entry in `lp.toml` are silently ignored.
All metadata slots in the newly packed LP image will be identical.
The partition images in `lp_images` are required even when packing an empty image. They can be sparse files that contain no data, but must have the proper size.
### Repacking an LP image
```bash
avbroot lp repack [-i <input LP image>] [-i <input LP image>]... -o <output LP image> [-o <output LP image>]...
```
This subcommand is logically equivalent to `avbroot lp unpack` followed by `avbroot lp pack`, except more efficient. Instead of unpacking and packing all partition images, the raw data is directly copied from the old LP image to the new LP image.
When `--slot` is specified, this is useful for discarding unwanted metadata slots and the partition data exclusive to them.
### Showing LP image metadata
```bash
avbroot lp info -i <first LP image>
```
This subcommand shows the LP image metadata, including all metadata slots. If there are multiple images, only the first one is needed because it is the only one that stores the metadata.
## `avbroot payload`
### Unpacking a payload binary
+6 -1
View File
@@ -11,7 +11,7 @@ publish = false
[dependencies]
anyhow = "1.0.75"
base64 = "0.22.1"
bitflags = "2.4.1"
bitflags = { version = "2.4.1", features = ["serde"] }
bstr = "1.6.2"
byteorder = "1.4.3"
cap-std = "3.0.0"
@@ -53,6 +53,8 @@ topological-sort = "0.2.2"
tracing = "0.1.40"
tracing-subscriber = "0.3.18"
x509-cert = { version = "0.2.4", features = ["builder"] }
zerocopy = "0.7.35"
zerocopy-derive = "0.7.35"
# There are multiple upstream bugs that cause infinite loops in the Drop
# implementation of write::BzDecoder. Unfortunately, the project is no longer
@@ -82,3 +84,6 @@ assert_matches = "1.5.0"
[features]
static = ["bzip2/static", "liblzma/static"]
[lints.rust]
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(fuzzing)'] }
+3 -1
View File
@@ -15,7 +15,7 @@ use clap::{Parser, Subcommand, ValueEnum};
use tracing::{debug, Level};
use tracing_subscriber::fmt::{format::Writer, time::FormatTime};
use crate::cli::{avb, boot, completion, cpio, fec, hashtree, key, ota, payload};
use crate::cli::{avb, boot, completion, cpio, fec, hashtree, key, lp, ota, payload};
#[allow(clippy::large_enum_variant)]
#[derive(Debug, Subcommand)]
@@ -27,6 +27,7 @@ pub enum Command {
Fec(fec::FecCli),
HashTree(hashtree::HashTreeCli),
Key(key::KeyCli),
Lp(lp::LpCli),
Ota(ota::OtaCli),
Payload(payload::PayloadCli),
/// (Deprecated: Use `avbroot ota patch` instead.)
@@ -130,6 +131,7 @@ pub fn main(logging_initialized: &AtomicBool, cancel_signal: &AtomicBool) -> Res
Command::Fec(c) => fec::fec_main(&c, cancel_signal),
Command::HashTree(c) => hashtree::hash_tree_main(&c, cancel_signal),
Command::Key(c) => key::key_main(&c),
Command::Lp(c) => lp::lp_main(&c, cancel_signal),
Command::Ota(c) => ota::ota_main(&c, cancel_signal),
Command::Payload(c) => payload::payload_main(&c, cancel_signal),
// Deprecated aliases.
+684
View File
@@ -0,0 +1,684 @@
/*
* SPDX-FileCopyrightText: 2024 Andrew Gunnerson
* SPDX-License-Identifier: GPL-3.0-only
*/
use std::{
ffi::OsStr,
fs::{self, File},
io::{Seek, SeekFrom},
path::{Path, PathBuf},
sync::atomic::AtomicBool,
};
use anyhow::{bail, Context, Result};
use cap_std::{ambient_authority, fs::Dir};
use clap::{CommandFactory, Parser, Subcommand};
use rayon::iter::{
IndexedParallelIterator, IntoParallelIterator, IntoParallelRefIterator, ParallelIterator,
};
use crate::{
format::lp::{Extent, ExtentType, ImageType, Metadata, SECTOR_SIZE},
stream::{self, FromReader, PSeekFile, Reopen, ToWriter},
};
fn open_lp_inputs(paths: &[impl AsRef<Path>]) -> Result<(Vec<PSeekFile>, Metadata)> {
let mut inputs = paths
.iter()
.map(|p| {
let p = p.as_ref();
File::open(p)
.map(PSeekFile::new)
.with_context(|| format!("Failed to open LP image for reading: {p:?}"))
})
.collect::<Result<Vec<_>>>()?;
let metadata = Metadata::from_reader(&mut inputs[0])
.with_context(|| format!("Failed to parse LP image metadata: {:?}", paths[0].as_ref()))?;
Ok((inputs, metadata))
}
fn open_lp_outputs(paths: &[impl AsRef<Path>]) -> Result<Vec<PSeekFile>> {
paths
.iter()
.map(|p| {
let p = p.as_ref();
File::create(p)
.map(PSeekFile::new)
.with_context(|| format!("Failed to open LP image for writing: {p:?}"))
})
.collect::<Result<Vec<_>>>()
}
fn read_info(path: &Path) -> Result<Metadata> {
let data = fs::read_to_string(path)
.with_context(|| format!("Failed to read metadata info TOML: {path:?}"))?;
let info = toml_edit::de::from_str(&data)
.with_context(|| format!("Failed to parse metadata info TOML: {path:?}"))?;
Ok(info)
}
fn write_info(path: &Path, metadata: &Metadata) -> Result<()> {
let data = toml_edit::ser::to_string_pretty(metadata)
.with_context(|| format!("Failed to serialize metadata info TOML: {path:?}"))?;
fs::write(path, data)
.with_context(|| format!("Failed to write metadata info TOML: {path:?}"))?;
Ok(())
}
fn display_metadata(cli: &LpCli, metadata: &Metadata) {
if !cli.quiet {
println!("{metadata:#?}");
}
}
struct CopyExtent {
device_index: usize,
lp_offset: u64,
out_offset: u64,
size: u64,
}
/// Split extents into smaller ones for parallelization.
fn split_extents(extents: &[Extent]) -> Vec<CopyExtent> {
// 64 MiB is the smallest size we'll parallelize by.
const CHUNK_SIZE: u64 = 64 * 1024 * 1024;
let mut result = vec![];
let mut out_offset = 0;
for extent in extents {
let mut remain = extent.num_sectors * u64::from(SECTOR_SIZE);
match extent.extent_type {
ExtentType::Linear {
start_sector,
block_device_index,
} => {
let mut lp_offset = start_sector * u64::from(SECTOR_SIZE);
// 64 MiB is the smallest size we'll parallelize by.
let num_chunks = remain.div_ceil(64 * 1024 * 1024);
for _ in 0..num_chunks {
let chunk_size = CHUNK_SIZE.min(remain);
result.push(CopyExtent {
device_index: block_device_index,
out_offset,
lp_offset,
size: chunk_size,
});
out_offset += chunk_size;
lp_offset += chunk_size;
remain -= chunk_size;
}
}
ExtentType::Zero => out_offset += remain,
}
}
result
}
/// Use the CLI-specified slot or automatically select one if all slots are
/// identical.
fn get_slot_number(metadata: &Metadata, cli_slot: Option<u32>) -> Result<usize> {
match cli_slot {
Some(n) => {
let n = n as usize;
if n >= metadata.slots.len() {
bail!("Slot out of range: {n}");
}
Ok(n)
}
None => {
if metadata.slots.windows(2).any(|w| w[0] != w[1]) {
bail!("A slot must be specified because they are not all identical");
}
Ok(0)
}
}
}
/// Remove all slots aside from the specified one and return the old slot count.
fn retain_slot(metadata: &mut Metadata, slot: usize) -> usize {
let slot_count = metadata.slots.len();
metadata.slots.swap(0, slot);
metadata.slots.truncate(1);
slot_count
}
/// Duplicate the first slot until the required number of slots is reached.
fn fill_slots(metadata: &mut Metadata) {
let required = match metadata.image_type {
ImageType::Normal => metadata.metadata_slot_count as usize,
ImageType::Empty => 1,
};
for _ in metadata.slots.len()..required {
metadata.slots.extend_from_within(0..=0);
}
}
fn unpack_subcommand(lp_cli: &LpCli, cli: &UnpackCli, cancel_signal: &AtomicBool) -> Result<()> {
let mut inputs = cli
.input
.iter()
.map(|p| {
File::open(p)
.map(PSeekFile::new)
.with_context(|| format!("Failed to open LP image for reading: {p:?}"))
})
.collect::<Result<Vec<_>>>()?;
let mut metadata = Metadata::from_reader(&mut inputs[0])
.with_context(|| format!("Failed to read LP image metadata: {:?}", cli.input[0]))?;
// Display and write only the selected slot.
let slot_number = get_slot_number(&metadata, cli.slot)?;
retain_slot(&mut metadata, slot_number);
display_metadata(lp_cli, &metadata);
write_info(&cli.output_info, &metadata)?;
let authority = ambient_authority();
Dir::create_ambient_dir_all(&cli.output_images, authority)
.with_context(|| format!("Failed to create directory: {:?}", cli.output_images))?;
let directory = Dir::open_ambient_dir(&cli.output_images, authority)
.with_context(|| format!("Failed to open directory: {:?}", cli.output_images))?;
let slot = &metadata.slots[0];
if slot.block_devices.len() != inputs.len() {
bail!(
"Need {} input images, but have {}",
slot.block_devices.len(),
inputs.len(),
);
}
// Preopen all image output files.
let mut paths = vec![];
let mut files = vec![];
for group in &slot.groups {
let mut group_paths = vec![];
let mut group_files = vec![];
for partition in &group.partitions {
// A partition name with unsafe characters fails during parsing.
let path = format!("{}.img", partition.name);
let file = directory
.create(&path)
.map(|f| PSeekFile::new(f.into_std()))
.with_context(|| format!("Failed to open for writing: {path:?}"))?;
file.set_len(partition.size()?)
.with_context(|| format!("Failed to truncate file: {path:?}"))?;
group_paths.push(path);
group_files.push(file);
}
paths.push(group_paths);
files.push(group_files);
}
// For empty images, the outputs have already been truncated and there's no
// data to unpack.
if metadata.image_type == ImageType::Empty {
return Ok(());
}
slot.groups
.par_iter()
.enumerate()
// Flatten grouped partitions.
.flat_map(|(g_index, g)| {
g.partitions
.par_iter()
.enumerate()
.map(move |(p_index, p)| (g_index, p_index, p))
})
// Flatten extents in all partitions and split them to smaller chunks
// for better parallelism.
.flat_map(|(g_index, p_index, p)| {
split_extents(&p.extents)
.into_par_iter()
.map(move |e| (g_index, p_index, e))
})
.map(|(g_index, p_index, extent)| {
// Never fails for PSeekFiles.
let mut reader = inputs[extent.device_index].reopen()?;
let mut writer = files[g_index][p_index].reopen()?;
let r_path = &cli.input[extent.device_index];
let w_path = &paths[g_index][p_index];
reader
.seek(SeekFrom::Start(extent.lp_offset))
.with_context(|| format!("Failed to seek file: {r_path:?}"))?;
writer
.seek(SeekFrom::Start(extent.out_offset))
.with_context(|| format!("Failed to seek file: {w_path:?}"))?;
stream::copy_n(&mut reader, &mut writer, extent.size, cancel_signal)
.with_context(|| format!("Failed to copy extent: {r_path:?} -> {w_path:?}"))?;
Ok(())
})
.collect::<Result<()>>()?;
Ok(())
}
fn pack_subcommand(lp_cli: &LpCli, cli: &PackCli, cancel_signal: &AtomicBool) -> Result<()> {
let mut metadata = read_info(&cli.input_info)?;
if metadata.slots.len() != 1 {
bail!("There must be exactly one metadata slot");
}
let slot = &mut metadata.slots[0];
let mut outputs = open_lp_outputs(&cli.output)?;
if slot.block_devices.len() != outputs.len() {
bail!(
"Need {} output images, but have {}",
slot.block_devices.len(),
outputs.len(),
);
}
if metadata.image_type == ImageType::Normal {
for (i, (block_device, output)) in slot.block_devices.iter().zip(&outputs).enumerate() {
output
.set_len(block_device.size)
.with_context(|| format!("Failed to truncate file: {:?}", cli.output[i]))?;
}
}
let authority = ambient_authority();
let directory = Dir::open_ambient_dir(&cli.input_images, authority)
.with_context(|| format!("Failed to open directory: {:?}", cli.input_images))?;
for group in &slot.groups {
for partition in &group.partitions {
let name = &partition.name;
if Path::new(name).file_name() != Some(OsStr::new(name)) {
bail!("Unsafe partition name: {name}");
}
}
}
// Preopen all image input files.
let mut paths = vec![];
let mut files = vec![];
for group in &mut slot.groups {
let mut group_paths = vec![];
let mut group_files = vec![];
for partition in &mut group.partitions {
let path = format!("{}.img", partition.name);
let mut file = directory
.open(&path)
.map(|f| PSeekFile::new(f.into_std()))
.with_context(|| format!("Failed to open for reading: {path:?}"))?;
let size = file
.seek(SeekFrom::End(0))
.with_context(|| format!("Failed to seek file: {path:?}"))?;
if size % u64::from(SECTOR_SIZE) != 0 {
bail!("File size is not {SECTOR_SIZE}B aligned: {size}: {path:?}");
}
// This will be filled out properly later during reallocation.
partition.extents.push(Extent {
num_sectors: size / u64::from(SECTOR_SIZE),
extent_type: ExtentType::Linear {
start_sector: 0,
block_device_index: 0,
},
});
group_paths.push(path);
group_files.push(file);
}
paths.push(group_paths);
files.push(group_files);
}
// Now that we have all the partition sizes, actually allocate extents for
// them on the block devices.
slot.reallocate_extents()
.context("Failed to allocate extents")?;
// Display only the selected slot and make the rest identical.
let _ = slot;
display_metadata(lp_cli, &metadata);
fill_slots(&mut metadata);
let slot = &metadata.slots[0];
// Write the new metadata.
metadata
.to_writer(&mut outputs[0])
.with_context(|| format!("Failed to write LP image metadata: {:?}", cli.output[0]))?;
// For empty images, there's no data to pack. The inputs were only used for
// computing the extents.
if metadata.image_type == ImageType::Empty {
return Ok(());
}
slot.groups
.par_iter()
.enumerate()
// Flatten grouped partitions.
.flat_map(|(g_index, g)| {
g.partitions
.par_iter()
.enumerate()
.map(move |(p_index, p)| (g_index, p_index, p))
})
// Flatten extents in all partitions and split them to smaller chunks
// for better parallelism.
.flat_map(|(g_index, p_index, p)| {
split_extents(&p.extents)
.into_par_iter()
.map(move |e| (g_index, p_index, e))
})
.map(|(g_index, p_index, extent)| {
// Never fails for PSeekFiles.
let mut reader = files[g_index][p_index].reopen()?;
let mut writer = outputs[extent.device_index].reopen()?;
let r_path = &paths[g_index][p_index];
let w_path = &cli.output[extent.device_index];
reader
.seek(SeekFrom::Start(extent.out_offset))
.with_context(|| format!("Failed to seek file: {r_path:?}"))?;
writer
.seek(SeekFrom::Start(extent.lp_offset))
.with_context(|| format!("Failed to seek file: {w_path:?}"))?;
stream::copy_n(&mut reader, &mut writer, extent.size, cancel_signal)
.with_context(|| format!("Failed to copy extent: {r_path:?} -> {w_path:?}"))?;
Ok(())
})
.collect::<Result<()>>()?;
Ok(())
}
fn repack_subcommand(lp_cli: &LpCli, cli: &RepackCli, cancel_signal: &AtomicBool) -> Result<()> {
// Show a clap-style error if the number of inputs and outputs aren't equal.
if cli.input.len() != cli.output.len() {
let (arg_id, actual_len, expected_len) = if cli.input.len() < cli.output.len() {
("input", cli.input.len(), cli.output.len())
} else {
("output", cli.output.len(), cli.input.len())
};
let mut command = RepackCli::command();
command.build();
let arg = command
.get_arguments()
.find(|a| a.get_id() == arg_id)
.expect("argument not found");
let mut error =
clap::Error::new(clap::error::ErrorKind::WrongNumberOfValues).with_cmd(&command);
error.insert(
clap::error::ContextKind::InvalidArg,
clap::error::ContextValue::String(arg.to_string()),
);
error.insert(
clap::error::ContextKind::ActualNumValues,
clap::error::ContextValue::Number(actual_len as isize),
);
error.insert(
clap::error::ContextKind::ExpectedNumValues,
clap::error::ContextValue::Number(expected_len as isize),
);
// We don't show the usage because only Command::_build_subcommand() can
// create an appropriate Command instance for showing the subcommand
// usage and there's no way to call that, directly or indirectly.
error.exit();
}
let (inputs, mut metadata) = open_lp_inputs(&cli.input)?;
let mut outputs = open_lp_outputs(&cli.output)?;
// Display only the selected slot and make the rest identical.
let slot_number = get_slot_number(&metadata, cli.slot)?;
retain_slot(&mut metadata, slot_number);
display_metadata(lp_cli, &metadata);
fill_slots(&mut metadata);
let slot = &metadata.slots[0];
if slot.block_devices.len() != inputs.len() {
bail!(
"Need {} images, but have {}",
slot.block_devices.len(),
inputs.len(),
);
}
// Write the new metadata.
metadata
.to_writer(&mut outputs[0])
.with_context(|| format!("Failed to write LP image metadata: {:?}", cli.output[0]))?;
// Explicitly set the file size in case there are dm-zero extents, which are
// ignored below.
if metadata.image_type == ImageType::Normal {
for (i, (block_device, output)) in slot.block_devices.iter().zip(&outputs).enumerate() {
output
.set_len(block_device.size)
.with_context(|| format!("Failed to truncate file: {:?}", cli.output[i]))?;
}
}
slot.groups
.par_iter()
// Flatten grouped partitions.
.flat_map(|group| &group.partitions)
// Flatten extents in all partitions and split them to smaller chunks
// for better parallelism.
.flat_map(|partition| split_extents(&partition.extents))
.map(|extent| {
// Never fails for PSeekFiles.
let mut reader = inputs[extent.device_index].reopen()?;
let mut writer = outputs[extent.device_index].reopen()?;
let r_path = &cli.input[extent.device_index];
let w_path = &cli.output[extent.device_index];
reader
.seek(SeekFrom::Start(extent.lp_offset))
.with_context(|| format!("Failed to seek file: {r_path:?}"))?;
writer
.seek(SeekFrom::Start(extent.lp_offset))
.with_context(|| format!("Failed to seek file: {w_path:?}"))?;
stream::copy_n(&mut reader, &mut writer, extent.size, cancel_signal)
.with_context(|| format!("Failed to copy extent: {r_path:?} -> {w_path:?}"))?;
Ok(())
})
.collect::<Result<()>>()?;
Ok(())
}
fn info_subcommand(lp_cli: &LpCli, cli: &InfoCli) -> Result<()> {
let (_, metadata) = open_lp_inputs(&[&cli.input])?;
// Unlike the other subcommands, we show all metadata slots here.
display_metadata(lp_cli, &metadata);
Ok(())
}
pub fn lp_main(cli: &LpCli, cancel_signal: &AtomicBool) -> Result<()> {
match &cli.command {
LpCommand::Unpack(c) => unpack_subcommand(cli, c, cancel_signal),
LpCommand::Pack(c) => pack_subcommand(cli, c, cancel_signal),
LpCommand::Repack(c) => repack_subcommand(cli, c, cancel_signal),
LpCommand::Info(c) => info_subcommand(cli, c),
}
}
/// Unpack an LP image.
///
/// Each partition is extracted to `<partition name>.img` in the output images
/// directory. This is the case even when unpacking empty LP images. In this
/// scenario, the partition images are sparse files containing no data.
///
/// The LP image metadata is written to the info TOML file.
///
/// If any partition names are unsafe to use in a path, the extraction process
/// will fail and exit. Extracted files are never written outside of the tree
/// directory, even if an external process tries to interfere.
#[derive(Debug, Parser)]
struct UnpackCli {
/// Path to input LP images.
///
/// If there are multiple images, they must be specified in order. If the
/// order is unknown, run `avbroot lp info` against the `super` image and
/// look at the `block_devices` field.
#[arg(short, long, value_name = "FILE", value_parser, required = true)]
input: Vec<PathBuf>,
/// Path to output info TOML.
#[arg(long, value_name = "FILE", value_parser, default_value = "lp.toml")]
output_info: PathBuf,
/// Path to output images directory.
#[arg(long, value_name = "DIR", value_parser, default_value = "lp_images")]
output_images: PathBuf,
/// The LP metadata slot to use.
///
/// This slot is the only slot where data extents are copied from. Any data
/// referenced exclusively by other slots (if any) will be ignored.
///
/// This option is required if not all slots are identical.
#[arg(short, long)]
slot: Option<u32>,
}
/// Pack an LP image.
///
/// For normal images, the number of metadata slots written is equal to the
/// `metadata_slot_count` value in the info TOML. Each slot has identical
/// metadata. It is not possible to write multiple slots with different metadata
/// using this tool. For empty images, only a single slot is written, regardless
/// of the value of `metadata_slot_count`, as required by the file format.
///
/// The new LP image will *only* contain images listed in the info TOML file and
/// they are added in the order listed. Note that the input partition images are
/// required even when packing an empty LP image. They can be sparse files that
/// contain no data, but must have the proper size.
#[derive(Debug, Parser)]
struct PackCli {
/// Path to output LP images.
///
/// If there are multiple images, they must be specified in the same order
/// as the block device entries are listed in the info TOML.
#[arg(short, long, value_name = "FILE", value_parser, required = true)]
output: Vec<PathBuf>,
/// Path to input info TOML.
#[arg(long, value_name = "FILE", value_parser, default_value = "lp.toml")]
input_info: PathBuf,
/// Path to input images directory.
#[arg(long, value_name = "DIR", value_parser, default_value = "lp_images")]
input_images: PathBuf,
}
/// Repack an LP image.
///
/// This command is equivalent to running `unpack` and `pack`, except without
/// storing the unpacked data to disk.
#[derive(Debug, Parser)]
struct RepackCli {
/// Path to input LP images.
///
/// If there are multiple images, they must be specified in order. If the
/// order is unknown, run `avbroot lp info` against the `super` image and
/// look at the `block_devices` field.
#[arg(short, long, value_name = "FILE", value_parser, required = true)]
input: Vec<PathBuf>,
/// Path to output LP images.
///
/// The number of output images must equal the number of input images.
#[arg(short, long, value_name = "FILE", value_parser, required = true)]
output: Vec<PathBuf>,
/// The LP metadata slot to use.
///
/// This slot is the only slot where data extents are copied to the output
/// images. Any data referenced exclusively by other slots (if any) will be
/// ignored.
///
/// This option is required if not all slots are identical.
#[arg(short, long)]
slot: Option<u32>,
}
/// Display LP image metadata.
#[derive(Debug, Parser)]
struct InfoCli {
/// Path to input LP image.
///
/// If there are multiple images, this should refer to the first one, which
/// is usually the `super` image. The other images are not needed when
/// inspecting the metadata because the metadata is only stored in the first
/// image.
#[arg(short, long, value_name = "FILE", value_parser)]
input: PathBuf,
}
#[derive(Debug, Subcommand)]
enum LpCommand {
Unpack(UnpackCli),
Pack(PackCli),
Repack(RepackCli),
Info(InfoCli),
}
/// Pack, unpack, and inspect LP images.
#[derive(Debug, Parser)]
pub struct LpCli {
#[command(subcommand)]
command: LpCommand,
/// Don't print LP metadata information.
#[arg(short, long, global = true)]
quiet: bool,
}
+1
View File
@@ -11,5 +11,6 @@ pub mod cpio;
pub mod fec;
pub mod hashtree;
pub mod key;
pub mod lp;
pub mod ota;
pub mod payload;
+1 -1
View File
@@ -461,7 +461,7 @@ enum PayloadCommand {
Info(InfoCli),
}
/// Inspect OTA payloads.
/// Pack, unpack, and inspect OTA payloads.
#[derive(Debug, Parser)]
pub struct PayloadCli {
#[command(subcommand)]
File diff suppressed because it is too large Load Diff
+1
View File
@@ -9,6 +9,7 @@ pub mod compression;
pub mod cpio;
pub mod fec;
pub mod hashtree;
pub mod lp;
pub mod ota;
pub mod padding;
pub mod payload;
+428
View File
@@ -0,0 +1,428 @@
/*
* SPDX-FileCopyrightText: 2024 Andrew Gunnerson
* SPDX-License-Identifier: GPL-3.0-only
*/
use std::{io::Cursor, num::NonZeroU64};
use avbroot::{
format::lp::{
BlockDevice, BlockDeviceFlags, Extent, ExtentType, HeaderFlags, ImageType, Metadata,
MetadataSlot, Partition, PartitionAttributes, PartitionGroup, PartitionGroupFlags,
},
stream::{FromReader, ToWriter},
};
fn round_trip(metadata: &Metadata, sha512: &[u8; 64]) {
let mut writer = Cursor::new(Vec::new());
metadata.to_writer(&mut writer).unwrap();
let data = writer.into_inner();
assert_eq!(
ring::digest::digest(&ring::digest::SHA512, &data).as_ref(),
sha512,
);
let mut reader = Cursor::new(&data);
let new_metadata = Metadata::from_reader(&mut reader).unwrap();
assert_eq!(&new_metadata, metadata);
}
#[test]
fn round_trip_empty_image() {
// Layout from Google Pixel 9 Pro XL stock factory image:
// komodo-ad1a.240530.047-factory-bb04e484.zip -> super_empty.img
let metadata = Metadata {
image_type: ImageType::Empty,
metadata_max_size: 65536,
metadata_slot_count: 3,
logical_block_size: 4096,
slots: vec![MetadataSlot {
major_version: 10,
minor_version: 2,
groups: vec![
PartitionGroup {
name: "default".into(),
flags: PartitionGroupFlags::empty(),
maximum_size: None,
partitions: vec![],
},
PartitionGroup {
name: "google_dynamic_partitions_a".into(),
flags: PartitionGroupFlags::empty(),
maximum_size: NonZeroU64::new(8527020032),
partitions: vec![
Partition {
name: "system_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "system_dlkm_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "system_ext_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "product_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "vendor_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "vendor_dlkm_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
],
},
PartitionGroup {
name: "google_dynamic_partitions_b".into(),
flags: PartitionGroupFlags::empty(),
maximum_size: NonZeroU64::new(8527020032),
partitions: vec![
Partition {
name: "system_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "system_dlkm_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "system_ext_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "product_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "vendor_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "vendor_dlkm_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
],
},
],
block_devices: vec![BlockDevice {
first_logical_sector: 2048,
alignment: 1048576,
alignment_offset: 0,
size: 8531214336,
partition_name: "super".into(),
flags: BlockDeviceFlags::empty(),
}],
flags: HeaderFlags::VIRTUAL_AB_DEVICE,
}],
};
// This is semantically equivalent, but not identical. The Metadata data
// structure only retains the order of partitions within a group, but not
// globally. This checksum is meant to protect against unintended future
// changes.
let sha512 = [
0xfa, 0xdf, 0xf2, 0xb6, 0x74, 0xec, 0x78, 0x7d, 0x0f, 0x7d, 0x17, 0x54, 0xcf, 0x1b, 0x53,
0x13, 0x66, 0x13, 0x5e, 0x8e, 0xcc, 0x84, 0xa2, 0x63, 0xaf, 0x0d, 0x68, 0x96, 0xc6, 0x40,
0x4e, 0x83, 0xe4, 0xe9, 0xef, 0x61, 0xdc, 0x2a, 0x25, 0x5f, 0xa2, 0x7d, 0x29, 0x0b, 0xb6,
0x26, 0x93, 0x59, 0xc9, 0xa8, 0x56, 0x3b, 0x3d, 0x3d, 0x15, 0x6b, 0xee, 0x78, 0x56, 0x78,
0xa1, 0x83, 0x6d, 0x70,
];
round_trip(&metadata, &sha512);
}
#[test]
fn round_trip_normal_image() {
// Layout from Google Pixel 9 Pro XL GrapheneOS factory image:
// komodo-install-2024082500.zip -> super_1.img
let slot = MetadataSlot {
major_version: 10,
minor_version: 2,
groups: vec![
PartitionGroup {
name: "default".into(),
flags: PartitionGroupFlags::empty(),
maximum_size: None,
partitions: vec![],
},
PartitionGroup {
name: "google_dynamic_partitions_a".into(),
flags: PartitionGroupFlags::empty(),
maximum_size: NonZeroU64::new(8527020032),
partitions: vec![
Partition {
name: "system_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![Extent {
num_sectors: 2465952,
extent_type: ExtentType::Linear {
start_sector: 2048,
block_device_index: 0,
},
}],
},
Partition {
name: "system_dlkm_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![Extent {
num_sectors: 23720,
extent_type: ExtentType::Linear {
start_sector: 2469888,
block_device_index: 0,
},
}],
},
Partition {
name: "system_ext_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![Extent {
num_sectors: 786144,
extent_type: ExtentType::Linear {
start_sector: 2494464,
block_device_index: 0,
},
}],
},
Partition {
name: "product_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![Extent {
num_sectors: 1396432,
extent_type: ExtentType::Linear {
start_sector: 3280896,
block_device_index: 0,
},
}],
},
Partition {
name: "vendor_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![Extent {
num_sectors: 1959024,
extent_type: ExtentType::Linear {
start_sector: 4677632,
block_device_index: 0,
},
}],
},
Partition {
name: "vendor_dlkm_a".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![Extent {
num_sectors: 55008,
extent_type: ExtentType::Linear {
start_sector: 6637568,
block_device_index: 0,
},
}],
},
],
},
PartitionGroup {
name: "google_dynamic_partitions_b".into(),
flags: PartitionGroupFlags::empty(),
maximum_size: NonZeroU64::new(8527020032),
partitions: vec![
Partition {
name: "system_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "system_dlkm_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "system_ext_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "product_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "vendor_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
Partition {
name: "vendor_dlkm_b".into(),
attributes: PartitionAttributes::READONLY,
extents: vec![],
},
],
},
],
block_devices: vec![BlockDevice {
first_logical_sector: 2048,
alignment: 1048576,
alignment_offset: 0,
size: 8531214336,
partition_name: "super".into(),
flags: BlockDeviceFlags::empty(),
}],
flags: HeaderFlags::VIRTUAL_AB_DEVICE,
};
let metadata = Metadata {
image_type: ImageType::Normal,
metadata_max_size: 65536,
metadata_slot_count: 3,
logical_block_size: 4096,
slots: vec![slot; 3],
};
// This is semantically equivalent, but not identical. The Metadata data
// structure only retains the order of partitions within a group, but not
// globally. This checksum is meant to protect against unintended future
// changes.
let sha512 = [
0x3b, 0xad, 0xd4, 0x22, 0xa1, 0x5a, 0xc5, 0xdf, 0x72, 0x7d, 0x92, 0x35, 0x04, 0x8a, 0x75,
0xd9, 0x33, 0x0d, 0xaa, 0x9e, 0x97, 0xd4, 0x13, 0x28, 0x5e, 0x0f, 0x12, 0x0c, 0xf2, 0xb3,
0xdc, 0x35, 0x89, 0x65, 0x40, 0xb0, 0x67, 0xb1, 0x54, 0x09, 0x52, 0x3e, 0x78, 0x3d, 0x3f,
0xa7, 0xf7, 0xa0, 0x77, 0xa8, 0xfc, 0xb7, 0x93, 0x19, 0xcd, 0x43, 0xea, 0x9a, 0x74, 0x65,
0x54, 0x3c, 0xaa, 0x12,
];
round_trip(&metadata, &sha512);
}
#[test]
fn round_trip_retrofit_image() {
// Layout from Google Pixel 3a XL stock factory image:
// bonito-ota-sp2a.220505.008-37a410d5.zip -> system.img
let slot = MetadataSlot {
major_version: 10,
minor_version: 0,
groups: vec![
PartitionGroup {
name: "default".into(),
flags: PartitionGroupFlags::empty(),
maximum_size: None,
partitions: vec![],
},
PartitionGroup {
name: "google_dynamic_partitions".into(),
flags: PartitionGroupFlags::SLOT_SUFFIXED,
maximum_size: NonZeroU64::new(4068474880),
partitions: vec![
Partition {
name: "system".into(),
attributes: PartitionAttributes::READONLY
| PartitionAttributes::SLOT_SUFFIXED,
extents: vec![Extent {
num_sectors: 1757416,
extent_type: ExtentType::Linear {
start_sector: 2048,
block_device_index: 0,
},
}],
},
Partition {
name: "vendor".into(),
attributes: PartitionAttributes::READONLY
| PartitionAttributes::SLOT_SUFFIXED,
extents: vec![Extent {
num_sectors: 991848,
extent_type: ExtentType::Linear {
start_sector: 1761280,
block_device_index: 0,
},
}],
},
Partition {
name: "product".into(),
attributes: PartitionAttributes::READONLY
| PartitionAttributes::SLOT_SUFFIXED,
extents: vec![
Extent {
num_sectors: 3627008,
extent_type: ExtentType::Linear {
start_sector: 2754560,
block_device_index: 0,
},
},
Extent {
num_sectors: 538240,
extent_type: ExtentType::Linear {
start_sector: 2048,
block_device_index: 1,
},
},
],
},
Partition {
name: "system_ext".into(),
attributes: PartitionAttributes::READONLY
| PartitionAttributes::SLOT_SUFFIXED,
extents: vec![Extent {
num_sectors: 490744,
extent_type: ExtentType::Linear {
start_sector: 540672,
block_device_index: 1,
},
}],
},
],
},
],
block_devices: vec![
BlockDevice {
first_logical_sector: 2048,
alignment: 1048576,
alignment_offset: 0,
size: 3267362816,
partition_name: "system".into(),
flags: BlockDeviceFlags::SLOT_SUFFIXED,
},
BlockDevice {
first_logical_sector: 2048,
alignment: 1048576,
alignment_offset: 0,
size: 805306368,
partition_name: "vendor".into(),
flags: BlockDeviceFlags::SLOT_SUFFIXED,
},
],
flags: HeaderFlags::empty(),
};
let metadata = Metadata {
image_type: ImageType::Normal,
metadata_max_size: 65536,
metadata_slot_count: 2,
logical_block_size: 4096,
slots: vec![slot; 2],
};
// First 274432 bytes of system.img. Unlike the other test cases, this is
// identical to the original image because there is only one partition group
// with partitions, so the group-level ordering is the same as the global
// ordering.
let sha512 = [
0xb9, 0x97, 0xf5, 0x83, 0x39, 0x37, 0x90, 0x0a, 0xb6, 0x46, 0xdd, 0x27, 0x57, 0xf1, 0xf3,
0xbd, 0x8f, 0xc4, 0x63, 0x07, 0x6f, 0xf4, 0x19, 0xc0, 0x02, 0x28, 0x48, 0x99, 0x54, 0xbb,
0xb3, 0xbf, 0x67, 0x95, 0xc4, 0xa7, 0x99, 0xf4, 0xa9, 0xc4, 0xf4, 0x1d, 0xf7, 0x59, 0x28,
0xeb, 0xbc, 0x85, 0x46, 0xd1, 0x7d, 0x65, 0x0f, 0xbe, 0x21, 0xf6, 0xf2, 0xa2, 0x20, 0x5b,
0xda, 0xde, 0xfa, 0x50,
];
round_trip(&metadata, &sha512);
}
+21
View File
@@ -0,0 +1,21 @@
#[cfg(not(windows))]
mod fuzz {
use std::io::Cursor;
use avbroot::{format::lp::Metadata, stream::FromReader};
use honggfuzz::fuzz;
pub fn main() {
loop {
fuzz!(|data: &[u8]| {
let reader = Cursor::new(data);
let _ = Metadata::from_reader(reader);
});
}
}
}
fn main() {
#[cfg(not(windows))]
fuzz::main();
}