WIP: mobile access over QUIC + iroh relays (p2p)

Prototype replacing the lapstone WebSocket tunnel with ACP carried over
iroh QUIC streams: relay-discoverable, direct path when possible, E2E
encrypted (relay sees ciphertext only).

- crates/goose/src/acp/transport/iroh.rs: iroh ACP transport (server)
- crates/goose-cli: `goose serve --iroh` launch + stable NodeId
- crates/goose-tunnel-ffi: goose-owned UniFFI client crate (not iroh-ffi)
- swift-driver: pure-Swift end-to-end + mobile suspend/resume harnesses
- documentation/notes: research + prototype status

Proven end-to-end Swift -> FFI -> iroh QUIC -> goosed ACP on both
direct and relay paths, across simulated suspend/resume cycles.

Signed-off-by: Michael Neale <michael.neale@gmail.com>
This commit is contained in:
Michael Neale
2026-06-02 09:17:40 +10:00
parent 25ff547487
commit 54d04f7970
16 changed files with 2887 additions and 49 deletions
Generated
+1608 -48
View File
File diff suppressed because it is too large Load Diff
+2
View File
@@ -64,6 +64,8 @@ sha2 = { workspace = true }
sigstore-verify = { version = "0.8", default-features = false, optional = true }
axum.workspace = true
clap_complete_nushell = { version = "4", default-features = false }
iroh = "1.0.0-rc.1"
hex = "0.4.3"
[target.'cfg(target_os = "windows")'.dependencies]
anstream = { version = "1", default-features = false, features = ["wincon"] }
+66 -1
View File
@@ -831,6 +831,10 @@ enum Command {
action = clap::ArgAction::Append
)]
builtins: Vec<String>,
/// Serve ACP over an iroh QUIC transport (relay + direct) instead of HTTP
#[arg(long)]
iroh: bool,
},
/// Start or resume interactive chat sessions
@@ -1368,6 +1372,60 @@ async fn handle_serve_command(host: String, port: u16, builtins: Vec<String>) ->
Ok(())
}
async fn handle_serve_iroh_command(builtins: Vec<String>) -> Result<()> {
use goose::acp::server_factory::{AcpServer, AcpServerFactoryConfig};
use goose::acp::transport::iroh as iroh_transport;
use goose::config::paths::Paths;
use std::sync::Arc;
use tracing::info;
let builtins = if builtins.is_empty() {
vec!["developer".to_string()]
} else {
builtins
};
let server = Arc::new(AcpServer::new(AcpServerFactoryConfig {
builtins,
data_dir: Paths::data_dir(),
config_dir: Paths::config_dir(),
goose_platform: GoosePlatform::GooseCli,
additional_source_roots: Vec::new(),
}));
// Persist a stable NodeId across restarts so the paired client keeps working.
let secret_key = load_or_create_iroh_secret_key()?;
let (endpoint, token) = iroh_transport::bind_server(secret_key).await?;
info!("iroh ACP server ready");
println!("\n=== goose iroh ACP server ===");
println!("NodeId: {}", endpoint.id());
println!("Connection token (QR payload):\n{token}\n");
iroh_transport::serve(endpoint, server).await
}
fn load_or_create_iroh_secret_key() -> Result<iroh::SecretKey> {
use goose::config::paths::Paths;
use std::io::Write as _;
let path = Paths::config_dir().join("iroh_acp_secret.key");
if let Ok(hex_str) = std::fs::read_to_string(&path) {
if let Ok(bytes) = hex::decode(hex_str.trim()) {
if let Ok(arr) = <[u8; 32]>::try_from(bytes.as_slice()) {
return Ok(iroh::SecretKey::from_bytes(&arr));
}
}
}
let key = iroh::SecretKey::generate();
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
let mut f = std::fs::File::create(&path)?;
write!(f, "{}", hex::encode(key.to_bytes()))?;
Ok(key)
}
async fn handle_session_subcommand(command: SessionCommand) -> Result<()> {
match command {
SessionCommand::List {
@@ -2091,7 +2149,14 @@ pub async fn cli() -> anyhow::Result<()> {
host,
port,
builtins,
}) => handle_serve_command(host, port, builtins).await,
iroh,
}) => {
if iroh {
handle_serve_iroh_command(builtins).await
} else {
handle_serve_command(host, port, builtins).await
}
}
Some(Command::Session {
command: Some(cmd), ..
}) => handle_session_subcommand(cmd).await,
+26
View File
@@ -0,0 +1,26 @@
[package]
name = "goose-tunnel-ffi"
version = "0.1.0"
edition = "2021"
publish = false
[lib]
name = "goose_tunnel_ffi"
crate-type = ["staticlib", "cdylib", "lib"]
[[bin]]
name = "uniffi-bindgen"
path = "src/uniffi-bindgen.rs"
[dependencies]
iroh = "1.0.0-rc.1"
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "io-util", "sync", "time"] }
tokio-util = { workspace = true, features = ["compat"] }
futures = { workspace = true }
anyhow = { workspace = true }
base64 = { workspace = true }
serde_json = { workspace = true }
uniffi = { version = "0.28", features = ["cli"] }
tracing = { workspace = true }
thiserror = { workspace = true }
hex = "0.4"
+226
View File
@@ -0,0 +1,226 @@
//! goose-owned FFI for the iroh ACP tunnel.
//!
//! Exposes a small, goose-shaped surface to Swift (and Kotlin) over UniFFI:
//! connect to a paired goosed over iroh QUIC, open one ACP bidi stream, send
//! newline-delimited JSON-RPC lines, and receive inbound lines via a callback.
//!
//! ACP itself lives in Swift — this crate is only the authenticated byte pipe
//! (iroh: relay + direct path + NodeId identity). No HTTP, no Rust ACP logic.
use std::sync::Arc;
use base64::Engine as _;
use iroh::{endpoint::Endpoint, RelayConfig, RelayMap, RelayMode, RelayUrl, SecretKey};
use tokio::io::AsyncWriteExt;
use tokio::sync::{mpsc, Mutex};
uniffi::setup_scaffolding!();
const ALPN_GOOSE_ACP_V1: &[u8] = b"goose-acp/1";
const DEFAULT_RELAYS: &[&str] = &[
"https://usw1-2.relay.michaelneale.mesh-llm.iroh.link./",
"https://aps1-1.relay.michaelneale.mesh-llm.iroh.link./",
];
#[derive(Debug, thiserror::Error, uniffi::Error)]
pub enum TunnelError {
#[error("invalid connection token: {0}")]
InvalidToken(String),
#[error("connection failed: {0}")]
ConnectFailed(String),
#[error("transport error: {0}")]
Transport(String),
}
/// Whether traffic is flowing direct (hole-punched) or via a relay.
#[derive(Debug, Clone, Copy, uniffi::Enum)]
pub enum PathKind {
Connecting,
Direct,
Relayed,
}
/// Implemented in Swift; receives each inbound newline-delimited JSON-RPC line.
#[uniffi::export(with_foreign)]
pub trait MessageListener: Send + Sync {
fn on_message(&self, line: String);
fn on_closed(&self, reason: String);
}
/// Generate a device keypair (hex-encoded 32-byte secret). The NodeId is the
/// public key; persist this in the Keychain so the device identity is stable.
#[uniffi::export]
pub fn generate_device_keypair() -> String {
let key = SecretKey::generate();
hex::encode(key.to_bytes())
}
fn relay_mode() -> RelayMode {
let configs = DEFAULT_RELAYS
.iter()
.filter_map(|u| u.parse::<RelayUrl>().ok())
.map(|url| RelayConfig::new(url, None));
RelayMode::Custom(RelayMap::from_iter(configs))
}
fn decode_addr(token: &str) -> Result<iroh::EndpointAddr, TunnelError> {
let json = base64::engine::general_purpose::URL_SAFE_NO_PAD
.decode(token)
.map_err(|e| TunnelError::InvalidToken(e.to_string()))?;
serde_json::from_slice(&json).map_err(|e| TunnelError::InvalidToken(e.to_string()))
}
fn device_key(hex_key: &str) -> Result<SecretKey, TunnelError> {
let bytes = hex::decode(hex_key).map_err(|e| TunnelError::InvalidToken(e.to_string()))?;
let arr: [u8; 32] = bytes
.as_slice()
.try_into()
.map_err(|_| TunnelError::InvalidToken("device key must be 32 bytes".into()))?;
Ok(SecretKey::from_bytes(&arr))
}
/// A connected ACP tunnel: one iroh QUIC bidi stream carrying JSON-RPC lines.
#[derive(uniffi::Object)]
pub struct GooseTunnel {
runtime: Arc<tokio::runtime::Runtime>,
send_tx: mpsc::UnboundedSender<String>,
endpoint: Endpoint,
server_addr: iroh::EndpointAddr,
closed: Arc<Mutex<bool>>,
}
#[uniffi::export]
impl GooseTunnel {
/// Send one newline-delimited JSON-RPC line to the agent.
pub fn send(&self, line: String) -> Result<(), TunnelError> {
self.send_tx
.send(line)
.map_err(|e| TunnelError::Transport(e.to_string()))
}
/// Direct vs relayed, observed from iroh's path info.
pub fn path_kind(&self) -> PathKind {
let info = self
.runtime
.block_on(async { self.endpoint.remote_info(self.server_addr.id).await });
match info {
Some(info) => {
let has_direct = info
.addrs()
.any(|a| matches!(a.addr(), iroh::TransportAddr::Ip(_)));
if has_direct {
PathKind::Direct
} else {
PathKind::Relayed
}
}
None => PathKind::Connecting,
}
}
pub fn disconnect(&self) {
self.runtime.block_on(async {
*self.closed.lock().await = true;
self.endpoint.close().await;
});
}
}
/// Connect to a paired goosed over iroh and open one ACP stream.
///
/// `server_token` is the base64url `EndpointAddr` from the QR. `device_key_hex`
/// is this device's persisted keypair. Inbound lines are delivered to `listener`.
#[uniffi::export]
pub fn connect(
server_token: String,
device_key_hex: String,
listener: Arc<dyn MessageListener>,
) -> Result<Arc<GooseTunnel>, TunnelError> {
let runtime = Arc::new(
tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()
.map_err(|e| TunnelError::Transport(e.to_string()))?,
);
let server_addr = decode_addr(&server_token)?;
let secret = device_key(&device_key_hex)?;
let (send_tx, mut send_rx) = mpsc::unbounded_channel::<String>();
let closed = Arc::new(Mutex::new(false));
let endpoint = runtime
.block_on(async {
Endpoint::builder(iroh::endpoint::presets::Minimal)
.secret_key(secret)
.alpns(vec![ALPN_GOOSE_ACP_V1.to_vec()])
.relay_mode(relay_mode())
.bind()
.await
})
.map_err(|e| TunnelError::ConnectFailed(e.to_string()))?;
// Connect and open the bidi stream.
let (mut send, mut recv) = runtime
.block_on(async {
let _ =
tokio::time::timeout(std::time::Duration::from_secs(10), endpoint.online()).await;
let conn = endpoint
.connect(server_addr.clone(), ALPN_GOOSE_ACP_V1)
.await
.map_err(|e| anyhow::anyhow!(e.to_string()))?;
conn.open_bi()
.await
.map_err(|e| anyhow::anyhow!(e.to_string()))
})
.map_err(|e: anyhow::Error| TunnelError::ConnectFailed(e.to_string()))?;
let closed_writer = closed.clone();
runtime.spawn(async move {
while let Some(line) = send_rx.recv().await {
if *closed_writer.lock().await {
break;
}
let mut buf = line.into_bytes();
buf.push(b'\n');
if send.write_all(&buf).await.is_err() {
break;
}
let _ = send.flush().await;
}
});
let listener_reader = listener.clone();
let closed_reader = closed.clone();
runtime.spawn(async move {
let mut buf = Vec::with_capacity(8192);
let mut chunk = [0u8; 4096];
loop {
match recv.read(&mut chunk).await {
Ok(None) | Ok(Some(0)) | Err(_) => {
listener_reader.on_closed("stream ended".into());
*closed_reader.lock().await = true;
break;
}
Ok(Some(n)) => {
buf.extend_from_slice(&chunk[..n]);
while let Some(pos) = buf.iter().position(|&b| b == b'\n') {
let line: Vec<u8> = buf.drain(..=pos).collect();
let line = String::from_utf8_lossy(&line[..line.len() - 1]).to_string();
if !line.is_empty() {
listener_reader.on_message(line);
}
}
}
}
}
});
Ok(Arc::new(GooseTunnel {
runtime,
send_tx,
endpoint,
server_addr,
closed,
}))
}
@@ -0,0 +1,3 @@
fn main() {
uniffi::uniffi_bindgen_main()
}
@@ -0,0 +1,66 @@
import Foundation
// End-to-end Swift driver: connect to goosed over iroh, speak ACP (newline JSON-RPC).
// Proves: pure Swift -> goose-owned FFI -> iroh QUIC (relay/direct) -> goosed ACP.
//
// Build/run via crates/goose-tunnel-ffi/swift-driver/run.sh
final class ACPListener: MessageListener {
let sema: DispatchSemaphore
var sawInitialize = false
init(_ sema: DispatchSemaphore) { self.sema = sema }
func onMessage(line: String) {
print("⬅️ \(line)")
if let data = line.data(using: .utf8),
let obj = try? JSONSerialization.jsonObject(with: data) as? [String: Any],
obj["result"] != nil, obj["id"] as? Int == 1 {
sawInitialize = true
print("✅ ACP initialize round-trip succeeded over iroh")
sema.signal()
}
}
func onClosed(reason: String) {
print("🔌 stream closed: \(reason)")
sema.signal()
}
}
guard CommandLine.arguments.count >= 2 else {
FileHandle.standardError.write("usage: driver <server-token>\n".data(using: .utf8)!)
exit(2)
}
let token = CommandLine.arguments[1]
let deviceKey = generateDeviceKeypair()
print("🔑 device key (NodeId pinned on pairing): \(deviceKey.prefix(16))")
let done = DispatchSemaphore(value: 0)
let listener = ACPListener(done)
do {
print("🌐 connecting over iroh…")
let tunnel = try connect(serverToken: token, deviceKeyHex: deviceKey, listener: listener)
print("🛣️ path: \(tunnel.pathKind())")
let initReq: [String: Any] = [
"jsonrpc": "2.0",
"id": 1,
"method": "initialize",
"params": ["protocolVersion": 1],
]
let data = try JSONSerialization.data(withJSONObject: initReq)
let line = String(data: data, encoding: .utf8)!
print("➡️ \(line)")
try tunnel.send(line: line)
let result = done.wait(timeout: .now() + 30)
if result == .timedOut { print("⏱️ timed out waiting for ACP response") }
print("🛣️ final path: \(tunnel.pathKind())")
tunnel.disconnect()
exit(listener.sawInitialize ? 0 : 1)
} catch {
print("\(error)")
exit(1)
}
@@ -0,0 +1,84 @@
import Foundation
// Mobile lifecycle harness: exercises the iOS suspend/resume cycle against a real
// goosed iroh server, the way the mobile app would actually behave.
//
// foreground -> connect -> ACP initialize
// background -> disconnect (iOS freezes the process; UDP socket dies)
// foreground -> reconnect (fresh tunnel) -> ACP initialize again [x3]
//
// This is the part a plain CLI run does NOT cover: connection teardown on
// suspend and fast re-establishment on resume, repeatedly, over real QUIC/relay.
final class CycleListener: MessageListener {
private let sema: DispatchSemaphore
private(set) var gotInit = false
init(_ s: DispatchSemaphore) { sema = s }
func onMessage(line: String) {
if let d = line.data(using: .utf8),
let o = try? JSONSerialization.jsonObject(with: d) as? [String: Any],
o["result"] != nil, (o["id"] as? Int) == 1 {
gotInit = true
sema.signal()
}
}
func onClosed(reason: String) { /* expected on background teardown */ }
}
@main
struct MobileLifecycle {
static func foregroundConnectAndInit(token: String, deviceKey: String) -> (ok: Bool, path: String, ms: Int) {
let sema = DispatchSemaphore(value: 0)
let listener = CycleListener(sema)
let start = Date()
do {
let tunnel = try connect(serverToken: token, deviceKeyHex: deviceKey, listener: listener)
let initReq: [String: Any] = ["jsonrpc": "2.0", "id": 1, "method": "initialize",
"params": ["protocolVersion": 1]]
let line = String(data: try JSONSerialization.data(withJSONObject: initReq), encoding: .utf8)!
try tunnel.send(line: line)
let waited = sema.wait(timeout: .now() + 20)
let ms = Int(Date().timeIntervalSince(start) * 1000)
let path = "\(tunnel.pathKind())"
// Simulate iOS backgrounding: clean-close so we don't leave a zombie.
tunnel.disconnect()
return (waited == .success && listener.gotInit, path, ms)
} catch {
return (false, "error: \(error)", Int(Date().timeIntervalSince(start) * 1000))
}
}
static func main() {
guard CommandLine.arguments.count >= 2 else {
FileHandle.standardError.write("usage: mobile-lifecycle <server-token>\n".data(using: .utf8)!)
exit(2)
}
let token = CommandLine.arguments[1]
// The device identity is generated once and persisted (Keychain on a real
// phone). Reused across every foreground/reconnect so the server sees a
// stable NodeId.
let deviceKey = generateDeviceKeypair()
print("📱 device identity (stable across suspend/resume): \(deviceKey.prefix(16))")
var allOk = true
let cycles = 3
for cycle in 1...cycles {
print("\n🔆 FOREGROUND #\(cycle): connecting over iroh…")
let r = foregroundConnectAndInit(token: token, deviceKey: deviceKey)
if r.ok {
print(" ✅ ACP initialize ok | path: \(r.path) | \(r.ms)ms")
} else {
print(" ❌ failed | \(r.path) | \(r.ms)ms")
allOk = false
}
if cycle < cycles {
print("🌙 BACKGROUND: app suspended, tunnel torn down (UDP frozen on real iOS)")
Thread.sleep(forTimeInterval: 2.0)
}
}
print("\n\(allOk ? "✅ PASS" : "❌ FAIL"): survived \(cycles) suspend/resume cycles with stable device identity")
exit(allOk ? 0 : 1)
}
}
@@ -0,0 +1,49 @@
import Foundation
// Path probe: connect with a relay-only token (the off-network mobile case) and
// sample path_kind() over time to observe iroh's relay -> direct upgrade.
//
// On a single host a loopback direct path always exists, so iroh correctly
// upgrades relay -> direct after the disco handshake. This probe documents that
// progression honestly rather than pretending the relay path is permanent.
final class Probe: MessageListener {
let sema = DispatchSemaphore(value: 0)
var gotInit = false
func onMessage(line: String) {
if let d = line.data(using: .utf8),
let o = try? JSONSerialization.jsonObject(with: d) as? [String: Any],
o["result"] != nil, (o["id"] as? Int) == 1 { gotInit = true; sema.signal() }
}
func onClosed(reason: String) {}
}
@main
struct PathProbe {
static func main() {
guard CommandLine.arguments.count >= 2 else {
FileHandle.standardError.write("usage: path-probe <server-token>\n".data(using: .utf8)!)
exit(2)
}
let token = CommandLine.arguments[1]
let deviceKey = generateDeviceKeypair()
let probe = Probe()
do {
let tunnel = try connect(serverToken: token, deviceKeyHex: deviceKey, listener: probe)
print("t=0.0s path: \(tunnel.pathKind()) (just connected)")
let initReq: [String: Any] = ["jsonrpc": "2.0", "id": 1, "method": "initialize",
"params": ["protocolVersion": 1]]
try tunnel.send(line: String(data: try JSONSerialization.data(withJSONObject: initReq), encoding: .utf8)!)
_ = probe.sema.wait(timeout: .now() + 20)
print(" ACP initialize: \(probe.gotInit ? "ok" : "FAILED")")
for t in [1.0, 2.0, 4.0] {
Thread.sleep(forTimeInterval: t == 1.0 ? 1.0 : (t == 2.0 ? 1.0 : 2.0))
print("t=\(t)s path: \(tunnel.pathKind())")
}
tunnel.disconnect()
exit(probe.gotInit ? 0 : 1)
} catch {
print("\(error)"); exit(1)
}
}
}
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
# Build + run the mobile-lifecycle Swift harness (suspend/resume cycles) against
# a goose iroh ACP server.
#
# 1. cargo build -p goose-cli --bin goose -p goose-tunnel-ffi
# 2. ./target/debug/goose serve --iroh # copy the printed connection token
# 3. crates/goose-tunnel-ffi/swift-driver/run-mobile.sh "<connection-token>"
set -euo pipefail
TOKEN="${1:?usage: run-mobile.sh <connection-token>}"
REPO_ROOT="$(cd "$(dirname "$0")/../../.." && pwd)"
LIBDIR="$REPO_ROOT/target/debug"
DRIVER_DIR="$(cd "$(dirname "$0")" && pwd)"
BUILD_DIR="$(mktemp -d)"
cargo build -p goose-tunnel-ffi --manifest-path "$REPO_ROOT/Cargo.toml" >/dev/null
cargo run -p goose-tunnel-ffi --bin uniffi-bindgen --manifest-path "$REPO_ROOT/Cargo.toml" -- \
generate --library "$LIBDIR/libgoose_tunnel_ffi.dylib" --language swift --out-dir "$BUILD_DIR" >/dev/null 2>&1
echo 'module goose_tunnel_ffiFFI { header "goose_tunnel_ffiFFI.h" export * }' > "$BUILD_DIR/module.modulemap"
swiftc -parse-as-library -o "$BUILD_DIR/mobile" \
"$DRIVER_DIR/mobile-lifecycle.swift" "$BUILD_DIR/goose_tunnel_ffi.swift" \
-I "$BUILD_DIR" \
-L "$LIBDIR" -lgoose_tunnel_ffi \
-Xcc -fmodule-map-file="$BUILD_DIR/module.modulemap"
DYLD_LIBRARY_PATH="$LIBDIR" "$BUILD_DIR/mobile" "$TOKEN"
+27
View File
@@ -0,0 +1,27 @@
#!/usr/bin/env bash
# Build + run the Swift end-to-end driver against a goose iroh ACP server.
#
# 1. cargo build -p goose-cli --bin goose -p goose-tunnel-ffi
# 2. ./target/debug/goose serve --iroh # copy the printed connection token
# 3. crates/goose-tunnel-ffi/swift-driver/run.sh "<connection-token>"
set -euo pipefail
TOKEN="${1:?usage: run.sh <connection-token>}"
REPO_ROOT="$(cd "$(dirname "$0")/../../.." && pwd)"
LIBDIR="$REPO_ROOT/target/debug"
DRIVER_DIR="$(cd "$(dirname "$0")" && pwd)"
BUILD_DIR="$(mktemp -d)"
cargo build -p goose-tunnel-ffi --manifest-path "$REPO_ROOT/Cargo.toml" >/dev/null
cargo run -p goose-tunnel-ffi --bin uniffi-bindgen --manifest-path "$REPO_ROOT/Cargo.toml" -- \
generate --library "$LIBDIR/libgoose_tunnel_ffi.dylib" --language swift --out-dir "$BUILD_DIR" >/dev/null 2>&1
echo 'module goose_tunnel_ffiFFI { header "goose_tunnel_ffiFFI.h" export * }' > "$BUILD_DIR/module.modulemap"
swiftc -o "$BUILD_DIR/driver" "$DRIVER_DIR/main.swift" "$BUILD_DIR/goose_tunnel_ffi.swift" \
-I "$BUILD_DIR" \
-L "$LIBDIR" -lgoose_tunnel_ffi \
-Xcc -fmodule-map-file="$BUILD_DIR/module.modulemap"
DYLD_LIBRARY_PATH="$LIBDIR" "$BUILD_DIR/driver" "$TOKEN"
+1
View File
@@ -215,6 +215,7 @@ pctx_code_mode = { version = "0.3", default-features = false, optional = true }
icu_calendar = { version = "=2.1.1", default-features = false }
icu_locale = { version = "=2.1.1", default-features = false }
llama-cpp-sys-2 = { workspace = true, optional = true }
iroh = "1.0.0-rc.1"
[target.'cfg(target_os = "windows")'.dependencies]
winapi = { workspace = true }
+139
View File
@@ -0,0 +1,139 @@
//! iroh/QUIC transport for ACP.
//!
//! Serves ACP as newline-delimited JSON-RPC directly on an iroh QUIC bidi stream
//! (no HTTP layer). Each accepted bidi stream gets its own agent via
//! `acp::server::serve`, exactly like the stdio and channel-backed transports —
//! only the byte pipe changes.
//!
//! iroh provides: peer discovery (relay + hole-punched direct path), NodeId
//! identity (ed25519 public key, mutually verified by QUIC TLS 1.3), and an
//! authenticated QUIC connection. We bake in default relays so a paired client
//! can find the server behind NAT.
use std::sync::Arc;
use anyhow::{Context, Result};
use iroh::{
endpoint::{Endpoint, RecvStream, SendStream},
RelayMap, RelayMode, SecretKey,
};
use tokio_util::compat::{TokioAsyncReadCompatExt, TokioAsyncWriteCompatExt};
use tracing::{error, info, warn};
use crate::acp::server_factory::AcpServer;
/// ALPN for goose ACP over QUIC. Bumped if the on-wire ACP framing changes.
pub const ALPN_GOOSE_ACP_V1: &[u8] = b"goose-acp/1";
/// Default relays the client and server share so a phone can find a laptop
/// behind NAT. Overridable via `GOOSE_IROH_RELAYS` (comma-separated URLs).
pub const DEFAULT_RELAYS: &[&str] = &[
"https://usw1-2.relay.michaelneale.mesh-llm.iroh.link./",
"https://aps1-1.relay.michaelneale.mesh-llm.iroh.link./",
];
fn effective_relay_urls() -> Vec<String> {
match std::env::var("GOOSE_IROH_RELAYS") {
Ok(v) if !v.trim().is_empty() => v
.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect(),
_ => DEFAULT_RELAYS.iter().map(|s| s.to_string()).collect(),
}
}
fn relay_mode() -> Result<RelayMode> {
let urls = effective_relay_urls();
let configs = urls
.iter()
.map(|u| {
u.parse::<iroh::RelayUrl>()
.map(|url| iroh::RelayConfig::new(url, None))
.with_context(|| format!("invalid relay URL: {u}"))
})
.collect::<Result<Vec<_>>>()?;
Ok(RelayMode::Custom(RelayMap::from_iter(configs)))
}
/// Bind an iroh endpoint for serving ACP, returning the endpoint and the
/// base64url-encoded `EndpointAddr` token a client uses to connect (the QR
/// payload). `secret_key` persists the server's NodeId across restarts.
pub async fn bind_server(secret_key: SecretKey) -> Result<(Endpoint, String)> {
let endpoint = Endpoint::builder(iroh::endpoint::presets::Minimal)
.secret_key(secret_key)
.alpns(vec![ALPN_GOOSE_ACP_V1.to_vec()])
.relay_mode(relay_mode()?)
.bind()
.await?;
// Wait until we're reachable via a relay so the advertised addr is usable.
let _ = tokio::time::timeout(std::time::Duration::from_secs(10), endpoint.online()).await;
let addr = endpoint.addr();
let token = encode_addr_token(&addr)?;
info!(node_id = %endpoint.id(), "iroh ACP endpoint bound");
Ok((endpoint, token))
}
pub fn encode_addr_token(addr: &iroh::EndpointAddr) -> Result<String> {
use base64::Engine as _;
let json = serde_json::to_vec(addr)?;
Ok(base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(json))
}
pub fn decode_addr_token(token: &str) -> Result<iroh::EndpointAddr> {
use base64::Engine as _;
let json = base64::engine::general_purpose::URL_SAFE_NO_PAD
.decode(token)
.context("invalid endpoint token encoding")?;
serde_json::from_slice(&json).context("invalid endpoint token JSON")
}
/// Accept loop: each incoming connection may open multiple ACP bidi streams;
/// each stream is served by its own agent.
pub async fn serve(endpoint: Endpoint, server: Arc<AcpServer>) -> Result<()> {
info!("iroh ACP transport accepting connections");
while let Some(incoming) = endpoint.accept().await {
let server = server.clone();
tokio::spawn(async move {
if let Err(e) = handle_connection(incoming, server).await {
warn!("iroh connection ended: {e}");
}
});
}
Ok(())
}
async fn handle_connection(
incoming: iroh::endpoint::Incoming,
server: Arc<AcpServer>,
) -> Result<()> {
let connection = incoming.await?;
let remote = connection.remote_id();
info!(peer = %remote, "iroh ACP connection established");
loop {
match connection.accept_bi().await {
Ok((send, recv)) => {
let server = server.clone();
tokio::spawn(async move {
if let Err(e) = handle_stream(send, recv, server).await {
error!("iroh ACP stream error: {e}");
}
});
}
Err(e) => {
info!(peer = %remote, "iroh connection closed: {e}");
break;
}
}
}
Ok(())
}
async fn handle_stream(send: SendStream, recv: RecvStream, server: Arc<AcpServer>) -> Result<()> {
let agent = server.create_agent().await?;
// QUIC streams are tokio AsyncRead/AsyncWrite; serve() wants futures-io.
crate::acp::server::serve(agent, recv.compat(), send.compat_write()).await
}
+1
View File
@@ -1,5 +1,6 @@
pub mod connection;
pub mod http;
pub mod iroh;
pub mod websocket;
use std::sync::Arc;
@@ -0,0 +1,150 @@
# ACP-over-iroh mobile tunnel — working prototype
> Status: **end-to-end proven** on 2026-06-01. Pure-Swift client → goose-owned
> FFI → iroh QUIC (relay + direct) → goosed ACP, both direct and relay paths.
> All changes are local prototype code; see "What's prototype-quality" below.
## What was built
### 1. Server: iroh ACP transport in goose
`crates/goose/src/acp/transport/iroh.rs` (new) — binds an iroh `Endpoint`
(ALPN `goose-acp/1`), registers with the two baked-in relays
(`*.relay.michaelneale.mesh-llm.iroh.link`, override via `GOOSE_IROH_RELAYS`),
accepts QUIC connections, and serves **ACP as newline-delimited JSON-RPC directly
on each bidi stream** via the existing `acp::server::serve` (Option A — no HTTP
layer). Also encodes/decodes the `EndpointAddr` connection token (the QR payload).
Launch: `crates/goose-cli/src/cli.rs``goose serve --iroh`. Persists a stable
NodeId in `~/.config/goose/iroh_acp_secret.key`, prints the NodeId + base64url
connection token.
### 2. Client FFI: goose-owned, not iroh-ffi
`crates/goose-tunnel-ffi/` (new crate) — a small UniFFI surface, **not** n0's broad
`iroh-ffi`/`IrohLib`. Exposes only:
- `generate_device_keypair() -> String` (hex; NodeId = the device identity)
- `connect(server_token, device_key_hex, listener) -> GooseTunnel`
- `GooseTunnel.send(line)` / `.path_kind() -> Direct|Relayed|Connecting` / `.disconnect()`
- `trait MessageListener { on_message(line); on_closed(reason) }`
Internally: iroh client `Endpoint`, relays baked in, opens one ACP bidi stream,
newline-frames JSON-RPC both directions. **ACP logic stays in the (Swift) caller**
the FFI is just the authenticated byte pipe.
### 3. Swift driver (end-to-end proof, in-repo)
`crates/goose-tunnel-ffi/swift-driver/{main.swift,run.sh}` — pure Swift, links the
generated UniFFI bindings + the dylib. Generates a device key, connects via a token,
sends ACP `initialize`, parses the response. `run.sh` regenerates bindings, compiles,
and runs in one step. (A proof harness; the real iOS app would reuse its existing ACP
client over the same FFI.)
## What was verified (live)
```
# direct path (same LAN — token has Relay + Ip):
🛣️ path: direct
➡️ {"jsonrpc":"2.0","id":1,"method":"initialize",...}
⬅️ {"jsonrpc":"2.0","result":{"protocolVersion":1,"agentCapabilities":{...}}}
✅ ACP initialize round-trip succeeded over iroh (exit 0)
# relay path (off-network sim — token stripped to Relay only):
🛣️ path: relayed
⬅️ {"jsonrpc":"2.0","result":{...}} (exit 0)
```
- **Direct path:** iroh hole-punched a direct QUIC path on the LAN automatically.
- **Relay path:** with a relay-only token (the real off-network mobile case),
traffic flowed through the relay — E2E encrypted, relay sees only ciphertext.
- **Same ACP, both paths:** identical `initialize` round-trip; real goose agent
capabilities returned.
- **Identity:** QUIC TLS 1.3 mutually authenticates NodeIds; the device key is the
client's identity (pin it on pairing for authorization — not yet enforced, below).
## How to reproduce
```bash
# 1. build
cargo build -p goose-cli --bin goose -p goose-tunnel-ffi
# 2. server — prints NodeId + connection token
./target/debug/goose serve --iroh
# 3. run the Swift driver end to end (regenerates bindings, compiles, runs)
crates/goose-tunnel-ffi/swift-driver/run.sh "<connection-token>"
```
Verified reproducing from a clean tree on 2026-06-01: direct path (LAN) and relay
path (relay-only token) both exit 0 with a real ACP `initialize` round-trip.
## Mobile lifecycle testing (tested like a phone, not just a CLI)
A plain end-to-end run is a desktop-style process. What makes it *mobile* is the
lifecycle: suspend kills the UDP socket, foreground must reconnect fast, and the
path may be relay or direct. These were exercised with two extra in-repo harnesses:
- `swift-driver/mobile-lifecycle.swift` (run via `run-mobile.sh`) — **3
foreground/background cycles** with a **stable device identity** (generated once,
reused on every reconnect, like a Keychain-persisted key). Each foreground:
connect → ACP `initialize` → clean `disconnect()` (simulating iOS background
teardown). **Result: PASS** — every cycle reconnects cold and re-establishes ACP
in ~1.32.7s.
- `swift-driver/path-probe.swift` — connect with a **relay-only token** (the true
off-network/cellular case: client is given only the relay, no LAN address) and
sample `path_kind()` over time.
### What this proved (honestly)
- **Suspend/resume works:** cold reconnect + ACP re-init succeeds repeatedly with a
stable NodeId. This is the core mobile loop.
- **Relay path is real:** with a relay-only token the path stays `relayed` for the
whole session and ACP completes over the relay (E2E encrypted, relay sees only
ciphertext).
- **Direct path is real:** with the full token (relay + LAN IP) iroh hole-punches to
`direct` automatically.
### Honest single-host caveat
Client and server run on **one machine**, so a loopback direct path always exists.
After a relay-bootstrapped connection, iroh's disco can learn that loopback
candidate and upgrade `relayed → direct` on later reconnects — correct iroh
behavior, but it means a *sustained* relay-only path can't be guaranteed on one
host. A truly permanent relay path (no direct possible) needs two machines / real
NAT / cellular, or network-level blocking of the direct path. The relay path itself
is proven (probe + cycle #1); its permanence under real NAT is not single-host
testable.
### Not yet tested (needs a real device / Xcode app)
- iOS `scenePhase`/`NWPathMonitor`-driven reconnect (here it's simulated by explicit
disconnect/reconnect).
- QUIC connection migration on a live Wi-Fi↔cellular switch.
- Behavior across a real OS process freeze (vs. simulated teardown).
- Streaming `session/prompt` interrupted mid-turn by suspend + resume cursor.
## What's prototype-quality (not production)
1. **No device authorization yet.** Any client that has the token can connect
(transport identity is verified, but we don't pin/allow-list the device NodeId
on goosed). Next: capture device NodeId at QR-pairing, enforce an allow-list.
2. **One stream per connection, no resume cursor.** Suspend/resume (§4 of the
research note) — event-id cursor + `resume(session, after_event=N)` — not built.
3. **Token strips to relay-only manually** in the test; real client just uses the
full token and lets iroh choose (verified it does).
4. **FFI runs its own tokio runtime per connect** — fine for one tunnel; revisit
if multiple.
5. **Server reuses `acp::server::serve` per stream**, bypassing the
`ConnectionRegistry`/replay buffer. To get replay-on-reconnect, route through the
registry instead (small change; the registry is already transport-agnostic).
6. **iOS lifecycle glue not built** (no Xcode app here) — `scenePhase` +
`NWPathMonitor` → reconnect, `beginBackgroundTask` clean-close. Design in §5 of
the research note.
7. **xcframework packaging not done** — bindings generated for macOS dylib only;
the iOS build pipeline (lipo + create-xcframework for arm64/sim) is the
`make_swift.sh`-style step from §8.
8. **iroh `1.0.0-rc.1`** pinned in both goose and the FFI crate — keep them in lock
step (shared relay/disco wire protocol).
## Files changed
- `crates/goose/src/acp/transport/iroh.rs` (new)
- `crates/goose/src/acp/transport/mod.rs` (+`pub mod iroh;`)
- `crates/goose/Cargo.toml` (+`iroh`)
- `crates/goose-cli/src/cli.rs` (`--iroh` flag, `handle_serve_iroh_command`, key persistence)
- `crates/goose-cli/Cargo.toml` (+`iroh`, `hex`)
- `crates/goose-tunnel-ffi/` (new crate: `Cargo.toml`, `src/lib.rs`, `src/uniffi-bindgen.rs`)
- `crates/goose-tunnel-ffi/swift-driver/` (in-repo Swift harnesses + run scripts:
`main.swift`/`run.sh` = basic e2e; `mobile-lifecycle.swift`/`run-mobile.sh` =
suspend/resume cycles; `path-probe.swift` = relay-vs-direct path observation)
@@ -0,0 +1,410 @@
# Replacing the goose Mobile Tunnel with ACP over iroh/QUIC
> Research note — exploratory, no implementation. Proposes throwing out the
> "lapstone" HTTP-over-WebSocket tunnel and replacing it with **ACP carried over
> iroh QUIC streams**, reachable from a **pure-Swift iOS app via a goose-owned FFI
> framework** (`Goose.xcframework`, built with the n0 `iroh-ffi` *recipe* but not
> shipping their broad binding — no Rust source in the app). Framing decision: ACP
> as newline-delimited JSON-RPC directly on the QUIC stream (Option A, §2).
## 0. Corrections to earlier assumptions (read first)
Two things I previously got wrong, now verified:
1. **iroh *does* have an official Swift binding.** `github.com/n0-computer/iroh-ffi`
ships:
- a **prebuilt `Iroh.xcframework`** (`ios-arm64`, `ios-arm64_x86_64-simulator`,
`macos-arm64`),
- an **`IrohLib` Swift Package** + CocoaPods podspecs (`IrohLib.podspec`,
`IrohLibFramework.podspec`),
- a documented Swift usage guide (`README.swift.md`) and a
"build-your-own-binding" route (docs.iroh.computer/deployment/other-languages).
The binding is UniFFI-generated. So Swift consumes iroh as an **opaque binary
framework** — no Rust toolchain or Rust source in the app project.
2. **You don't have to reimplement the relay/QUIC protocol, and you don't have to
author the FFI from scratch.** `iroh-ffi`/`IrohLib` is a reference/example with a
far-too-broad surface (blobs/docs/gossip) — we **don't ship it** (§8). Instead we
reuse its *build recipe* to produce a small **goose-owned** `Goose.xcframework`
exposing just connect + ACP stream + cancel, with our relays baked in.
The plain-QUIC-gateway idea (previous "Option A") is **dropped**: a gateway that
terminates the phone's QUIC is just lapstone with a nicer transport — no direct
path, no end-to-end encryption. Not worth doing.
## 1. What exists today (to be replaced)
- **lapstone tunnel** (`crates/goose-server/src/tunnel/{mod.rs,lapstone.rs}`,
`routes/tunnel.rs`): goosed dials an outbound WebSocket to a **personal
Cloudflare Worker**; the iOS app hits that Worker over HTTPS. Every phone HTTP
request is reframed as JSON `TunnelMessage`/`TunnelResponse` over WS, with manual
chunking and SSE-streaming handling.
- **The mobile client talks HTTP+SSE, not ACP.** It hits goosed's REST surface —
notably `POST /reply` which returns `text/event-stream`
(`routes/reply.rs`: `SseResponse`, `Content-Type: text/event-stream`). The tunnel
is a generic HTTP proxy; the phone speaks the same HTTP API the desktop does.
- **ACP already has network transports.** Beyond stdio, goose has a
transport-agnostic ACP server at **`crates/goose/src/acp/transport/`**:
`http.rs`, `websocket.rs`, `connection.rs`, `mod.rs`. It serves ACP over an axum
router on `/acp``POST` (JSON-RPC request), `GET` (WebSocket upgrade *or* SSE),
`DELETE` (teardown), scoped by `Acp-Connection-Id` / `Acp-Session-Id` headers.
**This is the integration point — not stdio.**
### The transport abstraction (this is what we hook into)
`connection.rs` defines a transport-agnostic `Connection` + `ConnectionRegistry`:
- `to_agent_tx: mpsc::Sender<String>` — client→agent JSON-RPC lines.
- An outbound fan-out (`OutboundStream`, broadcast) for agent→client, **with a
pre-subscribe replay buffer** (`subscribe_with_replay`): messages emitted before
a subscriber attaches are buffered and replayed on (re)subscribe.
- `adapters.rs` bridges mpsc ↔ `AsyncRead/AsyncWrite` with newline JSON-RPC framing.
A concrete transport (`websocket::run_ws`) is **~70 lines**: split the socket,
replay buffered messages, then `select!` { socket→`to_agent_tx` ; `outbound_rx`
socket }. **An iroh transport is the same ~70 lines over a QUIC bidi stream** — the
`Connection`, registry, session routing, and replay buffer are all reused unchanged.
The replay buffer is also **most of the suspend/resume story already** (reconnect →
replay); we only extend the cursor to survive a full reconnect (§4).
### Why replace it
- Single personal relay = SPOF + trust anchor; **Cloudflare terminates TLS and
sees plaintext** (no E2E).
- **Never a direct path** — even same-Wi-Fi traffic round-trips through the Worker.
- Auth is a **shared 32-byte bearer secret** in the QR (plus a buggy
`secure_compare` using non-constant-time 64-bit `DefaultHasher` — fix regardless).
- Bespoke HTTP-over-JSON-over-WS reframing split across 3 repos; binary is lossy.
## 2. Target architecture
```
iOS app (pure Swift) Goose.xcframework (goose-owned, vendored) laptop
┌────────────────┐ goose- ┌───────────────────────────┐ iroh QUIC ┌────────┐
│ SwiftUI + ACP │ tunnel │ iroh Endpoint (client) │ ──relay──► │ goosed │
│ client (Swift) │ Swift API │ ALPN goose-acp/1 │ ◄─direct──► │ +iroh │
└────────────────┘ ─────────► │ 2 relays baked in │ └────────┘
no Rust source └───────────────────────────┘ E2E encrypted; relay
sees ciphertext only
```
- **goosed** binds an iroh `Endpoint` (ALPN `goose-acp/1`), registers with the two
relays via `RelayMode::Custom`, runs an accept loop, and serves **ACP as
newline-delimited JSON-RPC on each accepted QUIC bidi stream** (Option A below).
- **iOS app** uses the goose-owned `Goose.xcframework` (§8) to dial the server's
`NodeAddr`, open a bidi QUIC stream, and speak the **same newline JSON-RPC ACP it
already uses** — only the byte pipe changes. The two relays are compiled-in defaults.
- On LAN: connects via relay, then **hole-punches to a direct path** automatically.
Off-net: rides the relay, **end-to-end encrypted** (relay can't read traffic).
**Server change is small because ACP transports are already pluggable.** goosed adds
a `crates/goose/src/acp/transport/iroh.rs` next to `websocket.rs`: bind an iroh
`Endpoint` (ALPN `goose-acp/1`, 2 relays via `RelayMode::Custom`), accept loop →
for each QUIC bidi stream call `registry.create_connection()` and run the same
`select!` bridge as `run_ws`. No new ACP semantics, no protocol redesign — the
agent loop, session routing, and replay buffer are reused verbatim.
### The layer stack — and the one real decision
iroh is **not** a peer of HTTP. These are stacked layers; the only open choice is
the *framing* layer:
```
┌──────────────────────────────────────────────────────────────┐
│ ACP JSON-RPC 2.0 (initialize, session/prompt, …) │ the protocol
├──────────────────────────────────────────────────────────────┤
│ FRAMING newline-delimited JSON-RPC ← THE choice (Option A) │ ← decided: A
│ (alt: HTTP/3 over the top — rejected, Option B) │
├──────────────────────────────────────────────────────────────┤
│ STREAM a reliable, ordered, bidi byte stream │
│ == iroh QUIC bidi stream (SendStream/RecvStream) │
├──────────────────────────────────────────────────────────────┤
│ QUIC streams + TLS 1.3 + multiplexing + migration │ (inside iroh)
├──────────────────────────────────────────────────────────────┤
│ iroh discover peer + NodeId identity + relay/holepunch │ finds + authenticates
│ → hands you an authenticated QUIC connection │ → a QUIC connection
└──────────────────────────────────────────────────────────────┘
```
**iroh's job:** find the goose server (relay or direct), authenticate it by NodeId,
and produce an authenticated QUIC connection. **ACP's job:** ride a byte stream as
JSON-RPC lines. A QUIC bidi stream *is* the byte stream — they meet directly.
### DECISION: Option A — ACP as newline-delimited JSON-RPC directly on the QUIC stream
**No HTTP layer.** This is not a close call:
1. **Zero impedance with the agent core.** `acp::server::serve(agent, read, write)`
already consumes a generic `AsyncRead`/`AsyncWrite` of newline-delimited JSON-RPC
— exactly what stdio uses. A QUIC `SendStream`/`RecvStream` *is* that pair, so the
QUIC stream feeds `serve()` directly (via the existing `ReceiverToAsyncRead`/
`compat` adapters). Option B would stand up an HTTP server on the QUIC connection
only to re-derive the same byte stream the agent then re-parses.
2. **HTTP framing solves problems we no longer have.** The `/acp` POST/GET-as-SSE/
DELETE routes + `Acp-*` headers + chunking + content-type sniffing exist to
smuggle a long-lived **bidirectional** JSON-RPC conversation through request/
response HTTP. A QUIC bidi stream is natively bidirectional and long-lived —
HTTP-over-the-top re-introduces the very SSE/chunking machinery we're deleting.
3. **Smallest, lowest-risk change.** One file, `transport/iroh.rs`, mirroring
`websocket.rs`'s ~70-line `select!` loop over QUIC streams. `Connection`,
`ConnectionRegistry`, session routing, and the replay buffer reused verbatim.
No HTTP/3 server, no route re-mounting, no SSE plumbing.
4. **Proven shape.** mesh-llm runs its protocol as frames straight on iroh QUIC
streams (no HTTP) — same shape as ACP-on-QUIC.
5. **Cleaner suspend/resume.** Owning the framing end-to-end lets us add a resume
cursor (event-id on notifications, "resume after N" on reconnect) as a protocol
decision we control, instead of leaning on HTTP/SSE `Last-Event-ID` semantics and
gateway-style buffering (§4).
**Framing sub-choice:** use **newline-delimited JSON-RPC** (identical to stdio + WS
today → maximum reuse). Length-prefixed frames (mesh-llm style) are marginally more
robust for large/binary payloads but unnecessary for line-oriented JSON ACP — don't
add unless a concrete need appears.
**Option B (HTTP/3 over QUIC) — rejected.** Only justifiable to reuse the exact
existing `/acp` HTTP handlers unchanged for a throwaway prototype; the transport
abstraction is already thin enough that the saving is tiny and the HTTP layer is
permanent overhead. Not worth it.
Net: **one protocol, one transport.** Mobile speaks ACP over an iroh QUIC stream;
desktop/CLI keep their existing transports. The HTTP REST surface (`/reply`, the
`/acp` HTTP routes) stays for the desktop, but mobile no longer depends on it.
## 3. Identity & end-to-end auth (replaces the shared secret)
- **Transport identity is free from iroh/QUIC.** Each endpoint is an ed25519
keypair; **NodeId == public key**, mutually verified by QUIC's TLS 1.3 handshake.
goosed learns the phone's verified public key; the phone learns goosed's. No
bearer secret needed to prove *who* you're talking to.
- **Authorization = per-device NodeId pinning.** At QR-pairing time, capture the
phone's NodeId into an allow-list in goose config. Only enrolled devices connect;
revoke one device without rotating a global secret. (mesh-llm's
`SignedNodeOwnership` certs over NodeIds are a ready blueprint if we want signed
enrollment rather than a plain allow-list.)
- **Pairing QR** carries `base64url(NodeAddr)` = server NodeId + relay URLs
(iroh's `EndpointAddr`/`NodeAddr` token, cf. mesh-llm
`encode/decode_endpoint_addr_token`), replacing `goosechat://configure?{url,secret}`.
- Local `server_secret` can still gate the loopback bridge as defense-in-depth; it
never leaves the machine.
## 4. Operational behavior on iOS — the suspend/resume lifecycle
**Fundamental constraint (transport-independent):** iroh runs over UDP on a tokio
runtime *inside* the xcframework. When iOS suspends the app it **freezes the
process** — tokio stops scheduling, keepalives stop, the relay session and any
direct path hit their idle timeout and are reaped server-side. **No entitlement
keeps a general data app's UDP socket alive** in the background (VoIP/PushKit is
for calls; Apple rejects misuse). Design rule: **don't survive suspension —
survive *resumption* fast.**
| Phase | Behavior | Action |
|---|---|---|
| Foreground, LAN | relay first → **hole-punch to direct QUIC** in ~1-2s | observe via `home_relay()`/`remote_info_list()` |
| Foreground, off-net | relay path, **ciphertext-only** through relay | normal operation |
| Backgrounding | ~seconds before freeze | **clean `disconnect()`** in `beginBackgroundTask`; persist stream cursor; mark `Suspended` |
| Frozen | nothing runs | fine — don't fight it |
| Foreground resume | re-`connect()`; QUIC **0-RTT session resume (~ms)**; auto direct-path re-upgrade | trigger on `scenePhase == .active` |
| Roaming (Wi-Fi↔cellular, foregrounded) | QUIC **connection migration**, often seamless (no reconnect) | trigger on `NWPathMonitor` change |
| Mid-stream across suspend | stream dies (any transport) | **app-level resume cursor** (see below) |
| Wake while asleep | impossible over iroh (frozen phone can't receive UDP) | needs **APNs push** to foreground first |
**vs. lapstone:** background-drop is the same (unavoidable on iOS), but resume is
**0-RTT QUIC vs. full TCP+TLS+WS reconnect**, roaming can be **seamless migration
vs. full reconnect**, and foreground gives **direct path + true E2E** that lapstone
never has.
### Streaming continuity across suspend (iroh does NOT solve this)
A QUIC stream carrying an in-flight ACP turn dies on freeze. Design ACP-over-iroh
with a **resume cursor**:
- goosed tags each `session/update` with a monotonic event id.
- On reconnect the client sends `resume(session_id, after_event=N)`.
- goosed replays buffered tail events, or returns the completed turn if it finished
while the app was away.
This is the same problem lapstone has today; it's an application-protocol concern.
### "Notify me when it's done while asleep"
Only achievable with **APNs**: a relay-side or goosed-side hook fires a push → app
foregrounds → iroh reconnects (0-RTT) → client pulls the result via resume cursor.
Orthogonal to transport; the only path to asleep-delivery on iOS.
## 5. Minimal Swift lifecycle glue (sketch)
```swift
// One "ensure connected" routine driven by both lifecycle and network triggers.
@MainActor final class GooseTunnel: ObservableObject {
@Published var path: PathKind = .disconnected // .direct / .relayed / .suspended
private let node: GooseTunnelNode // from goose-owned Goose.xcframework (§8)
private var conn: GooseAcpStream?
private var resumeCursor: UInt64 = 0
private let serverAddr: NodeAddr // decoded from paired QR (base64url)
func ensureConnected() async {
guard conn == nil else { return }
conn = try? await node.connectAcp(serverAddr,
resumeAfter: resumeCursor) // 0-RTT when possible
path = (try? await node.isDirect(serverAddr)) == true ? .direct : .relayed
}
func onForeground() { Task { await ensureConnected() } } // scenePhase == .active
func onPathChange(_ p: NWPath) { // NWPathMonitor
// QUIC migration usually handles this transparently when foregrounded;
// only re-dial if the stream actually dropped.
if conn == nil { Task { await ensureConnected() } }
}
func onBackground(_ task: UIBackgroundTaskID) { // beginBackgroundTask
conn?.closeCleanly() // don't leave a zombie
conn = nil
path = .suspended
// resumeCursor already persisted as updates arrive
}
}
```
The `connectAcp` / `isDirect` / `closeCleanly` / event-id cursor methods are what
the **goose-owned FFI surface** (`crates/goose-tunnel-ffi`, §8) exposes. Internally
it uses iroh's `Endpoint` + `node_addr`/`home_relay`/`remote_info_list` to decide
direct-vs-relayed and to open the ACP bidi stream — but the Swift app only sees the
small goose-shaped API, not iroh's full surface.
## 6. Implementation shape (when we act)
**Server (Rust, in goose) — small, because ACP transports are pluggable:**
1. Add **`crates/goose/src/acp/transport/iroh.rs`** next to `websocket.rs`: bind iroh
`Endpoint` (ALPN `goose-acp/1`, `RelayMode::Custom` with the two baked-in relays),
accept loop → per QUIC bidi stream `registry.create_connection()` + the same
`select!` bridge `run_ws` already uses. **Reuses `Connection`, registry, session
routing, and the replay buffer verbatim** — no new ACP semantics.
2. Wire it into goosed startup (where `tunnel/mod.rs` is today): bind endpoint,
carry over the watchdog/reconnect + single-instance lock.
3. Device enrollment: NodeId allow-list in goose config; QR emits `base64url(NodeAddr)`.
4. Extend the replay buffer with a **persistent per-session event cursor** so a full
reconnect (post-suspend) resumes instead of replaying-from-attach only (§4).
5. Delete lapstone (`tunnel/lapstone.rs`, Cloudflare Worker dependency,
`tunnel_secret` plumbing, `secure_compare`).
**Client (Swift app) — talks the *same* ACP it already speaks, just over iroh:**
1. Vendor the goose-owned `Goose.xcframework` (§8) — exposes connect + ACP stream.
2. The existing Swift ACP/JSON-RPC client logic is reused; only the byte transport
changes (iroh bidi stream instead of the HTTP/WS-via-Worker it uses today).
3. Lifecycle glue (§5): `scenePhase` + `NWPathMonitor``ensureConnected`.
4. QR pairing → store `NodeAddr` + device keypair (Keychain).
5. (Later) APNs for asleep-delivery.
**Relays:**
- Run our own iroh relays (cf. mesh-llm's `usw1-2 / aps1-1.relay.…iroh.link`),
fall back to n0 default relays, make configurable — mirror `effective_relay_urls`.
## 7. Trade-offs summary
| Concern | lapstone (today) | ACP over iroh |
|---|---|---|
| LAN datapath | always via Cloudflare | **direct (hole-punched)** |
| Off-net datapath | via Cloudflare (plaintext) | via relay (**ciphertext only**) |
| E2E encryption to app | no | **yes** |
| Identity/auth | shared 32-byte secret | **per-device ed25519 NodeId, mutually verified** |
| Relay trust | personal Worker, SPOF | self-hostable iroh relays, can't read traffic |
| Protocol | HTTP→JSON→WS reframe + manual chunk/stream | **native ACP over QUIC bidi stream** |
| Binary payloads | lossy (UTF-8 strings) | native bytes |
| Swift client | HTTP via Worker (works, no Rust) | **goose-owned `Goose.xcframework` — no Rust source in app** |
| Background drop | unavoidable | unavoidable (same) |
| Resume cost | full TCP+TLS+WS | **0-RTT QUIC** |
| Roaming | full reconnect | **QUIC migration (often seamless)** |
| Mid-stream resume | app-level cursor | app-level cursor (same) |
| Wake while asleep | needs push | needs push (same) |
| New infra | Cloudflare Worker | iroh relay server(s) (or n0 defaults) |
## 8. Roll our own FFI — don't ship `iroh-ffi`/`IrohLib`
`n0-computer/iroh-ffi` is explicitly a **reference/example**, not a product
dependency (`publish = false`; README says "for example"). Its surface is the
*whole* iroh toolkit — `src/{blob,doc,gossip,author,tag,node,net,endpoint,key,
ticket}.rs` (blobs, docs, gossip, authors…) — none of which we want on a mobile
ACP tunnel. We want a **tiny goose-owned crate** that exposes only: connect to a
paired goosed, open an ACP stream, push/pull ACP frames, resume cursor, cancel,
disconnect, observe path (direct/relayed). Pinning to their broad, fast-moving
binding (currently iroh `0.35`, UniFFI `0.28`) would drag in surface and version
churn we don't control.
**What we copy from `iroh-ffi` is the *recipe*, not the code:**
- Crate layout: `crate-type = ["staticlib", "cdylib"]`, `uniffi::setup_scaffolding!()`,
a `uniffi-bindgen` bin, `lto = true`, deps = just `iroh` + `tokio` + our ACP types.
- Build pipeline (`make_swift.sh`): `cargo build --release` for
`aarch64-apple-ios`, `aarch64-apple-ios-sim`, `x86_64-apple-ios`,
`aarch64-apple-darwin``uniffi-bindgen generate --language swift`
`lipo` the sim archs → assemble `Goose.xcframework` (ios-arm64 + sim + macos) →
generate the Swift interface → `swift package compute-checksum` for a binary
Swift Package target. ~80 lines of shell we adapt verbatim.
**Proposed `crates/goose-tunnel-ffi/`** (or live alongside the existing client SDK):
```
goose-tunnel-ffi/
Cargo.toml # iroh, tokio, uniffi, goose ACP types only
src/lib.rs # uniffi::setup_scaffolding!()
src/tunnel.rs # #[uniffi::export] GooseTunnel: connect/stream/cancel/status
uniffi-bindgen.rs
make_swift.sh # adapted from iroh-ffi
GooseTunnel.xcframework (generated artifact, or released as a zip)
```
Exported surface (UniFFI), goose-shaped, ALPN `goose-acp/1`, relays baked in:
```
generate_device_keypair() -> String
connect(server_addr_token, device_keypair, resume_after) -> GooseTunnel // async, 0-RTT capable
GooseTunnel.send_acp(json)
GooseTunnel.stream(req, listener: AcpEventListener) // callback iface, replaces SSE
GooseTunnel.cancel(request_id)
GooseTunnel.path_kind() -> Direct | Relayed // from remote_info_list/home_relay
GooseTunnel.disconnect()
```
The Swift app imports this single framework — no Rust source, no toolchain, and a
surface we own and version with goose, not with n0's example repo.
> Server side reuses the same `iroh` crate directly in goosed (no FFI there) — see §6.
### Where does the Rust live? (three viable placements)
The FFI crate is normal Rust; the only question is *which repo/build owns it*. The
Swift app always consumes a **binary `.xcframework`** regardless — so "Rust in the
Swift codebase" is really about whether the Rust *source* and its build sit in the
mobile repo, the goose monorepo, or standalone.
1. **In the goose monorepo** (`crates/goose-tunnel-ffi/`) — *recommended default.*
- Pros: one `iroh` version pinned across goosed + FFI (critical — §9.6); shares
ACP types with `crates/goose/src/acp`; CI builds the xcframework as a release
artifact; client/server protocol can't drift.
- Cons: mobile release cadence coupled to monorepo; iOS toolchain in goose CI.
- The Swift app references the published `Goose.xcframework` (versioned release
zip + checksum), exactly how mesh-llm's `Package.swift` pulls a remote
`MeshLLMFFI.xcframework.zip`.
2. **In the Swift app repo** (Rust crate as a subdirectory, built by an Xcode
build phase / `make_swift.sh`).
- Pros: mobile team owns cadence; Rust+Swift co-located, one PR changes both.
- Cons: must vendor/track the right `iroh` + goose ACP types out-of-tree; easy
for server and client iroh versions to drift (the §9.6 risk); duplicates the
Rust toolchain into the mobile repo. This is the literal "Rust in the Swift
codebase" option — it *works* (mesh-llm proves the build), but the version-skew
risk makes it weaker than (1) for a protocol shared with goosed.
3. **Its own repo/crate** (`goose-tunnel` published crate + released xcframework).
- Pros: clean dependency for both goosed and the app; independent versioning;
reusable by a future Android/Kotlin client (UniFFI already does Kotlin).
- Cons: a third repo to release-coordinate; still needs the iroh-version pin
enforced across three consumers.
**Recommendation:** start with (1) — crate in the monorepo, xcframework as a CI
release artifact the Swift app consumes by version. It eliminates the protocol/iroh
drift risk by construction. Promote to (3) only if/when a second client (Android)
or external consumers appear. (2) is fine for a fast prototype but I'd avoid it as
the long-term home because the client and server share the iroh wire protocol and
must move together.
## 9. Open questions / next steps
1. **ACP as a first-class network transport** — generalize the stdio ACP server
over a stream, or bridge-to-HTTP first for a faster cutover?
2. Own relays vs. n0 default vs. configurable (likely all three).
3. Enrollment UX: plain NodeId allow-list vs. signed ownership certs (mesh-llm style).
4. Migration: ship iroh transport behind a flag alongside lapstone, then delete.
5. APNs design for asleep-delivery (relay-side vs. goosed-side hook).
6. Pin a single `iroh` version across goosed + `goose-tunnel-ffi` (server and FFI
must speak the same relay/disco protocol — it changes between iroh versions).
7. Replace `secure_compare` now regardless (security fix to current code).