# Reproduction and measurement notes

## Hardware and software

One iPhone 18 Pro (A20 Pro) and one iPhone 17 Pro Max (A19 Pro), each sold with 12 GB RAM and 256 GB storage. Both report iOS 27.0 (24A437). This is a comparison of two complete phones; their chassis and cooling differ. It does not isolate a chip's causal contribution.

Both received the same optimized Release binary of Russet 2026.9.20 (26092002), built with Xcode 27.0 and the opt-in `RUSSET_RESEARCH` compilation condition. `build-provenance.json` records the binary hash, base revision, changed-source hashes and MLX dependency revisions. No debugger or Instruments recording is attached during measurement. Account gates and model memory checks remain active.

## Workload

- Qwen3.5 0.8B, six-bit MLX conversion, and Gemma 4 E2B Instruct, four-bit MLX conversion. Text-only inference; no images, tools, retrieval or network inference.
- Input targets: 256 and 2,048 tokens. Output ceiling: 256 tokens. Actual token counts are exported; targets are not assumed to be exact lengths.
- Three fixed prompt cases, each repeated in four loops. Each model/context/device cell therefore contains 12 observations, including repetitions of the same three prompts. These are not 12 independent phones.
- The existing Torrance runner pads prompts with a repeating list of benchmark-related words to approach the input target. The 2,048-token case is a synthetic context-length probe, not a realistic long-document comprehension or accuracy evaluation.
- The model is loaded before timing. Each prompt gets a fresh session, and each prompt is warmed up before the measured loops. No prefix-cache reuse across measured conversations is intended. Allocator cache is cleared after each run. Three prompts run sequentially, followed by a 15-second pause between loops.
- Research sampling explicitly overrides model-catalog defaults: temperature 0, top-p 1, top-k 0, min-p 0, presence penalty 0 and repetition penalty 1. Qwen reasoning is off. Greedy sampling does not itself guarantee bitwise-identical floating-point execution across devices; response hashes are checked separately.
- Initial exploratory Qwen sessions used model-default stochastic sampling because catalog settings took precedence over the session temperature. They are preserved locally but excluded from the controlled comparison.

## What the metrics mean

**First visible response:** wall-clock time from the app's generation call to its first nonempty response callback. This includes app/runtime overhead and prompt processing. It is not a GPU kernel timer or cold model-load latency.

**Visible output rate:** the visible response is retokenized, capped at the output ceiling, and divided by the interval from first nonempty callback to generation completion. The numerator can include the first visible token or batch. This is the existing Torrance metric, consistently applied to both devices; do not equate it with an engine's raw decode throughput. Raw engine counters are retained separately where available.

**Completion time:** app generation-call to return, excluding load, calibration, warm-up and cooldown. Comparisons need actual output lengths alongside them.

**Memory:** process footprint sampled at the start, callbacks and completion, reported as the largest sample in GiB (2^30 bytes). This includes the app and can miss peaks between samples. It is neither model weight size nor maximum memory capacity.

**Thermal state:** iOS's categorical state recorded before and after each measured prompt. Nominal does not establish a fixed temperature, clock frequency or absence of performance management. The built-in Markdown report's thermal label is sampled at report construction; per-prompt JSON is used for analysis.

## Controls and limitations

Both phones are connected to the Mac. Battery/charging, Low Power Mode and thermal state are exported. The runner requires nominal thermal state and Low Power Mode off at launch. Ambient temperature, case configuration and precise battery temperature were not instrumented. No external power meter was used; there is no inference-energy or battery-life conclusion.

Model manifests were collected outside timed sessions. Gemma's seven files are byte-identical across phones. Qwen's weights, model configuration and other runtime files match; its tokenizer configuration differs in JSON serialization but is structurally identical after parsing. The raw originals are retained. Do not interpret an unpinned repository URL as sufficient proof of identical weights.

Show all observations, medians and observed ranges; ranges are not confidence intervals. Between-phone repetition cannot estimate device-to-device manufacturing variation. Paused bursts cannot establish uninterrupted thermal saturation. No answer-quality score is inferred from speed.