A month ago we published the original plan for Body Twin: give the Digital Twin a second channel of truth by reading your body, on the iPhone alone, without breaking the promise that nothing is collected. This is the release retrospective. Some of that plan shipped in v1.5, and it is genuinely the part we are proud of. Some of it did not, and this post is at least as much about the second category as the first. If you only read the plan, you would come away expecting more than v1.5 delivers — so the honest thing is to say exactly what landed, what slipped, and why we think the half we shipped is the defensible half.
The engineering context that makes all of this unusual: DailyVox has no server-side anything. No feature store, no training cluster, no analytics pipeline you could inspect to see whether a model is misbehaving. The Twin is a small personal model that lives and dies on one device, seen by exactly one person. That framing is easy to say and removes almost every instrument an ML team normally leans on — which is why the validation half of this post exists at all.
Two-layer sampling, and why only one layer shipped
The design separates two very different physiological signals. The first is background context: sleep, morning heart-rate variability, resting heart rate, steps, mindful minutes — snapshotted at the moment you journal from health data that is already stable and settled. The second is a foreground recording-time signature: what your body is doing in the very moment you speak.
The second layer is the interesting one, and it is deliberately not in this build. Reading it properly needs a wrist sensor for live sampling during the recording; approximating it from phone-only data would produce a confident, wrong number, and a confident wrong number is worse than an honest absence. So v1.5 ships the background-context layer only. The bulk of this post's "what shipped" is really about making that one layer trustworthy — because a single physiological channel with no server to clean it up has to be right the first time, on-device.
Activity context is the field everything hinges on
A raised heart rate means nothing without knowing why. The same reading is stress at a desk and unremarkable on a jog. So every snapshot carries an activity-context value — at rest, active, post-workout, or unknown — derived from the phone's motion sensing. The critical decision here is that "unknown" is a first-class value we are willing to emit rather than guess. Without this tag the physiological signal is not merely noisy; it is systematically misleading, because it would read every workout as anxiety. An honest "unknown" is worth more than a precise lie, and treating it as a real category rather than a fallback is what keeps the background layer from poisoning the model with exercise mislabelled as emotion.
Heart signals are stored as deltas, not absolutes
We do not store a raw heartbeat. We store a deviation from your own hour-of-day baseline — this heart activity relative to what is normal for you at this time of day. The reason is standard but load-bearing: absolute heart rate is dominated by between-person variation (fitness, age, medication) that swamps the within-person signal we actually care about. Baselining per person and per hour removes most of that nuisance variance, and, as a side effect, makes the representation portable — a delta means roughly the same thing across two very different bodies, where a raw reading does not. It is per-subject standardisation applied at the point of capture, and because there is no server, the baseline it standardises against is computed and refined locally as your entries accumulate.
The review-and-discard queue as an honesty primitive
Passively-captured signals do not flow straight into the Twin. They land in a review queue and wait for you to explicitly keep or discard each one before it can ever reach the model. This inverts the usual passive-sensing default, where data is ingested first and maybe forgotten later. Here, nothing passive is trusted by default.
We built it as a reusable gate, not a one-off screen: every future passive signal — the deferred recording-time layer included, whenever it arrives — inherits the same keep-or-discard barrier. It is opt-in consent expressed as a data-flow primitive rather than a settings toggle, and it has a clean consequence for the model: the training set for the physiological channel is, by construction, only the data you chose to include. That is a stronger guarantee than a privacy policy paragraph, because it is enforced by where the data can and cannot go.
No server-side feature store — the real ML-systems constraint
Health-derived data stays local to the device: never synced to a cloud database, excluded from device backups, and able to leave only inside your own encrypted export. Per-signal opt-outs filter before the read, so a disabled signal is never even fetched. For an ML engineer the consequence is structural, not merely legal: there is no server-side feature store. Every feature is computed, stored, and consumed on one device. There is no place to reprocess history when you improve a transform, no backfill, no normalisation statistics shared across users, no way to recompute a baseline centrally when its definition changes. Baselines are personal and live locally; migrations run on-device or not at all. It rules out an entire, comfortable class of architecture — and it is the constraint that makes the rest of the promise credible.
Riding the same release train: a native Liquid Glass tab bar, a self-prediction fidelity feature we call Twin Resolution — a felt read on how well your Twin currently knows you, which is the maturation curve below surfaced honestly to the user — and a fix that stopped the constellation view from pinning the run loop while rendering.
How do you validate a model you can never see?
This is the part the on-device promise makes genuinely hard, and it is the payoff. We cannot watch the model in production, so validation happens before shipping, on synthetic and model-generated personas, and every result is reported against an explicit baseline. The approach follows the spirit of recent generative-agent and persona-simulation work: construct personas with known ground-truth traits, generate diary-like text from them, and check what the pipeline recovers.
Lift over an explicit baseline, or it does not count. The spine of the whole evaluation is a refusal to report raw agreement numbers. A "do-nothing" model — one that ignores the data and always predicts the prior — scores around 75% on a naive agreement metric for these tasks, purely because the label distribution is skewed. That kind of 75% flatters every model equally; it is a vanity metric. So every result is reported as lift over a named baseline plus a tracking correlation: a constant-prior baseline for fidelity, a climatology-plus-persistence baseline for forecasting, a population-mean baseline for personality. If a model cannot beat its baseline, we say so.
Every number is a lift over a baseline
| What we measured | Metric | Result | Measured against | Verdict |
|---|---|---|---|---|
| Self-prediction fidelity | 1−MAE lift over a do-nothing model | +15.5% | constant-prior (do-nothing ≈ 75% raw) | ✓ Signal |
| Short-horizon mood forecast | skill vs baselines | +5% vs climatology; −178% vs persistence | “tomorrow ≈ today” | ✗ No — a mirror, not an oracle |
| On-device Big Five | correlation with assigned traits | r 0.74–0.83 | population mean | ✓ Yes (synthetic) |
| Answer groundedness | claims backed by state | 100% (1,705 / 1,705) | — | ✓ Yes |
Self-prediction fidelity matures on a curve. Against synthetic personas with analytic ground truth, the Twin's recovery of a persona's traits beats the do-nothing baseline, and — usefully for product design — the self-model stabilises after roughly twenty entries. That maturation curve is not a nuisance to hide; it is the honest content behind Twin Resolution. Below about twenty entries the model genuinely does not know you well, and the product should say so rather than perform a confidence it has not earned.
On-device personality inference is feasible, and cheap. A light regressor over the platform's on-device sentence embeddings recovers assigned Big Five traits from diary text at roughly 0.74–0.83 correlation on a synthetic corpus — for a model measured in kilobytes, because the embeddings themselves already sit resident in the operating system. Two properties make this more than a curiosity. First, cross-corpus robustness: a model trained on one writing style still recovers personality from a deliberately different style at about 0.74.
On-device Big Five, per trait
| Trait | Within-corpus r | Cross-corpus r (different writing style) |
|---|---|---|
| Openness | 0.83 | 0.80 |
| Conscientiousness | 0.78 | 0.68 |
| Extraversion | 0.74 | 0.70 |
| Agreeableness | 0.75 | 0.79 |
| Neuroticism | 0.74 | 0.63 |
| Mean | 0.77 | 0.72 |
Cross-corpus = trained on one writing style, tested on a deliberately different one (the reverse direction holds too, mean r 0.77). Human text-to-Big-Five studies report roughly r 0.38 — our sanity ceiling for ‘a plausible amount of signal.’
That gap between train and test styles is what shows it learned a personality signal rather than a stylistic fingerprint. Second, it is honestly bounded: the labels are synthetic, so this demonstrates feasibility and style-invariance, not human accuracy. There is also a size discipline here worth stating: for plain valence and sentiment, the platform's built-in on-device language stack is good enough that a bespoke model is not worth the bytes. Custom ML earns its place specifically for personality, which the built-in stack cannot produce at all.
Multilingual means per-language heads, not translated rules. English keyword heuristics do not transfer to other languages — obviously — but neither does a naive "translate then apply" shortcut. The learned embedding path transfers, but only per language, because the embedding spaces differ; a model trained on one language is useless in another. So multilingual support means separately trained heads per language, not one model plus a translation layer. And on-device sentence embeddings exist for only a handful of languages today, which is a concrete constraint on which languages we can prioritise — the platform's coverage, not our ambition, sets the order.
Negative controls, so the metrics can say no. A validation suite that only ever produces good news is broken. The evaluation includes negative controls: shuffle the labels or the inputs and confirm the metrics collapse back to baseline.
When the signal is destroyed, the metrics collapse
| Control (signal destroyed on purpose) | Metric | Real | After shuffling |
|---|---|---|---|
| Shuffled labels (fidelity) | tracking r | +1.00 | −0.08 |
| Shuffled labels (fidelity) | lift over do-nothing | +16.5% | −6.8% |
| Shuffled entries (personality) | Big Five r | +0.77 | −0.02 |
| Permuted labels (personality) | Big Five r | +0.77 | −0.04 |
Against a 20-run label-permutation null (mean +0.005), the real r = +0.77 sits 7.2 standard deviations above chance — p = 0.000. The metric can report failure, which is what lets it report success.
When personality correlation falls to chance under shuffling and forecasting lift disappears, the positive results become trustworthy — we have shown the metric is capable of reporting failure, which is the only thing that makes it capable of reporting success.
Where v1.5 falls short of the plan
The plan post promised more than this release delivers. That is not something to bury under the good news, so here it is plainly. Three shortfalls, stated as shortfalls.
1. The embodied recording-time body layer did not ship
The plan promised a live physiological signature captured at the moment of recording — the foreground layer that reads what your body is doing as you actually speak. It is deferred. Reading it honestly requires the wrist sensor for live sampling, and rather than fabricate that layer from phone-only data, we shipped without it. v1.5 delivers the background-context layer alone. This was a deliberate choice — one honest layer over two where one would be invented — but it is still a promise from the plan that this release does not keep, and the more embodied half of "Body Twin" is not yet in your hands.
2. The Twin cannot forecast your mood
The roadmap leaned on a predictive framing: a Twin that could see a little way ahead. We measured that directly, and it does not hold. On short-horizon mood, the Twin does not beat a trivial persistence baseline — "tomorrow is about the same as today." This is not a bug we are working around; it is the expected result, and it aligns with an affect-forecasting literature in which persistence baselines are famously hard to beat and people are famously poor at predicting their own future feelings. So the Twin behaves as a conditional-mean estimator — "given a day like this, here is your typical response" — not a trajectory predictor. We let the honest negative result set the product boundary: DailyVox is positioned as reflection and precedent ("you have felt this before, and here is what tended to follow"), and never as mood prediction. It is a mirror, not an oracle, and we can prove it fails the oracle test.
3. "Measurable fidelity" is proven on synthetic data only
Everything in the validation section above runs on synthetic and model-generated personas. That establishes feasibility, style-invariance, and the shape of the learning curves — genuinely useful things to know before shipping. It does not establish human accuracy. No amount of clean synthetic performance substitutes for real people writing real diaries, and we have not yet done that study. Real-diary validation, under the same lift-over-baseline discipline and the same on-device, nothing-collected constraint, is unfinished, gated work. Until it is done, "measurable fidelity" is a claim about the measurement apparatus and the synthetic curves, not a claim about how well the Twin reflects an actual person. We would rather state that boundary than let a synthetic number stand in for a human one.
The half we shipped is the half we can defend
None of the three shortfalls above is comfortable to publish, and that is rather the point. The Twin is a mirror of your reflective self — the person who shows up in what you chose to write down — not a clone of you, and not a forecaster. The most important results in this release are the negative ones: forecasting that cannot beat persistence, accuracy claims fenced off as synthetic-only, a "do-nothing" baseline held up next to every number so that none of them can flatter us quietly. An engineering team that measures honestly ends up shipping the defensible half and naming the rest as unfinished. That is what v1.5 is. We would rather ship a mirror we can stand behind than an oracle we cannot — especially one we could never watch fail.
—
Related Articles
- The Body Twin: How DailyVox v1.5 Brings Apple Health and Apple Watch to Your Digital Twin (the original plan)
- DailyVox v1.4.1: Speak Your First Star — the Cold-Start Problem for a Model of One
- The On-Device AI Maturity Model
- Why Your Journal App Should Predict, Not Just Record
Download DailyVox on the App Store · Read the original Body Twin plan · getdailyvox.com