How Trevo decides
Trevo is an autonomous growth engineer that does the work while you stay in command of what ships. Here is exactly how it reaches its conclusions — and the permissions that bound what it can do.
Always-valid statistics (mSPRT)
Trevo evaluates experiments with a mixture Sequential Probability Ratio Test (mSPRT), not a fixed-horizon p-value. Classic A/B tests are only valid if you decide the sample size up front and never peek — every early glance inflates the false-positive rate. mSPRT produces always-valid p-values and confidence intervals: you can look at the result at any time, and the agent can act the moment significance is genuinely reached, without a peeking penalty.
Sample-ratio mismatch (SRM)
Before trusting any result, Trevo checks that traffic actually split the way it was configured. A sample-ratio mismatch — a chi-square test of observed vs. expected exposures — means something upstream is broken (caching, a redirect, a bot filter), and the numbers can't be trusted. An experiment that trips SRM is flagged and auto-paused rather than reported.
Guardrails and auto-pause
Every experiment runs behind safety brakes that check on each stats tick. Trevo auto-pauses a running experiment when it detects an SRM, a guardrail metric that degrades significantly (beyond a magnitude floor, not just a tiny blip), or a spike in SDK errors. A pause is reversible — it flips assignment back to control and alerts you; it never concludes the experiment or deletes data. Safety brakes run unconditionally, even when proactive autonomy is switched off.
What the agent may and may not do
The boundary is simple: the agent owns everything reversible; you own everything irreversible. It proposes hypotheses, writes variant code, runs experiments, measures results, and explains what happened. It only ever writes to branches — it opens pull requests for your review. It never merges a PR and never ships a variant. Merging and shipping are yours, always.
Autonomy you control
Autonomy is a simple per-workspace switch. When it's on, Trevo works between your visits — proposing experiments and opening pull requests for review. When it's off, it acts only when you ask. Either way it only ever prepares work; it never merges a pull request or ships a variant. An organization-wide kill switch halts all proactive autonomy instantly, and the safety auto-pause keeps running no matter what.