experiments.do
Ship Features Faster with A/B Testing
Run experiments at global edge locations with instant assignment. Test variations and feature releases with automatic statistical analysis.
import { Experiments } from 'experiments.do' // A/B test at the edge with <5ms assignmenton.Request.to.PricingPage($ => { const variant = Experiments.assign({ experimentId: 'pricing-redesign', userId: $.User.id, variants: { control: 50, newDesign: 50 } }) $.Analytics.track.ExperimentView({ variant: variant.id }) return variant.id === 'newDesign' ? $.Render('pricing-v2') : $.Render('pricing-v1')})
One of 48 primitives
experiments.do sits in Insight: measure, test and trace. Every .do shares one grammar, Business-as-Code, and one SDK.
Data
Records, vocabulary and search
Insight
Measure, test and trace
- Analytics
- Experiments
- Evals
- Benchmarks
- Trace
- Models
Frequently asked questions
Experiments.do assigns variants in <5ms at 300+ edge locations globally with deterministic user bucketing. No database lookups required—assignments cached at the edge based on userId + experimentId hash. Zero impact on page load performance, works for server-side rendering, client-side React apps, and API endpoints. Consistent assignments across devices and sessions for the same user.
Bayesian inference for continuous monitoring without peeking penalties, frequentist t-tests for fixed-horizon tests, sequential testing for early stopping when results are significant, and false discovery rate (FDR) control for multiple comparisons. Automatic sample size calculations, minimum detectable effect (MDE) estimation, and 95% confidence intervals for all metrics. Built-in protection against Simpson's paradox and regression to the mean.
Yes. Test multiple variables simultaneously (button color + headline + CTA text) with full factorial or fractional factorial designs. Run experiments across multiple pages (landing → signup → onboarding flow) with funnel tracking. Segment results by user cohorts, traffic source, device type, geography, or custom dimensions. Analyze interaction effects between variables automatically.
Define experiment layers to prevent conflicts—users can be in one experiment per layer simultaneously. Mutual exclusion groups ensure users never see conflicting variants (e.g., don't show discount banner and premium upsell together). Automatic traffic allocation balancing across competing experiments. Priority-based assignment for critical experiments that need clean traffic.
Built-in feature flag system with percentage-based rollouts (0% → 5% → 25% → 50% → 100%), user targeting rules (beta users, specific organizations), and kill switches for instant rollback. Schedule automatic rollouts (increase 10% daily until 100%). Monitor error rates, performance metrics, and user feedback during rollout. Integration with CI/CD for trunk-based development and continuous deployment.