How to Set Up an iOS Onboarding Experiment That Measures Revenue

Before writing any code, establish what you're testing and how you'll measure success. Your hypothesis should be specific: for example, showing social proof during onboarding will increase day-7 subscription rate by 15%. Choose one primary metric that reflects business value, typically a conversion rate or revenue metric within a defined timeframe. Secondary metrics might include completion rate, time to convert, or retention, but these shouldn't drive your decision.

Decide your sample size requirements based on your current conversion rate and minimum detectable effect. An app converting 5% of users to paid might need 8,000 users per variant to detect a 20% relative improvement with statistical confidence. This calculation determines how long your experiment needs to run and whether you can afford to split traffic evenly or need weighted allocation.

Variant assignment must occur before any onboarding screen renders, typically in your AppDelegate or SceneDelegate initialization. Generate or retrieve the variant assignment synchronously using a deterministic hash of the user identifier, or call your experiment API before presenting onboarding. Store this assignment in UserDefaults or a persistent database so the user sees consistent behavior across sessions.

The assignment should be sticky, meaning once a user receives a variant, they remain in that variant permanently for this experiment. Create a simple enum for your variants and a manager class that handles assignment, storage, and retrieval. Include logic to handle edge cases like app reinstalls, where you may want to assign a new variant or retrieve the previous assignment if you can match the user.

Add event tracking at each significant step in your onboarding flow: screen views, button taps, dismissals, and completions. Include the variant identifier as a property on every event so you can segment analytics by experiment group. Track both success events like advancing to the next screen and failure events like dismissals or skips, as these reveal where variants lose users.

Use your existing analytics framework but ensure events fire reliably before users can navigate away. For critical conversion events like reaching the paywall or completing onboarding, consider sending these synchronously or with retry logic. Batch less critical events to preserve performance, but never batch the variant assignment event itself, which should log immediately after assignment occurs.

Link subscription purchases to experiment variants by setting your variant identifier as a custom user attribute in your purchase validation system. When using RevenueCat, set the variant as an attribute immediately after assignment, then configure your analytics integration to include these attributes with purchase events. This allows you to calculate revenue per variant and measure the true business impact of onboarding changes.

Ensure your App User ID matches between your experiment tracking and purchase system. If you generate a UUID for experiment tracking, use that same ID when configuring RevenueCat or your purchase backend. Test the complete flow in sandbox mode: assign a variant, complete onboarding, make a test purchase, and verify the purchase event includes the correct variant identifier in your analytics.

Reserve 10-20% of traffic for a holdout group that sees your current production experience while you test new variants against it. This control group provides the baseline for measuring improvement and protects you if all new variants perform worse. When you identify a winning variant, you can gradually shift new user traffic toward it while keeping the holdout group constant to measure ongoing impact.

Tools like Spinosis handle sticky variant assignment with configurable traffic allocation, allowing you to shift only new users to a winner while maintaining experiment integrity for users already assigned. This approach lets you capture the benefit of a winning variant immediately without contaminating your experiment data. Monitor your rollout closely for the first few days to catch any implementation bugs before they affect a large user base.