Tool Evaluation
Part of Experimentation platforms
Comparing feature testing with marketing page testing
Compare how feature and marketing page tests are delivered, assigned, measured and controlled before choosing a workflow.
Use a feature testing workflow when the variation is delivered through application code or configuration. Use a marketing page testing workflow when the main task is editing and previewing a website page. Some products can do both. The useful distinction is how the change is delivered, who maintains it and how exposure and outcomes are recorded.
Compare the proposed change
Consider two ideas: changing a campaign landing page's headline and form layout, and changing the order of onboarding steps after sign-in. The page test needs reliable targeting and preview, plus a measure such as completed enquiries. The onboarding test may need a developer to place alternative behaviour behind a flag, maintain an appropriate assignment across visits and measure completion and errors.
The boundary is flexible. A page can contain coded behaviour, and a feature flag can change web copy. Classify the test by its hardest operational requirement.
| Question | Marketing page emphasis | Feature emphasis |
|---|---|---|
| Where is the change made? | Page content, styling or a page editor | Application code or configuration |
| Who prepares it? | Often a marketer with site support | Usually product or engineering staff |
| What needs previewing? | Rendering, forms and page targeting | Behaviour across users and release states |
| How is it stopped? | Pause or end the page variation | Change flag targeting or stop the experiment |
Follow assignment to the outcome
For either workflow, identify the unit assigned to a variation and when exposure is recorded. A metric event that cannot be connected to exposure may not answer the experiment question.
In LaunchDarkly's documented hosted-metrics route, a flag evaluation establishes exposure and tracked actions are recorded separately. Statsig's raw-event route also distinguishes exposure from custom events and uses unit identifiers to connect them.
For a page test, check that the experiment runs only on intended URLs and see what happens before a visitor makes a tracking choice. Optimizely Web documents URL or saved-page targeting, an experimentation editor and a site snippet. Some of its consent-gated implementations leave visitors on the original page until consent, excluding an unconsented first landing-page view from the test.
Steps to Create an Experiment (Optimizely)
- Define the experiment goale.g., increase completed enquiries on a campaign landing page
- Set up URL or saved-page targetingEnsure the test runs only on intended pages
- Create and preview variationsTest different headlines and form layouts
- Implement tracking eventsMeasure actions like form submissions or clicks
- Launch and monitorCheck for anomalies and ensure data integrity
Key Metrics and Data Handling in Experimentation Platforms
- Exposure Tracking (LaunchDarkly)
- Flag evaluation records exposure
- Event Separation (Statsig)
- Exposure and custom events are distinct
- Consent Handling (Optimizely)
- Unconsented views may be excluded from testing
Choose an operable workflow
Ask each candidate to show the same proposed change: creating and previewing a variation, identifying eligible people, defining an outcome and stopping a harmful experience. Include the person who would edit the page or code and the person responsible for measurement. A page editor does not solve missing event data; a flag workflow may demand developer time for a simple copy change.
The team may choose one platform, two connected workflows or a narrower first use case. Record which system owns each experiment and the hand-off between delivery and analysis.
