AgencyImported version
Design an Effective Product Experiment
/Users/fabien/Desktop/AI/PROMPTS/-agency/01 Research, Strategy and Reporting/Audits and research/Design an effective product experiment.odt
Folder-agency/01 Research, Strategy and Reporting/Audits and research
Versions0
Imported2026-11-10
StatusActive
## Design a Better Product Experiment Act as a senior product manager and experimentation strategist. Help me design a rigorous product experiment for [feature/change/hypothesis] that produces a clear decision, not just interesting data. ## Context - Product: [product] - Target users: [segment] - Problem: [problem] - Proposed change: [feature/variation] - Hypothesis: [what you expect to happen] - Primary business goal: [activation / retention / conversion / engagement / revenue] - Current baseline: [metric + value] - Traffic/sample available: [users] - Experiment window: [days/weeks] - Constraints: [engineering, legal, seasonality, dependencies] - Known risks: [risks] ## Experiment Design Design the experiment with: 1. A precise hypothesis 2. Control and treatment definitions 3. Primary success metric 4. Secondary metrics 5. Guardrail metrics 6. Target segment and exclusions 7. Minimum meaningful effect to detect 8. Recommended duration/sample approach 9. Key confounders or failure modes 10. Clear launch, stop, and rollback criteria Separate what would prove the hypothesis from what would merely look positive. Identify any ways the experiment could produce misleading results, including novelty effects, selection bias, instrumentation problems, or overlapping tests. ## Deliverables Finish with: - The experiment brief - A pre-launch checklist - A results interpretation framework - Clear Ship / Iterate / Reject / Run Longer decision rules