Was there a followup question to “Assistance to the poor or welfare question?” … something like …. “Would your opinion change if you were told that “Welfare” is just another term for “Assistance to the poor” ?
To prevent finding spurious results with A/B testing on subpopulations, couldn't you run the same tests on multiple randomly pulled samples of the study population? If the same subsets show significant test results in all the samples, you have more confidence it's a real signal, not noise.
first thing is when you run experiment on subpopulation the power decreases w/ less sample size; second thing there is a cost to rerun tests and sometimes not feasible (this is why different causal methods for observational data exist: randomized experiment cannot be run)
Great content! Thanks Susan & Brady
Great lecture I'll will follow with the tutorials. Thanks you in advance!
perfect, I was read this paper, that's nice.
Was there a followup question to “Assistance to the poor or welfare question?” … something like …. “Would your opinion change if you were told that “Welfare” is just another term for “Assistance to the poor” ?
To prevent finding spurious results with A/B testing on subpopulations, couldn't you run the same tests on multiple randomly pulled samples of the study population?
If the same subsets show significant test results in all the samples, you have more confidence it's a real signal, not noise.
first thing is when you run experiment on subpopulation the power decreases w/ less sample size; second thing there is a cost to rerun tests and sometimes not feasible (this is why different causal methods for observational data exist: randomized experiment cannot be run)