Statistics with Posterior Probability and a PHC Curve

This textbook reconstructs the statistics curriculum from the perspective of posterior probability.

In recent years, there have been several reports that the results of studies using significant tests cannot be reproduced. It is a problem called a “reproducibility crisis”. For example, suppose we could reject the null hypothesis that “the average number of days to recovery in patients who took a new drug was the same as that in the control group”. However, rejecting the null hypothesis is only a necessary condition for the new drug to be effective. Even if the necessary conditions are met, it does not necessarily mean that the new drug is effective. In fact, there are many cases where the effect is not reproduced. Sufficient conditions should be presented, such as “the average number of days until recovery in patients who take new drugs is sufficiently short compared to the control group, evaluated from a medical point of view”, without paying attention to necessary conditions. 

This book reconstructs statistics from the perspective of PHC, i.e., probability that a research hypothesis is correct. For example, the PHC curve shows the posterior probability that the statement “The average number of days until recovery for patients taking a new drug is at least θ days shorter than that of the control group” is correct as a function of θ. Using the PHC curve makes it possible to discuss the sufficient conditions rather than the necessary conditions for being an efficient treatment. The value of statistical research should be evaluated with concrete indicators such as “90% probability of being at least 3 days shorter”, not abstract metrics like the p-value.