
For each possible value of the theoretical mean, the Z-test statistic has a different probability distribution.

In the just mentioned example that would be the Z-statistic belonging to the one-sided one-sample Z-test.

For example, when testing the null hypothesis that a distribution is normal with a mean less than or equal to zero against the alternative that the mean is greater than zero (variance known), the null hypothesis does not specify the probability distribution of the appropriate test statistic. In contrast, in a composite hypothesis the parameter's value is given by a set of numbers. In parametric hypothesis testing problems, a simple or point hypothesis refers to a hypothesis where the parameter's value is assumed to be a single number. Ī p-curve can be used to assess the reliability of scientific literature, such as by detecting publication bias or p-hacking. The distribution of p-values is sometimes called a p-curve. when considering a group of studies on the same subject), When a collection of p-values are available ( e.g. Usually only a single p-value relating to a hypothesis is observed, so the p-value is interpreted by a significance test and no effort is made to estimate the distribution it was drawn from.

In statistics, every conjecture concerning the unknown probability distribution of a collection of random variables representing the observed data X when the alternative hypothesis is true. In 2016, the American Statistician Association (ASA) made a formal statement that " p-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone" and that "a p-value, or statistical significance, does not measure the size of an effect or the importance of a result" or "evidence regarding a model or hypothesis." That said, a 2019 task force by ASA has issued a statement on statistical significance and replicability, concluding with: " p-values and significance tests, when properly applied and interpreted, increase the rigor of the conclusions drawn from data." Basic concepts Even though reporting p-values of statistical tests is common practice in academic publications of many quantitative fields, misinterpretation and misuse of p-values is widespread and has been a major topic in mathematics and metascience. A very small p-value means that such an extreme observed outcome would be very unlikely under the null hypothesis. In null-hypothesis significance testing, the p-value is the probability of obtaining test results at least as extreme as the result actually observed, under the assumption that the null hypothesis is correct.
