Exact two-sided p-values in natural exponential families: coincidence, non-uniqueness, and sample-size stability
We study the non-uniqueness of exact two-sided $p$-values in continuous one-parameter natural exponential families (NEFs). For directed one-sided problems, the tail $p$-value agrees with the $p$-values using UMP, UMPU, and likelihood-ratio (LR) tests. For a two-sided simple null, we distinguish four constructions: equal-tail, density-ordered, UMPU, and LR $p$-values. At a fixed null parameter, UMPU and equal-tail $p$-values coincide if and only if the null law is symmetric about its mean; under a regular two-branch density-level condition, the same fixed-null symmetry characterization holds for UMPU versus density ordering and equal-tail versus density ordering. Requiring any of these coincidences throughout the NEF characterizes the Gaussian family. We combine these results with the theorem of Bar-Lev, Bshouty and Letac that UMPU and LR $p$-values coincide throughout a continuous NEF precisely for the normal, gamma and inverse-Gaussian families. We also investigate the two LR pairings not covered by those results. If equal-tail and LR $p$-values coincide throughout a NEF satisfying our standing regularity assumptions, then $(V^{2/3})^{\prime \prime \prime }=0$ on the mean domain. The same coincidence also forces an explicit density-at-the-mean identity. For an i.i.d.\ sample with canonical sufficient statistic $T_n=\sum_{i=1}^nX_i$, persistence of equal-tail-LR coincidence throughout the family along an unbounded sequence of sample sizes forces Gaussianity. A corresponding density-LR statement is given conditionally on an explicitly stated differentiated local Edgeworth expansion. Finally, inverse-Gaussian and hyperbolic-secant examples quantify numerical $p$-value differences, disagreement of rejection decisions, and differences in power.