On Rereading Stein’s Lemma: Its Intrinsic Connection with Cramér-Rao Identity and Some New Identities

Abstract

Now is an opportune time to revisit Stein’s (1973) beautiful lemma all over again. It is especially so since researchers have recently begun discovering a great deal of potential of closely related Stein’s unbiased risk estimate (SURE) in a number of directions involving novel applications. In recognition of the importance of the topic of Stein’s (1973; Ann Statist 9:1135–1151, 1981) research and its elegance, we include a selective review from the field. The process of rereading Stein’s lemma and reliving its awesome simplicity as well as versatility rekindled a number of personal thoughts, queries, and observations. A number of new and interesting insights are highlighted in the spirit of providing updated and futuristic versions of the celebrated lemma by largely focusing on univariate continuous distributions not belonging to an exponential family. In doing so, a number of new identities have emerged when the parent population is continuous, but they are highly non-normal. Last, but not the least, we have argued that there is no big foundational difference between the basic messages obtained via Stein’s identity and Cramer-Rao identity.