Preface

Young people today love luxury. They have bad manners, despise authority, have no respect for older people, and chatter when they should be working.

(Socrates, 470-399 BC)

This book on statistical inference can be viewed as a continuation of the author's previous two books on probability theory (Paolella, 2006, 2007), hereafter referred to as Books I and II. Of those two, Book I (or any book at a comparable level) is more relevant, in establishing the basics of random variables and distributions as required to understand statistical methodology. Occasional use of material from Book II is made, though most of that required material is reviewed in the appendix herein in order to keep this volume as self-contained as possible. References to those books will be abbreviated as I and II, respectively. For example, Figure 5.1 in (Chapter 5 of) Paolella (2006) is referred to as Figure I.5.1; and similarly for equation references, where (I.5.22) and (II.4.3) refer to equations (5.22) and (4.3) in Paolella (2006) and Paolella (2007) respectively (and both are the Cauchy-Schwarz inequality).

Further prerequisites are the same as those for Book I, namely a solid command of basic undergraduate calculus and matrix algebra, and occasionally very rudimentary concepts from complex analysis, as required for working with characteristic functions. As with Books I and II, a solutions manual to the exercises is available.

Certainly, no measure theory is required, nor any previous exposure to statistical inference, though it would be useful to have had an introductory course in statistics or data analysis. The book is aimed at beginning master's students in statistics, though it is written to be fully accessible to master's students in the social sciences. In particular, I have in mind students in economics and finance, as I provide introductory coverage of some nonstandard topics, notably Chapter 9 on heavy-tailed distributions and tail estimation, and detailed coverage of the mixed normal distribution in Chapter 5.

Naturally, the book can be also used for undergraduates in a mathematics program. For the intended audience of master's students in statistics or the social sciences, the instructor is welcome to skip material that uses concepts from convergence and limit theorems if the target audience is not ready for such mathematics. This is one of the points of this book: such material is included so that, for example, accessible, detailed proofs of the Glivenko-Cantelli theorem and the limiting distribution of the maximum likelihood estimator can be demonstrated at a reasonably rigorous level. The vast majority of the book only requires simple algebra and basic calculus.

In this book, I stick to the independent, identically distributed (i.i.d.) setting, using it as a platform for introducing the major concepts arising in statistics without the additional overhead and complexities associated with, say, (generalized) linear models, survival analysis, copula methods, and time series. This also allows for more in-depth coverage of important topics such as bootstrap techniques, nonparametric inference via the empirical c.d.f., numerical optimization, discrete mixture models, bias-adjusted estimators, tail estimation (as a nice segue into the study of extreme value theory), and the method of indirect inference. A future project, referred to as Book IV, builds on the framework in the present volume and is dedicated to the linear model (regression and ANOVA) and, primarily, time series analysis (univariate ARMAX models), GARCH, and multivariate distributions for modeling and predicting financial asset returns.

Before discussing the contents of this volume, it is important to mention that, similar to Books I and II, the overriding goals are:

1. to emphasize the practical side of matters by addressing computation issues;
2. to motivate students to actively engage in the material by replicating and extending reported results, and to read the literature on topics of their interest;
3. to go beyond the standard topics and examples traditionally taught at this level, albeit still within the i.i.d. framework; and
4. to set the stage for students intending to pursue further courses in statistical/econometric inference (and quantitative risk management), as well as those embarking on careers as modern data analysts and applied quantitative researchers.

Regarding point (i), I explain to students that computer programming skills are necessary, but far from sufficient, to be successful in applied research. In an occasional lecture dedicated to programming issues, I emphasize (not sarcastically - I do not test computer skills) that it is fully optional, and those students who are truly mathematically talented can skip it, explaining that they will always have programmers in their team (in industry) or PhD students and co-authors (in academics) as resources to do the computer grunt work implementing their theoretical constructs. Oddly, nobody leaves the room.

With respect to point (ii), the reader will notice that some chapters have few (or no) exercises (some have many). This is because I believe the nature of the material presented is such that it offers the student a judicious platform for self experimentation, particularly with respect to numerical implementation. Some of the material could have been packaged as exercises (and much is), though I prefer to illustrate important concepts, distributions, and methods in a detailed way, along with code and graphics, instead of banishing it to the exercises (or, far worse, littering the exercises with trite, useless algebraic manipulations devoid of genuine application) and instead encourage the student to replicate, complement, and extend the presented material. The reader will no doubt tire at my occasional explicit suggestions for this ("The reader is encouraged."). One of my role model authors is Hamilton (1994), whose book has no exercises, is twice the size of this book, and has been praised as an outstanding presentation of time series. Hamilton clearly intended to teach the material in a straightforward, clear way, with highly detailed and accessible derivations. I aspire to a similar approach, as well as adding numeric illustrations and Matlab code.1

Regarding point (iii), besides the obvious benefit of giving students a more modern viewpoint on methods and applications in statistics, having a large variety of such is useful for students (and instructors) looking for interesting, relevant topics for master's theses. An example of a nonstandard topic of interest is in Chapter 5, giving a detailed discussion on the problems associated with, and solutions to, estimating the (univariate) discrete mixed normal, via a variety of non-m.l.e. methods (empirical m.g.f., c.f., quantile-based methods, etc.), and the use of the EM algorithm with shrinkage, with its immediate extension to the multivariate case. For the latter, I refer to recent work of mine using the minimum covariance determinant (MCD) for parameter estimation, this also serving as an example of (i) what can be done when, here, the multivariate normal mixture is surely misspecified, and (ii) use of a most likely inconsistent estimator (which outperforms the m.l.e. in terms of density forecasting and portfolio allocation for financial returns data).

Particularly with the less common topics developed in Part III of this book, the result is, like Books I and II, a substantially larger project than some similarly positioned books. It is thus essential to understand that not everything in the text is supposed to be (or could be) covered in the classroom, at least not in one semester. In my opinion, students (even in mathematics departments, but particularly those in the social sciences) benefit from having clearly laid out explanations, detailed proofs, illustrative examples, a variety of approaches, introductions to modern techniques, and discussions of important, possibly controversial topics (e.g., the irrelevance of consistent estimators in light of the notion that, in realistic settings, the model is wrong anyway, and changing through time or space; and the arguable superfluousness, if not danger, of the typical hypothesis testing framework), as well as topics that could initially be skipped in a first course, but returned to later or assigned as outside reading, depending on the interests and abilities of the students.

I wish to emphasize that this book is for teaching, as (obviously) opposed to being a research monograph, or (less obviously) a dry regurgitation of traditional concepts and examples. An anonymous reviewer of Book I, when I initially submitted it to the publisher Wiley, remarked "it's too much material: It seems the author has written a brain dump." While I like to think I have much more in my head than what was written in that book, he (his gender was indeed disclosed to me) apparently believes that students (let alone instructors) are incapable of assessing what material is core, and what can be deemed "extra," or suitable for reading after the main concepts are mastered. It is trivial to just skip some material, whereas not having it at all results in an admittedly shorter book (who cares, besides arguably the publisher?) that accomplishes far less, and might even give the student a false sense of understanding and competence (which will be painfully revealed in a quant job interview). Fortunately, not everyone agrees with him: Besides heart-warming student feedback over the years on Book I (from...