Introduction to High-Dimensional Statistics
1st Edition
Published on 7. January 2015
Book
Hardback
270 pages
978-1-4822-3794-8 (ISBN)
Description
Ever-greater computing technologies have given rise to an exponentially growing volume of data. Today massive data sets (with potentially thousands of variables) play an important role in almost every branch of modern human activity, including networks, finance, and genetics. However, analyzing such data has presented a challenge for statisticians and data analysts and has required the development of new statistical methods capable of separating the signal from the noise.
Introduction to High-Dimensional Statistics is a concise guide to state-of-the-art models, techniques, and approaches for handling high-dimensional data. The book is intended to expose the reader to the key concepts and ideas in the most simple settings possible while avoiding unnecessary technicalities.
Offering a succinct presentation of the mathematical foundations of high-dimensional statistics, this highly accessible text:
Describes the challenges related to the analysis of high-dimensional data
Covers cutting-edge statistical methods including model selection, sparsity and the lasso, aggregation, and learning theory
Provides detailed exercises at the end of every chapter with collaborative solutions on a wikisite
Illustrates concepts with simple but clear practical examples
Introduction to High-Dimensional Statistics is suitable for graduate students and researchers interested in discovering modern statistics for massive data. It can be used as a graduate text or for self-study.
Introduction to High-Dimensional Statistics is a concise guide to state-of-the-art models, techniques, and approaches for handling high-dimensional data. The book is intended to expose the reader to the key concepts and ideas in the most simple settings possible while avoiding unnecessary technicalities.
Offering a succinct presentation of the mathematical foundations of high-dimensional statistics, this highly accessible text:
Describes the challenges related to the analysis of high-dimensional data
Covers cutting-edge statistical methods including model selection, sparsity and the lasso, aggregation, and learning theory
Provides detailed exercises at the end of every chapter with collaborative solutions on a wikisite
Illustrates concepts with simple but clear practical examples
Introduction to High-Dimensional Statistics is suitable for graduate students and researchers interested in discovering modern statistics for massive data. It can be used as a graduate text or for self-study.
Reviews / Votes
"Introduction to High-Dimensional Statistics by Christophe Giraud succeeds singularly at providing a structured introduction to this active field of research. ... it is arguably the most accessible overview yet published of the mathematical ideas and principles that one needs to master to enter the field of high-dimensional statistics. ... recommended to anyone interested in the main results of current research in high-dimensional statistics as well as anyone interested in acquiring the core mathematical skills to enter this area of research."-Journal of the American Statistical Association, December 2015
"This is an attractive textbook. It will prove a very useful addition to any library or personal reference collection. ... This book achieves well what it sets out to provide, an introduction to the mathematical foundations of high-dimensional statistics. ... likely to stand the test of time well."
-International Statistical Review, 83, 2015
"There is a real need for this book. It can quickly make someone new to the field familiar with modern topics in high-dimensional statistics and machine learning, and it is great as a textbook for an advanced graduate course."
-Marten H. Wegkamp, Cornell University, Ithaca, New York, USA
"As a mathematician, I am quite charmed by the book and its focus on getting the important ideas through in as short a form as possible, all the while sacrificing none of the mathematical correctness. I certainly plan to use it myself as a support in my own lectures!"
-Gilles Blanchard, University of Potsdam, Germany
"The book Introduction to High-Dimensional Statistics by Christophe Giraud succeeds singularly at providing a structured introduction to this active field of research. It describes a statistical pipeline where statistical principles enable the development of new methods, which, in turn, require a new mathematical analysis...A striking aspect of this book is the omnipresence of computational considerations across chapters. The author carefully points to potential implementations, R packages and algorithmic details that have now become inherent to modern high-dimensional statistical research...Giraud also offers informative and fairly comprehensive bibliographical notes that point to the main results of the field as well as connected work...It should be recommended to anyone interested in the main results of current research in high-dimensional statistics as well as anyone interested in acquiring the core mathematical skills to enter this area of research."
- Philippe Rigollet, Massachusetts Institute of Technology, USA
More details
Series
Language
English
Place of publication
Oakville
Canada
Target group
College/higher education
Professional and scholarly
Graduate students and researchers of statistics, data science, and machine learning.
Illustrations
scatter color on pages 2, 186, 204 and 206, 33 s/w Abbildungen, 2 s/w Tabellen
scatter color on pages 2, 186, 204 and 206; 2 Tables, black and white; 33 Illustrations, black and white
Dimensions
Height: 234 mm
Width: 159 mm
Weight
635 gr
ISBN-13
978-1-4822-3794-8 (9781482237948)
Copyright in bibliographic data and cover images is held by Nielsen Book Services Limited or by the publishers or by their respective licensors: all rights reserved.
Schweitzer Classification
Other editions
New editions
Christophe Giraud
Introduction to High-Dimensional Statistics
Book
08/2021
2nd Edition
Chapman & Hall/CRC
€122.10
Shipment within 15-20 days
Person
Christophe Giraud was a student of the Ecole Normale Superieure de Paris, and he received a Ph.D in probability theory from the University Paris 6. He was assistant professor at the University of Nice from 2002 to 2008. He has been associate professor at the Ecole Polytechnique since 2008 and professor at Paris Sud University (Orsay) since 2012. His current research focuses mainly on the statistical theory of high-dimensional data analysis and its applications to life sciences.
Content
<P>Preface</P>
<P>Acknowledgments</P><B>
<P>Introduction</P></B>
<P>High-Dimensional Data</P>
<P>Curse of Dimensionality</P><I>
<P>Lost in the Immensity of High-Dimensional Spaces</P>
<P>Fluctuations Cumulate</P>
<P>An Accumulation of Rare Events May Not Be Rare</P>
<P>Computational Complexity</P></I>
<P>High-Dimensional Statistics</P><I>
<P>Circumventing the Curse of Dimensionality</P>
<P>A Paradigm Shift</P>
<P>Mathematics of High-Dimensional Statistics</P></I>
<P>About This Book</P><I>
<P>Statistics and Data Analysis</P>
<P>Purpose of This Book</P>
<P>Overview</P></I>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Strange Geometry of High-Dimensional Spaces</P>
<P>Volume of a p-Dimensional Ball</P>
<P>Tails of a Standard Gaussian Distribution</P>
<P>Principal Component Analysis</P>
<P>Basics of Linear Regression</P>
<P>Concentration of the Square Norm of a Gaussian Random Variable</P></I><B>
<P>Model Selection</P></B>
<P>Statistical Setting</P>
<P>To Select among a Collection of Models</P><I>
<P>Models and Oracle</P>
<P>Model Selection Procedures</P></I>
<P>Risk Bound for Model Selection</P><I>
<P>Oracle Risk Bound</P></I>
<P>Optimality</P><I>
<P>Minimax Optimality</P>
<P>Frontier of Estimation in High Dimensions</P>
<P>Minimal Penalties</P></I>
<P>Computational Issues</P>
<P>Illustration</P>
<P>An Alternative Point of View on Model Selection</P>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Orthogonal Design</P>
<P>Risk Bounds for the Different Sparsity Settings</P>
<P>Collections of Nested Models</P>
<P>Segmentation with Dynamic Programming</P>
<P>Goldenshluger-Lepski Method</P>
<P>Minimax Lower Bounds</P></I><B>
<P>Aggregation of Estimators</P></B>
<P>Introduction</P>
<P>Gibbs Mixing of Estimators</P>
<P>Oracle Risk Bound</P>
<P>Numerical Approximation by Metropolis-Hastings</P>
<P>Numerical Illustration</P>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Gibbs Distribution</P>
<P>Orthonormal Setting with Power Law Prior</P>
<P>Group-Sparse Setting</P>
<P>Gain of Combining</P>
<P>Online Aggregation</P></I><B>
<P>Convex Criteria</P></B>
<P>Reminder on Convex Multivariate Functions</P><I>
<P>Subdifferentials</P>
<P>Two Useful Properties</P></I>
<P>Lasso Estimator</P><I>
<P>Geometric Insights</P>
<P>Analytic Insights</P>
<P>Oracle Risk Bound</P>
<P>Computing the Lasso Estimator</P>
<P>Removing the Bias of the Lasso Estimator</P></I>
<P>Convex Criteria for Various Sparsity Patterns</P><I>
<P>Group-Lasso (Group Sparsity)</P>
<P>Sparse-Group Lasso (Sparse-Group Sparsity)</P>
<P>Fused-Lasso (Variation Sparsity)</P></I>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>When Is the Lasso Solution Unique?</P>
<P>Support Recovery via the Witness Approach</P>
<P>Lower Bound on the Compatibility Constant</P>
<P>On the Group-Lasso</P>
<P>Dantzig Selector</P>
<P>Projection on the l<SUP>1</SUP>-Ball</P>
<P>Ridge and Elastic-Net</P></I><B>
<P>Estimator Selection</P></B>
<P>Estimator Selection</P>
<P>Cross-Validation Techniques</P>
<P>Complexity Selection Techniques</P><I>
<P>Coordinate-Sparse Regression</P>
<P>Group-Sparse Regression</P>
<P>Multiple Structures</P></I>
<P>Scaled-Invariant Criteria</P>
<P>References and Discussion</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Expected V-Fold CV l<SUP>2</SUP>-Risk</P>
<P>Proof of Corollary 5.5</P>
<P>Some Properties of Penalty (5.4)</P>
<P>Selecting the Number of Steps for the Forward Algorithm</P></I><B>
<P>Multivariate Regression</P></B>
<P>Statistical Setting</P>
<P>A Reminder on Singular Values</P>
<P>Low-Rank Estimation</P><I>
<P>If We Knew the Rank of A*</P>
<P>When the Rank of A* Is Unknown</P></I>
<P>Low Rank and Sparsity</P><I>
<P>Row-Sparse Matrices</P>
<P>Criterion for Row-Sparse and Low-Rank Matrices</P>
<P>Convex Criterion for Low Rank Matrices</P>
<P>Convex Criterion for Sparse and Low-Rank Matrices</P></I>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Hard-Thresholding of the Singular Values</P>
<P>Exact Rank Recovery</P>
<P>Rank Selection with Unknown Variance</P></I><B>
<P>Graphical Models</P></B>
<P>Reminder on Conditional Independence</P>
<P>Graphical Models</P><I>
<P>Directed Acyclic Graphical Models</P>
<P>Nondirected Models</P></I>
<P>Gaussian Graphical Models (GGM)</P><I>
<P>Connection with the Precision Matrix and the Linear Regression</P>
<P>Estimating g by Multiple Testing</P>
<P>Sparse Estimation of the Precision Matrix</P>
<P>Estimation of g by Regression</P></I>
<P>Practical Issues</P>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Factorization in Directed Models</P>
<P>Moralization of a Directed Graph</P>
<P>Convexity of -log(det(K))</P>
<P>Block Gradient Descent with the l<SUP>1 </SUP>/ l<SUP>2</SUP> Penalty</P>
<P>Gaussian Graphical Models with Hidden Variables</P>
<P>Dantzig Estimation of Sparse Gaussian Graphical Models</P>
<P>Gaussian Copula Graphical Models</P>
<P>Restricted Isometry Constant for Gaussian Matrices</P></I><B>
<P>Multiple Testing</P></B>
<P>An Introductory Example</P><I>
<P>Differential Expression of a Single Gene</P>
<P>Differential Expression of Multiple Genes</P></I>
<P>Statistical Setting</P><I>
<P>p-Values</P>
<P>Multiple Testing Setting</P>
<P>Bonferroni Correction</P></I>
<P>Controlling the False Discovery Rate</P><I>
<P>Heuristics</P>
<P>Step-Up Procedures</P>
<P>FDR Control under the WPRDS Property</P></I>
<P>Illustration</P>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>FDR versus FWER</P>
<P>WPRDS Property</P>
<P>Positively Correlated Normal Test Statistics</P></I><B>
<P>Supervised Classification</P></B>
<P>Statistical Modeling</P><I>
<P>Bayes Classifier</P>
<P>Parametric Modeling</P>
<P>Semi-Parametric Modeling</P>
<P>Nonparametric Modeling</P></I>
<P>Empirical Risk Minimization</P><I>
<P>Misclassification Probability of the Empirical Risk Minimizer</P>
<P>Vapnik-Chervonenkis Dimension</P>
<P>Dictionary Selection</P></I>
<P>From Theoretical to Practical Classifiers</P><I>
<P>Empirical Risk Convexification</P>
<P>Statistical Properties</P>
<P>Support Vector Machines</P>
<P>AdaBoost</P>
<P>Classifier Selection</P></I>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Linear Discriminant Analysis</P>
<P>VC Dimension of Linear Classifiers in R<SUP>d</SUP> </P>
<P>Linear Classifiers with Margin Constraints</P>
<P>Spectral Kernel</P>
<P>Computation of the SVM Classifier</P>
<P>Kernel Principal Component Analysis (KPCA)</P></I><B>
<P>Gaussian Distribution</P></B>
<P>Gaussian Random Vectors</P>
<P>Chi-Square Distribution</P>
<P>Gaussian Conditioning</P><B>
<P>Probabilistic Inequalities</P></B>
<P>Basic Inequalities</P>
<P>Concentration Inequalities</P><I>
<P>McDiarmid Inequality</P>
<P>Gaussian Concentration Inequality</P></I>
<P>Symmetrization and Contraction Lemmas</P><I>
<P>Symmetrization Lemma</P>
<P>Contraction Principle</P></I>
<P>Birge's Inequality</P><B>
<P>Linear Algebra</P></B>
<P>Singular Value Decomposition (SVD)</P>
<P>Moore-Penrose Pseudo-Inverse</P>
<P>Matrix Norms</P>
<P>Matrix Analysis</P><B>
<P>Subdifferentials of Convex Functions</P></B>
<P>Subdifferentials and Subgradients</P>
<P>Examples of Subdifferentials</P><B>
<P>Reproducing Kernel Hilbert Spaces</P></B>
<P>Notations</P>
<P>Bibliography</P>
<P>Index</P>
<P>Acknowledgments</P><B>
<P>Introduction</P></B>
<P>High-Dimensional Data</P>
<P>Curse of Dimensionality</P><I>
<P>Lost in the Immensity of High-Dimensional Spaces</P>
<P>Fluctuations Cumulate</P>
<P>An Accumulation of Rare Events May Not Be Rare</P>
<P>Computational Complexity</P></I>
<P>High-Dimensional Statistics</P><I>
<P>Circumventing the Curse of Dimensionality</P>
<P>A Paradigm Shift</P>
<P>Mathematics of High-Dimensional Statistics</P></I>
<P>About This Book</P><I>
<P>Statistics and Data Analysis</P>
<P>Purpose of This Book</P>
<P>Overview</P></I>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Strange Geometry of High-Dimensional Spaces</P>
<P>Volume of a p-Dimensional Ball</P>
<P>Tails of a Standard Gaussian Distribution</P>
<P>Principal Component Analysis</P>
<P>Basics of Linear Regression</P>
<P>Concentration of the Square Norm of a Gaussian Random Variable</P></I><B>
<P>Model Selection</P></B>
<P>Statistical Setting</P>
<P>To Select among a Collection of Models</P><I>
<P>Models and Oracle</P>
<P>Model Selection Procedures</P></I>
<P>Risk Bound for Model Selection</P><I>
<P>Oracle Risk Bound</P></I>
<P>Optimality</P><I>
<P>Minimax Optimality</P>
<P>Frontier of Estimation in High Dimensions</P>
<P>Minimal Penalties</P></I>
<P>Computational Issues</P>
<P>Illustration</P>
<P>An Alternative Point of View on Model Selection</P>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Orthogonal Design</P>
<P>Risk Bounds for the Different Sparsity Settings</P>
<P>Collections of Nested Models</P>
<P>Segmentation with Dynamic Programming</P>
<P>Goldenshluger-Lepski Method</P>
<P>Minimax Lower Bounds</P></I><B>
<P>Aggregation of Estimators</P></B>
<P>Introduction</P>
<P>Gibbs Mixing of Estimators</P>
<P>Oracle Risk Bound</P>
<P>Numerical Approximation by Metropolis-Hastings</P>
<P>Numerical Illustration</P>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Gibbs Distribution</P>
<P>Orthonormal Setting with Power Law Prior</P>
<P>Group-Sparse Setting</P>
<P>Gain of Combining</P>
<P>Online Aggregation</P></I><B>
<P>Convex Criteria</P></B>
<P>Reminder on Convex Multivariate Functions</P><I>
<P>Subdifferentials</P>
<P>Two Useful Properties</P></I>
<P>Lasso Estimator</P><I>
<P>Geometric Insights</P>
<P>Analytic Insights</P>
<P>Oracle Risk Bound</P>
<P>Computing the Lasso Estimator</P>
<P>Removing the Bias of the Lasso Estimator</P></I>
<P>Convex Criteria for Various Sparsity Patterns</P><I>
<P>Group-Lasso (Group Sparsity)</P>
<P>Sparse-Group Lasso (Sparse-Group Sparsity)</P>
<P>Fused-Lasso (Variation Sparsity)</P></I>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>When Is the Lasso Solution Unique?</P>
<P>Support Recovery via the Witness Approach</P>
<P>Lower Bound on the Compatibility Constant</P>
<P>On the Group-Lasso</P>
<P>Dantzig Selector</P>
<P>Projection on the l<SUP>1</SUP>-Ball</P>
<P>Ridge and Elastic-Net</P></I><B>
<P>Estimator Selection</P></B>
<P>Estimator Selection</P>
<P>Cross-Validation Techniques</P>
<P>Complexity Selection Techniques</P><I>
<P>Coordinate-Sparse Regression</P>
<P>Group-Sparse Regression</P>
<P>Multiple Structures</P></I>
<P>Scaled-Invariant Criteria</P>
<P>References and Discussion</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Expected V-Fold CV l<SUP>2</SUP>-Risk</P>
<P>Proof of Corollary 5.5</P>
<P>Some Properties of Penalty (5.4)</P>
<P>Selecting the Number of Steps for the Forward Algorithm</P></I><B>
<P>Multivariate Regression</P></B>
<P>Statistical Setting</P>
<P>A Reminder on Singular Values</P>
<P>Low-Rank Estimation</P><I>
<P>If We Knew the Rank of A*</P>
<P>When the Rank of A* Is Unknown</P></I>
<P>Low Rank and Sparsity</P><I>
<P>Row-Sparse Matrices</P>
<P>Criterion for Row-Sparse and Low-Rank Matrices</P>
<P>Convex Criterion for Low Rank Matrices</P>
<P>Convex Criterion for Sparse and Low-Rank Matrices</P></I>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Hard-Thresholding of the Singular Values</P>
<P>Exact Rank Recovery</P>
<P>Rank Selection with Unknown Variance</P></I><B>
<P>Graphical Models</P></B>
<P>Reminder on Conditional Independence</P>
<P>Graphical Models</P><I>
<P>Directed Acyclic Graphical Models</P>
<P>Nondirected Models</P></I>
<P>Gaussian Graphical Models (GGM)</P><I>
<P>Connection with the Precision Matrix and the Linear Regression</P>
<P>Estimating g by Multiple Testing</P>
<P>Sparse Estimation of the Precision Matrix</P>
<P>Estimation of g by Regression</P></I>
<P>Practical Issues</P>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Factorization in Directed Models</P>
<P>Moralization of a Directed Graph</P>
<P>Convexity of -log(det(K))</P>
<P>Block Gradient Descent with the l<SUP>1 </SUP>/ l<SUP>2</SUP> Penalty</P>
<P>Gaussian Graphical Models with Hidden Variables</P>
<P>Dantzig Estimation of Sparse Gaussian Graphical Models</P>
<P>Gaussian Copula Graphical Models</P>
<P>Restricted Isometry Constant for Gaussian Matrices</P></I><B>
<P>Multiple Testing</P></B>
<P>An Introductory Example</P><I>
<P>Differential Expression of a Single Gene</P>
<P>Differential Expression of Multiple Genes</P></I>
<P>Statistical Setting</P><I>
<P>p-Values</P>
<P>Multiple Testing Setting</P>
<P>Bonferroni Correction</P></I>
<P>Controlling the False Discovery Rate</P><I>
<P>Heuristics</P>
<P>Step-Up Procedures</P>
<P>FDR Control under the WPRDS Property</P></I>
<P>Illustration</P>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>FDR versus FWER</P>
<P>WPRDS Property</P>
<P>Positively Correlated Normal Test Statistics</P></I><B>
<P>Supervised Classification</P></B>
<P>Statistical Modeling</P><I>
<P>Bayes Classifier</P>
<P>Parametric Modeling</P>
<P>Semi-Parametric Modeling</P>
<P>Nonparametric Modeling</P></I>
<P>Empirical Risk Minimization</P><I>
<P>Misclassification Probability of the Empirical Risk Minimizer</P>
<P>Vapnik-Chervonenkis Dimension</P>
<P>Dictionary Selection</P></I>
<P>From Theoretical to Practical Classifiers</P><I>
<P>Empirical Risk Convexification</P>
<P>Statistical Properties</P>
<P>Support Vector Machines</P>
<P>AdaBoost</P>
<P>Classifier Selection</P></I>
<P>Discussion and References</P><I>
<P>Take-Home Message</P>
<P>References</P></I>
<P>Exercises</P><I>
<P>Linear Discriminant Analysis</P>
<P>VC Dimension of Linear Classifiers in R<SUP>d</SUP> </P>
<P>Linear Classifiers with Margin Constraints</P>
<P>Spectral Kernel</P>
<P>Computation of the SVM Classifier</P>
<P>Kernel Principal Component Analysis (KPCA)</P></I><B>
<P>Gaussian Distribution</P></B>
<P>Gaussian Random Vectors</P>
<P>Chi-Square Distribution</P>
<P>Gaussian Conditioning</P><B>
<P>Probabilistic Inequalities</P></B>
<P>Basic Inequalities</P>
<P>Concentration Inequalities</P><I>
<P>McDiarmid Inequality</P>
<P>Gaussian Concentration Inequality</P></I>
<P>Symmetrization and Contraction Lemmas</P><I>
<P>Symmetrization Lemma</P>
<P>Contraction Principle</P></I>
<P>Birge's Inequality</P><B>
<P>Linear Algebra</P></B>
<P>Singular Value Decomposition (SVD)</P>
<P>Moore-Penrose Pseudo-Inverse</P>
<P>Matrix Norms</P>
<P>Matrix Analysis</P><B>
<P>Subdifferentials of Convex Functions</P></B>
<P>Subdifferentials and Subgradients</P>
<P>Examples of Subdifferentials</P><B>
<P>Reproducing Kernel Hilbert Spaces</P></B>
<P>Notations</P>
<P>Bibliography</P>
<P>Index</P>