| 1 Introduction 1 |
| 1.1 Basic concepts in statistical learning 1 |
| 1.2 Statistical models and learning machines 10 |
| 1.3 Statistical estimation methods 18 |
| 1.4 Four main formulas 26 |
| 1.5 Overview of this book 41 |
| 1.6 Probability theory 42 |
| 2 Singularity theory 48 |
| 2.1 Polynomials and analytic functions 48 |
| 2.2 Algebraic set and analytic set 50 |
| 2.3 Singularity 53 |
| 2.4 Resolution of singularities 58 |
| 2.5 Normal crossing singularities 66 |
| 2.6 Manifold 72 |
| 3 Algebraic geometry 77 |
| 3.1 Ring and ideal 77 |
| 3.2 Real algebraic set 80 |
| 3.3 Singularities and dimension 86 |
| 3.4 Real projective space 87 |
| 3.5 Blow-up 91 |
| 3.6 Examples 99 |
| 4 Zeta function and singular integral 105 |
| 4.1 Schwartz distribution 105 |
| 4.2 State density function 111 |
| 4.3 Mellin transform 116 |
| 4.4 Evaluation of singular integral 118 |
| 4.5 Asymptotic expansion and b-function 128 |
| 5 Empirical processes 133 |
| 5.1 Convergence in law 133 |
| 5.2 Function-valued analytic functions 140 |
| 5.3 Empirical process 144 |
| 5.4 Fluctuation of Gaussian processes 154 |
| 6 Singular learning theory 158 |
| 6.1 Standard form of likelihood ratio function 160 |
| 6.2 Evidence and stochastic complexity 168 |
| 6.3 Bayes and Gibbs estimation 177 |
| 6.4 Maximum likelihood and a posteriori 203 |
| 7 Singular learning machines 217 |
| 7.1 Learning coefficient 217 |
| 7.2 Three-layered neural networks 227 |
| 7.3 Mixture models 230 |
| 7.4 Bayesian network 233 |
| 7.5 Hidden Markov model 234 |
| 7.6 Singular learning process 235 |
| 7.7 Bias and variance 239 |
| 7.8 Non-analytic learning machines 245 |
| 8 Singular statistics 249 |
| 8.1 Universally optimal learning 249 |
| 8.2 Generalized Bayes information criterion 252 |
| 8.3 Widely applicable information criteria 253 |
| 8.4 Singular hypothesis test 258 |
| 8.5 Realization of a posteriori distribution 264 |
| 8.6 From regular to singular 274 |
| Bibliography 277 |
| Index 284 |
| 1 Introduction 1 |
| 1.1 Basic concepts in statistical learning 1 |
| 1.2 Statistical models and learning machines 10 |
| 1.3 Statistical estimation methods 18 |
| 1.4 Four main formulas 26 |
| 1.5 Overview of this book 41 |
