Outline

Below is a tentative outline of the course that with probability 1 will change during the course of the course¶

0 Introduction

1 Essentials of Signals, Systems and Stochastic Processes 1.1 Probability Theory .1.2 Stochastic Processes1.3 Quasi-stationary signals 1.4 Stochastic Convergence

2 Estimation Methods2.1 Minimum Mean Square Error Estimation2.2 Maximum A Posteriori Estimation 2.3 Unbiased Parameter Estimation

Sufficient statistics, score function, Cramér-Rao lower bound, Maximum Likelihood, Rao-Blackwell3 Minimum Mean Square Error Parameter Estimation3.1 The Bias-Variance Error Trade-Off3.2 Risk and Average Risk. The Bayes Estimator Risk estimation methods

SURE, Empirical Bayes, Variational Bayes

3.3 Linear in the Parameters Models

4 Numerical algorithms

4.1 Optimization

4.2 Likelihood optimization - the EM-method

4.2 Sampling

Markov Chain Monte Carlo methods - Metropolis Hastings, Gibbs

5 Linear in the Parameters Models

6 Dynamical Models6.1 Model Structures and Probabilistic Models6.2 Estimation Methods6.2.1 Maximum Likelihood Estimation6.2.2 The Extended Invariance Principle 6.2.3 The Prediction Error Method

Optimal filtering, the Kalman filter, particle filtering and smoothing

6.2.4 Multi-Step Least-Squares Methods6.2.5 Instrumental Variable Methods6.2.6 Indirect Inference6.3 Linear Models6.3.1 Maximum Likelihood Estimation6.3.2 The Prediction Error Method6.4 Multi-Step Least-Squares Methods6.5 Subspace Identification6.6 Instrumental Variable Methods6.7 Bayesian Methods6.8 Time versus Frequency Domain Identification6.9 Continuous Time Model Identification

6.10 Grey-Box Identification

7 Model Quality7.1 Variance Quantification7.1.1 Fundamental Geometric Principles 7.1.2 Fundamental Structural Results 7.1.3 Variability of Estimated Frequency Response7.1.4 Variability of Nonlinear System Estimates7.1.5 Bootstrap Methods8 Experiment Design8.1 Identifiability 8.2 Persistence of Exciation8.3 Input Signal Design8.3.1 Common Input Signals 8.4 Application Oriented Experiment Design8.5 Adaptive Experiment Design

9 Model Validation9.1 Residual whiteness Tests 9.2 Input to residual correlation tests9.3 Model Error Modelling

10 Application Examples10.1 Closed Loop Identification10.2 Network Models10.3 Errors-in-Variables Models10.4 Block-structured Nonlinear Models

10.5 Identification for Control