Outline

0 Introduction

1 Essentials of Signals, Systems and Stochastic Processes 
1.1 Probability Theory .
1.2 Stochastic Processes
1.3 Quasi-stationary signals 
1.4 Stochastic Convergence 

2 Estimation Methods
2.1 Minimum Mean Square Error Estimation
2.2 Maximum A Posteriori Estimation 
2.3 Unbiased Parameter Estimation

       Sufficient statistics, score function, Cramér-Rao lower bound, Maximum Likelihood, Rao-Blackwell

3 Minimum Mean Square Error Parameter Estimation
3.1 The Bias-Variance Error Trade-Off
3.2 Risk and Average Risk. 
     The Bayes Estimator
      Risk estimation methods

           SURE, Empirical Bayes, Variational Bayes

3.3 Linear in the Parameters Models

4 Numerical algorithms

4.1 Optimization

 4.2 Likelihood optimization - the EM-method

4.2 Sampling 

      Markov Chain Monte Carlo methods - Metropolis Hastings, Gibbs

5 Linear in the Parameters Models 

6 Dynamical Models
6.1 Model Structures and Probabilistic Models
6.2 Estimation Methods
6.2.1 Maximum Likelihood Estimation
6.2.2 The Extended Invariance Principle 
6.2.3 The Prediction Error Method

        Optimal filtering, the Kalman filter, particle filtering and smoothing

6.2.4 Multi-Step Least-Squares Methods
6.2.5 Instrumental Variable Methods
6.2.6 Indirect Inference
6.3 Linear Models
6.3.1 Maximum Likelihood Estimation
6.3.2 The Prediction Error Method
6.4 Multi-Step Least-Squares Methods
6.5 Subspace Identification
6.6 Instrumental Variable Methods
6.7 Bayesian Methods
6.8 Time versus Frequency Domain Identification
6.9 Continuous Time Model Identification

6.10 Grey-Box Identification

7 Model Quality
7.1 Variance Quantification
7.1.1 Fundamental Geometric Principles 
7.1.2 Fundamental Structural Results 
7.1.3 Variability of Estimated Frequency Response
7.1.4 Variability of Nonlinear System Estimates
7.1.5 Bootstrap Methods

8 Experiment Design
8.1 Identifiability 
8.2 Persistence of Exciation
8.3 Input Signal Design
8.3.1 Common Input Signals 
8.4 Application Oriented Experiment Design
8.5 Adaptive Experiment Design 

9 Model Validation
9.1 Residual whiteness Tests 
9.2 Input to residual correlation tests
9.3 Model Error Modelling 

10 Application Examples
10.1 Closed Loop Identification
10.2 Network Models
10.3 Errors-in-Variables Models
10.4 Block-structured Nonlinear Models 

10.5 Identification for Control