Schedule

Preliminary schedule

Homeworks within parentheses are additional problems for FEL3202.

1. Introduction. Updated slides (Friday 15/5, 15-17) . Chapter 1-2 in Lecture Notes (LN). Chapter 1-2 in Ljung 
• Signals and systems
• The basic problem
• Some examples
• Introduction to parameter estimation
• Some pitfalls
• HW: 1.1 a-d  (1.1e). 2.1 (2.2, 2.5) ) Deadline Tuesday 26/5.
2. Probabilistic models Updated slides (Tuesday 19/5, 10-12). Chapter 3 in LN. Chapter 4 in Ljung. 
   • Models and model structures
   • Estimators
   • A probabilistic toolshed
   • HW: 3.3 a-f (g,h), 3.4 a (b,c) . Deadline: Friday 29/5.
3. Probabilistic models continued, and estimation theory (Tuesday 26/5, 10-12). Sections 3.4.7-4.2 in LN. Chapter 3 in Ljung 
   • A probabilistic toolshed continued
     • Stationary processes
     • Wide-sense stationarity
     • Quasi-stationarity
     • Frequency domain characterization
     • A swatch of building blocks
   • Estimation theory
     • Information contents in random variables
     • Estimation of random variables
   • HW 4.1, 4.3, 4.7f,h (3.6,4.5,4.7g). Deadline: See excel file
4. Wold decomposition and unbiased parameter estimation (Friday 29/5, 15-17). Sections 4.3-5.9 in LN. Chapter 7 in Ljung.
   • Wold decomposition
     • Linearly regular processes
     • Wold decomposition
     • Multivariable considerations
     • Spectral distribution function
     • Spectral factorization
     • Full rank processes
   • Unbiased parameter estimation
     • The Cramér-Rao lower bound
     • Efficient estimators
     • The maximum likelihood estimator
     • Data compression
     • Uniform minimum variance unbiased estimators
     • Best linear unbiased estimator (BLUE)
   • HW: 5.1, 5.5, 5.7 (4.7,5.3)
5. Biased parameter estimation (Tuesday 2/6, 10-12) . Chapter 6 in LN.
   • The bias-variance trade-off
   • The Cramér-Rao lower bound
   • Average risk minimization
   • Minimax estimation
   • Pointwise risk minimization
   • HW: 6.2a,b (6.2c, 6.3)
6. Estimating LTI models (Friday 5/6, 15-17). Chapter 7 in LN, Chapter 7 in Ljung.
   • LTI models
   • Maximum likelihood estimation
   • Prediction error methods
   • HW: C.7.1, C.7.2, 7.1a-e (7.1f-g, 7.2, the remaining problems are also very illuminating for  the intricacies of filtering, if you have time do more)
7. Asymptotic theory (Tuesday 9/6, 08-10). Chapter 8 in L.N. Chapters 8-9 in Ljung (we will cover these chapters in Lecture 9 though)
   • Limits of random variables
   • Large sample properties of estimators
   • Large sample properties of biased estimators
   • HW: 8.1 (8.2)
8. Modeling and estimation using Gaussian Processes (Friday 12/6, 15-17) Guest lecture by Dr Riccardo Sven Risuleo, Klarna AB
   
   • Basics, including Mercers theorem
   • Impulse response estimation
   • Estimation of nonlinear systems, including Hammerstein and Wiener models
   • Modeling and estimation of uncertain input systems
9. Asymptotic theory for the PEM (Friday 26/6, 10-12) Chapter 8 in L.N. Chapter 8, Sections 13.2, 13.4 and 13.5 in Ljung 
   • Identifiability
   • Informative experiments
   • Persistence of excitation
   • Consistency
   • Closed loop identification
   • HW: 9.2, 9.4, C.9.1, (9.1, 9.3)
10. Asymptotic theory for the PEM  (Friday 28/8, 15-17) Section 9.5 in L.N. Chapter 9  in Ljung 
    • Estimation criteria and the corresponding asymptotic covariance matrices
    • Geometric analysis
    • Reproducing kernel approach
    • SISO LTI systems
    • HW: 9.7, 9.8 C.9.2a-d, (9.5, 9.6, C.9.2e-g)
11. Experiment design  (Wednesday 2/9). Slides + Chapter 13 in Ljung.
12. Model structure selection and model validation  (Friday 4/9). Slides + Chapter 16 in Ljung
    • HW: C.9.3, C.9.4, C.9.6d-f, C.9.7, C.9.8  (C.9.6.a-c). Data can be found here.  These exercises do not have to be corrected. Just make sure you understand what is going on.
13. Computational aspects. Chapter 10 in Ljung (except subspace id) + slides
    • Gradient based optimization
    • Convex relaxations
    • Integration by Markov Chain Monte Carlo (MCMC) methods
    • Nonlinear filtering using particle filters and smoothers
14. Additional methods
    • Correlation based methods
    • Subspace identification
    • Nonlinear stochastic state-space models
    • Continuous time identification