FEO3250 Information Theory and Source Coding 12.0 credits

Information theory of discrete and continuous variables: Entropy, Kraft inequality, relative entropy, entropy rate, redundancy rate, mutual information, asymptotic equipartition.

Lossless coding: nonadaptive codes: Shannon, Huffmann, arithmetic codes. Universal and adaptive codes. Ziv-Lempel codes.

Rate-distortion theory: the rate-distortion function, Shannon lower bound, rate distribution over independent variables, reverse waterfilling, Blahut algorithm.

High-rate quantization: resolution-constrained and entropy-constrained quantization. Scalar and vector quantization.

Low-rate quantization: Lloyd training algorithm for resolution-constrained and entropy-constrained cases.

Transform coding: Orthonormal transforms, transform coding gain, Karhunen-Loeve transform, Energy concentration.

Predictive coding: MSE-optimal linear prediction, linear prediction gain, coding of prediction error, closed-loop prediction.

After the course, the student should be able to:

• describe the general principles of rate distortion theory
• assess given source coding schemes in the light of these principles
• devise coding schemes for given sources
• apply these principles to accomplish a project task
• contribute to the research frontier in the area

EQ1220 Signal Theory or equivalent

EQ1220 Signal Theory or equivalent

If the course is discontinued, students may request to be examined during the following two academic years.

• EXA1 - Examination, 12.0 credits, grading scale: P, F

Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.

The examiner may apply another examination format when re-examining individual students.

• Homework problems: pass/fail
• Written examination: pass/fail
• Project: pass/fail

Homework, examination, and project

Markus Flierl

• All members of a group are responsible for the group's work.
• In any assessment, every student shall honestly disclose any help received and sources used.
• In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.