Skip to main content
Till KTH:s startsida

FEO3250 Information Theory and Source Coding 12.0 credits

Administer About course

Information per course offering

Course offerings are missing for current or upcoming semesters.

Course syllabus as PDF

Please note: all information from the Course syllabus is available on this page in an accessible format.

Course syllabus FEO3250 (Spring 2019–)
Headings with content from the Course syllabus FEO3250 (Spring 2019–) are denoted with an asterisk ( )

Content and learning outcomes

Course contents

Information theory of discrete and continuous variables: Entropy, Kraft inequality, relative entropy, entropy rate, redundancy rate, mutual information, asymptotic equipartition.

Lossless coding: nonadaptive codes: Shannon, Huffmann, arithmetic codes. Universal and adaptive codes. Ziv-Lempel codes.

Rate-distortion theory: the rate-distortion function, Shannon lower bound, rate distribution over independent variables, reverse waterfilling, Blahut algorithm.

High-rate quantization: resolution-constrained and entropy-constrained quantization. Scalar and vector quantization.

Low-rate quantization: Lloyd training algorithm for resolution-constrained and entropy-constrained cases.

Transform coding: Orthonormal transforms, transform coding gain, Karhunen-Loeve transform, Energy concentration.

Predictive coding: MSE-optimal linear prediction, linear prediction gain, coding of prediction error, closed-loop prediction.

Intended learning outcomes

After the course, the student should be able to:

  • describe the general principles of rate distortion theory
  • assess given source coding schemes in the light of these principles
  • devise coding schemes for given sources
  • apply these principles to accomplish a project task
  • contribute to the research frontier in the area

Literature and preparations

Specific prerequisites

EQ1220 Signal Theory or equivalent

Recommended prerequisites

EQ1220 Signal Theory or equivalent

Literature

You can find information about course literature either in the course memo for the course offering or in the course room in Canvas.

Examination and completion

Grading scale

P, F

Examination

  • EXA1 - Examination, 12.0 credits, grading scale: P, F

Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.

The examiner may apply another examination format when re-examining individual students.

If the course is discontinued, students may request to be examined during the following two academic years.

  • Homework problems: pass/fail
  • Written examination: pass/fail
  • Project: pass/fail

Other requirements for final grade

Homework, examination, and project

Examiner

Profile picture Markus Flierl

Ethical approach

  • All members of a group are responsible for the group's work.
  • In any assessment, every student shall honestly disclose any help received and sources used.
  • In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.

Further information

Course room in Canvas

Registered students find further information about the implementation of the course in the course room in Canvas. A link to the course room can be found under the tab Studies in the Personal menu at the start of the course.

Offered by

EECS/Information Science and Engineering

Education cycle

Third cycle

Postgraduate course

Postgraduate courses at EECS/Information Science and Engineering