This course introduces the principles of information theory and source coding, discusses fundamental source coding concepts, and provides hands-on experience for selected popular source coding algorithms. The course includes topics on information and entropy, lossless coding, Shannon's noiseless source coding theorem, lossy coding, rate distortion, Shannon's noisy source coding theorem, scalar and vector quantization, transform and predictive coding.
EQ2845 Information Theory and Source Coding 7.5 credits
The course treats the principles underlying the encoding of speech, audio, video, and images at low bit rates. Source coding techniques such as scalar and vector quantization, orthogonal transforms, and linear prediction are introduced and their performance is analyzed theoretically. The theoretical bounds on the performance of source coders are discussed.
Information per course offering
Information for Spring 2025 Start 14 Jan 2025 programme students
- Course location
KTH Campus
- Duration
- 14 Jan 2025 - 16 Mar 2025
- Periods
- P3 (7.5 hp)
- Pace of study
50%
- Application code
60427
- Form of study
Normal Daytime
- Language of instruction
English
- Course memo
- Course memo is not published
- Number of places
Min: 10
- Target group
See connected programs.
Open to all programmes as long as it can be included in your programme.
- Planned modular schedule
- [object Object]
- Schedule
- Part of programme
Master's Programme, ICT Innovation, åk 1, VCCN, Recommended
Master's Programme, Information and Network Engineering, åk 1, Recommended
Master's Programme, Information and Network Engineering, åk 1, COE, Recommended
Master's Programme, Information and Network Engineering, åk 1, INF, Recommended
Master's Programme, Information and Network Engineering, åk 1, MMB, Recommended
Contact
Markus Flierl (mflierl@kth.se)
Course syllabus as PDF
Please note: all information from the Course syllabus is available on this page in an accessible format.
Course syllabus EQ2845 (Spring 2019–)Headings with content from the Course syllabus EQ2845 (Spring 2019–) are denoted with an asterisk ( )
Content and learning outcomes
Course contents
Intended learning outcomes
After passing this course, participants should be able to
- describe and use the principles of information theory, like entropy, mutual information, asymptotic equipartition, data processing, prefix codes, Kraft inequality, noiseless source coding, maximum entropy, rate distortion, noisy source coding, Shannon lower bound, backward channel, reverse waterfilling, energy concentration, etc. to develop source coding algorithms,
- develop source coding schemes for lossless coding, like Huffman coding, arithmetic coding, Lempel-Ziv coding, universal source coding,
- develop source coding schemes for lossy coding, like scalar and vector quantization, Lloyd-Max quantization, entropy-constrained quantization, high-rate quantization, transform coding, predictive coding,
- implement (for example with MatLab) and assess the developed source coding schemes / algorithms,
- explain coding design choices using the principles of information theory,
- develop source coding schemes for a given source coding problem,
- model and assess source coding schemes using the principles of information theory,
- analyze given source coding problems, identify and explain the challenges, propose possible solutions, and explain the chosen design.
To achive higher grades, participants should also be able to
- solve more advanced problems in all areas mentioned above.
Literature and preparations
Specific prerequisites
For single course students: 120 credits and documented proficiency in English B or equivalent.
Recommended: EQ1220 Signal Theory or equivalent
Recommended prerequisites
EQ1220 Signal Theory or equivalent.
Equipment
No information inserted
Literature
T.M. Cover and J.A. Thomas, “Elements of Information Theory,” John Wiley & Sons, Inc., New York.
Examination and completion
If the course is discontinued, students may request to be examined during the following two academic years.
Grading scale
A, B, C, D, E, FX, F
Examination
- INL1 - Assignment, 1.5 credits, grading scale: P, F
- TEN1 - Exam, 6.0 credits, grading scale: A, B, C, D, E, FX, F
Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.
The examiner may apply another examination format when re-examining individual students.
Homework assignments 1.5 ECTS (P/F). Written exam 6 ECTS (A-F).
Opportunity to complete the requirements via supplementary examination
No information inserted
Opportunity to raise an approved grade via renewed examination
No information inserted
Examiner
Ethical approach
- All members of a group are responsible for the group's work.
- In any assessment, every student shall honestly disclose any help received and sources used.
- In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.
Further information
Course room in Canvas
Registered students find further information about the implementation of the course in the course room in Canvas. A link to the course room can be found under the tab Studies in the Personal menu at the start of the course.
Offered by
Main field of study
Electrical Engineering
Education cycle
Second cycle
Add-on studies
No information inserted
Contact
Markus Flierl (mflierl@kth.se)
Supplementary information
In this course, the EECS code of honor applies, see: http://www.kth.se/en/eecs/utbildning/hederskodex.