Skip to main content
Till KTH:s startsida

IK2221 Networked Systems for Machine Learning 7.5 credits

Administer About course

Machine learning inference (model serving) is becoming vital for every aspect of the society. Unfortunately, inference can have very tight latency bounds and requires unsustainable amounts of resources. This course teaches students the most recent techniques for effectively serving machine learning workloads with datacenters.

Information per course offering

Termin

Information for Spring 2025 Start 17 Mar 2025 programme students

Course location

KTH Campus

Duration
17 Mar 2025 - 2 Jun 2025
Periods
P4 (7.5 hp)
Pace of study

50%

Application code

60061

Form of study

Normal Daytime

Language of instruction

English

Course memo
Course memo is not published
Number of places

Places are not limited

Target group

Open to all master's programmes as long as it can be included in the program.

Planned modular schedule
[object Object]

Contact

Examiner
No information inserted
Course coordinator
No information inserted
Teachers
No information inserted

Course syllabus as PDF

Please note: all information from the Course syllabus is available on this page in an accessible format.

Course syllabus IK2221 (Spring 2025–)
Headings with content from the Course syllabus IK2221 (Spring 2025–) are denoted with an asterisk ( )

Content and learning outcomes

Course contents

Network functions. Virtualisation. Kernel bypass technologies (e.g. DPDK) for networks with over 100 gigabits per second. Offloading to Smart Network Interface Cards (SmartNIC). Fast networking with little or no CPU intervention using remote direct memory access (RDMA). Network aspects of machine learning inference using graphics processing units (GPUs). Load estimation and load balancing Request for dispatch and scheduling. Efficient, large-scale machine learning inference. Inference with large language models (LLM).

Intended learning outcomes

After passing the course, the student should be able to:

  • describe Network Functions Virtualization components and how they work together
  • configure an example of a service using virtualisation of network functions (NFV)
  • in detail explain the important advantages of remote direct memory access (RDMA) and how it operates
  • analyse methods for performing network I/O directly to/from graphics processors (GPU)
  • in detail, explain methods to improve inference latency
  • describe the role of the load balancing for servers
  • give examples of and describe current research problems with serving machine learning workloads in data centers

Literature and preparations

Specific prerequisites

Knowledge in advanced Internet technique, 7.5 higher education credits, equivalent completed course IK2215. Knowledge and skills in programming in C++, Java or Python, 6 higher education credits, equivalent completed course DD1310-DD1319/DD1331/DD1337/DD100N/ID1018.

Equipment

No information inserted

Literature

No information inserted

Examination and completion

If the course is discontinued, students may request to be examined during the following two academic years.

Grading scale

A, B, C, D, E, FX, F

Examination

  • PRO1 - Project assignments, 2.5 credits, grading scale: A, B, C, D, E, FX, F
  • SEM1 - Paper summaries, 2.5 credits, grading scale: A, B, C, D, E, FX, F
  • TEN1 - Written exam, 2.5 credits, grading scale: A, B, C, D, E, FX, F

Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.

The examiner may apply another examination format when re-examining individual students.

Opportunity to complete the requirements via supplementary examination

No information inserted

Opportunity to raise an approved grade via renewed examination

No information inserted

Examiner

Profile picture Dejan Kostic

Ethical approach

  • All members of a group are responsible for the group's work.
  • In any assessment, every student shall honestly disclose any help received and sources used.
  • In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.

Further information

Course room in Canvas

Registered students find further information about the implementation of the course in the course room in Canvas. A link to the course room can be found under the tab Studies in the Personal menu at the start of the course.

Offered by

EECS/Computer Science

Main field of study

Computer Science and Engineering

Education cycle

Second cycle

Add-on studies

No information inserted