PRO1 – Project Assignments, 2.5 credits, grading scale: P/F
SEM1 – Paper Summaries, 2.5 credits, grading scale: P/F
TEN1 – Written Exam, 2.5 credits, grading scale: P/F
FIK3221 Networked Systems for Machine Learning 7.5 credits
Machine learning inference (model serving) is becoming vital for every aspect of the society.
Unfortunately, inference can have very tight latency bounds and requires unsustainable amounts of
resources. This course teaches students the most recent techniques for effectively serving machine
learning workloads with datacenters.
Information per course offering
Information for Spring 2025 Start 17 Mar 2025 programme students
- Course location
KTH Kista
- Duration
- 17 Mar 2025 - 2 Jun 2025
- Periods
- P4 (7.5 hp)
- Pace of study
50%
- Application code
61208
- Form of study
Normal Daytime
- Language of instruction
English
- Course memo
- Course memo is not published
- Number of places
Places are not limited
- Target group
- No information inserted
- Planned modular schedule
- No information inserted
- Schedule
- Schedule is not published
- Part of programme
- No information inserted
Contact
Course syllabus as PDF
Please note: all information from the Course syllabus is available on this page in an accessible format.
Course syllabus FIK3221 (Spring 2025–)Headings with content from the Course syllabus FIK3221 (Spring 2025–) are denoted with an asterisk ( )
Content and learning outcomes
Course disposition
Course contents
Network functions. Virtualisation. Kernel bypass technologies (e.g. DPDK) for networks with over 100 gigabits per second. Offloading to Smart Network Interface Cards (SmartNIC). Fast networking
with little or no CPU intervention using remote direct memory access (RDMA). Network aspects of machine learning inference using graphics processing units (GPUs). Load estimation and load balancing. Request for dispatch and scheduling. Efficient, large-scale machine learning inference.
Inference with large language models (LLM).
Intended learning outcomes
After passing the course, the student should be able to:
- describe and analyze an example of a service using virtualization of network functions (NFV)
- list and analyze an example of a service using virtualization of network functions (NFV)
- explain and differentiate the important advantages of remote direct memory access (RDMA) and how it operates
- analyze methods for performing network I/O directly to/from graphics processors (GPU)
- explain methods to improve inference latency in detail
- describe and analyze the role of the load balancing for servers
- describe and analyze examples of current research problems with serving machine learning workloads in data centers
- apply the knowledge from the course to analyze your research domain, demonstrating its practical use and impact
- analyze the connections between the course material and your own research, emphasizing their significance
- argue for the validity of these connections, providing clear and evidence-based reasoning
Literature and preparations
Specific prerequisites
Knowledge in advanced Internet technique, 7.5 higher education credits, equivalent completed course IK2215. Knowledge and skills in programming in C++, Java or Python, 6 higher education credits, equivalent completed course DD1310-DD1319/DD1331/DD1337/DD100N/ID1018.
Recommended prerequisites
Knowledge in advanced Internet technique, 7.5 higher education credits, equivalent completed
course IK2215. Knowledge and skills in programming in C++, Java or Python, 6 higher education
credits, equivalent completed course DD1310-DD1319/DD1331/DD1337/DD100N/ID1018.
Literature
You can find information about course literature either in the course memo for the course offering or in the course room in Canvas.
Examination and completion
If the course is discontinued, students may request to be examined during the following two academic years.
Grading scale
P, F
Examination
- PRO1 - Project assignments, 2.5 credits, grading scale: P, F
- SEM1 - Paper summaries, 2.5 credits, grading scale: P, F
- TEN1 - Written exam, 2.5 credits, grading scale: P, F
Based on recommendation from KTH’s coordinator for disabilities, the examiner will decide how to adapt an examination for students with documented disability.
The examiner may apply another examination format when re-examining individual students.
Examiner
Ethical approach
- All members of a group are responsible for the group's work.
- In any assessment, every student shall honestly disclose any help received and sources used.
- In an oral assessment, every student shall be able to present and answer questions about the entire assignment and solution.
Further information
Course room in Canvas
Registered students find further information about the implementation of the course in the course room in Canvas. A link to the course room can be found under the tab Studies in the Personal menu at the start of the course.
Offered by
Main field of study
This course does not belong to any Main field of study.
Education cycle
Third cycle