2:35 Resource Sharing: Mutlu emphasizes the need for shared resources (caches, buses, interconnects) in computer systems for efficiency and utilization, but highlights the resulting problem of contention.
6:00 Contention Issues: Uncontrolled resource sharing can lead to performance degradation, performance isolation (variability depending on co-running threads), and quality of service (QoS) issues.
9:48 Unfairness & Starvation: "Free for all" resource allocation like Least Recently Used (LRU) caching can lead to unfairness and starvation of some threads. Example given of a memory-intensive thread (Matlab) significantly slowing down another thread (GCC).
11:35 Programmer Challenges: Unpredictable performance due to resource contention makes it hard for programmers to optimize their applications effectively.
17:11 Goal: Efficient & Controllable Sharing: Mutlu outlines the need for mechanisms to achieve efficient, controllable, and partitionable resource sharing. He introduces two approaches: "smart" resources with built-in QoS mechanisms, and "dumb" resources managed by external techniques.
19:21 Memory Performance Hogs: Mutlu revisits the problem of memory performance hogs - threads that, due to memory-intensive and streaming access patterns, monopolize memory bandwidth and starve other threads. A real-world example is shown where a streaming application significantly slows down a random access application.
24:49 Solving the Problem: Four fundamental techniques for interference control are presented:
Prioritization/request scheduling
Data mapping to different resources
Core/source throttling
Application and thread scheduling
24:56 Stall Time Fair Memory Scheduling (STFM): This policy aims to equalize the slowdown experienced by threads, measured as the relative increase in stall time compared to running alone. STFM dynamically switches between throughput-oriented and fairness-oriented scheduling to maintain acceptable fairness levels.
41:13 STFM Challenges: Accurately estimating a thread's stall time when running alone is a major challenge for STFM, as is its complexity of implementation.
53:50 Oldest First is Unfair: Analogy of a revolving door illustrates how an oldest-first scheduling policy can be unfair, as it implicitly favors intensive threads that continuously generate requests.
58:07 Exploiting Bank-Level Parallelism: Mutlu highlights the importance of memory controllers being aware of and exploiting bank-level parallelism in multi-threaded applications. A naive scheduler may serialize requests from different threads, even if they target different banks, leading to unnecessary stall time.
59:22 Parallelism-Aware Batch Scheduler (ParBS): Introduced to exploit bank-level parallelism and improve fairness. Two key principles are:
Parallelism awareness: Scheduling requests from a thread to different banks back-to-back.
Request batching: Grouping requests from different threads into batches and servicing each batch in order to prevent starvation.
1:06:29 Within-Batch Scheduling: ParBS employs shortest stall time first (SSTF) ranking to prioritize threads within a batch. Threads with lower estimated stall times are ranked higher.
1:07:16 ParBS Prioritization Order: The scheduler prioritizes marked (batched) requests, then row hits, higher-ranked threads, and finally older requests.
1:10:59 Shortest Stall Time First Ranking: SSTF ranking aims to maximize throughput by prioritizing threads with lower expected stall times, leading to faster completion of less memory-intensive threads.
1:26:15 Identifying "Limited Threads" in Multi-threaded Applications: Mutlu introduces the concept of identifying "limited threads" in multi-threaded applications - threads on the critical path that slow down the overall application. Prioritizing these threads can improve performance. Three methods are suggested:
Identifying the most contended critical section
Identifying the slowest pipeline stage
Identifying the thread falling most behind in execution
2:15:44 Blacklisting Memory Scheduler (Bliss): Aims for simplicity by grouping threads into two categories: interference-vulnerable and interference-causing. "Hogging" threads are blacklisted and deprioritized.
2:31:49 Heterogeneous Systems: Mutlu briefly discusses resource allocation in heterogeneous systems with CPUs, GPUs, and accelerators, highlighting the need to mitigate interference and provide predictable performance. Several research papers tackling this problem are presented, but not discussed in detail.
2:38:43 Service Guarantees: Mutlu touches upon the concept of service guarantees and predictable performance, highlighting the need for memory controllers to support performance isolation and service level agreements (SLAs). Several research papers are mentioned, but detailed discussion is deferred to a future lecture.
2:39:49 Memory Channel Partitioning (MCP): Presents a technique for partitioning memory channels between applications to isolate their memory requests and reduce interference. Two key principles are:
Separating low and high memory intensity applications
Separating low and high row buffer locality applications
Note: The lecture abruptly ends with a discussion of MCP due to time constraints.