This live stream documents the process of developing a Linux kernel USB driver for the front panel LEDs of an Intel-based Apple Xserve. The goal is to display CPU load on these LEDs, a feature missing in Linux for this hardware. The developer starts by adapting Greg Kroah-Hartman's USB skeleton driver, removing unnecessary bulk-in functionality and focusing solely on bulk-out transfers to control the 16 LEDs via 32-byte USB packets. Initial attempts to write to the device via user space fail, leading to a strategy of direct in-kernel control. After successfully sending static test patterns, the developer integrates CPU load monitoring logic heavily inspired by Benjamin Herrenschmidt's driver for the older G5 Xserve (which used I2S, not USB). This involves using kernel statistics (get_cpu_idle_time) and per-CPU kernel workqueues (schedule_delayed_work) to calculate the load for each of the 16 CPU threads and map it to the brightness of a corresponding LED. An attempt to refactor the code to use a single workqueue instead of 16 results in kernel crashes, leading to a reversion to the per-CPU workqueue approach, which ultimately proves successful. The stream highlights the complexities of kernel driver development, debugging challenges (including kernel panics), and the process of adapting existing code.
Linux Kernel USB Driver Development for Apple Xserve Front Panel LEDs
0:00:12 Introduction & Goal: The developer aims to create a Linux USB driver for the front panel LEDs on an Intel Apple Xserve, noting a user-space attempt exists but is flawed and macOS-specific. The G5 Xserve driver (by Benjamin Herrenschmidt) uses I2S, not USB.
0:01:31 Driver Strategy: Copying: Due to the complexity of kernel APIs, the developer decides to adapt ("steal") an existing driver, choosing Greg Kroah-Hartman's USB skeleton driver after finding others unsuitable.
0:03:11 Microkernel Tangent: Expresses frustration with monolithic kernel complexity for simple tasks, wishing for a microkernel approach that would simplify driver development (predicting 1000+ lines vs. an ideal ~32).
0:06:00 Adapting the Skeleton Driver: Begins modifying the skeleton code, setting the correct USB Vendor/Product IDs (Apple: 0x05AC, Product: 0x8241).
0:16:14 Removing Bulk-In: Identifies that the device only needs bulk-out transfers (writing LED data) and removes the skeleton's bulk-in (reading) functionality, including related URBs (USB Request Brokers) and file operations.
0:23:00 Understanding URBs & Callbacks: Explains that USB transfers use URBs and rely on asynchronous callbacks to notify when a transfer is complete, managing resources like buffers and reference counts.
0:31:00 Initial Build & Test Setup: Compiles the modified driver, boots the target Apple Xserve, copies the kernel module (.ko file), and loads it using insmod.
0:37:28 User-Space Write Failure: The driver successfully creates a device node (/dev/frontpanel0), but attempts to write data using echo fail with "Permission denied," even as root.
0:46:27 Shifting to In-Kernel Control: Abandons the user-space file interface approach for now. Modifies the code to send data directly from within the kernel, starting with a static test pattern called from the probe function.
1:01:15 Basic Write Success: After recompiling and reloading, the static test pattern is successfully sent, and the front panel LEDs light up for the first time under Linux control.
1:03:07 Code Cleanup: Removes now-unused code related to file operations, minor device numbers, and the USB class driver registration.
1:14:11 Integrating CPU Load Logic: Plans to incorporate CPU load monitoring by adapting code from Benjamin Herrenschmidt's G5 Xserve driver, the only known similar kernel implementation. Discusses the lack of a generic kernel CPU load trigger.
1:26:16 Handling Multi-Core/Threading: Realizes the Intel Xserve has many more CPU threads (16) than the G5 driver handled (max 2). Decides to map each of the 16 threads directly to one of the 16 LEDs, modulating brightness based on individual thread load.
1:31:58 Implementing Per-Thread Monitoring: Adapts the G5 driver's logic using get_cpu_idle_time, kernel jiffies/vclocks, and schedule_delayed_work, creating a separate workqueue/timer for each CPU thread to calculate its load.
1:50:45 Sending Load Data via USB: Modifies the timer callback to store the calculated load for each CPU into a shared buffer and triggers the skeleton_write function (now sending the dynamic load buffer instead of a static pattern) from CPU 0's callback.
2:03:09 CPU Load Display Success: After fixing compilation errors and reloading the module, the front panel LEDs successfully display the load of each individual CPU thread.
2:12:30 Refactoring Attempt (Single Workqueue): Tries to optimize the code by using only one central timer/workqueue that iterates through all CPUs, aiming to reduce overhead.
2:36:21 Kernel Crash & Debugging: The refactored code causes kernel panics (page faults) upon loading. Debugging points towards issues in the workqueue scheduling or accessing CPU state incorrectly. Requires rebooting the machine.
2:47:47 Reverting Refactoring: After failed fixes, the developer reverts the code back to the previous working state with one workqueue per CPU thread, concluding this complexity is likely necessary due to kernel internals.
2:57:32 Final Working State: The version with per-thread workqueues is confirmed to work reasonably well, displaying individual thread load on the LEDs.
3:00:38 Further Cleanup & Conclusion: Identifies remaining cleanup tasks (renaming variables from "skeleton" to "frontpanel," disabling debug logging). Concludes the 3-hour live development session, showcasing a functional, albeit complex, solution.