Okay, here is the abstract and summary for the provided transcript.

Abstract:

This article details an investigation into an eBPF mystery where IPv4 traffic from the dotnet CLI was unexpectedly being handled as IPv6 by eBPF programs. The author initially set up eBPF hooks (BPF_CGROUP_INET4_CONNECT and BPF_PROG_TYPE_CGROUP_SKB) to redirect DNS requests and block non-IPv4 traffic within a cgroup, assuming no IPv6 was in use. However, dotnet commands caused the egress hook to block perceived IPv6 traffic, while the IPv4 connect hook remained untriggered. Through debugging with bpf_ringbuf_output and adding an IPv6 connect hook (cgroup/connect6), the author discovered that dotnet (since .NET 5) utilizes "DualMode sockets." This feature allows IPv4 communication over an IPv6 socket by employing "IPv4-mapped IPv6 addresses" (e.g., ::ffff:x.x.x.x). The kernel internally processes this as IPv6 initially, triggering IPv6-specific eBPF hooks, before translating it to actual IPv4 network traffic. The solution involved updating the connect6 eBPF program to parse these mapped addresses and modifying the egress filter to differentiate true IPv6 from IPv4-mapped IPv6 using the skb->protocol field.

eBPF Mystery: Unraveling How IPv4 Can Masquerade as IPv6

• Initial Setup: DNS Redirection with eBPF

  • Goal: Transparently redirect DNS requests (port 53) for specific programs/containers using eBPF.
  • Method: Used BPF_CGROUP_INET4_CONNECT to inspect and redirect syscall.connect for IPv4 within a cgroup.
  • Assumption: Machines lacked IPv6 support, so only IPv4 handling was considered necessary.
  • Egress Filter: A BPF_PROG_TYPE_CGROUP_SKB eBPF program was added to block direct IP calls, specifically denying traffic if skb->family == AF_INET6.
  • Debugging Tool: bpf_ringbuf_output was used to log events from eBPF programs, proving crucial for troubleshooting.

• The Problem: dotnet CLI Hangs

  • Symptom: Running dotnet add package x would hang indefinitely.
  • Observation: The egress eBPF program logged many PACKET_IPV6_TYPE blocked messages.
  • Initial Hypothesis: dotnet is somehow making IPv6 requests.

• Deep Dive and Contradictions

  • connect4 hook: The BPF_CGROUP_INET4_CONNECT eBPF program was not being triggered by dotnet.
  • Wireshark: Network sniffing confirmed dotnet was not making IPv6 calls off the machine; only IPv4 traffic was observed.
  • The Bafflement: Network traffic showed IPv4, the egress eBPF program saw IPv6, and the connect4 eBPF program (for IPv4) didn't fire.

• Further Investigation: Adding connect6

  • New Hook: A cgroup/connect6 eBPF program was added to log syscall.connect attempts on IPv6.
  • Key Finding: The connect6 hook was immediately hit when running dotnet add package x, even though Wireshark still showed only IPv4 traffic exiting the VM.
  • Conclusion: The kernel must be interpreting the traffic as IPv6 internally, even if it's ultimately IPv4.

• The Discovery: DualMode Sockets and IPv4-Mapped IPv6 Addresses

  • dotnet Source: Investigation into dotnet repositories revealed that since .NET 5, "DualMode sockets" are used in SocketsHttpHandler.
  • RFC 1933: This practice allows handling IPv4 traffic from an IPv6 socket.
  • Kill Switch: Disabling DualMode sockets in dotnet made everything work as expected (connect4 hit, no IPv6 seen by egress).
  • Mechanism: dotnet uses "IPv4-mapped IPv6 addresses." These encode an IPv4 address within an IPv6 address (e.g., ::ffff:1.1.1.1).
  • eBPF Observation: The connect6 eBPF program, when modified to output the IPv6 address, confirmed it was an IPv4-mapped address.
  • Kernel Behavior: When using DualMode sockets, dotnet requests an IPv6 socket and populates user_ip6 with an IPv4-mapped address. The Linux kernel supports this and routes the request as IPv4, making the IPv6 nature only visible internally to eBPF/kernel.

• The Fix: Adapting eBPF Programs

  • Updating connect6:
    • The cgroup/connect6 eBPF program was modified to check if ctx->user_ip6 represents an IPv4-mapped address (first 10 bytes zero, next 2 bytes 0xffff).
    • If mapped, it extracts the IPv4 address from ctx->user_ip6[3].
    • The existing handle_connect_redirect logic is then called with this extracted IPv4 address.
    • If redirected, ctx->user_ip6[3] is updated with the new IP.
  • Updating Egress Filter:
    • The egress eBPF program's check skb->family == AF_INET6 was insufficient as it flagged IPv4-mapped addresses as IPv6.
    • Solution: Changed the check to skb->protocol == bpf_htons(ETH_P_IPV6). This allows distinguishing true IPv6 from IPv4 mapped onto IPv6, as IPv4-mapped traffic does not hit this condition.

• Key Takeaway:

  • IPv4 traffic can appear as IPv6 to eBPF programs when applications like dotnet (using DualMode sockets) employ IPv4-mapped IPv6 addresses. This is an internal kernel representation before the traffic becomes standard IPv4 on the wire.
  • eBPF programs interacting with network connections need to be aware of and potentially handle IPv4-mapped IPv6 addresses to function correctly with such applications.
  • Checking skb->protocol can be more specific than skb->family for differentiating true IPv6 from IPv4-mapped IPv6 in egress scenarios.