Trace System Calls With strace and BPF
Trace System Calls With strace and BPF
kairenner-gh/slates
Last update 2 w. agoCreated on the 23rd of March 2026

Every Abstraction Eventually Leaks

Your Nextcloud container is a Go binary compiled from source, running on a glibc runtime, making calls to a Linux kernel, on hardware with its own firmware. When something fails with a cryptic error message, the most reliable way to find the actual cause is to go to the boundary between userspace and kernel space — the system call interface — and watch what the process is asking the kernel to do. Every file read, every network connection, every memory allocation goes through a syscall. strace puts you at that boundary in real time.

"

Every abstraction eventually leaks — strace is how you see through it.

"
KaiRenner
KaiRenner
24th of March 2026

Understand What a System Call Is

User programs cannot directly access hardware, write to disk, or open network connections. The Linux kernel provides a stable interface — the system call ABI — through which programs request these privileged operations. On x86-64, a program places a sysca

Attach strace to a Running Process

Install strace: sudo apt install strace. Find the PID of the process you want to trace: ps aux | grep your-process-name. Attach strace to it: sudo strace -p 1234, where 1234 is the PID. strace will print each syscall the process makes as it happens: the s

Read strace Output for a Web Request

When you strace a process making an HTTP request, you see the full sequence: socket(AF_INET, SOCK_STREAM, 0) creates a TCP socket and returns a file descriptor number. connect(fd, {sa_family=AF_INET, sin_addr="93.184.216.34"}, 16) initiates the TCP connec

Understand Why strace Has an Overhead Problem

strace uses the ptrace kernel mechanism, which works by stopping the traced process before and after every syscall so the tracer can inspect it. This stop-and-start adds significant latency to every syscall — traced processes run five to ten times slower

Get Started with eBPF and bpftrace

eBPF is a kernel facility that lets you run small verified programs at specific hook points inside the kernel with near-zero overhead. Unlike ptrace, eBPF programs do not stop the traced process — they run asynchronously in a kernel buffer and collect dat

Debug a Docker Container That Fails to Start

A Docker container that exits immediately after starting is a common frustrating problem. The logs might show nothing useful. Use strace to find the actual cause: run docker run --rm your-image your-binary under strace with strace -f -e trace=file,process

What Comes Next

System calls are the interface between your programs and the kernel — and the kernel itself is a massive piece of software with its own internal architecture. Processes, virtual memory, the scheduler, the VFS layer, the TCP stack — each is a subsystem worth understanding in its own right. The self-hosting rabbit hole goes all the way down.

Understand the Linux Kernel Architecture

A conceptual guide to how the Linux kernel is structured — the VFS, the scheduler, memory management, and how every system call you just traced eventually arrives here.