Re: Compiler warnings with --enable-dtrace

On Wed, May 23, 2018 at 5:17 AM, Christoph Berg <myon(at)debian(dot)org> wrote:
> I'm not aware of any guidelines - I guess this stuff is still too new
> to make general statements about it. Which other software offers
> custom probes?

As the LWN article mentions, user space probes are available for a
number of other widely used open source software projects, including
MySQL and even glibc. I'm pretty sure that these are not new; they
were added for DTrace quite a few years back, and then perhaps
forgotten about. I'm pretty sure that these are generally not enabled
on standard Linux packages. I bet they're enabled on packages for
Solaris/illumos, though, since DTrace is considered an important part
of the Solaris ecosystem.

In recent years, the Linux kernel gained the ability to expose probes
that can be used in much the same way as the --enable-dtrace
user-space probes, as well as the ability to run a custom eBPF
JIT-compiled program in kernel space, to do the actual
instrumentation. BCC is an externally maintained toolkit that is used
to write most eBPF-based instrumentation in practice, since it makes
the process far easier. BCC isn't a separate technology to eBPF,
though.

BCC/eBPF make the old DTrace/SystemTap probes much more useful on
Linux for a number of reasons. SystemTap had problems with reliability
in the past, owing to the fact that it builds kernel modules rather
than building eBPF code that runs in the eBPF in-kernel virtual
machine (eBPF code is generally considered "safe", which is a huge
advantage). Also, eBPF has less overhead for certain types of
aggregation which are apparently quite important, and benefits from
being JIT compiled (there is an LLVM backend, which is what BCC uses).
Finally, eBPF looks like it might emerge as a kind of defacto standard
on Linux, which SystemTap clearly never managed to do.

The ecosystem questions clearly aren't worked out yet, but there is a
lot of investment in and momentum behind eBPF. There is good reason to
expect that these teething issues be worked out. Linux should be able
to do this stuff without the user having to read a tedious explanation
of how the different parts fit together.

--
Peter Geoghegan