import site.body

Tidbits: TimerFD as an interval timer

While i was working on butter to add TimerFD support i noticed something odd. The TimerFD struct allowed specifying intervals down to one nano second. Not being one to pass up an opportunity to break my own machine by doing something stupid i decided to try this out and see if it would hard lock my machine in a flurry of IRQ attempts. What happened next turned out to be quite interesting.


Below is a example of the code i tried and some brief instructions in case you wish to replicate this.

from butter.timerfd import Timer
timer = Timer().every(nano_seconds=1)
for i in range(5):

To set up a virtual throw-away Python environment:

pyvenv-3.4 venv
. venv/bin/activate
pip install butter

This will set you up in a virtual Python environment where invoking 'python' will use python-3.4 with all the libraries installed. You may need to install some extra packages such as libffi and its headers as well as a c compiler and kernel headers, Refer to your operating systems documentation for how to install these.

To delete the environment use the following commands:

rm -rf venv

So what happened?

Nothing, Absolutely nothing. The CPU meter i have on the CLI/screen did not move an inch. However reading/waiting on the FD did give me the number of elapsed events since the last read. After dividing this number by 1,000,000 i found most of the numbers to be in the millisecond range with each subsequent read giving me the amount of nano seconds since the last read.

Having worked with embedded hardware i know that this type of timer is great for measuring input pulse or periods of time and hence its name 'Interval Timer'. This mode of TimerFD is handy for me as i measure a lot of timer intervals for a variety of things (eg benchmarking) and currently 'snapshot' the start and end times and then subtract them. While this is fine this new TimerFD based model is a tad bit nicer.

As TimerFD is FD based a number of odd possibilities pop up including sending the FD over unix socket, Inheriting across fork and other things i am sure i have not thought about. As i am about to start work on multi-process cooperative daemons these possibilities are all on the table (more details below).

So why does it work?

Honestly i don't know. The current theory i am working on is that a wakeup is not scheduled until there is a processes waiting on the timer and that the hardware supports reporting the amount of elapsed events (I intend to dig into the kernel source when i get a minute). Digging into dmesg i came across this interesting snippet:

[    0.000000] sched_clock: 32 bits at 24MHz, resolution 41ns, wraps every 178956ms

Before 2.6.21 it appears that 'jiffies' were the scheduling mechanism for time based events (reference), However more recent kernels have lifted this restriction and are instead backed by 'high resolution timers' that have platform specific granularity.

To see the timer granularity the file /proc/timer_list is provided which contains information about all the timers in the system and the processes waiting on them, A cut down version with systemd using TimerFD is shown below.

cpu: 0
 clock 0:
  .base:       816d7970
  .index:      0
  .resolution: 1 nsecs
  .get_time:   ktime_get
  .offset:     0 nsecs
active timers:
 #9: <ed528b40>, timerfd_tmrproc, S:01, hrtimer_start, systemd-logind/931
 # expires at 787263558964000-787263558964000 nsecs [in 19603039954 to 19603039954 nsecs]

As you can see from the above, The timer advertises itself with a nano second granularity. After checking several machines (x86_64 and arm) i found all timers had identical granularity. On older platforms the resolution may be less and the above results (when specifying an interval smaller than the granularity).

The time man page also mentions the Timer Slack feature which can be used to dictate how much 'slippage' in kernel scheduling the process can tolerate and is helpful in a real time context where you need to tightly control the time period at which your code runs (Not too early, Not too late)

I am currently looking into real time features for building a bike computer/motor controller for an electric bike and interval timers will be a heavily used feature for things such as process supervision and watchdogs as well as interval timing and pulse measuring and as such i am glad spending 5 minutes to try and crash my machine turned up an interesting feature of TimerFDs.