PTPd Source Code Documentation

This is a brief overview of PTPd's source code intended to aid developers looking through PTPd.

This documentation is incomplete, and it will be worked on as time permits. More information, particularly on PTPd's clock servo, is available in this paper written for the 2005 Conference on IEEE 1588.

The IEEE 1588 Precision Time Protocol

The PTP daemon (PTPd) implements the Precision Time protocol (PTP) as defined by the IEEE 1588 standard. PTP was developed to provide very precise time coordination of LAN connected computers. The IEEE 1558 specification dictates most of PTPd's operation, so one should obtain a copy of the spec before going through PTPd's source code. PTPd keeps all of the names used in the spec, so it should be easy to match up PTPd's source code with the spec. Descriptions herein will assume knowledge of the spec.

Playing with PTPd

The best way to become familiar with a program is to use it. One should be able to run PTPd on standard Linux systems without much trouble. To better observe the operation of PTPd, compile it with the PTPD_DBG flag to have the program produce debugging output, and run PTPd with the '-c' argument to prevent it from backgrounding itself.

PTPd coordinates the local clock by adjusting the effective tick rate, or slewing, the clock. This is a slow but precise process. To quickly coordinate a local clock that is off by more than one second, PTPd will reset the clock instead of slewing. This results in a step change in the time base, which can can cause problems for applications that require a smooth and monotonically increasing time base. The '-x' option will prevent PTPd from resetting the clock, while still allowing it to adjust the clock frequency. (This is different from the '-t' option that entirely prevents PTPd from adjusting the clock.)

For PTPd to operate, it must be run on at least two computers connected through an Ethernet hub or switch. It is possible to have PTPd coordinate two (and only two) computers through a routed connection by running PTPd on each computer with the '-u' argument followed by the IP address of the other computer.

PTPd can send management messages to query PTP nodes on a LAN or a PTPd server on the same machine. To send a management message, run PTPd with the '-k NUMBER' option, where the number is the management message key to send, or zero to send a sweep of different management messages. To query only a local PTPd server, bind the program to the loopback device with '-b lo'.

PTPd should be able to coordinate the clocks of your computers within tens of microseconds. The default distribution achieved coordination below ten microseconds running on fairly busy embedded systems with 66MHz m68k processors. For the best performance, you will need to tweak the servo gains with the '-a' command line argument. Also, you must compensate for asymmetric inbound versus outbound message latency with the '-l' argument. This will remove any constant offset in the time coordination. As a rule of thumb, you can eliminate a time offset T by specifying an inbound and outbound latency that have a difference of 2*T.

PTPd's precision is dependent upon the precision of message send and receive time stamps. PTPd previously required a kernel patch to improve its precision, but now it needs only a Berkeley sockets interface that has the SO_TIMESTAMP socket option. PTPd uses a few platform specific APIs to query network interfaces, but otherwise it should be very portable to any Posix platform that implements David Mills' ntp_adjtime/adtimex system call. PTPd runs on Linux, uClinux, NetBSD, and FreeBSD.

While it is fairly easy to get PTPd to provide a precisely coordinated time base, it is much more difficult to use the coordinated time base. This is because platforms like Linux do not provide interfaces to generate precisely timed events. Projects like RTAI are working on 'real-time' turnaround in Linux.

PTPd's Source Code Organization

PTPd's source is grouped into a few components. The component delineations are based on the functionality defined by the spec, but the delineations are not specifically defined by the spec.

The following is a block diagram of PTPd's major components, in which occlusion indicates interfaces between components.

In general, PTPd's source is divided into platform-independent and platform-dependent code. Platform-independent code is located in the top level of the PTPd source tree, and platform-dependent code is located the dep/ sub-directory.

PTPd's major source code components are:

Protocol Engine, protocol.c

The main protocol state machine defined in the spec. The state machine is implemented as a forever loop with cases for each state. It is called with protocol() after start-up, and only returns on an error. Normal execution is expected to be halted externally and asynchronously. The loop sleeps in a select() call on the network sockets. The primary states, handled by doState(), are master, slave, inactive, and faulty. State transitions, handled by toState(), occur primarily due to the results of the BMC algorithm. The primary events are message receives, which are checked for in handle(), and timer expiration. The primary actions are message sends, which are done in issue(), timer resets, regular runs of the BMC algorithm, foreign master data updates, and system clock servo updates after sync message receipts.

BMC, bmc.c

The Best Master Clock algorithm defined by the spec. It is called with bmc(), and it returns the proper state, master or slave, based on the reported clock statistics of other PTP clocks that have sent Sync messages (foreign masters).

Clock Servo, dep/servo.c

The clock servo computes the offset-from-master from the master-to-slave delay and slave-to-master delays. It uses the offset-from-master to compute clock tick rate adjustments to minimize the offset-from-master. The clock servo also performs filtering to remove jitter from its input.

Message Packer, dep/msg.c

Gathers data into and extracts data from PTP messages, the format of which is defined by the spec.

Network Layer, dep/net.c

Initializes connections, sends, and receives data between PTP clocks. The network layer also retrieves time stamps for Event messages from the Time Stamp component.

Time Stamp

Records message send and receive times. The send and receive times are used to compute the master-to-slave and slave-to-master delay. The delays are then used to coordinate slave clocks with master clock time. Time stamps should be recorded as close to the networking hardware as possible. This minimizes jitter in the time stamps, which is the most significant detriment to PTPd's clock coordination.

Timer, dep/timer.c

Low resolution interval timers used in the protocol engine. The timers control periodic Sync message sends by masters, Delay Request sends by slaves, periodic runs of the BMC (state change events), and Sync receive timeouts.

Start-up, dep/startup.c

Sets the program's execution state, and retrieves run-time options from the user.

Other code

The ptpd.c file contains the execution entry point. The ptpd header files contain function prototype declarations. The datatypes header files contain structure and data type declarations. Finally, the constants header files contain constant declarations.

PTPd's Clock Servo

The IEEE 1588 spec does not define how the offset from master produced by a PTP slave is used to bring the slave clock into coordination with master clock time. This procedure, called clock discipline, is not trivial, and there are many possible design approaches and trade-offs. For these reasons, this documentation devotes a section exclusively to the clock servo component.

The following shows the message send and receive paths in a typical system running PTPd, along with the associated time stamps that form the basis of the master-to-slave and slave-to-master delay measurements.

The following is a system diagram of PTPd's clock servo. The FIR filtering of the offset from master input is a simple, two-sample average. The IIR filtering of the input one-way delay is described below. The PI controller that mediates the tick rate adjustment output has the difference equation: y[n] = e[n]/Ap + a[n], a[n] = e[n]/Ai + a[n-1].

The following are plots of the frequency response of the one-way delay filter. It is a variable cutoff/delay low-pass, infinite impulse response (IIR) filter. The one-way delay filter has the difference equation: s*y[n] - (s-1)*y[n-1] = x[n]/2 + x[n-1]/2, where increasing the stiffness (s) lowers the cutoff and increases the delay.