Notes about time in UNIX and Linux systems (II): NTP

The second part of this post about time management I will write about the NTP and its daemon configuration. As I mentioned in the previous post, if you need a very accurate time the best option is using the ntp.org implementation of the protocol. If you need security over accuracy, then you can use OpenBSD project implementation. OpenNTPd is not a complete implementation of the protocol, but as usual in the OpenBSD software, it’s a good, well-documented, audited code.

NTP configuration

Tip: If you run GNU/Linux on virtual infraestructure, review the kernel boot parameters

Some years ago I had a problem with virtual machines that they weren’t able to syncrhonize with the NTP servers. The problem was solved reviewing this matrix at VMware.

Tip: Don’t forget opening the 123 port towards the NTP servers in your firewall.

There is a very simple /etc/ntp.conf file:

driftfile /var/lib/ntp/drift/ntp.drift # path for drift file
logfile   /var/log/ntp          # alternate log file
server server1
server server2

After “serverX” you can add some options on boot like iburst (RHEL6/7,SLES12) or dynamic (SLES11). These options help you to improve synchronization when the network is temporalily down and/or there is not name resolution.

Another interesting command is the driftfile, it helps to adjust the clock frequency on ntpd boot. Remember this file must be writtable by ntp user.

If you are configuring a SLES node, it’s easy to run yast. But maybe you are interested in doing a simple automated configuration, so you only want to touch the /etc/ntp.conf. You must disable NTP configuration at /etc/sysconfig/network/config, setting the policy parameter empty:

[...]
## Type:        string
## Default:     "auto"
#
# Defines the NTP merge policy as documented in netconfig(8) manual page.
# Set to "" to disable NTP configuration.
#
NETCONFIG_NTP_POLICY="auto"

## Type:        string
## Default:     ""
#
# List of NTP servers.
#
NETCONFIG_NTP_STATIC_SERVERS=""
[...]

As I said about configuring timezone in Exadata (RHEL5, 6?), the standard procedure is running /opt/oracle.cellos/ipconf tool.

But if you are tempted to reconfigure on /etc/ntp.conf and you make changes about ntp servers, you must restart the cellwall service after doing it. This is the firewall daemon enabled by default at the storage cells. When cellwall boots it scans /etc/ntp.conf file looking for the ntp servers in order to open the ports.

How to configure the NTP daemon

Tip: If you are running databases, you must use the slewing option (-x).

The slewing option is for avoiding abrupt time synchronizations. Time changes with great jumps are bad for db consistency, and very dangerous for some related services. As example, if you are running Oracle CRS and you have some seconds of error, you must stop all CRS processes (it’s not enough taking the node off the cluster) before making an on-hand NTP synchronization. If you don’t stop the CRS processes the synchronization can cause an outage.

SLES

The NTP daemon configuration is at /etc/sysconfig/ntp:

## Path:           Network/NTP
## Description:    Network Time Protocol (NTP) server settings
## Type:           string
## Default:        "-g -u ntp:ntp"
#
# Additional arguments when starting ntpd. The most
# important ones would be
# -u user[:group]   to make ntpd run as a user (group) other than root.
#
NTPD_OPTIONS="-g -u ntp:ntp"

## Type:           yesno
## Default:        yes
## ServiceRestart: ntp
#
# Shall the time server ntpd run in the chroot jail /var/lib/ntp?
#
# Each time you start ntpd with the init script, /etc/ntp.conf will be
# copied to /var/lib/ntp/etc/.
#
# The pid file will be in /var/lib/ntp/var/run/ntpd.pid.
#
NTPD_RUN_CHROOTED="yes"

## Type:           string
## Default:        ""
## ServiceRestart: ntp
#
# If the time server ntpd runs in the chroot jail these files will be
# copied to /var/lib/ntp/ besides the default of /etc/{localtime,ntp.conf}
#
NTPD_CHROOT_FILES=""

[...]

## Type:           boolean
## Default:        "no"
#
# Force time synchronization befor start ntpd
#
NTPD_FORCE_SYNC_ON_STARTUP="yes"

[...]

There are more options, but I think these are the most interesting: the ntpd options (there you can include the -x slewing option), chrooting (it improves the security of the daemon), and hard synchronization before booting the daemon.

If there is a difference between the current time in the machine and ntp servers larger than the tinker panic parameter sets (1000 secs by default), ntpd exits with error. But if you add the -g option means the daemon will synchronize on boot regardless the jump (only once at boot).

Be careful with NTPD_FORCE_SYNC_ON_STARTUP, your sensitive applications must boot after ntp to avoid time jumps.

It can be interesting too to enable the option NTPD_FORCE_SYNC_HWCLOCK_ON_STARTUP (if you enabled the last one), in order to have an accurate time at the hardware clock. Remember that’s the time the operating system takes on boot before starting the NTP daemon.

As you can see, in SLES chrooting is active by default. Remember this option needs some copied files in /var/lib/ntp and /proc bind mounted in the jail. Sometimes I use mondorescue for bare metal recovery, and I experienced some issues when I didn’t avoid the ntp jail in the backup.

After the daemon configuration, you have some options to run the daemon:

root@SLES10_or_11:~ # rcntp start
root@SLES12:~ # systemctl start ntpd
root@SLES10_11_12:~ # service ntp start

Don’t forget to enable the daemon by default on OS boot:

root@SLES10_or_11:~ # chkconfig ntp 35 
root@SLES12:~ # systemctl enable ntpd

RHEL

The RHEL config file /etc/sysconfig/ntpd is less documented by default than SLES one. This is the RHEL6 file:

# Drop root to id 'ntp:ntp' by default.
OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid -g"

With the -x option (or if you added servers in /etc/ntp/step-tickers) the daemon won’t try to synchronize before booting the daemon. So, in RHEL6 if you want to do a hard sync before booting the ntpd, you must enable the ntpdate daemon too.

It’s a good idea to add the SYNC_HWCLOCK=yes to /etc/sysconfig/ntpd (or /etc/sysconfig/ntpdate if you enable ntpdate daemon) as we did with NTPD_FORCE_SYNC_HWCLOCK_ON_STARTUP option in SLES.

In RHEL7 the use of ntpdate is deprecated in this way, and it is used as time-sync.target provider like sntp. In the documentation, Red Hat advises to add After=time-sync.target in your sensitive services in order to avoid important jumps with the inital synchronization with these tools.

ntpd chrooting is disabled by default in RHEL. I found a procedure for RHEL6, it’s not automagic than SLES. You must:

create the jail directory tree structure, files and devices
add -i /chroot/ntp to the ntpd options
copy (and adapt for new routes) the /etc/init.d/ntpd script

And… after the configuration, you can enable and start the daemon:

root@RHEL5_or_6:~ # chkconfig ntpd on 
root@RHEL7:~ # systemctl enable ntpd 

root@RHEL5_or_6:~ # service ntpd start 
root@RHEL7:~ # systemctl start ntpd

HP-UX

In HP-UX 11.31 coexists xntpd (by HP) and ntpd (free software) implementations. xntpd is not supported after April 1, 2014.

There is a configuration called /etc/rc.config.d/netdaemons. As you guess, you will find (x)ntpd daemon configuration there:

[...]
XNTPD_NAME=ntpd
export NTPDATE_SERVER=
export XNTPD=1
export XNTPD_ARGS="-x"
[...]

In order to enable the service, you can activate editing the file and setting XNTPD=1. The other way is running

root@myHPUX:/# ch_rc -a -p XNTP=1 
root@myHPUX:/# ch_rc -l -p XNTP   # show the status of xntp service on boot

And you start/stop the daemon in the classic way:

root@myHPUX:/# /sbin/init.d/xntpd start

AIX

In AIX the NTP daemon is enabled at the /etc/rc.tcpip with the main OS network daemons.

[...]
# Start up Network Time Protocol (NTP) daemon
start /usr/sbin/xntpd "$src_running" "-x"
[...]

As you can see, I added the -x option there. I could do it too in this way:

[root@myAIX /]# chssys -s xntpd -a "-x" # add the slewing option

[root@myAIX /]# chrctcp -S -a xntpd # -S start and -a enable the service

Start and check the xntpd status:

[root@myAIX /]# startsrc -s xntpd
[root@myAIX /]# lssrc -ls xntpd # check the service

Updated November 5th, 2015: If you upgrade from SLES11SP3 to SLES11SP4 and you have your ntpd chrooted, you will have a problem with the name resolution of the NTP servers. The cause is the update to ntpd > 4.2.7. You can fix it copying the needed files to the jail, but SUSE provided a /etc/ntp.conf default file with the needed options for backward compatibility doing nothing else.