hdiethelm · Oct 31, 2026 · Oct 31, 2026 · Nov 5, 2026 · Nov 9, 2026 · Nov 9, 2026
diff --git a/debian/linuxcnc.install.in b/debian/linuxcnc.install.in
@@ -40,6 +40,7 @@ usr/bin/kinslimits
 usr/bin/latency-histogram
 usr/bin/latency-plot
 usr/bin/latency-test
+usr/bin/latency-ping
 usr/bin/lineardelta
 usr/bin/linuxcnc
 usr/bin/linuxcnc_check_ini
@@ -84,6 +85,8 @@ usr/bin/qtvcp
 usr/bin/rotarydelta
 usr/bin/rs274
 usr/bin/rtapi_app
+usr/bin/realtime-config-kernel
+usr/sbin/realtime-config-ethernet
 usr/bin/scaragui
 usr/bin/schedrmt
 usr/bin/scorbot-er-3

diff --git a/docs/po4a.cfg b/docs/po4a.cfg
@@ -137,6 +137,7 @@
 [type: AsciiDoc_def] src/hal/tutorial.adoc $lang:build/adoc/$lang/hal/tutorial.adoc
 [type: AsciiDoc_def] src/hal/twopass.adoc $lang:build/adoc/$lang/hal/twopass.adoc
 [type: AsciiDoc_def] src/install/latency-test.adoc $lang:build/adoc/$lang/install/latency-test.adoc
+[type: AsciiDoc_def] src/install/latency-tuning.adoc $lang:build/adoc/$lang/install/latency-tuning.adoc
 [type: AsciiDoc_def] src/integrator/steppers.adoc $lang:build/adoc/$lang/integrator/steppers.adoc
 [type: AsciiDoc_def] src/integrator/stepper-timing.adoc $lang:build/adoc/$lang/integrator/stepper-timing.adoc
 [type: AsciiDoc_def] src/integrator/wiring.adoc $lang:build/adoc/$lang/integrator/wiring.adoc

diff --git a/docs/src/Master_Documentation.adoc b/docs/src/Master_Documentation.adoc
@@ -64,6 +64,8 @@ include::config/integrator-concepts.adoc[]
 
 include::install/latency-test.adoc[]
 
+include::install/latency-tuning.adoc[]
+
 include::motion/tweaking-steppers.adoc[]
 
 include::config/ini-config.adoc[]

diff --git a/docs/src/Submakefile b/docs/src/Submakefile
@@ -238,6 +238,7 @@ DOC_SRCS_EN := \
 	hal/tutorial.adoc \
 	hal/twopass.adoc \
 	install/latency-test.adoc \
+	install/latency-tuning.adoc \
 	integrator/steppers.adoc \
 	integrator/stepper-timing.adoc \
 	integrator/wiring.adoc \

diff --git a/docs/src/config/images/latency-ping.png b/docs/src/config/images/latency-ping.png
diff --git a/docs/src/getting-started/system-requirements.adoc b/docs/src/getting-started/system-requirements.adoc
@@ -54,7 +54,11 @@ such as checking G-code, testing config files and learning the system.
 To work with these kernel versions there are two versions of LinuxCNC distributed.
 The package names are "linuxcnc" and "linuxcnc-uspace".
 
-The realtime kernel options are preempt-rt, RTAI and Xenomai.
+The realtime kernel options are PREEMPT_RT, RTAI, Xenomai3 and Xenomai4 EVL.
+
+For most cases, PREEMPT_RT is the simplest option. For better latency, Xenomai4 EVL can help.
+
+RTAI is not maintained any more but still works on some systems.
 
 You can discover the kernel version of your system with the command:
 
@@ -63,41 +67,52 @@ uname -a
 ----
 
 If you see (as above) `-rt-` in the kernel name then you are running the
-preempt-rt kernel and should install the "uspace" version of LinuxCNC.
+PREEMPT_RT kernel and should install the "uspace" version of LinuxCNC.
 You should also install uspace for "sim" configs on non-realtime kernels.
 
 If you see `-rtai-` in the kernel name then you are running RTAI
 realtime. See below for the LinuxCNC version to install.
 
 === Preempt-RT with 'linuxcnc-uspace' package
 
-Preempt-RT is the newest of the realtime systems, and is also the version
+Preempt-RT is the version
 that is closest to a mainline kernel. Preempt-RT kernels are available
 as precompiled packages from the main repositories. The search term
 "PREEMPT_RT" will find them, and one can be downloaded and installed
 just like any other package.
-Preempt-RT will generally have the best driver support and is the only
-option for systems using the Mesa ethernet-connected hardware driver
-cards. In general preempt-rt has the worst latency of the available
+Preempt-RT will generally have the best support.
+ In general Preempt-RT has the worst latency of the available
 systems, but there are exceptions.
 
+=== Xenomai3 with 'linuxcnc-uspace' package
+
+Xenomai3 is supported, but you will have to find or build the kernel
+and compile LinuxCNC from source to utilise it.
+
+Prebuilt experimental kernels can be found here: https://github.com/hdiethelm/xenomai3-linuxcnc
+
+=== Xenomai4 EVL with 'linuxcnc-uspace' package
+
+Xenomai4 is supported, but you will have to find or build the kernel
+and compile LinuxCNC from source to utilise it. It has support for
+Mesa ethernet-connected hardware driver with out of band realtime ethernet.
+
+Prebuilt experimental kernels can be found here: https://github.com/hdiethelm/xenomai4-linuxcnc
+
 === RTAI with 'linuxcnc' package
 
 RTAI has been the mainstay of LinuxCNC distributions for many years. It
 will generally give the best realtime performance in terms of low
 latency, but might have poorer peripheral support and not as many screen
 resolutions. An RTAI kernel is available from the LinuxCNC package
 repository. If you installed from the Live/Install image then switching
-kernel and LinuxCNC flavour is described in [Installing-RTAI].
+kernel and LinuxCNC flavour is described in <<cha:Installing-RTAI>>.
 
-=== Xenomai with 'linuxcnc-uspace' package
-
-Xenomai is also supported, but you will have to find or build the kernel
-and compile LinuxCNC from source to utilise it.
+Prebuilt experimental kernels can be found here: https://github.com/NTULINUX/RTAI
 
 === RTAI with 'linuxcnc-uspace' package
 
-It is also possible to run LinuxCNC with RTAI in user-space mode.
+It is possible to run LinuxCNC with RTAI in user-space mode.
 As with Xenomai you will need to compile from source to do this.
 
 == Problematic Hardware

diff --git a/docs/src/index.tmpl b/docs/src/index.tmpl
@@ -67,6 +67,7 @@
 	<ul>
 		<li><a href="config/integrator-concepts.html">Integrator Concepts</a></li>
 		<li><a href="install/latency-test.html">Latency Test</a></li>
+		<li><a href="install/latency-tuning.html">Latency Tuning</a></li>
 		<li><a href="motion/tweaking-steppers.html">Stepper Tuning</a></li>
 		<li><a href="config/ini-config.html">INI Configuration</a></li>
 		<li><a href="config/ini-homing.html">Homing Configuration</a></li>

diff --git a/docs/src/install/latency-test.adoc b/docs/src/install/latency-test.adoc
@@ -33,6 +33,48 @@ However, software step pulses also have some disadvantages:
 - jitter in the generated pulses
 - loads the CPU
 
+[[sec:latency-to-achive]]
+== What to Achieve?
+This depends on your application. If step pulses are directly generated by the PC, minimal
+latency is key to achieve a hight step rate. See <<cha:stepper-config>>
+for requirements.
+
+As soon as the step pulses are generated using an external hardware like Mesa cards, the latency
+constraints are looser.
+
+If the servo-thread has period of 1 ms, latency of 100-200 us can still be used.
+
+However, in this case, the total thread runtime is more important. The time from where
+the thread should have started to the next cycle should never exceed the thread's period.
+If this happens, the following warning is shown:
+----
+Unexpected realtime delay on task 0 with period 1000000
+This Message will only display once per session.
+Run the Latency Test and resolve before continuing.
+----
+
+Especially for ethernet connected cards, the ethernet latency is often the limiting factor,
+not the pure thread start latency. When using ethernet, LinuxCNC sends a request to the card
+and waits for the response. This has to happen in one thread period.
+
+Often, bad thread latency correlates with bad ethernet latency, due to the LinuxCNC and ethernet
+kernel latency adds up.
+
+[[sec:latency-load]]
+== Perform Latency Tests
+Having a system that has low latency when mostly idle but behaves badly as soon
+as some load is added will result in an unreliable system.
+
+While the test is running, you should 'abuse' the computer.
+Move windows around on the screen. Surf the web. Copy some large files
+around on the disk. Play some music.
+Run an OpenGL program such as glxgears.
+Run a load and stress tool such as stress-ng.
+The idea is to put the PC through its paces while
+the latency test checks to see what the worst case numbers are.
+
+For quick comparisons between different settings, a test of a few minutes are fine.
+However, for final testing, a several hours long test should be performed.
 
 [[sec:latency-tests]]
 == Latency Tests
@@ -87,13 +129,6 @@ After starting a latency test you should see something like this:
 .HAL Latency Test
 image::../config/images/latency-test_en.png["HAL Latency Test",align="center"]
 
-While the test is running, you should 'abuse' the computer.
-Move windows around on the screen. Surf the web. Copy some large files
-around on the disk. Play some music.
-Run an OpenGL program such as glxgears.
-The idea is to put the PC through its paces while
-the latency test checks to see what the worst case numbers are.
-
 The important number for software stepping is the 'max jitter' of the base thread.
 In the example above, that is 6693 nanoseconds (ns), or 6.693 microseconds (µs).
 Record this number, and enter it in StepConf when it is requested.
@@ -103,7 +138,7 @@ You should run the test for at least several minutes; sometimes
 the worst case latency doesn't happen very often, or only happens
 when you do some particular action. For instance, one Intel
 motherboard worked pretty well most of the time, but every 64
-seconds it had a very bad 300 µs latency. Fortunately that was fixable, see <<_latency_tuning>>.
+seconds it had a very bad 300 µs latency. Fortunately that was fixable, see <<cha:latency_tuning>>.
 
 So, what do the results mean?
 
@@ -156,7 +191,7 @@ Options:
 .`latency-plot` Window
 image::../config/images/latency-plot.png["latency-plot Window"]
 
-
+[[sec:latency-histogram]]
 === Latency Histogram
 
 The application latency-histogram displays a histogram of latency (jitter) for
@@ -192,6 +227,36 @@ Use `--show` to show count for the off-chart [pos|neg] bin.
 .`latency-histogram` Window
 image::../config/images/latency-histogram.png["latency-histogram Window"]
 
+[[sec:latency-ping]]
+=== Latency Ping
+
+The application latency-ping displays a histogram of the round trip time for
+a realtime thread pinging an ip address. This useful for network latency tuning where
+ip_address should be a Mesa card, an EtherCAT device or any other device which
+has a realtime capable ethernet interface to be used together with LinuxCNC.
+
+Due to ping is used, it should work with virtually any device without any setup.
+
+It is recommended to use the `if_name` argument to allow for static arp, so arp
+renewal is disabled and does not influence the response time.
+
+----
+Usage:
+       latency-ping [-t timeout] [-p period] [-n nbins ] [-b binsize ] ip_address [if_name]
+
+Example:
+       latency-ping 192.168.1.121 enp1s0
+
+Defaults:     timeout=800000nS period=1000000nS nbins=200 binsize=period/2/nbins
+Equivalently: timeout=800µs period=1ms nbins=200 binsize=2500
+
+Times may be specified with suffix "s", "ms", "us" "µs", or "ns"
+Times without a suffix and less than 1000 are taken to be in us;
+other times without a suffix are taken to be in ns
+----
+
+.`latency-ping` Window
+image::../config/images/latency-ping.png["latency-ping Window"]
 
 [[sec:latency-status-bar]]
 === Realtime status bar
@@ -227,95 +292,4 @@ isolcpus::
     for latency>>.
 
 
-[[_latency_tuning]]
-== Latency tuning
-
-LinuxCNC can run on many different hardware platforms and with many
-different realtime kernels, and they all may benefit from tuning for
-optimal latency.
-
-A primary goal in tuning the system for LinuxCNC is to reserve a CPU for the exclusive use of LinuxCNC's realtime tasks, so that other tasks (both user programs and kernel threads) do not interfere with LinuxCNC's access to that CPU.
-
-When specific tuning options are believed to be universally helpful
-LinuxCNC does this tuning automatically at startup, but many tuning
-options are machine-specific and cannot be done automatically.  The person
-installing LinuxCNC will need to experimentally determine the optimal
-tuning for their system.
-
-
-=== Tuning the BIOS for latency
-
-PC BIOSes vary wildly in their latency behavior.
-
-Tuning the BIOS is tedious because you have to reboot the computer,
-make one small tweak in the BIOS, boot Linux, and run the latency test
-(potentially for a long time) to see what effects your BIOS change had.
-Then repeat for all the other BIOS settings you want to try.
-
-Because BIOSes are all different and non-standard, providing a detailed
-BIOS tuning guide is not practical.  In general, some things to try
-tuning in the BIOS are:
-
-* Disable ACPI, APM, and any other power-saving features.  This includes
-  anything related to power saving, suspending, CPU sleep states, CPU
-  frequency scaling, etc.
-
-* Disable CPU "turbo" mode.
-
-* Disable CPU hyperthreading.
-
-* Disable (or otherwise control) System Management Interrupt (SMI).
-
-* Disable any hardware you do not intend to use.
-
-
-[[sec:latency-governor]]
-=== CPU frequency governor
-
-Frequency scaling lets the CPU lower its clock speed to save power, which adds
-latency while the CPU ramps back up to service a realtime thread. For the best
-latency, set the governor to `performance` on every CPU:
-
-----
-echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
-----
-
-This setting does not survive a reboot. To make it permanent, apply it at boot
-with a small systemd service, or with a CPU frequency management tool such as
-`cpupower` (Debian/Ubuntu package `linux-cpupower`) or `tuned`.
-
-
-[[sec:tuning-preempt-rt]]
-=== Tuning Preempt-RT for latency
-
-The Preempt-RT kernel may benefit from tuning in order to provide the
-best latency for LinuxCNC.  Tuning may be done via the kernel command
-line, sysctl, and via files in `/proc` and `/sys`.
-
-Some tuning parameters to look into:
-
-Kernel command line::
-
-    Details here: <https://www.kernel.org/doc/Documentation/admin-guide/kernel-parameters.txt>
-
-    * `isolcpus`: Prevent most non-LinuxCNC processes from using these
-      CPUs, leaving more CPU time available for LinuxCNC.
-
-    * `irqaffinity`: Select which CPUs service interrupts, so that the
-      CPUs reserved for LinuxCNC realtime don't have to perform this task.
-
-    * `rcu_nocbs`: Prevent RCU callbacks from running on these CPUs.
-
-    * `rcu_nocb_poll`: Poll for RCU callbacks instead of using sleep/wake.
-
-    * `nohz_full`: Disable clock tick on these CPUs.
-
-Sysctl::
-
-    Details here: <https://www.kernel.org/doc/html/latest/scheduler/sched-rt-group.html>
-
-    * `sysctl.kernel.sched_rt_runtime_us`: Set to -1 to remove the limit
-      on how much time realtime tasks may use.
-
-
 // vim: set syntax=asciidoc: