Skip to content

netutils/ptpd: IEEE 1588 P2P delay mechanism and hardware clock phase-lock - #3782

Open
daniel-p-carvalho wants to merge 10 commits into
apache:masterfrom
daniel-p-carvalho:feat/ptpd-p2p-hw-sync
Open

daniel-p-carvalho wants to merge 10 commits into
apache:masterfrom
daniel-p-carvalho:feat/ptpd-p2p-hw-sync

Conversation

@daniel-p-carvalho

Copy link
Copy Markdown
Contributor

Depends on #3779 (not yet merged). This branch is built on top of
fix/ptpd-e2e-delay-drift, so the diff below includes that PR's 4
commits until it merges into master - only the last 6 commits below
are new to this PR. Companion kernel/driver-side changes (hardware TX
timestamping, SIOCS_PTP_ADJFREQ/SIOCS_PTP_ADJPHASE ioctls) are in
apache/nuttx#20148.

Summary

Why change is necessary (fix, update, new feature)?

apps/netutils/ptpd only implemented the E2E delay mechanism
(Delay_Req/Delay_Resp). The IEC/IEEE 61850-9-3:2016 profile (the
target profile for substation automation) requires the P2P
(peer-delay) mechanism instead, so a real GPS-disciplined Grandmaster
running that profile (Toradex Colibri iMX7 + linuxptp's ptp4l)
could not be synchronized against without first switching it to E2E
for testing purposes (which is how #3779 was validated). This PR adds
P2P support and, together with apache/nuttx#20148, lets the PTP
hardware counter - and therefore the physical PPS output pin - actually
phase-lock onto the master instead of getting stuck at a fixed offset
after the first sync.

What functional part of the code is being changed?

apps/netutils/ptpd/ptpd.c, apps/netutils/ptpd/ptpv2.h,
apps/include/netutils/ptpd.h, apps/system/ptpd/ptpd_main.c - the
PTP client path, delay-mechanism selection, and hardware clock
discipline.

How does the change exactly work (what will change and how)?

  1. Fix Delay_Resp consumption by sendmsg and use canonical path delay. - prep fix in the existing E2E path before adding P2P.
  2. Implement IEEE 1588 peer-to-peer (P2P) delay mechanism. -
    Pdelay_Req/Pdelay_Resp/Pdelay_Resp_Follow_Up message types,
    -P CLI flag (mutually exclusive with -E), a
    PTP_DELAY_NONE/_E2E/_P2P config enum replacing the old
    bool delay_e2e, requester and responder logic, and dispatch in
    the RX loop. Verified against IEEE 1588-2019 clause 11.4.2: the
    two-step responder always operates as the standard's "Option B"
    (actual t2/t3 carried in the timestamp fields, not the
    correctionField), and the requester's two-step path reconstructs
    t2/t3 by adding the received correctionField, so the
    meanPathDelay = ((t4-t1) - (t3-t2)) / 2 formula closes correctly
    against a peer using either Option A or Option B. The one-step
    requester path matches clause 11.4.2(d)(3) directly. delayAsymmetry
    compensation (optional in the standard) is not implemented - not
    applicable to the bench's symmetric electrical link.
  3. Warn when P2P is selected without CONFIG_SCHED_TICKLESS. -
    P2P's independent, physical-link-rate Pdelay_Req cadence needs
    tickless timing precision to be meaningful; warn instead of
    silently degrading.
  4. Query hardware TX timestamp via SIOCG_TX_HW_TIMESTAMP. - consumes
    the ioctl added in arch/arm/stm32, sched, net: hardware PTP timestamping, ADJFREQ/ADJPHASE, and tickless fixes nuttx#20148 so Pdelay_Req's t1 (and the
    E2E path's Delay_Req t1) reflect the actual hardware TX time
    instead of a pre-send software estimate.
  5. Drive PTP hardware frequency trim via SIOCS_PTP_ADJFREQ. -
    consumes the ioctl from arch/arm/stm32, sched, net: hardware PTP timestamping, ADJFREQ/ADJPHASE, and tickless fixes nuttx#20148 so the local clock's
    frequency-drift estimate also disciplines the MAC's own hardware
    counter, not just software CLOCK_REALTIME.
  6. Phase-lock the MAC's PTP hardware counter to the master. - the
    real fix behind the PPS-offset bug found while validating this on
    the HIL bench: the initial clock jump omitted path_delay_ns from
    both the CLOCK_REALTIME step and the hardware phase nudge
    (SIOCS_PTP_ADJPHASE), so the hardware counter (and PPS edge)
    landed at, and then stayed at, an arbitrary permanent offset. The
    jump now targets the master's timestamp directly when hardware
    timestamping is enabled (local_timestamp lives in the hardware
    counter's own free-running epoch, not CLOCK_REALTIME, so
    subtracting it was never meaningful), and SIOCS_PTP_ADJFREQ now
    carries a phase-proportional term (not just frequency drift) so the
    hardware counter keeps converging between jumps instead of only
    matching the master's rate.

Impact

  • New -P CLI flag for ptpd, mutually exclusive with -E; existing
    -E/no-flag behavior unchanged.
  • struct ptpd_config_s.delay_e2e (bool) becomes
    delay_mechanism (enum PTP_DELAY_NONE/_E2E/_P2P) - source
    change for anything constructing this struct directly (in-tree,
    only ptpd_main.c does).
  • Hardware timestamping / SIOCS_PTP_ADJFREQ/SIOCS_PTP_ADJPHASE
    paths are all gated behind state->config->hardware_ts and the
    corresponding #ifdef SIOCG_TX_HW_TIMESTAMP / #ifdef SIOCS_PTP_ADJFREQ / #ifdef SIOCS_PTP_ADJPHASE; no effect on
    targets without arch/arm/stm32, sched, net: hardware PTP timestamping, ADJFREQ/ADJPHASE, and tickless fixes nuttx#20148 or without hardware timestamping
    enabled.

Testing

Same HIL bench as apache/nuttx#20148: STM32F4Discovery
(stm32f4discovery-ext custom board) as PTP slave against a real
GPS-disciplined Grandmaster (Toradex Colibri iMX7 + X-NUCLEO-GNSS1A1,
linuxptp's ptp4l, IEC/IEEE 61850-9-3 P2P profile).

  • ptpd -2 -s -S -r -B -P -i eth0 on the Discovery; ptpd -t <pid>
    shows path_delay_ns populating via the P2P path.
  • pmc -u -b 0 "GET PORT_STATS_NP" on the Grandmaster confirms
    tx_Pdelay_Resp/rx_Pdelay_Req incrementing.
  • Oscilloscope: the Discovery's hardware ETH_PPS_OUT (PB5) phase-
    locks onto the Grandmaster's 1PPS edge after the daemon's initial
    clock jump and stays aligned, instead of freezing at a fixed offset.

Host: Ubuntu 24.04, arm-none-eabi-gcc toolchain,
stm32f4discovery-ext:ethraw custom out-of-tree board config.

- Fix typo state->conifg -> state->config in BMCA announce check.
- Fix typo state->n_identity -> state->own_identity in BMCA announce check.
- Define ETHERTYPE_PTP as 0x88f7 for IEEE 802.3 Layer 2 transport.

Assisted-by: Claude:claude-sonnet-5
Assisted-by: Gemini:gemini-3.8-flash-medium
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
1. Post-jump drift bootstrap: on the first Sync packet following a step
   jump, do not compute frequency drift against a synthesized zero delta,
   which previously caused the entire residual phase offset (~ms) to be
   mistaken for frequency drift (~million ppb) and immediately absorbed.
2. Drift rate formula: normalize the adjustment contribution by the actual
   measurement interval instead of the adjtime slew period, and compute
   natural delta rate as (delta - last_delta + last_adjtime) / interval.
3. Remove broken last_delta > delta comparison that prevented offsets from
   converging and applied inverted corrections on negative overshoots.
4. Correct ptp_adjtime() invocation to always pass adjustment_ns for
   CLOCK_REALTIME slewing rather than dropping drift compensation when
   delta exceeds threshold. Clamp adjustment_ns to the hardware slew limit
   so last_adjtime_ns accurately mirrors the true slew applied.
5. Enable Delay_Req in E2E mode once clock is tracking (not jumping) and
   allow software timestamping latency in ptp_process_delay_resp().
6. Propagate initialization return code from ptpd_start() and avoid
   unconditional failure print in do_ptpd_start().

Assisted-by: Claude:claude-sonnet-5
Assisted-by: Gemini:gemini-3.8-flash-medium
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
1. On AF_PACKET SOCK_RAW sockets in NuttX, msg_name must be NULL because
   the destination MAC address is already contained in the Ethernet header.
   Previously, passing sockaddr_in caused sendmsg() to fail immediately
   with -EAFNOSUPPORT, completely blocking transmission of Delay_Req.
2. Correct PTP primary multicast MAC address to 01:1b:19:00:00:00
   (IEEE 1588 Annex F) and ensure ether_type is in network byte order.
3. Initialize delayreq_interval to 1 second default and guard against 0.
4. Set logmessageinterval to 0x7f (IEEE 1588-2008 Table 23 sentinel for
   Delay_Req) instead of inheriting 0 from the announce header template.
5. Update PTP version to 0x12 (2.1, minorVersionPTP=1) to match the
   value used by mature implementations such as linuxptp.

Assisted-by: Claude:claude-sonnet-5
Assisted-by: Gemini:gemini-3.8-flash-medium
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
Add ptpinfo()/ptpwarn() calls, gated by the existing
CONFIG_DEBUG_PTP_INFO/_WARN symbols (zero cost when disabled), at
points that previously failed silently: an unrecognized L2 protocol,
a domain mismatch, and a Delay_Resp rejected by the source/requester
identity check. These were essential to diagnosing the drift and
Delay_Req bugs fixed in the two preceding commits on real hardware,
and are kept for future maintainers debugging this path.

Assisted-by: Claude:claude-sonnet-5
Assisted-by: Gemini:gemini-3.8-flash-medium
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
…ath delay

ptp_sendmsg() called a blocking recvmsg(state->tx_socket, ...) right
after sending a Delay_Req whenever hardware_ts was set, assuming a
Linux-style MSG_ERRQUEUE/loopback semantics NuttX does not have.
Since tx_socket and event_socket share the same underlying
connection, this call instead blocked on and consumed whatever PTP
packet arrived next on the wire — almost always the Delay_Resp,
which typically arrives within milliseconds of the request. Its
payload was read into a local buffer that went out of scope on
return, so the packet never reached ptp_process_rx_packet() and
path_delay_ns stayed at 0 in -H mode. t3 is now always captured
locally via ptp_gettime(), the same way -S mode already did, until
M3 adds real hardware TX timestamping via SIOCG_TX_HW_TIMESTAMP.

Also replaces the path delay heuristic in ptp_process_delay_resp()
(which derived an approximation of (t2-t1) from path_delay_ns and
last_delta_ns, only valid once the clock had already converged) with
the canonical IEEE 1588-2008 §11.3 formula: store (t2-t1) directly
from Sync/Follow_Up as sync_diff_ns, then average it with (t4-t3)
from the Delay_Req/Delay_Resp exchange. Relaxes the M2 path delay
ceiling to 10ms unconditionally (previously only in -S mode), since
Delay_Req's t3 is software-timestamped in both modes until M3 lands.

Assisted-by: Claude:claude-sonnet-5
Assisted-by: Gemini:gemini-3.8-flash-medium
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
(cherry picked from commit 13193ca)
Implements the Peer-to-Peer (P2P) transparent clock delay measurement
mechanism (IEEE 1588-2008 §11.4 / IEEE 802.1AS / IEC/IEEE 61850-9-3)
in apps/netutils/ptpd:

- Add PTP_MSGTYPE_PDELAY_REQ, PTP_MSGTYPE_PDELAY_RESP, and
  PTP_MSGTYPE_PDELAY_RESP_FOLLOW_UP definitions and structs in ptpv2.h.
- Define IEEE 1588-2008 Annex F peer delay multicast MAC address
  01:80:c2:00:00:0e and Annex D peer delay IP address 224.0.0.107.
- Replace bool delay_e2e with enum ptp_delay_mechanism_e (PTP_DELAY_NONE,
  PTP_DELAY_E2E, PTP_DELAY_P2P) in include/netutils/ptpd.h.
- Add -P CLI option in system/ptpd/ptpd_main.c with mutual exclusion
  check against -E, and display last_transmitted_pdelayreq in status.
- Implement responder logic in ptp_process_pdelay_req() sending
  Pdelay_Resp (t2) and Pdelay_Resp_Follow_Up (t3) regardless of master
  or slave state.
- Implement requester logic in ptp_send_pdelay_req() gated on the
  physical link without requiring prior BMCA master selection.
- Implement ptp_process_pdelay_resp() and
  ptp_process_pdelay_resp_followup() using canonical mean path delay
  formula ((t4 - t1) - (t3 - t2)) / 2.
- Refactor path delay bounds checking and moving average filter into
  ptp_record_path_delay() shared across E2E and P2P mechanisms.

Assisted-by: Claude:claude-sonnet-5
Assisted-by: Gemini:gemini-3.8-flash-medium
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
(cherry picked from commit 049fbba)
Without a tickless (hardware timer-backed) clock, clock_gettime() only
advances once per CONFIG_USEC_PER_TICK scheduler tick, with no
interpolation. The P2P peer delay formula subtracts two local
timestamps (t1, t4) typically captured microseconds apart, which on a
tick-driven clock almost always fall inside the same tick: (t4 - t1)
comes out exactly 0, or a full tick jump on the rare occasions a tick
boundary falls in between. Either way path_delay_ns is rejected as out
of range and never converges. Warn at startup so this isn't mistaken
for a logic bug.

Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
(cherry picked from commit b725654)
In ptp_sendmsg(), query the hardware transmit timestamp via ioctl
SIOCG_TX_HW_TIMESTAMP when sendts is requested (Milestone M4).

Fallback transparently to software timestamp capture via ptp_gettime()
if the ioctl fails or is not supported by the network driver.

Assisted-by: Gemini:gemini-3.8-flash-medium
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
(cherry picked from commit 8efe9ee)
In ptp_adjtime(), when hardware timestamping is enabled, send the
drift estimate to the MAC's PTP counter via SIOCS_PTP_ADJFREQ
alongside the existing software phase-slew adjtime() call.

Assisted-by: Gemini:gemini-3.8-flash-medium
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
(cherry picked from commit 29c75fe)
The PTP hardware counter (used for RX/TX timestamps and the physical
PPS output pin) is a separate clock domain from CLOCK_REALTIME. Two
gaps kept it from ever locking onto the master's time:

- The initial clock jump computed the new time from raw local/remote
  timestamps without accounting for path_delay_ns, and the hardware
  phase nudge (SIOCS_PTP_ADJPHASE) mirrored that same omission - so
  any nonzero path delay became a permanent, uncorrected hardware
  offset. Earlier attempts at this fix only made the two paths
  consistent with each other without fixing the underlying omission;
  this instead makes the jump target - and, when hardware timestamping
  is enabled, the hardware phase nudge itself - use the master's
  timestamp directly, sidestepping local_timestamp altogether (it
  lives in the hardware counter's own free-running epoch, not
  CLOCK_REALTIME, so subtracting it was never meaningful to begin
  with).
- SIOCS_PTP_ADJFREQ only ever received the frequency drift estimate,
  with no phase term, so once past the one-time jump there was no way
  to pull the hardware counter's phase back into line. ptp_adjtime()
  now folds the residual phase error into the ppb sent to the driver
  (clamped to the configured slew limit), turning the frequency trim
  into a proportional phase servo for the hardware counter between
  jumps.

Validated on the HIL bench: the STM32's ETH_PPS_OUT pin now phase-locks
to the GPS Grandmaster's 1PPS edge and stays aligned, instead of
freezing at a fixed offset after the initial jump.

Assisted-by: Gemini:gemini-3.8-flash-medium
Assisted-by: Claude:claude-sonnet-5
Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
@xiaoxiang781216

Copy link
Copy Markdown
Contributor

please fix the conflict. @daniel-p-carvalho

Comment thread netutils/ptpd/ptpd.c

ret_ts = ioctl(state->tx_socket, SIOCG_TX_HW_TIMESTAMP,
(unsigned long)&req);
if (ret_ts < 0)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (ioct(...) < 0) and remove line 866

Comment thread netutils/ptpd/ptpd.c
{
int64_t max_path_delay;

max_path_delay = CONFIG_NETUTILS_PTPD_MAX_PATH_DELAY_NS;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge with line 1482

Comment thread netutils/ptpd/ptpd.c
if (path_delay >= 0 && path_delay < max_path_delay)
{
if (state->path_delay_avgcount <
CONFIG_NETUTILS_PTPD_DELAYREQ_AVGCOUNT)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge into one line

Comment thread netutils/ptpd/ptpd.c
/ state->path_delay_avgcount;

ptpinfo("Path delay: %ld ns (avg: %ld ns)\n",
(long)path_delay, (long)state->path_delay_ns);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove all cast

Comment thread netutils/ptpd/ptpd.c
else
{
ptpwarn("Path delay out of range: %lld ns\n",
(long long)path_delay);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove too

Comment thread netutils/ptpd/ptpd.c
{
interval = (1 << msg->header.logmessageinterval);
ptpwarn("Ignoring out-of-sequence Pdelay_Resp (%d vs. expected %d)\n",
(int)sequence, (int)state->pdelay_req_seq);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove the cast

Comment thread netutils/ptpd/ptpd.c
{
state->pdelay_waiting_followup = true;
ptpinfo("Waiting for Pdelay_Resp_Follow_Up, seq %d\n",
(int)sequence);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Comment thread netutils/ptpd/ptpd.c

t4_t1_ns = timespec_delta_ns(&state->pdelayresp_rx_time,
&state->pdelayreq_tx_time);
t3_t2_ns = (int64_t)correction_time;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove the cast

Comment thread netutils/ptpd/ptpd.c
{
ptpwarn("Ignoring out-of-sequence Pdelay_Resp_Follow_Up "
"(%d vs. expected %d)\n",
(int)sequence, (int)state->pdelay_req_seq);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove too

Comment thread netutils/ptpd/ptpd.c
if (drift_ppb > slew_limit_ppb || drift_ppb < -slew_limit_ppb)
{
ptpwarn("Drift estimate out of range: %lld\n",
(long long)drift_ppb);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove the cast

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants