Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Large diffs are not rendered by default.

176 changes: 176 additions & 0 deletions docs/components/datarouter/detailed_design/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,185 @@ Detailed Design
:security: NO
:realizes: wp__sw_implementation[version==1]

Description
===========

The Datarouter (:need:`comp__datarouter`) is the DLT daemon. It reads the log
records of the clients from shared memory, filters them, and sends them as DLT
messages over UDP multicast. A client (``mw::log``) connects through a message
passing channel. The Datarouter asks the client to hand over a buffer by a
notification and reads it from the shared memory. It never blocks on a client.

The design documents and PlantUML diagrams are in ``score/datarouter/doc/design``.
This folder links them by symbolic links, so that the files stay at their place in
the source tree. The source code is in ``score/datarouter``. Paths in this document
are relative to this directory unless noted otherwise.

Design Decisions
----------------

The diagrams show how the design addresses the requirements and concerns.

.. list-table::
:header-rows: 1
:widths: 45 55

* - Requirement or concern
- Diagram
* - Freedom from interference: the Datarouter is a separate process and only
reads the shared memory of the clients
- ``detail-ipc.puml`` and ``context-highlevel.puml`` in
:doc:`logging_architecture`
* - A client that does not answer must not stall the other clients
- ``client_session_interaction_sequence.puml`` in
`Dynamic Diagrams for Unit Interactions`_
* - Stale sessions are cleaned up without losing readable data
- ``activity_watchdog_session_lifecycle.puml`` in
:doc:`logging_architecture`
* - A client that crashes while its session is created
- ``sequence_early_disconnect_race.puml`` in :doc:`logging_architecture`
* - Reduce the traffic early, own level per channel
- ``dlt_message_filtering_backend.puml`` in :doc:`logging_architecture`
* - Bandwidth quota per source
(:need:`comp_req__data_router__dlt_quota_bw_reduction`)
- No diagram shows the quota, see IMPL_01_03 in
:doc:`chklst_impl_inspection`
* - One code base for different deployments, stubs keep the interfaces
- ``file_transfer_feature_class_diagram.puml`` in
:doc:`logging_architecture` and ``package-datarouter.puml`` in
`Static Diagrams for Unit Interactions`_
* - Deployment-specific configuration without a rebuild
(:need:`comp_req__data_router__dlt_multiple_channels`)
- ``dlt_message_filtering_backend.puml`` in :doc:`logging_architecture` and
``seq_kvs_threads_sequence_diagram.puml`` in
`Dynamic Diagrams for Unit Interactions`_
* - The persistent storage is provided by the platform
- ``seq_kvs_threads_sequence_diagram.puml`` in
`Dynamic Diagrams for Unit Interactions`_

Design Constraints
------------------

All requirements of the component apply, see :need:`doc__data_router_requirements`.
The diagrams show the constraints per building block.

.. list-table::
:header-rows: 1
:widths: 20 40 40

* - Building block
- Constraint
- Diagram
* - DLT output
- UDP multicast only (:need:`comp_req__data_router__dlt_server`)
- ``context-highlevel.puml`` in :doc:`logging_architecture`
* - Configuration
- ``log-channels.json`` is mandatory, ``class-id.json`` is optional
- ``dlt_message_filtering_backend.puml`` in :doc:`logging_architecture`
* - Sessions
- One worker thread serves all sessions, the notification is non-blocking
- ``client_session_interaction_sequence.puml`` in
`Dynamic Diagrams for Unit Interactions`_
* - Watchdog
- A session is removed after a limited number of missed acquire deadlines
- ``activity_watchdog_session_lifecycle.puml`` in
:doc:`logging_architecture`
* - Configuration sessions
- Changes are not persistent in this repository (stub dictionary)
- ``seq_kvs_threads_sequence_diagram.puml`` in
`Dynamic Diagrams for Unit Interactions`_

External Dependencies
---------------------

External dependencies and their relationships are defined in the Bazel BUILD
targets. The dependency analysis and its findings are part of the checklist item
IMPL_01_04 in :doc:`chklst_impl_inspection`.

Rationale Behind Decomposition into Units
-----------------------------------------

The component is split by responsibility so that each unit can be tested and
replaced independently. The units and their relations are shown in
``datarouter_units.puml`` and ``package-datarouter.puml`` in
`Static Diagrams for Unit Interactions`_.

Static Diagrams for Unit Interactions
=====================================

.. uml:: _assets/package-datarouter.puml

.. uml:: _assets/context-sw-units.puml

The units that appear in the sequence diagrams and their bindings:

.. uml:: _assets/datarouter_units.puml

The diagrams are validated by the ``architectural_design`` target
``//score/datarouter/doc/design:datarouter_architectural_design``. The interface
names in the unit diagram follow the convention of the validator (``I<caller><callee>``)
and are not interfaces of the code.

Further static diagrams are part of :doc:`logging_architecture`.

Dynamic Diagrams for Unit Interactions
======================================

The session handling between a client and the Datarouter:

.. uml:: _assets/client_session_interaction_sequence.puml

The configuration session:

.. uml:: _assets/seq_kvs_threads_sequence_diagram.puml

Further dynamic diagrams (early disconnect, watchdog, filtering) are part of
:doc:`logging_architecture`.

Units within the Component
==========================

The decomposition into units is given by the directory and file structure; a unit
belongs to this component through its location. Attributes, behavior and the
interfaces of a unit are documented in the source code (public headers). Paths are
relative to ``score/datarouter`` of this repository.

Interfaces of the Component
---------------------------

The interfaces of the component architecture and the external interfaces are
realized by the following source code interfaces. The interface documentation is
part of the headers.

.. list-table::
:header-rows: 1
:widths: 30 35 35

* - Architecture interface
- Source code interface
- Realizing units
* - :need:`logic_arc_int__logging__shm`
- ``ISharedMemoryReader`` (``score_logging`` ``score/mw/log``), used by
``SourceSession``
- Data router core
* - :need:`logic_arc_int__logging__session_ctrl_channel`
- ``MessagePassingServer`` with ``MessagePassingServer::ISession`` and
``ISessionHandle``, messages in ``data_router_messages.h``
- Message passing server, data router core
* - DLT over UDP multicast (:need:`comp_req__data_router__dlt_server`)
- ``UdpStreamOutput``, ``DltLogChannel``
- DLT server and channels
* - Configuration client (Unix domain socket)
- ``UnixDomainServer::ISession``, ``IDiagnosticJobHandler``
- Unix domain server, configuration
* - Command line (``-c``, ``-C``)
- ``Options``
- Configuration

.. toctree::
:titlesonly:
:maxdepth: 1

logging_architecture
shm_apis
chklst_impl_inspection
33 changes: 33 additions & 0 deletions score/datarouter/doc/design/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# *******************************************************************************
# Copyright (c) 2026 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************
load(
"@score_tooling//bazel/rules/rules_score:rules_score.bzl",
"architectural_design",
)

architectural_design(
name = "datarouter_architectural_design",
dynamic = [
"uml/client_session_interaction_sequence.puml",
"uml/sequence_early_disconnect_race.puml",
"uml/seq_kvs_threads_sequence_diagram.puml",
],
static = [
"uml/context-sw-units.puml",
"uml/datarouter_units.puml",
"uml/detail-ipc.puml",
"uml/file_transfer_feature_class_diagram.puml",
"uml/package-datarouter.puml",
],
visibility = ["//visibility:public"],
)
42 changes: 33 additions & 9 deletions score/datarouter/doc/design/logging_architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,8 @@ The following constraints influenced the logging infrastructure design:
## Context

The diagram below illustrates the logging framework context:
<img alt="CONTEXT_ECU" src="https://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/eclipse-score/logging/refs/heads/main/score/datarouter/doc/design/uml/context-ecu.puml">
```{uml} _assets/context-ecu.puml
```

Applications write log data through logging interfaces.

Expand All @@ -79,7 +80,8 @@ The logging framework transmits data to the following sinks:

### Process structure

<img alt="CONTEXT_HIGHLEVEL" src="https://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/eclipse-score/logging/refs/heads/main/score/datarouter/doc/design/uml/context-highlevel.puml">
```{uml} _assets/context-highlevel.puml
```

The logging framework implements multiple components to meet system goals:

Expand Down Expand Up @@ -130,7 +132,16 @@ On the first call for a given type `T`, `LogEntry<T>::Instance()` (a Meyers sing

The diagram below illustrates the high-level class structure of the datarouter component.

<img alt="PACKAGE_DATAROUTER" src="https://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/eclipse-score/logging/refs/heads/main/score/datarouter/doc/design/uml/package-datarouter.puml">
```{uml} _assets/package-datarouter.puml
```

The next diagram shows the software units of the datarouter, the shared memory and the two IPC paths: the message passing channel to the clients and, if the build flag `enable_dynamic_configuration` is set, the Unix domain server for configuration clients.

```{uml} _assets/context-sw-units.puml
```

```{uml} _assets/detail-ipc.puml
```

## Runtime view

Expand All @@ -140,12 +151,14 @@ Applications access logging functionality through `mw::log`.

Initialization stage 1 executes when the first log request occurs. This may occur in global object constructors before the `main()` function executes, causing implicit initialization that creates necessary singletons automatically.
The activity diagram below depicts the first-run process:
<img alt="SEQ_TRACE" src="https://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/eclipse-score/logging/refs/heads/main/score/datarouter/doc/design/uml/seq-trace.puml">
```{uml} _assets/seq-trace.puml
```

### mw::log implementation

The `mw::log` implementation creates log records and commits them atomically to shared memory on flush.
<img alt="DLT_MESSAGE_FILTERING_FRONTEND" src="https://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/eclipse-score/logging/refs/heads/main/score/datarouter/doc/design/uml/dlt_message_filtering_frontend.puml">
```{uml} _assets/dlt_message_filtering_frontend.puml
```

### Ring buffer and linear allocator buffer

Expand All @@ -158,19 +171,30 @@ The Datarouter-Client Session uses [message_passing](https://github.com/eclipse-

The main idea is to keep clients independent, so that one non-responsive client should not stall the datarouter. This matters because the datarouter serves all clients from a single thread, one after another on each periodic tick, so a single blocking call would hold up every other client. To avoid this, the datarouter asks for buffers (to read) using non-blocking QNX pulses (`IServerConnection::Notify()`), which return right away and never wait on the client. So if a client is slow or stuck, its pulse simply stays unanswered while the healthy clients keep getting served. See below for the full flow.

<img alt="CLIENT_SESSION_INTERACTION_SEQUENCE" src="https://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/eclipse-score/logging/refs/heads/main/score/datarouter/doc/design/uml/client_session_interaction_sequence.puml">
```{uml} _assets/client_session_interaction_sequence.puml
```

However, with such a design there are risks of stale sessions. A small per-session watchdog limits how long the datarouter waits for an acquire response. If a client keeps missing its deadline, the watchdog cleans up the stale session, but first it does a best-effort read of the client's last buffer so no logs are lost unnecessarily. See below.

<img alt="ACTIVITY_WATCHDOG_SESSION_LIFECYCLE" src="https://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/eclipse-score/logging/refs/heads/main/score/datarouter/doc/design/uml/activity_watchdog_session_lifecycle.puml">
```{uml} _assets/activity_watchdog_session_lifecycle.puml
```

The setup also handles an early-disconnect race, wherein if a client crashes while the datarouter is still building its session, the connect and disconnect paths work together to throw away the half-built session instead of keeping a dangling connection pointer. The client can then reconnect cleanly afterwards. See below.

<img alt="SEQUENCE_EARLY_DISCONNECT_RACE" src="https://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/eclipse-score/logging/refs/heads/main/score/datarouter/doc/design/uml/sequence_early_disconnect_race.puml">
```{uml} _assets/sequence_early_disconnect_race.puml
```

### datarouter

<img alt="DLT_MESSAGE_FILTERING_BACKEND" src="https://www.plantuml.com/plantuml/proxy?src=https://raw.githubusercontent.com/eclipse-score/logging/refs/heads/main/score/datarouter/doc/design/uml/dlt_message_filtering_backend.puml">
```{uml} _assets/dlt_message_filtering_backend.puml
```

### File transfer

If the build flag `file_transfer` is set (default true), `FileTransferStreamHandler` is registered in the `LogParser` as a `TypeHandler` for the file transfer messages that `dlt_filetransfer_trigger_lib` (`IFileTransfer`, `FileTransfer`) writes. The handler queues a `FileTransferEntry` and a thread of the handler reads the file and sends it as DLT file transfer messages (file header, file data in packages, file end, error) through `DltLogServer::SendFtVerbose`. Without the flag `StubFileTransferStreamHandler` ignores the messages.

@rmaddikery rmaddikery Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This text block is not necessary. Linked uml is fine


```{uml} _assets/file_transfer_feature_class_diagram.puml
```

### Application-side library configuration

Expand Down
4 changes: 2 additions & 2 deletions score/datarouter/doc/design/uml/context-sw-units.puml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
frame ApplicationProcess {
node AdaptiveApplication {
component ApplicationLogic
component "score::mw::log" {
component "score::mw::log" as mwlog {
component log_entry
component "logging_serializer" as serializer
component logger {
Expand All @@ -42,7 +42,7 @@ frame ApplicationProcess {

ConfigReader ..> LogConfig

ApplicationLogic --> "score::mw::log" : verbose
ApplicationLogic --> mwlog : verbose
ApplicationLogic --> TRACE : non-verbose
}

Expand Down
Loading
Loading