Skip to content

dashboard: get rid of oatpp - #419

Open
MariusBgm wants to merge 5 commits into
mainfrom
dev/get_rid_of_oatpp
Open

dashboard: get rid of oatpp#419
MariusBgm wants to merge 5 commits into
mainfrom
dev/get_rid_of_oatpp

Conversation

@MariusBgm

@MariusBgm MariusBgm commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Drop oat++ from the SIL Kit Dashboard client

  • removed: ThirdParty/oatpp submodule, its CMake and license entries
  • added: minimal HTTP/1.1 client over the bundled standalone asio (dashboard/http/)
  • changed: dashboard JSON now emitted with the bundled rapidyaml
  • changed: wire format identical bar 3 cosmetics: space after :, no \/, UTF-8 over \uXXXX; unescapable control chars become U+FFFD
  • fixed: registry hung on shutdown when the dashboard accepted but never answered; requests now have connect/write/read deadlines and abort after a grace period
  • fixed: use-after-free, RegistryInstance destroyed the dashboard before the registry pointing at it
  • fixed: one unmappable event killed the worker thread, silently ending all dashboard reporting; failures are now logged and skipped per event and per flush
  • fixed: unhandled controller types threw instead of being skipped, reachable by 11 of 21 types
  • removed: bulk-update probe, which killed the worker on a negative result; also drops a startup round-trip and a ~7 s stall when unreachable
  • fixed: events pending at shutdown were discarded, now flushed
  • fixed: dashboard log lines had no Dashboard topic and escaped topic filtering
  • changed: SILKIT_BUILD_DASHBOARD=OFF with --dashboard-uri now reports the missing feature plainly
  • added: warning when the dashboard is on but CollectFromRemote is off, which silently yields no metrics or attributes
  • changed: SilKitEvent type erasure to std::variant, worker split out; net −348 lines
  • tests: new suites for HTTP client, parser, retry, JSON writer, DTO mapper, worker, shutdown; IRestClient mocked, batching had no coverage
  • tests: no sleeps or wall-clock assertions, latch handover, stable under CI load

Signed-off-by: Marius Börschig <Marius.Boerschig@vector.com>
@MariusBgm
MariusBgm force-pushed the dev/get_rid_of_oatpp branch from 713ff9a to 8701f38 Compare September 1, 2026 16:47
refactor and cleanups

Signed-off-by: Marius Börschig <Marius.Boerschig@vector.com>
Signed-off-by: Marius Börschig <Marius.Boerschig@vector.com>
Signed-off-by: Marius Börschig <Marius.Boerschig@vector.com>
Signed-off-by: Marius Börschig <Marius.Boerschig@vector.com>
@MariusBgm MariusBgm added the needs reviewer This issue is looking for a reviewer. label Sep 2, 2026
@MariusBgm
MariusBgm marked this pull request as ready for review September 2, 2026 12:50

@VDanielEdwards VDanielEdwards left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heres a mixed set of comments made by Claude and me.


asio::io_context ioContext{1};
std::optional<asio::ip::tcp::socket> socket;
asio::streambuf readBuffer;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Default max_size is SIZE_MAX, so ReadHead's async_read_until and ReadUntilClose's async_read never self-terminate — maxHeadSize/maxHttpBodySize are only checked once the data is already buffered. A server that streams headers without the blank line, or a response with neither Content-Length nor Transfer-Encoding, grows RSS for the whole 30s read window.

asio::streambuf readBuffer{maxHeadSize + maxHttpBodySize}; bounds both (async_read_until then completes with error::not_found). Content-Length and chunked framing are already bounded before reading.

{
return asio::error::message_size;
}
const auto* data = asio::buffer_cast<const char*>(readBuffer.data());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

asio::buffer_cast is deprecated in asio 1.30 — compiles only while ASIO_NO_DEPRECATED is unset. static_cast<const char*>(readBuffer.data().data()) is the current spelling. Also lines 277, 298, 368 and in FakeHttpServer.hpp.


TEST_F(Test_AsioHttpClient, Post_ReportsATransportErrorWhenNothingIsListening)
{
// Port 1 is reserved and never has a listener. The connect deadline is out of reach, so a

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1h connect deadline + port 1 assumes the host refuses rather than drops. Where loopback port 1 is filtered this hangs to the CTest timeout. Same in service/Test_DashboardShutdown.cpp:79.

// Parse eagerly so that a malformed --dashboard-uri is reported when the instance is created,
// rather than later on the registry's thread. The result is discarded; DashboardRestClient
// parses it again once it is built.
(void)SilKit::Core::Uri::Parse(dashboardUri);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider making sure that the URI scheme is http. The Uri::Parse function accepts any scheme.

Suggested change
(void)SilKit::Core::Uri::Parse(dashboardUri);
const auto uri = SilKit::Core::Uri::Parse(dashboardUri);
if (uri.Scheme() != "http")
{
SilKit::SilKitError{"Dashboard URI must have the http scheme!"};
}

Alternatively the check could be done in DashboardRestClient.

Comment on lines +99 to +123
# Every dashboard test links the same set, so register them from one list.
set(SILKIT_DASHBOARD_TEST_SOURCES
Test_DashboardEventQueueWorker.cpp
Test_DashboardSilKitEventQueue.cpp
client/Test_DashboardSystemServiceClient.cpp
http/Test_AsioHttpClient.cpp
http/Test_HttpResponseParser.cpp
http/Test_RetryingHttpClient.cpp
json/Test_DashboardJsonWriter.cpp
service/Test_DashboardDtoMapper.cpp
service/Test_DashboardRestClient.cpp
service/Test_DashboardShutdown.cpp
)

add_silkit_test_to_executable(SilKitDashboardTests
SOURCES client/Test_DashboardSystemServiceClient.cpp
LIBS
S_SilKitImpl
O_SilKit_Dashboard
I_SilKit
oatpp
)
target_sources(SilKitDashboardTests PRIVATE http/FakeHttpServer.hpp Mocks/MockRestClient.hpp)

foreach(testSource IN LISTS SILKIT_DASHBOARD_TEST_SOURCES)
add_silkit_test_to_executable(SilKitDashboardTests
SOURCES ${testSource}
LIBS
S_SilKitImpl
O_SilKit_Dashboard
I_SilKit
)
endforeach()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using target_sources is only valid when SILKIT_BUILD_TESTS=ON. Otherwise the target does not exist. All sources can be combined into a single call to add_silkit_test_to_executable. This does the check internally.

Suggested change
# Every dashboard test links the same set, so register them from one list.
set(SILKIT_DASHBOARD_TEST_SOURCES
Test_DashboardEventQueueWorker.cpp
Test_DashboardSilKitEventQueue.cpp
client/Test_DashboardSystemServiceClient.cpp
http/Test_AsioHttpClient.cpp
http/Test_HttpResponseParser.cpp
http/Test_RetryingHttpClient.cpp
json/Test_DashboardJsonWriter.cpp
service/Test_DashboardDtoMapper.cpp
service/Test_DashboardRestClient.cpp
service/Test_DashboardShutdown.cpp
)
add_silkit_test_to_executable(SilKitDashboardTests
SOURCES client/Test_DashboardSystemServiceClient.cpp
LIBS
S_SilKitImpl
O_SilKit_Dashboard
I_SilKit
oatpp
)
target_sources(SilKitDashboardTests PRIVATE http/FakeHttpServer.hpp Mocks/MockRestClient.hpp)
foreach(testSource IN LISTS SILKIT_DASHBOARD_TEST_SOURCES)
add_silkit_test_to_executable(SilKitDashboardTests
SOURCES ${testSource}
LIBS
S_SilKitImpl
O_SilKit_Dashboard
I_SilKit
)
endforeach()
add_silkit_test_to_executable(SilKitDashboardTests
SOURCES
http/FakeHttpServer.hpp
Mocks/MockRestClient.hpp
Test_DashboardEventQueueWorker.cpp
Test_DashboardSilKitEventQueue.cpp
client/Test_DashboardSystemServiceClient.cpp
http/Test_AsioHttpClient.cpp
http/Test_HttpResponseParser.cpp
http/Test_RetryingHttpClient.cpp
json/Test_DashboardJsonWriter.cpp
service/Test_DashboardDtoMapper.cpp
service/Test_DashboardRestClient.cpp
service/Test_DashboardShutdown.cpp
LIBS
S_SilKitImpl
O_SilKit_Dashboard
I_SilKit
)

Comment on lines +55 to +57
char buffer[64];
const int length = std::snprintf(buffer, sizeof buffer, "%.16g", value);
node << ryml::csubstr{buffer, static_cast<size_t>(length < 0 ? 0 : length)};

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since snprintf(..., "%.16g", ...) can emit nan or inf (for which JSON has no represantation in any case) we should handle this case here. I'd just emit 0. We could also add a log message in this cae, but then we'd have to keep a logger pointer around.

Suggested change
char buffer[64];
const int length = std::snprintf(buffer, sizeof buffer, "%.16g", value);
node << ryml::csubstr{buffer, static_cast<size_t>(length < 0 ? 0 : length)};
if (std::isfinite(value))
{
char buffer[64];
const int length = std::snprintf(buffer, sizeof buffer, "%.16g", value);
node << ryml::csubstr{buffer, static_cast<size_t>(length < 0 ? 0 : length)};
}
else
{
node << 0;
}

*
* Implementations must never throw out of Post(): every failure is reported as
* HttpResult::transportError. Post() is expected to be called from a single thread; Abort() may be
* called concurrently from another.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can Reset() also only be called from the same, singular thread as Post(...)?

// asio types are kept out of this header so that the 22k-line asio headers are confined to the
// single translation unit that implements the client.
namespace asio {
class io_context;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this forward declare neccessary? AFAICT its not used in this header, and the implementation includes the asio headers themselves.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs reviewer This issue is looking for a reviewer.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants