Sanitize XML 1.0 illegal control bytes in printer output#10
Merged
Conversation
Element values and attributes containing C0 controls (e.g. 0x16 SYN) were
written through libxml2's xmlTextWriterWrite{String,Attribute} unchanged.
libxml2 escapes &<>"' but does not validate against the XML 1.0 Char
production, so these bytes reached consumers and caused strict parsers
(Python's lxml: "PCDATA invalid Char value 22") to reject the document.
Fix:
- Add hasInvalidXmlChars / sanitizeXmlChars helpers in xml_printer_helper.hpp
plus thin write wrappers (writeXmlString, writeXmlAttribute, writeXmlRaw).
Illegal C0 bytes are replaced with spaces to preserve token boundaries
("1.25 \x16 3" -> "1.25 3", not "1.253"). Multi-byte UTF-8 is untouched
(continuation bytes are >= 0x80).
- Route every direct libxml2 write callsite in entity/xml_printer.cpp and
printer/xml_printer.cpp through the wrappers.
- Add IllegalControlCharactersAreSanitized regression test reproducing the
exact <Block>-with-0x16 payload that broke lxml.
Side effects of including xml_printer_helper.hpp from printer/xml_printer.cpp:
- Remove file-local duplicates of XmlWriter, AutoElement, openElement,
closeElement, and the THROW_IF_XML2_{ERROR,NULL} macros. They were
byte-identical to the canonical versions in the helper header.
Build / packaging changes needed to ship the fix to a stock Ubuntu 22.04
target without bumping libstdc++:
- Switch the matrix in both ubuntu workflows to shared=False and add
-o development=True so CI also runs the test suite (it wasn't before).
- conanfile.py: add boost*:without_stacktrace=True. Under static linking,
libboost_stacktrace_from_exception.a collides with libstdc++.a over
__cxa_allocate_exception; the agent does not use boost::stacktrace.
New local-build helpers:
- docker/Dockerfile.ubuntu-22.04 + docker/build.sh mirror the GHA
workflow inside a container, with a deb [version] subcommand that
replicates jiro4989/build-deb-action's output via dpkg-deb --build.
Verified end-to-end: 753/753 tests pass, resulting agent binary depends
only on libc/libm (no libstdc++.so), runs on Ubuntu 22.04 stock.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conan 2.x renamed the profile section; the latest 2.x rejects the old name with "ConfigParser: Unrecognized field 'system_tools'", which broke CI on this branch. Semantics are identical. Drops the on-the-fly sed workaround in docker/build.sh now that the in-tree profile is correct. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
johnathan-arsenault
requested review from
gmaentz,
gmchaleDNX,
npalmerDNX and
nsargentDNX
May 8, 2026 12:58
npalmerDNX
approved these changes
May 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
\x16SYN) with spaces in element values, attributes, and raw fragments before handing off to libxml2. This was the root cause of lxml/etree refusing to parse agent output (PCDATA invalid Char value 22). libxml2 escapes&<>"'but does not validate against the XML Char production, so callers must.shared=Falseand excludeboost::stacktrace, producing a fully static agent binary that runs on stock Ubuntu 22.04 without needing the gcc-13 libstdc++ ABI (GLIBCXX_3.4.32).-o development=Trueto both Ubuntu workflows so CI now runs the test suite — it wasn't before.What changed
XML escape fix (the actual bug):
src/mtconnect/printer/xml_printer_helper.hpp—sanitizeXmlChars(returnsstd::optional<std::string>; clean inputs avoid allocation) andwriteXmlString/writeXmlAttribute/writeXmlRawwrappers. Replacement-with-space (not deletion) preserves token boundaries:"1.25 \x16 3"→"1.25 3"rather than"1.253". UTF-8 continuation bytes (≥0x80) pass through untouched.src/mtconnect/entity/xml_printer.cpp,src/mtconnect/printer/xml_printer.cpp— every directxmlTextWriterWrite{String,Attribute,Raw}callsite routed through the wrappers.printer/xml_printer.cppcleanup: removed file-local duplicates ofXmlWriter,AutoElement,openElement,closeElement, and theTHROW_IF_XML2_{ERROR,NULL}macros — they were byte-identical to the canonical versions inxml_printer_helper.hpp.Build / packaging (needed for the fix to actually ship to the customer's device):
conanfile.py—boost*:without_stacktrace=True. Under static linking,libboost_stacktrace_from_exception.acollides withlibstdc++.aover__cxa_allocate_exception. The agent does not use boost::stacktrace..github/workflows/build-deb.yml,.github/workflows/build-ubuntu-22.04.yml— matrixshared: ["False"]and-o development=True.Local dev tooling (new):
docker/Dockerfile.ubuntu-22.04+docker/build.sh— mirror the GHA workflow inside a container so devs can reproduce CI failures locally.docker/build.sh deb [version]produces a.debmatchingjiro4989/build-deb-action@v3output.~/.cache/cppagent-conanpersists the conan cache between runs..gitignoreupdated to exclude the resulting build artifacts (*.deb,_CPack_Packages/,pkgroot/{home,DEBIAN/control}).Verification
XmlPrinterTest.IllegalControlCharactersAreSanitizedregression that uses the exact<Block>-with-\x16payload from the lxml traceback.ldd agent→ onlylibm.so.6,libc.so.6,ld-linux. Nolibstdc++.so.6, nolibboost_*.strings | grep GLIBCXX→ onlyGLIBCXX_TUNABLES(an unversioned token, not a symbol requirement).GLIBC_2.35(Ubuntu 22.04 ships 2.35).mtconnect-agent_2.6.0.2.2-dnx_all.deblocally, installed on the failing customer device (rolarproducts-edge1, Ubuntu 22.04.5 LTS) — the agent now starts cleanly.Test plan
docker/build.shruns the full ctest suite locally (753/753 pass).dpkg -i.build-deb.ymlandbuild-ubuntu-22.04.yml.Notes / follow-ups (not in this PR)
conan/profiles/gccstill uses[system_tools], deprecated in Conan 2.x →[platform_tool_requires].docker/build.shrewrites the section name on the fly with aTODOpointer so the upstream rename is findable.make-agent-deb.shis stale (still expects thepkgroot/DEBIAN/controlthat was removed inda3e934e); the deb path is now the GHA action /docker/build.sh deb. Cleanup is out of scope here.🤖 Generated with Claude Code