Feat/12129 log library v4#30
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces suite-wide structured logging to improve observability of test execution and TRex/Suricata operations, replacing most ad-hoc console output with consistent logger emissions and adding user-facing configuration/documentation.
Changes:
- Added centralized logging utilities (
util/log_util.py) and initialized suite logging via new pytest CLI options inconftest.py. - Replaced/augmented output in core utilities (Suricata/TRex helpers, stats saving, config builder, VLAN tool) with structured logs.
- Updated test cases and helper scripts to use the new logging approach; documented usage in
README.mdandpytest_start.sh.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| util/trex_util.py | Adds debug logging around remote operations and TRex mode selection. |
| util/suricata_manager.py | Adds structured logs for Suricata lifecycle actions and remote syncs. |
| util/suri_util.py | Adds debug/info logs around stats saving and graph generation helpers. |
| util/multiplier_iterator.py | Replaces warning/info prints with logging and introduces PROGRESS-style output. |
| util/make-graphs.py | Replaces prints with logging; configures basic console logging for script use. |
| util/make-bars.py | Replaces prints with logging; configures basic console logging for script use. |
| util/log_util.py | Introduces logging setup (console + optional file), formatting, and PROGRESS level support. |
| util/config_builder.py | Removes noisy print and adds debug logs for config builder operations. |
| util/add_vlan.py | Adds debug logs for VLAN-tagged pcap reuse/creation. |
| tests/web_50_sites/test_web_50_sites.py | Switches progress/finish output to logger PROGRESS/INFO. |
| tests/pcap_replay/test_pcap_replay.py | Switches progress/finish output to logger PROGRESS/INFO. |
| tests/nfs_smb_simple/test_nfs_smb_simple.py | Switches progress/finish output to logger PROGRESS/INFO. |
| tests/https_simple/test_https_simple.py | Switches progress/finish output to logger PROGRESS/INFO. |
| tests/http_simple/test_http_simple.py | Switches progress/finish output to logger PROGRESS/INFO. |
| tests/http_https_smb_simple/test_http_https_smb_simple.py | Switches progress/finish output to logger PROGRESS/INFO. |
| README.md | Documents new logging behavior and CLI options. |
| pytest_start.sh | Adds CLI plumbing for --suite-log-level and --suite-log-file. |
| conftest.py | Adds CLI options and initializes suite logging during pytest configuration. |
| assets/trex/traffic_profiles/trex_client_manager.py | Adds structured logging throughout TRex client manager operations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| for logger_name in _NOISY_LOGGERS: | ||
| lib_logger = logging.getLogger(logger_name) | ||
| for handler in list(lib_logger.handlers): | ||
| lib_logger.removeHandler(handler) | ||
| lib_logger.propagate = False |
There was a problem hiding this comment.
Not relevant due to loggers not having handlers attached yet. Also these handlers should not be creating file handlers regardless, but writing to StreamHandler by default ->StreamHandler.close() only calls flush() regardless.
| self._cycle += 1 | ||
| self._current_multiplier = (self._mini + self._maxi) / 2 | ||
| print(f"\n[PROGRESS] ------ Cycle: {self._cycle} ------- [PROGRESS]") | ||
| print() | ||
| logger.progress(f" ------ Cycle: {self._cycle} ------- ") | ||
| return self._current_multiplier |
There was a problem hiding this comment.
logger.progress() is available for this module, implemented in log_util.py. If someone would really want to apply this to other codebases, replacing progress with valid call is somewhat effortless. On top of that, this is a instantiated only from test functions through pytest. Print calls are intentional as separators for visual clarity on the CLI.
| parser.addoption( | ||
| "--suite-log-level", | ||
| type=str, | ||
| default="INFO", | ||
| action="store", | ||
| help=( | ||
| "Logging level for suite logger (DEBUG, INFO, PROGRESS, WARNING, ERROR, CRITICAL)." | ||
| ), | ||
| ) |
There was a problem hiding this comment.
This is valid but not neccesary needed change.
This pull request introduces structured logging throughout the test suite, enhancing debuggability, transparency, and operational insight. It adds a new logging section to the documentation, implements configurable logging levels and log file output, and replaces print statements with logger calls in both the main test runner and the TRex client manager. The changes make it easier to trace test execution, monitor progress, and troubleshoot issues.
Documentation updates:
README.md, explaining log levels, configuration options, and usage tips. The test suite now supports structured logging with configurable verbosity and optional log file output.--suite-log-leveland--suite-log-fileoptions.Core logging infrastructure:
conftest.py, including new command-line options for log level and log file, and initialized logging at test session start.TRex client manager enhancements:
trex_client_manager.pyfor all major operations, such as initialization, pcap upload, traffic property setting, preparation, execution, stopping, and updating run info. This replaces print statements and provides consistent, structured logs for TRex-related actions.Overall, these changes significantly improve the observability and maintainability of the test suite by standardizing logging practices and making logs more accessible and useful for troubleshooting.
Changes from V3