Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -633,6 +633,8 @@ generate_summary_report("report-sum") # totals + per-name p50/p90/p95/p99
| JUnit | `<base>-junit.xml` (CI-native) | single |
| Summary | `<base>.json` (per-name p50/p90/p95/p99) | single |

Under [TestPioneer](https://github.com/Integration-Automation/TestPioneer), which sets `TEST_PIONEER_ARTIFACT_DIR` for the runners it starts, a report asked for under a relative name is written below that directory, and the path each generator returns is the one it wrote. An absolute name is used as given, and without the variable nothing changes.

## Observability

```python
Expand Down
2 changes: 2 additions & 0 deletions README/README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -627,6 +627,8 @@ generate_summary_report("report-sum") # totals + per-name p50/p90/p95/p99
| JUnit | `<base>-junit.xml`(CI 原生) | single |
| Summary | `<base>.json`(per-name p50/p90/p95/p99) | single |

由 [TestPioneer](https://github.com/Integration-Automation/TestPioneer) 启动时(它会为启动的 runner 设置 `TEST_PIONEER_ARTIFACT_DIR`),以相对名称要求的报告会写到该目录底下,各生成器返回的路径就是实际写入的路径。绝对路径照原样使用;没有设置这个变量时,行为不变。

## 可观测性

```python
Expand Down
2 changes: 2 additions & 0 deletions README/README_zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -627,6 +627,8 @@ generate_summary_report("report-sum") # totals + per-name p50/p90/p95/p99
| JUnit | `<base>-junit.xml`(CI 原生) | single |
| Summary | `<base>.json`(per-name p50/p90/p95/p99) | single |

由 [TestPioneer](https://github.com/Integration-Automation/TestPioneer) 啟動時(它會為啟動的 runner 設定 `TEST_PIONEER_ARTIFACT_DIR`),以相對名稱要求的報告會寫到該目錄底下,各產生器回傳的路徑就是實際寫入的路徑。絕對路徑照原樣使用;沒有設定這個變數時,行為不變。

## 可觀測性

```python
Expand Down
5 changes: 4 additions & 1 deletion architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,10 @@ the published dependency floor. The old action-executor record contract is separ
generates scripts that use `from je_load_density import start_test` (`pybreeze/utils/curl_import/script_templates.py`).
- **TestPioneer**: `with: load-runner` imports `je_load_density.execute_action` in-process
(`test_pioneer/executor/run/utils.py`) after setting `LOCUST_SKIP_MONKEY_PATCH=1`. `parallel_run`
spawns `python -m je_load_density --execute_file <script>` (`parallel_run.py`). Anyone embedding
spawns `python -m je_load_density --execute_file <script>` (`parallel_run.py`) and reads its exit
status and the `<name>_success.json` / `<name>_failure.json` pair. It sets `TEST_PIONEER_ARTIFACT_DIR`
for the runner, in a child process and in-process alike; `utils/generate_report/report_path.py` places
a report with a relative name below it (`test/test_report_path.py`). Anyone embedding
this package must keep gevent patching in mind; the socket server calls `monkey.patch_all()`.
- **APITestka (subprocess)**: `apitestka load run` / `AT_run_load_test`
(`APITestka/je_api_testka/integrations/load_density_runner.py`) runs `python -m je_load_density --execute_file <file>`
Expand Down
9 changes: 9 additions & 0 deletions docs/updates/2026-10.md
Original file line number Diff line number Diff line change
Expand Up @@ -342,3 +342,12 @@ Added a stdlib smoke harness that exercises local HTTP requests in isolated proc
| `test/test_parameter_context.py:355` | `6dbba09e54cf028095340bbabf4fec5e` |
| `test/smoke/install_extra.py:22` | `21c71abada62748a6f66033481b743e5` |
| `test/smoke/run_cell.py:16` | `314e2c58a5430731933f6e0481342758` |

## U-20261008-01 · 2026-10-08 · Reports go below TestPioneer's artifact directory · #feature #report #testpioneer

- **What**: TestPioneer starts a runner with `TEST_PIONEER_ARTIFACT_DIR` set to the directory that run's output belongs in (its `docs/artifacts.rst`, "Native Support"). With the variable set, a report asked for under a relative name is now written below that directory, and missing folders are created (`utils/generate_report/report_path.py`). All fifteen report generators use it: JSON, XML, HTML, JUnit, summary, CSV, cost, CycloneDX, SARIF, service map, Excel, PDF, chart, histogram and Allure. The path a generator returns is the path it wrote.
- **Unchanged**: without the variable, every path is as before. An absolute name is used as given, which is what APITestka's `load run` passes; so are a name that would leave the directory and a directory that cannot be created.
- **Why**: the workflow then needs no `artifacts:` entry for the runner's report, and nothing is left in the working directory. The other half of TestPioneer's contract, a non-zero exit status after a failed action, has been here since PR #118.
- **Tests**: `test/test_report_path.py`.
- **Docs**: `README.md`, `README/README_zh-TW.md`, `README/README_zh-CN.md` (Reports), `architecture.md` §6.
- **Files**: `je_load_density/utils/generate_report/` (`report_path.py` new; fifteen generators), the test and docs above.
3 changes: 2 additions & 1 deletion docs/updates/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ In the same commit: delete the item from `progress.md`, add a `#done` entry here

| ID | Date | Title | Tags | Batch |
|---|---|---|---|---|
| U-20261008-01 | 2026-10-08 | Reports go below TestPioneer's artifact directory | #feature #report #testpioneer | [2026-10](2026-10.md) |
| U-20261003-07 | 2026-10-03 | Align security annotations with argument-expression scan locations | #done #quality #security #ci | [2026-10](2026-10.md) |
| U-20261003-06 | 2026-10-03 | Harden smoke XML and document reviewed Codacy false positives | #done #security #quality #ci | [2026-10](2026-10.md) |
| U-20261003-05 | 2026-10-03 | Expose the shared worker stop query without private access | #done #quality #async | [2026-10](2026-10.md) |
Expand Down Expand Up @@ -138,5 +139,5 @@ In the same commit: delete the item from `progress.md`, add a `#done` entry here

| File | Period | Entries |
|---|---|---:|
| [2026-10.md](2026-10.md) | 2026-10 | 30 |
| [2026-10.md](2026-10.md) | 2026-10 | 31 |
| [2026-09.md](2026-09.md) | 2026-09 | 45 |
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
run inside any existing Allure dashboard.
"""

from je_load_density.utils.generate_report.report_path import report_path
import json
import os
import time
Expand Down Expand Up @@ -57,6 +58,7 @@ def generate_allure_report(
label: Optional[str] = None,
) -> str:
"""Write one ``*-result.json`` file per record. Returns the directory path."""
output_dir = report_path(output_dir)
os.makedirs(output_dir, exist_ok=True)
if label:
env_path = os.path.join(output_dir, "environment.properties")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
or via the ``[charts]`` extra.
"""

from je_load_density.utils.generate_report.report_path import report_path
import os
from itertools import chain
from typing import Dict
Expand Down Expand Up @@ -83,9 +84,9 @@ def generate_chart_report(
if not any(item["p50_ms"] is not None for item in windows):
raise ChartDependencyError("no records to render charts")

latency_path = _render_latency_chart(plt, f"{report_name}-latency.png",
latency_path = _render_latency_chart(plt, report_path(f"{report_name}-latency.png"),
windows)
rps_xs = [item["start_time"] for item in windows]
rps_counts = [item["rps"] for item in windows]
rps_path = _render_rps_chart(plt, f"{report_name}-rps.png", rps_xs, rps_counts)
rps_path = _render_rps_chart(plt, report_path(f"{report_name}-rps.png"), rps_xs, rps_counts)
return {"latency": latency_path, "rps": rps_path}
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
cost breakdown JSON.
"""

from je_load_density.utils.generate_report.report_path import report_path
import json
import os
from typing import Any, Dict, Optional
Expand Down Expand Up @@ -55,7 +56,7 @@ def generate_cost_report(
egress_rate_usd_per_gb=egress_rate_usd_per_gb,
extra_costs_usd=extras_usd,
)
file_path = f"{report_name}.json"
file_path = report_path(f"{report_name}.json")
with open(file_path, "w", encoding="utf-8") as handle:
json.dump(breakdown, handle, ensure_ascii=False, indent=2)
return os.path.abspath(file_path)
3 changes: 2 additions & 1 deletion je_load_density/utils/generate_report/generate_csv_report.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from je_load_density.utils.generate_report.report_path import report_path
import csv
import sys
from typing import Iterable, List, Optional
Expand Down Expand Up @@ -33,7 +34,7 @@ def generate_csv_report(csv_name: str = "default_name") -> Optional[str]:
Generate a CSV report containing both success and failure records.
Returns the path written, or None on failure.
"""
csv_path = f"{csv_name}.csv"
csv_path = report_path(f"{csv_name}.csv")
try:
with open(csv_path, "w", newline="", encoding="utf-8") as fh:
writer = csv.DictWriter(fh, fieldnames=_FIELDS)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
process.
"""

from je_load_density.utils.generate_report.report_path import report_path
import json
import os
import uuid
Expand Down Expand Up @@ -53,7 +54,7 @@ def generate_cyclonedx_report(
},
"components": components,
}
file_path = f"{report_name}.cdx.json"
file_path = report_path(f"{report_name}.cdx.json")
with open(file_path, "w", encoding="utf-8") as handle:
json.dump(document, handle, ensure_ascii=False, indent=2)
return os.path.abspath(file_path)
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
sheet containing every test record.
"""

from je_load_density.utils.generate_report.report_path import report_path
import math
import os
import zipfile
Expand Down Expand Up @@ -122,7 +123,7 @@ def generate_excel_report(
failure_rows = _build_rows(test_record_instance.error_record_list, "failure")
sheet_bytes = _sheet_xml(header_row + success_rows + failure_rows)

file_path = f"{report_name}.xlsx"
file_path = report_path(f"{report_name}.xlsx")
with zipfile.ZipFile(file_path, "w", zipfile.ZIP_DEFLATED) as archive:
archive.writestr("[Content_Types].xml", _CONTENT_TYPES_XML)
archive.writestr("_rels/.rels", _ROOT_RELS_XML)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
Renders a histogram + log-scale CDF of latency in milliseconds.
"""

from je_load_density.utils.generate_report.report_path import report_path
import os
from typing import Dict, List

Expand Down Expand Up @@ -74,7 +75,7 @@ def generate_histogram_report(
if not latencies:
raise HistogramDependencyError("no records to render histogram")
histogram_path = _render_histogram(
plt, f"{report_name}-histogram.png", latencies, bins,
plt, report_path(f"{report_name}-histogram.png"), latencies, bins,
)
cdf_path = _render_cdf(plt, f"{report_name}-cdf.png", latencies)
cdf_path = _render_cdf(plt, report_path(f"{report_name}-cdf.png"), latencies)
return {"histogram": histogram_path, "cdf": cdf_path}
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from je_load_density.utils.generate_report.report_path import report_path
import sys
from html import escape
from typing import List, Tuple
Expand Down Expand Up @@ -119,7 +120,7 @@ def generate_html_report(html_name: str = "default_name") -> str:
success_list, failure_list = generate_html()

try:
html_path = f"{html_name}.html"
html_path = report_path(f"{html_name}.html")
with open(html_path, "w+", encoding="utf-8") as file_to_write:
file_to_write.write(_HTML_STRING_HEAD)
file_to_write.writelines(success_list)
Expand Down
5 changes: 3 additions & 2 deletions je_load_density/utils/generate_report/generate_json_report.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from je_load_density.utils.generate_report.report_path import report_path
import json
import sys
from threading import Lock
Expand Down Expand Up @@ -59,8 +60,8 @@ def generate_json_report(json_file_name: str = "default_name") -> Tuple[str, str
"""
success_dict, failure_dict = generate_json()

success_path = f"{json_file_name}_success.json"
failure_path = f"{json_file_name}_failure.json"
success_path = report_path(f"{json_file_name}_success.json")
failure_path = report_path(f"{json_file_name}_failure.json")

try:
with _json_report_lock:
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from je_load_density.utils.generate_report.report_path import report_path
import sys
from html import escape
from typing import Optional
Expand Down Expand Up @@ -45,7 +46,7 @@ def generate_junit_report(report_name: str = "loaddensity-junit") -> Optional[st
parts.append("</testsuite>")
xml_body = "".join(parts)

path = f"{report_name}.xml"
path = report_path(f"{report_name}.xml")
try:
with open(path, "w", encoding="utf-8") as fh:
fh.write(xml_body)
Expand Down
3 changes: 2 additions & 1 deletion je_load_density/utils/generate_report/generate_pdf_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
Renders the build_summary output as a one-page printable PDF.
"""

from je_load_density.utils.generate_report.report_path import report_path
import os
from html import escape
from typing import Any, Dict, Optional
Expand Down Expand Up @@ -77,7 +78,7 @@ def generate_pdf_report(
)
summary = build_summary()

file_path = f"{report_name}.pdf"
file_path = report_path(f"{report_name}.pdf")
document = rl["SimpleDocTemplate"](file_path, pagesize=rl["letter"])
styles = rl["styles"]()
flow = [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
GitHub code scanning, GitLab security dashboards, or SonarQube.
"""

from je_load_density.utils.generate_report.report_path import report_path
import json
import os
from typing import Any, Dict, List, Optional
Expand Down Expand Up @@ -62,7 +63,7 @@ def generate_sarif_report(
"results": [_result_for(record) for record in failures],
}],
}
file_path = f"{report_name}.sarif.json"
file_path = report_path(f"{report_name}.sarif.json")
with open(file_path, "w", encoding="utf-8") as handle:
json.dump(document, handle, ensure_ascii=False, indent=2)
return os.path.abspath(file_path)
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
Mermaid, or Cytoscape.
"""

from je_load_density.utils.generate_report.report_path import report_path
import json
import os
from collections import defaultdict
Expand Down Expand Up @@ -51,7 +52,7 @@ def generate_service_map(
) -> str:
"""Write the service map JSON. Returns its path."""
graph = build_service_map(min_weight=min_weight)
file_path = f"{report_name}.json"
file_path = report_path(f"{report_name}.json")
with open(file_path, "w", encoding="utf-8") as handle:
json.dump(graph, handle, ensure_ascii=False, indent=2)
return os.path.abspath(file_path)
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from je_load_density.utils.generate_report.report_path import report_path
import json
import sys
from typing import Any, Dict, Iterable, List, Optional
Expand Down Expand Up @@ -80,7 +81,7 @@ def build_summary(success=None, failures=None) -> Dict[str, Any]:

def generate_summary_report(report_name: str = "loaddensity-summary") -> Optional[str]:
summary = build_summary()
path = f"{report_name}.json"
path = report_path(f"{report_name}.json")
try:
with open(path, "w", encoding="utf-8") as fh:
json.dump(summary, fh, indent=2)
Expand Down
5 changes: 3 additions & 2 deletions je_load_density/utils/generate_report/generate_xml_report.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
from je_load_density.utils.generate_report.report_path import report_path
import sys
from threading import Lock
from defusedxml.minidom import parseString
Expand Down Expand Up @@ -40,8 +41,8 @@ def generate_xml_report(xml_file_name: str = "default_name") -> Tuple[str, str]:
failure_xml = parseString(failure_xml_str).toprettyxml()

lock = Lock()
success_path = f"{xml_file_name}_success.xml"
failure_path = f"{xml_file_name}_failure.xml"
success_path = report_path(f"{xml_file_name}_success.xml")
failure_path = report_path(f"{xml_file_name}_failure.xml")

try:
with lock: # 使用 with 確保自動 acquire/release
Expand Down
43 changes: 43 additions & 0 deletions je_load_density/utils/generate_report/report_path.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
"""
報告檔案寫到哪裡
Where a report file is written.

TestPioneer starts a runner with ``TEST_PIONEER_ARTIFACT_DIR`` set to the directory that run's
output belongs in. A report asked for under a relative name goes there, so the workflow does
not have to say which files to collect. Outside TestPioneer nothing changes.
"""
import os
from pathlib import Path

from je_load_density.utils.logging.loggin_instance import load_density_logger

# Set by TestPioneer for every runner it starts, in a child process and in-process alike.
ARTIFACT_DIR_VARIABLE: str = "TEST_PIONEER_ARTIFACT_DIR"


def report_path(name: str) -> str:
"""
回傳報告實際寫入的路徑
Return where the report called ``name`` is written.

With ``TEST_PIONEER_ARTIFACT_DIR`` set, a relative ``name`` is placed below that directory
and the folders it needs are created. ``name`` is returned as given when the variable is
not set, when ``name`` is absolute, when it would leave the directory (``../x``), and when
the folders cannot be created.

:param name: 報告檔名或路徑 / the report's file name or path, as the caller gave it
:return: 寫入路徑 / the path to write to
"""
directory = os.environ.get(ARTIFACT_DIR_VARIABLE, "").strip()
if not directory or Path(name).is_absolute():
return name
base = Path(directory).resolve()
target = (base / name).resolve()
if base not in target.parents:
return name
try:
target.parent.mkdir(parents=True, exist_ok=True)
except OSError as error:
load_density_logger.error(f"report_path, name: {name}, the artifact directory is not usable: {error!r}")
return name
return str(target)
Loading
Loading