Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,15 @@ updates:
default-days: 7
schedule:
interval: weekly
groups:
docker-dependencies:
patterns: ["*"]
- package-ecosystem: pip
directory: /tests/fixture/ansible
cooldown:
default-days: 7
schedule:
interval: weekly
groups:
python-dependencies:
patterns: ["*"]
7 changes: 6 additions & 1 deletion .github/workflows/__shared-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,14 @@ jobs:
contents: read
pull-requests: write
uses: $/.github/workflows/__test-actions-vm.yml
ansible:
permissions:
contents: read
pull-requests: write
uses: $/.github/workflows/__test-actions-ansible.yml
checks:
if: ${{ always() }}
needs: [tests, linter, vm]
needs: [tests, linter, vm, ansible]
runs-on: ubuntu-latest
steps:
- name: Require all checks
Expand Down
64 changes: 64 additions & 0 deletions .github/workflows/__test-actions-ansible.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
name: Internal - Test Ansible action
on:
workflow_call:
permissions: {}
jobs:
scenarios:
runs-on: ubuntu-latest
timeout-minutes: 30
permissions:
contents: read
pull-requests: write
strategy:
fail-fast: false
matrix:
mode: [playbook, sanity]
result: [success, failure]
steps:
- name: Checkout fixture
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Prepare selected scenario without a consumer Makefile
env:
MODE: ${{ matrix.mode }}
RESULT: ${{ matrix.result }}
run: |
rm Makefile
if [ "$RESULT" = failure ] && [ "$MODE" = sanity ]; then
printf '%s\n' '---' 'invalid: [' > tests/fixture/ansible/ansible_collections/example/smoke/roles/smoke/tasks/main.yml
fi
- name: Test Ansible with upstream actions
id: test
continue-on-error: true
uses: $/actions/ansible/test
env:
FIXTURE_RESULT: ${{ matrix.result }}
with:
ansible-requirements: tests/fixture/ansible/requirements.txt
test-playbook: ${{ matrix.mode == 'playbook' && 'tests/fixture/ansible/valid.yml' || '' }}
collection: ${{ matrix.mode == 'sanity' && 'tests/fixture/ansible/ansible_collections/example/smoke' || '' }}
report-name: Ansible tests - ${{ matrix.mode }} - ${{ matrix.result }}
- name: Verify check outcomes and native reports
if: ${{ always() }}
env:
MODE: ${{ matrix.mode }}
EXPECTED: ${{ matrix.result }}
OUTCOME: ${{ steps.test.outcome }}
REPORTS: ${{ steps.test.outputs.report-files }}
PARSED: ${{ steps.test.outputs.parsed-files }}
ARTIFACT: ${{ steps.test.outputs.artifact-id }}
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const assert = require('node:assert/strict');
const fs = require('node:fs');
assert.equal(process.env.OUTCOME, process.env.EXPECTED);
const files = process.env.REPORTS.split('\n').filter(Boolean);
assert.ok(files.length > 0);
assert.deepEqual(JSON.parse(process.env.PARSED).sort(), files.sort());
assert.ok(process.env.ARTIFACT);
assert.ok(files.every(file => file.endsWith('.xml')));
const reports = files.map(file => fs.readFileSync(file, 'utf8')).join('\n');
assert.equal(/<(?:failure|error)\b/.test(reports), process.env.EXPECTED === 'failure');
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
.cache/
.reports/
node_modules/
tests/fixture/ansible/ansible_collections/*/*/tests/output/
.env
*.tfstate*
*.tfplan
Expand Down
9 changes: 9 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,15 @@ comments. Consumer workflows grant comment permissions and always schedule clean
Unit tests exercise the action entrypoints and internal modules directly.
Keep hosted boot checks alongside the mocked unit contracts.

`__test-actions-ansible.yml` exercises playbook tests and collection sanity success/failure
scenarios through maintained playbook execution and Ansible community testing actions.
It verifies native report publication after failures. Adjacent JavaScript only
validates inputs and collects reports; upstream actions own test execution,
collection Python setup and containers. Playbook tests use a temporary virtual
environment and the built-in JUnit callback. Consumers supply their Ansible Core pin,
Galaxy requirements and selected targets. Dependabot maintains upstream action
pins and the fixture's requirements file. Linting remains in the separate linter workflow.

## Releases and dependencies

Use Conventional Commits and a reviewed pull request. Publish semantic versions
Expand Down
2 changes: 2 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ ci: ## Execute local CI tasks

docs: ## Generate action documentation
npm exec --yes --package=@ci-dokumentor/cli@0.5.0 -- ci-dokumentor generate --source actions/vm/create/action.yml actions/vm/test/action.yml actions/vm/destroy/action.yml --repository git
# The new Ansible action has no released version for automatic usage examples.
npm exec --yes --package=@ci-dokumentor/cli@0.5.0 -- ci-dokumentor generate --source actions/ansible/test/action.yml --repository git --exclude-sections usage

#############################
# Helpers
Expand Down
15 changes: 13 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

## Overview

GitHub Actions for testing infrastructure changes in disposable virtual machines.
GitHub Actions for validating Ansible and testing infrastructure changes in disposable virtual machines.
Create a Lima and QEMU guest from your configuration, run your checks, publish
test and coverage reports, and verify that the guest is removed afterward.

Expand All @@ -24,6 +24,13 @@ can run several test steps against the same guest.

## Actions

### Ansible

#### - [Test Ansible](actions/ansible/test/README.md)

Execute test playbooks and first-party collection sanity through maintained
Ansible actions. Publish test summaries, annotations, PR comments and report artifacts.

### Virtual machines

#### - [Create VM](actions/vm/create/README.md)
Expand All @@ -44,7 +51,11 @@ cleanup remains visible and preserves the runtime for a retry.

## Getting started

Use an ephemeral Linux x86-64 runner with KVM and network access. Check out your
For Ansible, check out your repository on a Linux runner. Pass its pinned
Ansible requirements file and test playbook or collection paths to
[ansible/test](actions/ansible/test/README.md).

For VMs, use an ephemeral Linux x86-64 runner with KVM and network access. Check out your
repository with `persist-credentials: false`, and provide a Lima YAML configuration
and Bash test commands or a script. Checkout files are not mounted into the guest.

Expand Down
115 changes: 115 additions & 0 deletions actions/ansible/test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
<!-- header:start -->

# ![Icon](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0IiBmaWxsPSJub25lIiBzdHJva2U9ImN1cnJlbnRDb2xvciIgc3Ryb2tlLXdpZHRoPSIyIiBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS1saW5lam9pbj0icm91bmQiIGNsYXNzPSJmZWF0aGVyIGZlYXRoZXItY2hlY2stY2lyY2xlIiBjb2xvcj0icHVycGxlIj48cGF0aCBkPSJNMjIgMTEuMDhWMTJhMTAgMTAgMCAxIDEtNS45My05LjE0Ij48L3BhdGg+PHBvbHlsaW5lIHBvaW50cz0iMjIgNCAxMiAxNC4wMSA5IDExLjAxIj48L3BvbHlsaW5lPjwvc3ZnPg==) GitHub Action: Ansible - Test

<!-- header:end -->
<!-- badges:start -->

[![Marketplace](https://img.shields.io/badge/Marketplace-ansible------test-blue?logo=github-actions)](https://github.com/marketplace/actions/ansible---test)
[![Release](https://img.shields.io/github/v/release/hoverkraft-tech/ci-github-iac)](https://github.com/hoverkraft-tech/ci-github-iac/releases)
[![Stars](https://img.shields.io/github/stars/hoverkraft-tech/ci-github-iac?style=social)](https://img.shields.io/github/stars/hoverkraft-tech/ci-github-iac?style=social)

<!-- badges:end -->
<!-- overview:start -->

## Overview

Run Ansible test playbooks and collection sanity with native reports, PR comments and artifacts.

Run Ansible tests with maintained upstream actions and native JUnit reports.

### Run tests

Check out the consumer on a fresh Linux GitHub-hosted runner. Supply its existing
requirements file containing an exact `ansible-core==X.Y.Z` pin and either an
explicit `test-playbook` or one `collection`. Use a matrix for multiple targets.
Linting and playbook syntax checks belong in separate consumer jobs.

Test playbooks are executed by
[dawidd6/action-ansible-playbook](https://github.com/dawidd6/action-ansible-playbook).
The pinned Ansible Core is installed in a temporary virtual environment using
the runner's Python; that Python must be supported by the selected Ansible Core.
Ansible's built-in JUnit callback writes task results, including failed assertions.
Provide test assertions and disposable targets, never a deployment playbook.
`inventory`, `ansible-config`, `collections-path` and `galaxy-requirements`
configure test execution. The action does not provision test environments.

Collection sanity uses the maintained Ansible community
[ansible-test action](https://github.com/ansible-community/ansible-test-gh-action),
which owns Python setup, test containers and native JUnit generation.
Its Ansible Core Git tag comes from the same requirements pin. Coverage is
disabled for sanity checks. The selected upstream revision pins nested actions
to full commit SHAs. Source stays in the consumer's checked-out collection.
Sanity covers all supported Python versions unless `target-python-version`
selects the consumer's supported runtime. These are sanity results, not code coverage.

Paths must stay inside the checkout and use letters, digits, dots, underscores,
hyphens and slashes. Each collection needs `galaxy.yml`. Use a fresh checkout per
collection job so native `tests/output/junit` contains only that run's reports.

### Publish results

Hoverkraft's `parse-ci-reports` publishes native JUnit as summaries, annotations
and a stable PR comment. Test output and task results appear in these reports;
use synthetic data suitable for the repository's visibility. The collection
action also publishes its own summary.

Report collection runs after upstream failures, including when the upstream
action does not export its report output. Reports are uploaded with a unique
artifact name and seven-day retention, including after parsing or commenting
errors. Missing reports fail the action. Failed tests remain failures. No custom
test runner, Docker lifecycle or JUnit generator is maintained here.

Grant `pull-requests: write` for comments. Fork and Dependabot comments are skipped.
Never run untrusted tests through `pull_request_target`. Use distinct report names
for matrix entries.

<!-- overview:end -->
<!-- inputs:start -->

## Inputs

| **Input** | **Description** | **Required** | **Default** |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | ------------ | ---------------------- |
| **`ansible-requirements`** | Checkout-relative requirements file containing exactly one stable ansible-core==X.Y.Z pin. | **true** | - |
| **`test-playbook`** | Checkout-relative playbook containing test assertions. This playbook is executed. Supply test-playbook or collection, exclusively. | **false** | - |
| **`collection`** | One checked-out collection directory containing galaxy.yml. Use a matrix for multiple collections. | **false** | - |
| **`target-python-version`** | Optional Python major.minor for collection sanity checks. Empty tests all versions supported by Ansible Core. | **false** | - |
| **`inventory`** | Optional checkout-relative inventory file for test playbook execution. | **false** | - |
| **`ansible-config`** | Optional checkout-relative ansible.cfg file for test playbook execution. | **false** | - |
| **`collections-path`** | Optional checkout-relative first-party collection search directory containing ansible_collections. | **false** | - |
| **`galaxy-requirements`** | Optional checkout-relative Galaxy requirements file for third-party role and collection dependencies. | **false** | - |
| **`report-name`** | Stable summary and PR comment title, distinct for each test step or matrix entry. | **false** | `Ansible test results` |
| **`github-token`** | Token for PR comments; requires pull-requests write. Fork and Dependabot comments are skipped. | **false** | `${{ github.token }}` |

<!-- inputs:end -->
<!-- secrets:start -->
<!-- secrets:end -->
<!-- outputs:start -->

## Outputs

| **Output** | **Description** |
| ------------------ | -------------------------------------------------------------------------------------------- |
| **`report-files`** | Newline-separated absolute paths of completed native reports, including after check failure. |
| **`summary`** | Parsed validation summary. |
| **`markdown`** | Markdown used for the PR comment. |
| **`parsed-files`** | JSON array of parsed report paths. |
| **`artifact-id`** | Uploaded report artifact identifier. |

<!-- outputs:end -->
<!-- examples:start -->
<!-- examples:end -->
<!-- contributing:start -->
<!-- contributing:end -->
<!-- security:start -->
<!-- security:end -->
<!-- license:start -->
<!-- license:end -->
<!-- generated:start -->

---

This documentation was automatically generated by [CI Dokumentor](https://github.com/hoverkraft-tech/ci-dokumentor).

<!-- generated:end -->
Loading
Loading