Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Repository development

## Project conventions

- This is the NI SystemLink Python client library. Service clients and Pydantic
models live in `nisystemlink/clients`; usage examples, tests, and Sphinx
documentation live in `examples`, `tests`, and `docs`.
- Follow [CONTRIBUTING.md](../CONTRIBUTING.md) and the existing neighboring code.
Consult the [decision index](../docs/decisions/README.md) for applicable ADRs
before changing design or compatibility policy.
Preserve public client/model contracts and keep package `__init__.py` exports
and `__all__` consistent. Run the documented export contract tests when changing
public exports.
- Use the Python floor and dependency constraints in
[pyproject.toml](../pyproject.toml). Preserve type annotations, Google-style
docstrings, and existing formatting/import conventions; do not suppress quality
checks or make unrelated refactors.
- Keep `pyarrow` optional. Verify both base-install behavior and the extra when
modifying Arrow support.
- Update relevant examples and documentation when behavior changes.

## Development and validation

- Use Poetry (`poetry install`, `poetry run ...`) and the existing Poe tasks.
Never hand-edit `poetry.lock`; follow the contribution guide for dependency
regeneration and auditing.
- Add a regression test for behavior changes and run the smallest relevant test
selection during iteration. Established offline checks are:

```sh
poetry run poe test
poetry run poe check
poetry run poe lint
poetry run poe types
poetry build
```

- Integration tests require SystemLink services and credentials. Prefer the
existing PR workflow for the Python-version matrix and credentialed integration
testing. Skipped tests are not proof that integration behavior works.
- Never retrieve, print, or hard-code API keys. Preserve the integration job's
serialization to prevent shared-service collisions.
- Use Conventional Commits and the
[pull request template](PULL_REQUEST_TEMPLATE.md). Report exact validation
commands and outcomes, including checks not run.
8 changes: 4 additions & 4 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']
python-version: ['3.11', '3.12', '3.13', '3.14', '3.15']
steps:
- uses: actions/checkout@v4
- name: Install poetry
Expand All @@ -29,7 +29,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12', '3.13', '3.14']
python-version: ['3.11', '3.12', '3.13', '3.14', '3.15']
# Do not run in parallel to limit parallel integration tests stomping on each other
max-parallel: 1
steps:
Expand All @@ -40,7 +40,7 @@ jobs:
with:
python-version: ${{ matrix.python-version }}
cache: "poetry"
- run: poetry install
- run: poetry install --extras pyarrow
- run: >
poetry run pytest
-m integration
Expand Down Expand Up @@ -68,7 +68,7 @@ jobs:
run: pipx install poetry
- uses: actions/setup-python@v4
with:
python-version: "3.10"
python-version: "3.11"
cache: "poetry"
- run: poetry install
- name: Semantic release
Expand Down
2 changes: 1 addition & 1 deletion .readthedocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ version: 2
build:
os: ubuntu-lts-latest
tools:
python: "3.10"
python: "3.11"

# Build documentation in the docs/ directory with Sphinx
sphinx:
Expand Down
52 changes: 51 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,8 @@ To contribute to this project, it is recommended that you follow these steps:

## Testing

Before running any tests, you must have a supported version of Python (3.10+) and [Poetry](https://python-poetry.org/docs/) installed locally.
Before running any tests, you must have a supported version of Python (3.11+) and [Poetry](https://python-poetry.org/docs/) installed locally.
The tested Python versions are currently 3.11-3.15.

It is also helpful to install SystemLink Server and configure the NI Web Server
to run on localhost.
Expand Down Expand Up @@ -96,6 +97,55 @@ To only run the tests in one particular folder, run
pytest tests/myfolder
```

## Decision records

Before changing compatibility policy or significant design behavior, consult
the [decision index](docs/decisions/README.md) and read the relevant records.
Use its templates and lifecycle guidance to capture durable maintainer decisions
in the same PR as the related changes.

## Maintaining Python support and dependencies

Review Python support annually against the
[CPython lifecycle](https://devguide.python.org/versions/). Support stable releases
that have not reached end of life, including releases receiving security-only
updates. Keep the Python constraint and classifiers in `pyproject.toml`, both CI
matrices, the release and Read the Docs interpreters, and the documented
requirements consistent. Package metadata allows future Python 3.x releases;
classifiers and CI identify the versions that have been tested.
The [pandas major-version decision](docs/decisions/ADR-001-python-specific-pandas-majors.md)
retains pandas 2 on Python 3.11-3.14 and uses native pandas 3 behavior on 3.15+;
do not replace that split with a global major upgrade.

To refresh all dependency resolutions within the declared constraints, run:

```sh
poetry lock --regenerate
poetry check --lock
```

Ordinary `poetry lock` retains existing resolutions; `poetry check --lock` checks
consistency, not vulnerabilities. Change dependency constraints only when needed
for supported-Python compatibility or a security fix.

Audit every distinct package/version pair in `poetry.lock`, including development
dependencies, optional extras, and all Python/platform marker branches. An audit
of one installed environment does not cover the complete lock. Use isolated
`pip-audit` tooling with fully pinned requirements derived from the lock; split
alternate versions of a package into separate inputs and audit each input:

```sh
pip-audit --no-deps --disable-pip --strict -r requirements-audit.txt
```

Do not treat skipped packages or collection errors as a clean audit, or suppress
advisories to make checks pass. Keep temporary audit inputs and reports outside
the repository. Validate the base install and the `pyarrow` extra, and use the PR
workflow for the supported-version matrix and credentialed integration tests.
Integration jobs are deliberately serialized to avoid shared-service collisions.
When retiring a CI version, ask a repository administrator to replace its required
branch-protection check with checks for the newly supported versions.

## Maintaining package exports

Package `__init__.py` files are part of the public API surface for this library.
Expand Down
10 changes: 9 additions & 1 deletion README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,15 @@ Requirements
**nisystemlink-clients** has the following requirements:

* Access to a SystemLink Server or SystemLink Enterprise installation
* CPython 3.10+
* CPython 3.11+ (tested on 3.11-3.15)

DataFrame compatibility
-----------------------
Python 3.11-3.14 uses pandas 2; Python 3.15+ uses pandas 3. DataFrame conversion
helpers follow the installed pandas version's native behavior. Moving to Python
3.15 can therefore change inferred string dtypes, timestamp units (for example,
microseconds instead of nanoseconds), and missing-value representations. Review
downstream dtype assumptions and serialization when upgrading.

.. _installation_section:

Expand Down
72 changes: 72 additions & 0 deletions docs/decisions/ADR-001-python-specific-pandas-majors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
id: ADR-001
date: 2026-10-29
title: Use Python-specific pandas majors
type: implementation
status: accepted
supersedes: []
---

# ADR-001: Use Python-specific pandas majors

## Context

- Adding Python 3.15 support requires deciding which pandas major to install
and whether DataFrame helpers should preserve older pandas defaults.
Without a policy, dependency resolution and output expectations are ambiguous.
- [pandas 2.3.3's published artifacts](https://pypi.org/pypi/pandas/2.3.3/json)
have no CPython 3.15 wheels. Local Python 3.15 validation of that version
crashed when constructing a timestamp column; pandas 3.1 completed the same
operation and [provides 3.15 wheels](https://pypi.org/pypi/pandas/3.1.0/json).
- The repository maintainer established during this maintenance that a global
pandas major upgrade would also require upgrading NbExec (notebook execution)
image versions, a customer-facing disruptive change. Those image upgrades
are outside this repository's Python-support maintenance.
- DataFrame tests currently assume pandas 2 timestamp units and missing-value
comparisons. pandas 3 changes timestamp-unit inference, inferred string
dtypes, and some missing-value behavior.

## Decision

- Retain pandas 2 on Python 3.11-3.14 and use pandas 3.1 or later within major 3
on Python 3.15 and later, expressed as Python-specific dependency constraints.
- Let DataFrame helpers use their installed pandas version's native defaults;
do not emulate pandas 2 output on Python 3.15.
- Do not upgrade existing NbExec images as part of this maintenance.

### Rationale

- The Python boundary confines the necessary major migration to the newly
supported runtime without forcing it into existing notebook environments.
This directly addresses the maintainer's NbExec deployment constraint.
- Native pandas behavior avoids a compatibility layer for dtype and missing
value inference. Tests must verify exact values, structure, and dtypes for
the installed major rather than suppress differences.
- Revisit the split when NbExec images can adopt pandas 3 through a coordinated
customer-facing rollout, or when security or Python support makes pandas 2
untenable.

## Alternatives Considered

- **Upgrade all supported Python versions to pandas 3:** a single major would
simplify testing, but would require the disruptive NbExec image migration.
Reconsider with a coordinated image rollout.
- **Emulate pandas 2 output on Python 3.15:** would reduce output differences
across runtimes, but adds normalization and maintenance that this decision
explicitly avoids. Reconsider if a concrete consumer contract requires it.
- **Defer Python 3.15 support:** would avoid the immediate dependency migration,
but would miss the goal of covering all stable, non-EOL CPython releases.
Reconsider only if compatible dependencies cannot support that runtime.

## Consequences

- The lock and CI must exercise both pandas majors. Keep the Python-specific
constraints aligned with the documented support window.
- Consumers moving to Python 3.15 can observe microsecond rather than nanosecond
timestamp dtypes for these inputs, inferred string rather than object dtypes,
and different missing-value representations. Do not promise uniform pandas
inference across runtimes.
- DataFrame tests retain strict dtype/value comparisons using each supported
major's expectations; omitted fields and explicit nulls remain distinct inputs.
- Future Python 3.x releases remain installable under the package's permissive
Python constraint, but are not certified until added to the test matrix.
92 changes: 92 additions & 0 deletions docs/decisions/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Decision records

This directory records durable choices and why they were made. Use it for
compatibility promises, architectural boundaries, and implementation decisions
that future changes need to understand, not routine coding details or task lists.
All records share one repository-wide index and `ADR-NNN` sequence.

## Index

| ID | Date | Status | Type | Summary | Superseded by |
| --- | --- | --- | --- | --- | --- |
| [ADR-001](ADR-001-python-specific-pandas-majors.md) | 2026-10-29 | `accepted` | implementation | Retain pandas 2 below Python 3.15; use native pandas 3 behavior on Python 3.15+ without forcing NbExec image upgrades. | |

## Finding applicable decisions

Before designing or changing behavior, scan the index for relevant topics and
open the linked records. Search record titles and bodies when the summary is
not enough. The record is authoritative; the index is a discovery aid.

Accepted records govern future work once merged to `master`. Other statuses
preserve context, not implementation instructions. A PR can implement the
decision it records, with acceptance reviewed alongside that implementation.
Specifications, public contracts, and tests still define product behavior.
If they conflict with a record, two accepted records conflict, or work depends
on an unresolved choice, report the conflict and obtain a maintainer decision
rather than silently choosing a direction.

## Creating a record

1. Search for an existing record about the question before allocating an ID.
Resolve an existing undecided record in place; do not duplicate it.
2. Use the next unused number after the highest existing `ADR-NNN`. Never reuse
an ID, including one belonging to a rejected or superseded record.
3. Copy [decision.md](templates/decision.md) when a direction has been selected,
or [decision-needed.md](templates/decision-needed.md) when it has not.
Save it as `ADR-NNN-<lowercase-slug>.md` directly in this directory.
4. Fill the metadata and sections, match the H1 to the ID/title, and remove
instructional comments and unused optional sections. Use the date the status
or chosen direction was established, not the date of an editorial correction.
5. Keep one independently changeable decision per record. State the problem
and constraints in Context, the choice in Decision, why it fits in Rationale,
and accepted trade-offs and limits in Consequences. Cite evidence and identify
assumptions that would reopen the choice. Record the maintainer's rationale
when it comes from discussion rather than a checked-in source.
6. Describe only significant interfaces, data shapes, or error/concurrency
boundaries in an optional implementation subsection. Leave file lists,
task order, and method-by-method instructions in the implementation plan.
7. Search existing documentation and code for both the new direction and the
values it replaces. Reconcile contradictions explicitly. Link the record from
relevant contributor documentation rather than copying its rationale. Keep
internal decision references and deployment rationale out of public user docs.
8. Add or update the index row in the same PR and obtain normal maintainer review.

## Status and lifecycle

| Status | Meaning |
| --- | --- |
| `decision-needed` | A question needs human judgment; no direction has been selected. |
| `blocked` | The question cannot progress until a named decision or external dependency resolves. |
| `proposed` | A direction is selected for discussion but not accepted. |
| `accepted` | The maintainer has selected this direction; merging the record makes it governing. |
| `deferred` | The choice is deliberately postponed; Consequences names a revisit trigger. |
| `rejected` | The proposed direction was declined; retain the reasoning. |
| `superseded` | A newly accepted record replaces this one. |

Use the undecided template for `decision-needed` and `blocked`; use the decision
template for all other statuses. A recommendation or review comment alone does
not accept a decision. State the intended status explicitly in the PR and have
the maintainer confirm it through review.

- **Clarify in place** for corrections, additional evidence, or wording that
does not change the required behavior. Keep the original date.
- **Resolve an undecided record** in place when a direction is selected. Preserve
its original context and resolved question; replace Question, Options, and
Blocking Dependencies with Decision and Consequences. Remove `blocked_by`.
- **Supersede** when the approach, compatibility promise, or material trade-off
changes. Create a new record with `supersedes` listing the old record's
repository-relative path. Only mark the old record `superseded` when the new
one becomes `accepted`, in the same change; preserve the old reasoning.
- Update the date whenever the status or direction changes. Update the index
for changes to metadata, summary, or supersession. Keep IDs and paths stable.

## Review checklist

- Metadata, filename, heading, and index agree; IDs are unique.
- Links resolve and `supersedes` contains file paths, not bare IDs.
- Context identifies the problem and sources without presenting the solution.
- The choice, rationale, alternatives actually weighed, and consequences are
distinct and concise. Deferred/blocked records name their revisit conditions
or dependencies.
- No placeholder text, guidance comments, conflicting active decisions, or
implementation task lists remain in a completed record.
35 changes: 35 additions & 0 deletions docs/decisions/templates/decision-needed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
---
id: ADR-NNN
date: YYYY-MM-DD
title: <Short decision title>
type: <architectural | implementation | informational>
status: <decision-needed | blocked>
supersedes: []
---

<!--
Use while no direction has been selected. Save as ADR-NNN-<slug>.md directly in
docs/decisions/ and update its README index. Add blocked_by only for status
blocked: list decision file paths, external links, or "external: <description>".
When resolved, keep the ID and original context, convert to decision.md's
sections, remove blocked_by, and follow the README lifecycle guidance.
-->

# <ID>: <Short decision title>

## Context

<!-- State the problem, constraints, source discussion or artifacts, and the
repository behavior or interfaces affected. Do not select a solution here. -->

## Question

<!-- One question that needs human judgment. -->

## Options

<!-- Give each realistic option and its main trade-off. -->

## Blocking Dependencies

<!-- Decision paths, external dependencies that must resolve, or "None". -->
Loading
Loading