diff --git a/docs/new-op.md b/docs/new-op.md index 296ff324..c2025093 100644 --- a/docs/new-op.md +++ b/docs/new-op.md @@ -130,8 +130,8 @@ The constructor takes no tuning parameter. To tune, construct the op, then call A different input schema, different math or a different output schema is a separate op class; a semantic mode with the same schemas is a construction parameter. Conventions every -op follows, such as the softmax scale `sm_scale`, `validate_inputs` for packed offsets and -the pairing of a `Bwd` op with its forward, are in +op follows, such as the softmax scale `sm_scale`, the `validate_inputs` check of packed +offsets and `in_range` bounds, and the pairing of a `Bwd` op with its forward, are in [Op Interfaces § Shared semantics](design/ops-design.md#shared-semantics). ### `kernel_for`, and choosing among kernels {#kernel-selection} diff --git a/docs/new-op.zh.md b/docs/new-op.zh.md index c1e22195..ce62cdb3 100644 --- a/docs/new-op.zh.md +++ b/docs/new-op.zh.md @@ -109,7 +109,7 @@ class GEMMFwdOp(Op): 构造函数不接收调优参数。需要调优时,先构造 op,再调用 `op.request_tune()`。 -输入 schema、数学定义或输出 schema 不同的,是另一个 op 类;schema 相同的语义模式是构造参数。所有 op 共同遵守的约定,例如 softmax 缩放 `sm_scale`、打包 offsets 的 `validate_inputs`、`Bwd` op 与其前向的配对,见 [Op Interfaces § Shared semantics](design/ops-design.md#shared-semantics)。 +输入 schema、数学定义或输出 schema 不同的,是另一个 op 类;schema 相同的语义模式是构造参数。所有 op 共同遵守的约定,例如 softmax 缩放 `sm_scale`、`validate_inputs` 对打包 offsets 与 `in_range` 上下界的检查、`Bwd` op 与其前向的配对,见 [Op Interfaces § Shared semantics](design/ops-design.md#shared-semantics)。 ### `kernel_for` 与 kernel 的选择 {#kernel-selection} diff --git a/docs/stability.zh.md b/docs/stability.zh.md index ab72794b..e90bd2c3 100644 --- a/docs/stability.zh.md +++ b/docs/stability.zh.md @@ -21,13 +21,15 @@ ## 兼容性 -在 0.x 版本内,minor 与 patch 发布对公开 API 只做增加: +在 0.x 版本内,patch 发布对公开 API 只做增加,minor 发布还可以带有经过评审的不兼容改动: -- 公开的名字、签名、参数默认值、返回结构与文档写明的行为保持不变。 +- 公开的名字、签名、参数默认值、返回结构与文档写明的行为保持不变,经过评审的不兼容改动除外。 - 一次发布可以增加模块、名字、可选参数,或新接受的输入。 -- 其他改动等到 major 发布。 +- 发布说明列出每一项经过评审的不兼容改动。从 1.0 起,不兼容改动等到 major 发布。 -`scripts/ci/check_api_compat.py` 在每个 PR 上对照最新发布做检查,一个调用若按最新发布写成、在 PR 上会被破坏,检查就失败。 +`scripts/ci/check_api_compat.py` 在每个 PR 上对照最新发布做检查:按最新发布写成的调用在 PR 上被破坏时,检查失败。 + +检查报出的不兼容改动经过评审后,列在 [`scripts/ci/api_compat_allow.txt`](https://github.com/tile-ai/TileOPs/blob/main/scripts/ci/api_compat_allow.txt) 中,检查跳过这些条目;下一次发布时清空该文件。检查发现不了的不兼容改动,例如 spec 格式的改动,在 PR 中写明。 ## 支持范围 diff --git a/docs/user-guide/manifest/calls.md b/docs/user-guide/manifest/calls.md index a05e204e..6138b9ac 100644 --- a/docs/user-guide/manifest/calls.md +++ b/docs/user-guide/manifest/calls.md @@ -22,7 +22,7 @@ In the **call** phase, the checks generated from the signature around `forward` 4. infer indices, and check the remaining refinements; 5. check the preconditions of output buffers; 6. check that no tensor the call writes shares a byte with another call tensor, except the inputs its manifest entry lists in `overlaps`; -7. when the op was constructed with `validate_inputs=True`, check the packed offsets on the CPU; +7. when the op was constructed with `validate_inputs=True`, check the packed offsets and `in_range` bounds on the CPU; 8. run the implementation on an aligned copy of any call tensor off a 16-byte boundary, and copy a written copy back after step 9; 9. check the outputs: the number of outputs and whether each is `None`, shape, dtype, device and memory layout; `out` and `alias` outputs must be the corresponding tensor objects, and other outputs must not share storage with inputs. diff --git a/docs/user-guide/manifest/calls.zh.md b/docs/user-guide/manifest/calls.zh.md index c9f4a98c..88fd6c9b 100644 --- a/docs/user-guide/manifest/calls.zh.md +++ b/docs/user-guide/manifest/calls.zh.md @@ -22,7 +22,7 @@ 4. 推断 index,并检查其余的 refinement; 5. 检查输出缓冲的前置条件; 6. 检查被写入的张量不与其他调用张量共享字节,manifest 条目在 `overlaps` 中列出的输入除外; -7. op 以 `validate_inputs=True` 构造时,在 CPU 上检查打包的 offsets; +7. op 以 `validate_inputs=True` 构造时,在 CPU 上检查打包的 offsets 与 `in_range` 上下界; 8. 执行实现,起始地址不在 16 字节边界上的调用张量以对齐的副本传入,被写入的副本在第 9 步之后拷回; 9. 检查输出:输出的数量与是否为 `None`、形状、dtype、设备与内存布局;`out` 与 `alias` 输出必须就是对应的张量对象,其他输出不能与输入共享存储。 diff --git a/docs/user-guide/manifest/extensions.md b/docs/user-guide/manifest/extensions.md index 260c098e..0495ff96 100644 --- a/docs/user-guide/manifest/extensions.md +++ b/docs/user-guide/manifest/extensions.md @@ -25,7 +25,7 @@ shape_rules: | 1 | `Tensor` | a tensor | the declaration's `Tensor[T, s]` | | 2 | `list[Tensor]` | a list or tuple of tensors | a tensor sequence, see [§ 11](#sequences) | | 3 | `int` | an `int` that is not a `bool` | `Int`, a call-time scalar | -| 4 | `float` | a `float` | a value, a call-time scalar | +| 4 | `float` | a `float`; also an `int` (not a `bool`) when the type has no `int` or `Number` member | a value, a call-time scalar | | 5 | `Number` | an `int` or a `float` | a value, a call-time scalar | | 6 | `bool` | a `bool` | `Bool`, a call-time scalar | | 7 | `None` | `None` | `None` | @@ -34,7 +34,7 @@ The rules for argument types: - `type` defaults to `Tensor`, which stands for the declaration's `Tensor[dtype, shape]`. `Number` already holds `int` and `float`, so it is never listed together with them. - An argument with a `Tensor` or `list[Tensor]` member is a call-time input, declared in `inputs`; every other argument is a construction parameter, declared in `params`. A tensor is never a construction parameter. -- A value takes a member by its Python type: `None` takes `None`; a list or tuple takes `list[Tensor]`; a tensor takes `Tensor`; a `bool` takes only `bool`; an `int` takes `int`, else `Number`; a `float` takes `float`, else `Number`. A value that no member takes is rejected. +- A value takes a member by its Python type: `None` takes `None`; a list or tuple takes `list[Tensor]`; a tensor takes `Tensor`; a `bool` takes only `bool`; an `int` takes `int`, else `Number`, else `float`; a `float` takes `float`, else `Number`. A value that no member takes is rejected. - An input without `default` is passed on every call. An input with `default` may be omitted; `forward` takes the `default` as its default, and omitting the input has the same effect as passing that value. - Among positional inputs, an input without `default` never follows one with `default`. A `kw_only` input follows every positional one, and may be required. - A required argument that may be `None` declares the `None` member and no `default`, for example `bias: {type: "Tensor | None", dtype: Wb, shape: "[N]"}`. @@ -105,7 +105,7 @@ signature: - A tensor's `shape` is written `[argument, ...]`, and the arguments map to `params` in declaration order. - The subject of `match` must be a discriminant with finitely many values: `Bool`, an enum, an ADT, `present(...)`, `member(...)`, a boolean expression of these such as `member(min) == 'Tensor'`, or a tuple of these. When matching a tuple, `when` is written as a list, for example `when: [true, false]`. -- A `Maybe` discriminant matches by its value; only `_` matches it when it is absent. +- A `Maybe[ADT]` discriminant matches by its value; only `_` matches it when it is absent. - `cases` must have no gaps and no overlaps over all values the spec accepts. - When several tensors apply the same type family, they always take the same branch. - References between type families must not form a cycle, and the validator reports an error if a type family is not used by any shape. @@ -280,7 +280,9 @@ The rules for `requires`: - A predicate argument can be another metadata tensor, so that one predicate constrains the relation between two tensors. For example, `batch_offsets` in GroupedGEMM declares `requires: ["exclusive_prefix_of(batch_sizes)"]`, which requires it to be the exclusive prefix sum of `batch_sizes`. On every branch where the constrained tensor exists, the tensor used as the argument must also exist. - The validator checks `requires` at instantiation against the generated values. In an actual call, the caller guarantees these constraints and the op does not check tensor contents by default; the validator therefore also checks that the predicate is well defined on every branch where the constrained tensor exists. - A tensor that declares `requires` must also declare `values`. -- An op with an input whose `requires` holds `prefix_offsets` declares `validate_inputs: {type: bool, default: false, kw_only: true}` in `params`. When the caller sets it, a check generated from the manifest reads the offsets on the CPU before the implementation runs; it synchronizes the device, so it cannot run inside CUDA Graph capture. +- An op with an input whose `requires` holds `prefix_offsets` declares `validate_inputs: {type: bool, default: false, kw_only: true}` in `params`. An op whose inputs' `requires` hold `in_range` but no `prefix_offsets` may declare it too. +- When the caller sets `validate_inputs`, a check generated from the manifest reads the inputs on the CPU before the implementation runs and checks their `prefix_offsets` and `in_range` predicates. Every other predicate stays the caller's obligation. +- The check synchronizes the device, so it cannot run inside CUDA Graph capture. ## 8. dtype combinations and packed dtypes {#dtype-combos} diff --git a/docs/user-guide/manifest/extensions.zh.md b/docs/user-guide/manifest/extensions.zh.md index a8e5b043..355e60a1 100644 --- a/docs/user-guide/manifest/extensions.zh.md +++ b/docs/user-guide/manifest/extensions.zh.md @@ -25,7 +25,7 @@ shape_rules: | 1 | `Tensor` | 张量 | 该声明的 `Tensor[T, s]` | | 2 | `list[Tensor]` | 由张量组成的 list 或 tuple | 张量序列,见[第 11 节](#sequences) | | 3 | `int` | 不是 `bool` 的 `int` | `Int`,调用期标量 | -| 4 | `float` | `float` | 一个值,调用期标量 | +| 4 | `float` | `float`;类型中没有 `int` 和 `Number` 成员时,也接受 `int`(`bool` 除外) | 一个值,调用期标量 | | 5 | `Number` | `int` 或 `float` | 一个值,调用期标量 | | 6 | `bool` | `bool` | `Bool`,调用期标量 | | 7 | `None` | `None` | `None` | @@ -34,7 +34,7 @@ shape_rules: - `type` 默认为 `Tensor`,表示该声明的 `Tensor[dtype, shape]`。`Number` 已经包含 `int` 与 `float`,因此不与它们同时列出。 - 有 `Tensor` 或 `list[Tensor]` 成员的实参是调用期输入,声明在 `inputs` 中;其他实参都是构造参数,声明在 `params` 中。张量不作为构造参数。 -- 值按它的 Python 类型对应成员:`None` 对应 `None`;list 或 tuple 对应 `list[Tensor]`;张量对应 `Tensor`;`bool` 只对应 `bool`;`int` 对应 `int`,没有 `int` 成员时对应 `Number`;`float` 对应 `float`,没有 `float` 成员时对应 `Number`。没有成员可以对应的值被拒绝。 +- 值按它的 Python 类型对应成员:`None` 对应 `None`;list 或 tuple 对应 `list[Tensor]`;张量对应 `Tensor`;`bool` 只对应 `bool`;`int` 对应 `int`,没有 `int` 成员时对应 `Number`,两者都没有时对应 `float`;`float` 对应 `float`,没有 `float` 成员时对应 `Number`。没有成员可以对应的值被拒绝。 - 没有 `default` 的输入每次调用都要传入。有 `default` 的输入可以省略:`forward` 以 `default` 作为它的默认值,省略这个输入与传入该值效果相同。 - 在按位置传入的输入中,没有 `default` 的输入不排在有 `default` 的输入之后。`kw_only` 的输入排在所有按位置传入的输入之后,也可以是必须传入的。 - 必须传入、但可以为 `None` 的实参声明 `None` 成员,不声明 `default`,例如 `bias: {type: "Tensor | None", dtype: Wb, shape: "[N]"}`。 @@ -105,7 +105,7 @@ signature: - 张量的 `shape` 写作 `[实参, ...]`,实参按 `params` 的声明顺序对应。 - `match` 的对象必须是取值有限的 discriminant,即 `Bool`、枚举、ADT、`present(...)`、`member(...)`、由它们构成的布尔表达式(例如 `member(min) == 'Tensor'`),或由它们组成的元组。对元组做 match 时,`when` 写成列表,例如 `when: [true, false]`。 -- `Maybe` 类型的 discriminant 按它的值匹配;只有 `_` 能匹配它缺省的情况。 +- `Maybe[ADT]` 类型的 discriminant 按它的值匹配;只有 `_` 能匹配它缺省的情况。 - `cases` 在 spec 接受的所有取值上必须既无遗漏也无重叠。 - 多个张量应用同一个 type family 时,它们总是取同一个分支。 - type family 之间的引用不能成环;如果某个 type family 没有被任何形状使用,validator 会报错。 @@ -280,7 +280,9 @@ generator 的规则如下: - 谓词的实参可以是另一个 metadata 张量,这样一条谓词就能约束两个张量之间的关系,例如 GroupedGEMM 的 `batch_offsets` 声明 `requires: ["exclusive_prefix_of(batch_sizes)"]`,要求它是 `batch_sizes` 的 exclusive 前缀和。在被约束的张量存在的每个分支上,作为实参的那个张量也必须存在。 - validator 在实例化时用生成的取值检查 `requires`。在实际调用中,这些约束由调用方保证,op 默认不检查张量的内容;因此 validator 还会检查,在被约束的张量存在的每个分支上,谓词都是良定义的。 - 声明了 `requires` 的张量也必须声明 `values`。 -- 有输入的 `requires` 含 `prefix_offsets` 的 op,在 `params` 中声明 `validate_inputs: {type: bool, default: false, kw_only: true}`。调用方把它设为 true 时,由 manifest 生成的检查在实现运行前于 CPU 上读取这些 offsets;这项检查会同步设备,不能在 CUDA Graph 捕获中运行。 +- 有输入的 `requires` 含 `prefix_offsets` 的 op,在 `params` 中声明 `validate_inputs: {type: bool, default: false, kw_only: true}`。输入的 `requires` 含 `in_range`、不含 `prefix_offsets` 的 op 也可以声明它。 +- 调用方设置 `validate_inputs` 时,由 manifest 生成的检查在实现运行前于 CPU 上读取输入,检查其中的 `prefix_offsets` 与 `in_range` 谓词。其他谓词仍由调用方保证。 +- 这项检查会同步设备,不能在 CUDA Graph 捕获中运行。 ## 8. dtype 组合与打包 dtype {#dtype-combos} diff --git a/docs/user-guide/op/lifecycle.md b/docs/user-guide/op/lifecycle.md index fc81995e..e3e0fa21 100644 --- a/docs/user-guide/op/lifecycle.md +++ b/docs/user-guide/op/lifecycle.md @@ -138,7 +138,7 @@ Where the seven steps sit in time is shown in Figure 1 of | No. | Step | What happens | | --- | --- | --- | | 1 | Begin the call | • Observe the call's specialization and look up its plan in `_plans`
• Push the current thread's call stack
• Collect the calls sub-ops complete during this call | -| 2 | Check the signature | • On a plan miss: run the checks `_SignaturePlan` generated, and hold what they decide as the plan
• On a hit: reuse the plan and run no check again
• On every call: raise `CheckError` when a tensor the call writes shares a byte with another call tensor its manifest entry does not list in `overlaps`
• On every call of an op constructed with `validate_inputs=True`: check the packed offsets on the CPU | +| 2 | Check the signature | • On a plan miss: run the checks `_SignaturePlan` generated, and hold what they decide as the plan
• On a hit: reuse the plan and run no check again
• On every call: raise `CheckError` when a tensor the call writes shares a byte with another call tensor its manifest entry does not list in `overlaps`
• On every call of an op constructed with `validate_inputs=True`: check the packed offsets and `in_range` bounds on the CPU | | 3 | Detect an empty write | When the plan says there is at least one write and every write holds no elements, run no implementation and build the result from the signature | | 4 | Select the target | Select one when the instance has none yet and this call needs to run an implementation; see [§ 4](#target) | | 5 | Run | • Run one of the three: the empty-write result, the target's kernel, or `forward`
• A call tensor off a 16-byte boundary runs as an aligned copy; a written copy is copied back after step 6 | diff --git a/docs/user-guide/op/lifecycle.zh.md b/docs/user-guide/op/lifecycle.zh.md index 145b9513..e6f5a69e 100644 --- a/docs/user-guide/op/lifecycle.zh.md +++ b/docs/user-guide/op/lifecycle.zh.md @@ -96,7 +96,7 @@ | No. | 步骤 | 内容 | | --- | --- | --- | | 1 | 开启调用 | • 确定这次调用的特化,在 `_plans` 中查找它的 plan
• 压入当前线程的调用栈
• 收集本次调用期间子 op 完成的调用 | -| 2 | 检查签名 | • plan 未命中:运行 `_SignaturePlan` 生成的检查,把检查的结论保存为 plan
• 命中:复用 plan,不再运行检查
• 每次调用:被写入的张量与另一个调用张量有共享字节、且 manifest 条目的 `overlaps` 没有列出后者时,抛出 `CheckError`
• 以 `validate_inputs=True` 构造的 op,每次调用都在 CPU 上检查打包的 offsets | +| 2 | 检查签名 | • plan 未命中:运行 `_SignaturePlan` 生成的检查,把检查的结论保存为 plan
• 命中:复用 plan,不再运行检查
• 每次调用:被写入的张量与另一个调用张量有共享字节、且 manifest 条目的 `overlaps` 没有列出后者时,抛出 `CheckError`
• 以 `validate_inputs=True` 构造的 op,每次调用都在 CPU 上检查打包的 offsets 与 `in_range` 上下界 | | 3 | 判断空写 | plan 表明至少有一个写入、且所有写入都没有元素时,不运行任何实现,按签名构造结果 | | 4 | 选择 target | 实例尚未选定 target、且这次调用需要运行实现时选择,见[第 4 节](#target) | | 5 | 执行 | • 运行空写结果、target 的 kernel 或 `forward`,三者取一
• 起始地址不在 16 字节边界上的调用张量以对齐的副本运行,被写入的副本在第 6 步之后拷回 |