diff --git a/docs/source/images/nanotabicl_dark.svg b/docs/source/images/nanotabicl_dark.svg new file mode 100644 index 0000000000..c673300e30 --- /dev/null +++ b/docs/source/images/nanotabicl_dark.svg @@ -0,0 +1,4 @@ + + +y +y +y +123Column-wise Induced AttentionConRow-wise Self-Attention[CLS]Cony +y +y +123 Self-AttentionLogitsxxx diff --git a/docs/source/images/nanotabicl_light.svg b/docs/source/images/nanotabicl_light.svg new file mode 100644 index 0000000000..e90d5d8a11 --- /dev/null +++ b/docs/source/images/nanotabicl_light.svg @@ -0,0 +1,4 @@ + + +y +y +y +123Column-wise Induced AttentionConRow-wise Self-Attention[CLS]Cony +y +y +123 Self-AttentionLogitsxxx diff --git a/docs/source/nanotabicl.md b/docs/source/nanotabicl.md index 99535ceb2c..12e88c661b 100644 --- a/docs/source/nanotabicl.md +++ b/docs/source/nanotabicl.md @@ -4,10 +4,20 @@ The [`sdm.nn`](api/nn) package provides reusable, GPU-accelerated building block In this guide, we show how to build a minimal [TabICLv2](https://arxiv.org/abs/2602.11139)-inspired Tabular Foundation Model (TFM) with it. Like other [in-context learning (ICL)](icl) models, it receives labeled context rows together with unlabeled query rows, and predicts the query targets in a single forward pass. -TabICL pioneered the **"Compress-then-ICL"** approach which has since been adopted by other TFMs such as [TabPFN-3](https://arxiv.org/abs/2605.13986), [TabFM](https://arxiv.org/abs/2609.37959), and [KumoTabular](https://huggingface.co/blog/nvidia/kumo-tabular). +[TabICL](https://arxiv.org/abs/2502.05564) pioneered the **"Compress-then-ICL"** approach which has since been adopted by other TFMs such as [TabPFN-3](https://arxiv.org/abs/2605.13986), [TabFM](https://arxiv.org/abs/2609.37959), and [KumoTabular](https://huggingface.co/blog/nvidia/kumo-tabular). Its main advantage is its efficient ICL stage that operates on compressed row representations rather than individual cells. It consists of three stages: +```{figure} images/nanotabicl_light.svg +:figclass: light-only +:width: 100% +``` + +```{figure} images/nanotabicl_dark.svg +:figclass: dark-only +:width: 100% +``` + 1. A **row-wise compression block** that maps each table row to a fixed-size embedding. 2. An **in-context learning block** that models interactions among labeled context rows and uses that context to update query representations. 3. A final **prediction head**.