Skip to content

How to handle data-dict's display: restricted column flag? #237

Description

@jat255

data-dict's spec gives a column an optional display property, whose only supported value is restricted. It marks PII and other sensitive data. A restricted column must be excluded from default user interfaces and other user-facing output, including tables, plots, and downloads. The spec is explicit that this is a steer rather than a guarantee. See https://data-dict.tidyverse.org/spec.html#display for details.

Currently, commons doesn't do anything special with this value. The string restricted does not appear in pkg-r/R, pkg-r/inst/prompts, or pkg-py/src (verified 2026-09-01).

Where this is relevant

dictionary_column_line() in pkg-r/R/data-dictionary.R builds a column's prompt line from its type, its description, and its examples. It does not read display. A restricted column is therefore described to the model exactly like every other column.

The agent gets no steer at all. It can select an SSN column and put the values in a table it shows the user, and nothing in the prompt suggests otherwise. This is the outcome the flag exists to make less likely.

Potential implementation direction

Two places, and they are separable: mark the column in the ambient text and the first-touch entry so the model knows not to show its values by default; and respect the flag in any user-facing output commons renders itself, if it renders column values outside the model's control.

Some other thoughts

An important note: this is more just steering for an agent and isn't really access control or a security feature (like data-dict's spec says). An agent that computes COUNT(DISTINCT ssn) or joins on a restricted column is doing legitimate work, and the result carries no restricted value. Refusing queries that name a restricted column would go beyond what the spec asks and would break useful analysis.

Related: data-dict's JSON export already strips sample_values, common_values, range, and histogram from a restricted column's profile and keeps only counts. Whatever commons reads from the export is already gated. The gap is the prompt text commons writes itself.

Both packages behave the same way today, so an eventual answer definition belongs in tests/shared/ and implemented on both sides.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pyAffects the Python implementationrAffects the R implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions