HQL

Hyper Query Language

A typed functional language for querying and computing over structured knowledge.

HQL is being developed alongside HyperMarkDown and is intended to operate over its cards, document structure, metadata, and knowledge graph. HyperMarkDown remains the authored/storage representation; HQL is a separate language and repository.

In development. What runs today is a typed expression core, a vault of Markdown and HyperMarkDown documents, pipelines over its cards, semantic retrieval and graph traversal. Much of doc/models/ specifies more language than the binary implements yet, and the implementation says so rather than faking it.

Install

A release ships a binary for each supported target, with a checksum beside it. Download the pair from the releases page, check it, and put hql somewhere on your PATH:

shasum -a 256 -c hql-<version>-<target>.tar.gz.sha256   # sha256sum -c on Linux
tar -xzf hql-<version>-<target>.tar.gz

The targets are x86_64-unknown-linux-gnu, aarch64-apple-darwin and x86_64-apple-darwin. Verifying the checksum is the point of publishing it: an archive that does not match is not the one that was built.

To build it yourself instead, with Rust 1.88 or newer from rustup — the floor is rust-version in Cargo.toml, and CI builds on it so the declared floor stays true:

cargo install hql --locked                                   # the last release
cargo install --git https://github.com/ewiger/hql --locked   # the latest commit
cargo install --path . --locked                              # from a clone

CHANGELOG.md records what changed in each release. While the major version is 0, a minor bump may break a program that ran before — the language is still in design, and the changelog says so release by release rather than promising otherwise.

Develop and run

Install Rust through rustup, then run from this repository:

cargo build --locked
cargo test --locked
cargo fmt --check
cargo clippy --locked --all-targets -- -D warnings

cargo run -- eval '40 + 2'                       # 42
cargo run -- --vault <dir> eval 'cards | count'
cargo run -- builtins                            # the steps a pipeline may use

Querying a vault

tests/fixtures/birds/ is a worked example: fifty-six encyclopedia articles about birds, with an embedding index committed beside them.

hql --vault tests/fixtures/birds eval '
import semantic

cards
| semantic("night hunting birds")
| take(3)
| expand(depth = 1)
| graph
| table'

That returns the owls. Not one of their articles contains the word "night" — they are nocturnal, and abroad after dark — so the same query through the other retrieval finds nothing of the kind:

hql --vault tests/fixtures/birds eval '
import lexical

cards
| lexical("night hunting birds")
| take(3)
| map(h => h.card.name)'

[mute-swan, passeriformes, alcedinidae]

Two retrievals, each named for what it does. lexical matches shared spellings, offline and with nothing to build. semantic scores against vectors a language model computed ahead of time, which contrib/semantics/ produces and the binary only reads — the model is not in here and will not be.

Either way the answer keeps its evidence. Every hit carries the retrieval that produced it — query, index, model, model revision, metric, and whether the search was approximate — because a score is evidence rather than relevance. take needs elements that carry an order of their own, which a card does, so a prefix is reproducible without the vault's file order ever being observable. expand traverses outwards and records why each node is present, so a graph of three matches and their neighbours does not claim that all of them matched.

Commands

Command	What it does
hql eval <program>	check and evaluate an argument
hql check <file>	print the type without evaluating
hql run <file>	check and evaluate a file
hql repl	read, evaluate and print, keeping bindings
hql render <file>	run the HQL blocks in a document, transcluding the answers
hql builtins	list the steps
hql config	the reporting mode in force, and where it came from

- reads standard input wherever a file is taken. --format json emits the type, the value and the report queue for another program.

Queries inside documents

A card can carry a query, and rendering it runs the query against the vault:

```hql#eval
cards
| filter(c => c.metadata.status == "todo")
| sort(by = c => c.title)
| map(c => c.title)
| table
```

hql render --write board.md writes the answer beneath it as an hql#result block, replacing the previous one, so a to-do list is a query rather than a list somebody maintains.

Reporting

A run yields a value and a queue of everything it had to say: warnings that change nothing, errors that make a result wrong, failures that make one impossible. --report strict stops at the first error and is the default; --report collect reports every error findable. The mode comes from the query, then the vault's hql.toml, then $HQL_REPORT, then the default.

Success is exit 0, a language or I/O failure 1, a usage error 2. Diagnostics carry zero-based UTF-8 byte ranges and are rendered with a line, a column and a caret.

Example corpus

The birds wiki pairs linked HyperMarkDown cards with runnable collection queries: repeated sightings, distinct species, keyed counts, ordered routes, and sorted keys. It also includes intentional errors for duplicate keys and non-orderable sorted-map keys.

Project knowledge

• System requirements
• Architecture
• The implemented grammar
• The command-line host and reporting
• The knowledge, graph and semantic search domains
• HQL overview
• HyperMarkDown integration
• Open design questions
• Bootstrap requirements

Scaffolded with grem's Rust template using grem init . -t rust --name hql in an empty hql directory.

License

MIT — see LICENSE.