diff --git a/CLAUDE.md b/CLAUDE.md index 4b43a27..d9c43c6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -5,12 +5,13 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co ## Commands ```bash -npm test # run all tests once +npm test # run all tests once (vitest, typecheck enabled) npm run test:watch # watch mode -npm run coverage # coverage report +npm run test:coverage # coverage report +npm run test:types # check the published type resolution (attw) npm run build # compile to dist/ -npm run watch # build in watch mode +npm run build:watch # build in watch mode npm run lint # oxlint npm run lint:fix # auto-fix lint issues @@ -29,11 +30,11 @@ npx vitest run src/combinators/map.test.ts ## Architecture -ESM-only TypeScript parser combinator library. Five separate entry points, each compiled to its own `.mjs` bundle: +ESM-only TypeScript parser combinator library. Five entry points, built together so that code shared between them is emitted once as a common chunk rather than inlined into each: | Entry | Path | Purpose | | -------------------- | ---------------------------- | ----------------------------------------------------------------------------------------------- | -| `unitas` | `src/index.ts` → `src/core/` | Core types: `Parser`, `Result`, `success`, `failure`, `run`, `grammar`, `lazy`, `memoize` | +| `unitas` | `src/index.ts` → `src/core/` | Core types: `Parser`, `Result`, `success`/`failure`, `run`/`parse`, `grammar`, `lazy`, `memoize` | | `unitas/terminals` | `src/terminals/` | Factory functions that match input directly (`char`, `regex`, `string`, `satisfy`, …) | | `unitas/primitives` | `src/primitives/` | Pre-built parser instances (`digit`, `letters`, `whitespace`, …) | | `unitas/combinators` | `src/combinators/` | Higher-order parsers that compose other parsers (`map`, `choice`, `sequence`, `many`, …) | @@ -42,13 +43,20 @@ ESM-only TypeScript parser combinator library. Five separate entry points, each ### Core types (`src/core/`) ```typescript -type Parser = (input: string) => Result; +type Parser = (input: string, index?: number, ctx?: Context) => Result; type Result = Success | Failure; -type Success = { ok: true; value: T; remaining: string }; -type Failure = { ok: false; error?: string }; +type Success = { ok: true; value: T; index: number }; +type Failure = { ok: false; index: number; expected: readonly string[] }; +type Context = { furthest: number; expected: readonly string[] }; ``` -`grammar` enables mutual recursion — rules receive a proxy object `p` so they can reference sibling rules by name without forward declarations. `lazy` defers evaluation for self-referential parsers. +Parsers read `input` from an offset and never slice it, so a position stays meaningful all the way up to the error message. + +`ctx` is the parse context, created per `run`/`parse`. Failures record themselves into it via `failure(ctx, …)`, and because it only moves forward, backtracking cannot lose what a discarded branch expected — that is what lets an error point at the furthest position reached rather than wherever parsing stopped. **Any parser you call must be passed `ctx`**; omitting it still parses correctly but silently degrades the error message. `src/core/propagation.test.ts` checks every call site for this. + +`not` and `label` deliberately run their inner parser against an isolated context, since they suppress the failure rather than report it. + +`grammar` enables mutual recursion — rules receive a proxy object `p` so they can reference sibling rules by name without forward declarations, and a left-recursive rule is reported by name instead of overflowing the stack. `lazy` defers evaluation for self-referential parsers. ### File conventions @@ -71,4 +79,4 @@ type Failure = { ok: false; error?: string }; - `src/**/*.test.ts` — unit tests co-located with source (vitest, typecheck enabled) - `test/examples.test.ts` — generated from README examples, do not edit by hand -- `src/combinators/node.test.ts` — newly added, covers `node` (alias for `ast`) +- `src/core/propagation.test.ts` — guards that every parser call site forwards `ctx`; covers new files automatically diff --git a/README.md b/README.md index 7d52ae5..fea4479 100644 --- a/README.md +++ b/README.md @@ -33,8 +33,9 @@ run(csv.row, '"a,b",c'); // ['a,b', 'c'] - **Zero dependencies**, ESM-only, fully typed - **Tree-shakeable** — five entry points, import only what you touch -- **126 composable exports** — grouped by what they do, indexed below +- **134 composable exports** — grouped by what they do, indexed below - **Mutual recursion out of the box** via `grammar`, no forward declarations +- **Errors that point at the problem** — line, column and what was expected > **Note:** This library is in active development. The API may change before v1.0.0. @@ -43,12 +44,14 @@ run(csv.row, '"a,b",c'); // ['a,b', 'c'] - [Entry points](#entry-points) - [Which function do I need?](#which-function-do-i-need) - [Core concepts](#core-concepts) +- [Error reporting](#error-reporting) - [More examples](#more-examples) - [API index](#api-index) +- [Migrating from 0.2.x](#migrating-from-02x) ## Entry points -Nothing is re-exported across entry points, so your bundler only ever sees what you import. +Nothing is re-exported across entry points, so your bundler only ever sees what you import. Code shared between them ships once, however many you import from. | Import from | Contains | | -------------------- | ------------------------------------------------------------------ | @@ -78,7 +81,7 @@ The library is broad on purpose: each function does one small thing, so grammars | a character class | `letter`, `digit`, `alphaNum`, `hexDigit`, `octDigit`, `lowercase`, `uppercase`, `anyChar` | | whitespace or a line break | `whitespace`, `space`, `tab`, `nl`, `crlf`, `eol`, `eof` | -Most primitives come in a singular and a plural form: `letter` matches one, `letters` matches a run of them. The same holds for `digit`/`digits`, `alphaNum`/`alphaNums`, `hexDigit`/`hexDigits`, `octDigit`/`octDigits`, `lowercase`/`lowercases`, `uppercase`/`uppercases`, `space`/`spaces` and `whitespace`/`whitespaces`. Need to know how much input is left? `position`. +Most primitives come in a singular and a plural form: `letter` matches one, `letters` matches a run of them. The same holds for `digit`/`digits`, `alphaNum`/`alphaNums`, `hexDigit`/`hexDigits`, `octDigit`/`octDigits`, `lowercase`/`lowercases`, `uppercase`/`uppercases`, `space`/`spaces` and `whitespace`/`whitespaces`. Need to know how far into the input you are? `position`. ### Repeating @@ -127,54 +130,61 @@ Most primitives come in a singular and a plural form: `letter` matches one, `let ### Grammars, recursion and errors -| I want to… | Reach for | -| ------------------------------------------ | ------------------------------------------------------ | -| let rules reference each other by name | `grammar` | -| refer to a parser defined further down | `lazy` | -| cache results and avoid exponential blowup | `memoize` | -| attach a readable error message | `label` | -| parse infix operators with precedence | `chainLeft`, `chainLeft1`, `chainRight`, `chainRight1` | -| parse prefix / postfix operators | `prefix`, `postfix` | -| handle both outcomes of a `Result` | `match` | +| I want to… | Reach for | +| -------------------------------------------- | ------------------------------------------------------ | +| let rules reference each other by name | `grammar` | +| refer to a parser defined further down | `lazy` | +| cache results and avoid exponential blowup | `memoize` | +| attach a readable error message | `label` | +| parse without throwing, with line and column | `parse` | +| turn an offset into a line and column | `locate` | +| render a failure with a caret under it | `format`, `ParseError` | +| record what a hand-written parser expected | `context`, `record`, `union` | +| parse infix operators with precedence | `chainLeft`, `chainLeft1`, `chainRight`, `chainRight1` | +| parse prefix / postfix operators | `prefix`, `postfix` | +| handle both outcomes of a `Result` | `match` | ## Core concepts ### The `Parser` type -A `Parser` is a function from an input string to a `Result`. That is the whole abstraction — everything else in this library either produces one or wraps one. +A `Parser` reads `input` from an offset and returns a `Result`. That is the whole abstraction — everything else in this library either produces one or wraps one. Both extra arguments are optional, so a parser can still be called with just an input. ```typescript -type Parser = (input: string) => Result; +type Parser = (input: string, index?: number, ctx?: Context) => Result; ``` +Parsers never slice the input. They pass the same string down and move an integer instead, which is what lets an error know where in the original source it happened. + +`ctx` is the parse context, created by `run` and `parse`. It is where error information accumulates — see [Error reporting](#error-reporting). If you write your own combinator, pass it on to every parser you call. + ### The `Result` type ```typescript -type Success = { ok: true; value: T; remaining: string }; -type Failure = { ok: false; error?: string }; +type Success = { ok: true; value: T; index: number }; +type Failure = { ok: false; index: number; expected: readonly string[] }; type Result = Success | Failure; ``` ```typescript -{ ok: true, value: 'hello', remaining: ' world' } - │ │ │ - │ │ └── what is left to parse +{ ok: true, value: 'hello', index: 5 } + │ │ │ + │ │ └── how far this parser got │ └── the parsed value └── always true for success ``` -`remaining` is the important part: it is how input gets consumed and how parsers chain. A failure carries an optional message, which you can always supply later with `label`. +`index` is the important part: it is how input gets consumed and how parsers chain. A failure reports the same `index`, plus the set of things that would have matched there: ```typescript -{ ok: false } // generic failure -{ ok: false, error: 'expected a' } // failure with a message +{ ok: false, index: 6, expected: ['digit'] } ``` ### Backtracking is free -Note what a `Failure` does _not_ carry: a position. There is nowhere to record how much input a failed parser got through, and combinators hand every alternative the same string they started with, so a branch that fails can never leave the cursor moved. `choice(a, b)` always offers `b` the full input, however far `a` got. +Combinators hand every alternative the same offset they started with, so a branch that fails can never leave the cursor moved. `choice(a, b)` always offers `b` the same starting offset, however far `a` got. -If you are coming from Parsec, this is why there is no `try`/`attempt` here — backtracking is unconditional, so there is nothing to opt into. +If you are coming from Parsec, this is why there is no `try`/`attempt` here — backtracking is unconditional, so there is nothing to opt into. What a failed branch _does_ leave behind is its expectation, recorded in the parse context at the offset it reached, so a later error can still mention it. ### Writing one by hand @@ -183,15 +193,30 @@ Most of the time you compose existing pieces, but nothing stops you from droppin ```typescript import { create, success, failure } from 'unitas'; -const parser = create((input) => { - if (input.startsWith('hello')) { - return success('hello', input.slice(5)); +const parser = create((input, index = 0, ctx) => { + if (input.startsWith('hello', index)) { + return success('hello', index + 5); } - return failure('expected "hello"'); + return failure(ctx, index, '"hello"'); }); ``` +Passing `ctx` to `failure` is what lets this parser's expectation appear in an error message even when a `choice` backtracks past it. Do the same for any parser you call: + +```typescript +const twice = (parser: Parser) => + create<[T, T]>((input, index = 0, ctx) => { + const first = parser(input, index, ctx); + if (!first.ok) return first; + + const second = parser(input, first.index, ctx); + if (!second.ok) return second; + + return success([first.value, second.value], second.index); + }); +``` + ### Grammars `grammar` hands every rule a proxy `p`, so rules can reference each other — including themselves — without forward declarations or `lazy` boilerplate. @@ -222,6 +247,60 @@ run(g.expr, '1+2+3'); // 6 run(g.expr, '(1+2)'); // 3 ``` +## Error reporting + +When a parse fails, `run` throws a `ParseError` that says where and what was expected: + +```typescript +import { run } from 'unitas'; +import { separatedBy, sequence } from 'unitas/combinators'; +import { char } from 'unitas/terminals'; +import { digits, letters } from 'unitas/primitives'; + +const pair = sequence(letters, char('='), digits); +const config = separatedBy(pair, char('\n')); + +run(config, 'host=1\nport=8080\ndebug=yes'); +// ParseError: 3:7 expected digit, found 'y' +// 3 | debug=yes +// | ^ +``` + +The position is the _furthest_ offset any branch reached, not wherever the parse happened to stop. That distinction is what makes the message useful: `separatedBy` succeeded with two pairs and left the rest unconsumed, but the error still points at the `y` that actually broke it. + +Alternatives are reported together, since `choice` unions the expectations of every branch it tried at the same offset: + +```typescript +run(sequence(choice(string('let'), string('const')), char(' ')), 'lot x'); +// ParseError: 1:1 expected 'const' or 'let', found 'l' +// 1 | lot x +// | ^ +``` + +Use `parse` when you would rather branch on the failure than catch it: + +```typescript +import { parse } from 'unitas'; + +const result = parse(config, 'host=1\nport=oops'); + +if (!result.ok) { + result.line; // 2 + result.column; // 6 + result.expected; // ['digit'] + result.message; // the formatted excerpt above +} +``` + +`label` replaces the expectations of a whole parser with one description, so errors can talk about your grammar instead of its characters: + +```typescript +import { label } from 'unitas'; + +run(label(pair, 'a key=value pair'), '!!!'); +// ParseError: 1:1 expected a key=value pair, found '!' +``` + ## More examples **JSON value** — a tagged union of scalars @@ -305,19 +384,27 @@ One line per export. Every name links to its full description and runnable examp ### Core — `unitas` -Types, constructors and the grammar runner. **9** exports — [full reference →][api-core] - -| | | -| ------------------------- | --------------------------------------------------------------------------- | -| [`failure`][core-failure] | Creates a failed result with an optional error message. | -| [`grammar`][core-grammar] | Creates a recursive grammar where rules can reference each other. | -| [`label`][core-label] | Labels a parser with a custom error message on failure. | -| [`lazy`][core-lazy] | Defers parser creation, useful for recursive grammars. | -| [`match`][core-match] | Pattern matching on a Result to handle success and failure cases. | -| [`memoize`][core-memoize] | Memoizes a parser to cache results by input string. | -| [`parser`][core-parser] | Creates a parser from a parser function. | -| [`run`][core-run] | Runs a parser and returns the value, throws on failure or unconsumed input. | -| [`success`][core-success] | Creates a successful result with a value and remaining input. | +Types, constructors and the grammar runner. **17** exports — [full reference →][api-core] + +| | | +| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- | +| [`context`][core-context] | Creates a fresh context. `run` and `parse` make one per call. | +| [`create`][core-create] | Creates a parser from a parser function. | +| [`failure`][core-failure] | Creates a failed result at an offset and notes it in the parse context. | +| [`format`][core-format] | Renders a parse failure as a source excerpt with a caret under the offset. | +| [`grammar`][core-grammar] | Creates a recursive grammar where rules can reference each other. | +| [`label`][core-label] | Replaces the expectations of a parser with a single description. | +| [`lazy`][core-lazy] | Defers parser creation, useful for recursive grammars. | +| [`locate`][core-locate] | Converts a character offset into a 1-based line and column. | +| [`match`][core-match] | Pattern matching on a Result to handle success and failure cases. | +| [`memoize`][core-memoize] | Memoizes a parser so each offset is parsed at most once (packrat caching). | +| [`parse`][core-parse] | Runs a parser over the whole input without throwing. | +| [`ParseError`][core-parseerror] | Error thrown by `run` when a parse fails, carrying the offset, the 1-based line/column and the set of expectations at that offset. | +| [`record`][core-record] | Notes what was expected at an offset, keeping only the furthest one seen. | +| [`reject`][core-reject] | Creates a failed result from an expectation list that is already allocated, avoiding a fresh array on every failure. | +| [`run`][core-run] | Runs a parser over the whole input and returns the value. | +| [`success`][core-success] | Creates a successful result with a value and the offset reached in the input. | +| [`union`][core-union] | Adds the entries of `b` to `a`, returning `a` itself when it already covers them. | ### Terminals — `unitas/terminals` @@ -345,109 +432,109 @@ Ready-made parsers for the usual suspects. **33** exports — [full reference | | | | --------------------------------------- | ------------------------------------------------------------------------------------ | | [`alphaNum`][primitives-alphanum] | Parse a single alphanumeric character. | -| [`alphaNums`][primitives-alphanums] | Parse one or more alphanumeric characters. | +| [`alphaNums`][primitives-alphanums] | Parse one or more alphanumeric characters as a string. | | [`anyChar`][primitives-anychar] | Parse any single character. | | [`bool`][primitives-bool] | Parse a boolean literal. | | [`crlf`][primitives-crlf] | Parse CRLF line ending. | | [`digit`][primitives-digit] | Parse a single digit and return as number. | -| [`digits`][primitives-digits] | Parse one or more digits and return as number. | -| [`eof`][primitives-eof] | Parse end of file (succeeds only on empty input). | +| [`digits`][primitives-digits] | Parse one or more digits and return as a number. | +| [`eof`][primitives-eof] | Match the end of the input. | | [`eol`][primitives-eol] | Parse end of line (\\n, \\r\\n, or EOF). | | [`float`][primitives-float] | Parse a floating point number. | | [`hexDigit`][primitives-hexdigit] | Parse a single hexadecimal digit. | -| [`hexDigits`][primitives-hexdigits] | Parse one or more hexadecimal digits. | +| [`hexDigits`][primitives-hexdigits] | Parse one or more hex digits as a string. | | [`identifier`][primitives-identifier] | Parse an identifier — starts with letter or underscore, no leading digit, no hyphen. | | [`integer`][primitives-integer] | Parse a signed integer. | | [`letter`][primitives-letter] | Parse a single letter. | -| [`letters`][primitives-letters] | Parse one or more letters. | +| [`letters`][primitives-letters] | Parse one or more letters as a string. | | [`line`][primitives-line] | Parse until end of line. | | [`literal`][primitives-literal] | Parse a word-like value including hyphens. | | [`lowercase`][primitives-lowercase] | Parse a single lowercase letter. | -| [`lowercases`][primitives-lowercases] | Parses one or more lowercase letters. | +| [`lowercases`][primitives-lowercases] | Parse one or more lowercase letters as a string. | | [`nl`][primitives-nl] | Parse a newline character. | | [`number`][primitives-number] | Parse an integer or float. | | [`octDigit`][primitives-octdigit] | Parse a single octal digit. | -| [`octDigits`][primitives-octdigits] | Parse one or more octal digits. | -| [`position`][primitives-position] | Get current position (remaining input length). | -| [`rest`][primitives-rest] | Parse the rest of the input. | +| [`octDigits`][primitives-octdigits] | Parse one or more octal digits as a string. | +| [`position`][primitives-position] | Get the current offset into the input, without consuming anything. | +| [`rest`][primitives-rest] | Consume and return everything left in the input. | | [`space`][primitives-space] | Parse a single space character. | -| [`spaces`][primitives-spaces] | Parse one or more space characters. | +| [`spaces`][primitives-spaces] | Parse one or more spaces as a string. | | [`tab`][primitives-tab] | Parse tab character. | | [`uppercase`][primitives-uppercase] | Parses a single uppercase letter. | -| [`uppercases`][primitives-uppercases] | Parses one or more uppercase letters. | +| [`uppercases`][primitives-uppercases] | Parse one or more uppercase letters as a string. | | [`whitespace`][primitives-whitespace] | Parses a single whitespace character. | -| [`whitespaces`][primitives-whitespaces] | Parses one or more whitespace characters. | +| [`whitespaces`][primitives-whitespaces] | Parse one or more whitespaces as a string. | ### Combinators — `unitas/combinators` Take parsers, return a new parser. **64** exports — [full reference →][api-combinators] -| | | -| -------------------------------------------------------- | --------------------------------------------------------------------------- | -| [`bind`][combinators-bind] | Chain parsers where the second parser depends on the first result. | -| [`braced`][combinators-braced] | Parse content surrounded by braces. | -| [`bracketed`][combinators-bracketed] | Parse content surrounded by brackets. | -| [`chainLeft`][combinators-chainleft] | Chain left-associative operations (right-to-left for same precedence). | -| [`chainLeft1`][combinators-chainleft1] | Chain left-associative operations (fails on empty input). | -| [`chainRight`][combinators-chainright] | Chain right-associative operations (right-to-left grouping). | -| [`chainRight1`][combinators-chainright1] | Chain right-associative operations (fails on empty input). | -| [`choice`][combinators-choice] | Try each parser in order, return first success. | -| [`concat`][combinators-concat] | Join string array parser result into a single string. | -| [`consume`][combinators-consume] | Consume input but discard the result (return null). | -| [`endBy`][combinators-endby] | Zero or more items separated and ending with terminator. | -| [`endBy1`][combinators-endby1] | One or more items separated and ending with terminator. | -| [`exactly`][combinators-exactly] | Parse exactly n occurrences. | -| [`first`][combinators-first] | Extract the first element from a parser result array. | -| [`flag`][combinators-flag] | Return true if parser succeeds, false otherwise. | -| [`fold`][combinators-fold] | Parse zero or more and fold into a single value. | -| [`fold1`][combinators-fold1] | Parse one or more and fold into a single value. | -| [`foldRight`][combinators-foldright] | Parse zero or more and fold right-to-left. | -| [`foldRight1`][combinators-foldright1] | Parse one or more and fold right-to-left. | -| [`fuse`][combinators-fuse] | Fuse multiple string parsers into a single one. | -| [`guard`][combinators-guard] | Conditionally apply parser based on a condition. | -| [`inner`][combinators-inner] | Extract inner value from surrounded content (like inner of braced). | -| [`interleaved`][combinators-interleaved] | Parse items with interleaved separators. | -| [`last`][combinators-last] | Extract the last element from a parser result array. | -| [`left`][combinators-left] | Keep only the left result from a sequence. | -| [`lexeme`][combinators-lexeme] | Parser that consumes trailing whitespace. | -| [`many`][combinators-many] | Zero or more occurrences (never fails). | -| [`many1`][combinators-many1] | One or more occurrences (fails if no matches). | -| [`manyAtLeast`][combinators-manyatleast] | Parse at least n occurrences. | -| [`manyAtMost`][combinators-manyatmost] | Parse at most n occurrences. | -| [`manyBetween`][combinators-manybetween] | Parse between min and max occurrences. | -| [`manyTill`][combinators-manytill] | Parse zero or more until terminator matches. | -| [`map`][combinators-map] | Transform the parsed value. | -| [`node`][combinators-node] | Create a node from parser fields. | -| [`not`][combinators-not] | Succeed if parser fails (without consuming input). | -| [`nth`][combinators-nth] | Extract the nth element from a parser result array. | -| [`optional`][combinators-optional] | Make parser optional (return null on failure, without consuming input). | -| [`optionalConsume`][combinators-optionalconsume] | Optionally consume input (always succeeds, returns void). | -| [`optionalSeparatedBy`][combinators-optionalseparatedby] | Zero or more items separated by a separator, with optional null values. | -| [`outer`][combinators-outer] | Extract outer values from a sequence of 3 parsers (skip middle). | -| [`padded`][combinators-padded] | Parse content with optional whitespace on both sides. | -| [`parenthesized`][combinators-parenthesized] | Parse content surrounded by parentheses. | -| [`peek`][combinators-peek] | Parse without consuming input. | -| [`postfix`][combinators-postfix] | Parse postfix operators (chains atom with operators that return functions). | -| [`prefix`][combinators-prefix] | Parse prefix operators (like - in -5). | -| [`pure`][combinators-pure] | Always return a value without consuming input. | -| [`quoted`][combinators-quoted] | Parse content surrounded by single or double quotes. | -| [`recover`][combinators-recover] | Use fallback value when parser fails. | -| [`right`][combinators-right] | Keep only the right result from a sequence. | -| [`separatedBy`][combinators-separatedby] | Zero or more items separated by a separator. | -| [`separatedBy1`][combinators-separatedby1] | One or more items separated by a separator. | -| [`separatedEndBy`][combinators-separatedendby] | Zero or more items separated by and ending with a terminator. | -| [`separatedEndBy1`][combinators-separatedendby1] | One or more items separated by and ending with a terminator. | -| [`separatedUntil`][combinators-separateduntil] | Parse items separated by separator until terminator matches. | -| [`sequence`][combinators-sequence] | Parse a sequence of parsers and return all results as an array. | -| [`skip`][combinators-skip] | Skip a parser n times. | -| [`skipMany`][combinators-skipmany] | Skip zero or more occurrences (never fails, returns null). | -| [`skipMany1`][combinators-skipmany1] | Skip one or more occurrences (fails if no matches). | -| [`surrounded`][combinators-surrounded] | Parse content surrounded by delimiters. | -| [`unless`][combinators-unless] | Parse unless condition is true (inverse of guard). | -| [`until`][combinators-until] | Parse until terminator matches (fails if terminator never matches). | -| [`validate`][combinators-validate] | Validate parsed value with a predicate. | -| [`value`][combinators-value] | Replace parsed value with a constant. | -| [`when`][combinators-when] | Branch on a boolean parser result. | +| | | +| -------------------------------------------------------- | ------------------------------------------------------------------------------- | +| [`bind`][combinators-bind] | Chain parsers where the second parser depends on the first result. | +| [`braced`][combinators-braced] | Parse content surrounded by braces. | +| [`bracketed`][combinators-bracketed] | Parse content surrounded by brackets. | +| [`chainLeft`][combinators-chainleft] | Chain left-associative operations (right-to-left for same precedence). | +| [`chainLeft1`][combinators-chainleft1] | Chain left-associative operations (fails on empty input). | +| [`chainRight`][combinators-chainright] | Chain right-associative operations. | +| [`chainRight1`][combinators-chainright1] | Chain right-associative operations (fails on empty input). | +| [`choice`][combinators-choice] | Try each parser in order, return first success. | +| [`concat`][combinators-concat] | Join string array parser result into a single string. | +| [`consume`][combinators-consume] | Consume input but discard the result (return null). | +| [`endBy`][combinators-endby] | Zero or more items separated and ending with terminator. | +| [`endBy1`][combinators-endby1] | One or more items separated and ending with terminator. | +| [`exactly`][combinators-exactly] | Parse exactly n occurrences. | +| [`first`][combinators-first] | Extract the first element from a parser result array. | +| [`flag`][combinators-flag] | Return true if parser succeeds, false otherwise. | +| [`fold`][combinators-fold] | Fold zero or more occurrences into a single value. | +| [`fold1`][combinators-fold1] | Fold one or more occurrences into a single value. | +| [`foldRight`][combinators-foldright] | Fold zero or more occurrences from the right into a single value. | +| [`foldRight1`][combinators-foldright1] | Fold one or more occurrences from the right into a single value. | +| [`fuse`][combinators-fuse] | Fuse multiple string parsers into a single one. | +| [`guard`][combinators-guard] | Conditionally apply parser based on a condition. | +| [`inner`][combinators-inner] | Extract inner value from surrounded content (like inner of braced). | +| [`interleaved`][combinators-interleaved] | Parse items separated by separators, keeping both in the result. | +| [`last`][combinators-last] | Extract the last element from a parser result array. | +| [`left`][combinators-left] | Keep only the left result from a sequence. | +| [`lexeme`][combinators-lexeme] | Parser that consumes trailing whitespace. | +| [`many`][combinators-many] | Zero or more occurrences (never fails). | +| [`many1`][combinators-many1] | One or more occurrences. | +| [`manyAtLeast`][combinators-manyatleast] | Parse at least n occurrences. | +| [`manyAtMost`][combinators-manyatmost] | Parse at most n occurrences (never fails). | +| [`manyBetween`][combinators-manybetween] | Parse between min and max occurrences. | +| [`manyTill`][combinators-manytill] | Parse zero or more until terminator matches. | +| [`map`][combinators-map] | Transform a parsed value through one or more functions. | +| [`node`][combinators-node] | Create a node from parser fields. | +| [`not`][combinators-not] | Succeed if parser fails (without consuming input). | +| [`nth`][combinators-nth] | Extract the nth element from a parser result array. | +| [`optional`][combinators-optional] | Make parser optional (return null on failure, without consuming input). | +| [`optionalConsume`][combinators-optionalconsume] | Consume input if the parser matches, discarding the result. | +| [`optionalSeparatedBy`][combinators-optionalseparatedby] | Parse items separated by a separator, allowing empty slots. | +| [`outer`][combinators-outer] | Extract outer values from a sequence of 3 parsers (skip middle). | +| [`padded`][combinators-padded] | Parse content surrounded by optional whitespace. | +| [`parenthesized`][combinators-parenthesized] | Parse content surrounded by parentheses. | +| [`peek`][combinators-peek] | Look ahead without consuming input. | +| [`postfix`][combinators-postfix] | Apply zero or more postfix operators to an atom. | +| [`prefix`][combinators-prefix] | Apply zero or more prefix operators to an atom. | +| [`pure`][combinators-pure] | Always succeed with a value without consuming input. | +| [`quoted`][combinators-quoted] | Parse content surrounded by single or double quotes. | +| [`recover`][combinators-recover] | Use fallback value when parser fails. | +| [`right`][combinators-right] | Keep only the right result from a sequence. | +| [`separatedBy`][combinators-separatedby] | Parse zero or more items separated by a separator. | +| [`separatedBy1`][combinators-separatedby1] | Parse one or more items separated by a separator. | +| [`separatedEndBy`][combinators-separatedendby] | Parse zero or more items separated by a separator, allowing a trailing one. | +| [`separatedEndBy1`][combinators-separatedendby1] | Parse one or more items separated by a separator, allowing a trailing one. | +| [`separatedUntil`][combinators-separateduntil] | Parse items separated by a separator, up to a terminator. | +| [`sequence`][combinators-sequence] | Parse a sequence of parsers and return all results as an array. | +| [`skip`][combinators-skip] | Skip a parser n times. | +| [`skipMany`][combinators-skipmany] | Skip zero or more occurrences (never fails, returns null). | +| [`skipMany1`][combinators-skipmany1] | Skip one or more occurrences (fails if no matches). | +| [`surrounded`][combinators-surrounded] | Parse content surrounded by delimiters. | +| [`unless`][combinators-unless] | Parse unless condition is true (inverse of guard). | +| [`until`][combinators-until] | Parse zero or more until terminator matches, leaving the terminator unconsumed. | +| [`validate`][combinators-validate] | Validate parsed value with a predicate. | +| [`value`][combinators-value] | Replace parsed value with a constant. | +| [`when`][combinators-when] | Branch on a boolean parser result. | ### Utils — `unitas/utils` @@ -465,15 +552,23 @@ Plain helpers for `map` callbacks. **8** exports — [full reference →][api-ut | [`spread`][utils-spread] | Collect spread arguments into an array. | [api-core]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md +[core-context]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#context +[core-create]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#create [core-failure]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#failure +[core-format]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#format [core-grammar]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#grammar [core-label]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#label [core-lazy]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#lazy +[core-locate]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#locate [core-match]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#match [core-memoize]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#memoize -[core-parser]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#parser +[core-parse]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#parse +[core-parseerror]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#parseerror +[core-record]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#record +[core-reject]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#reject [core-run]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#run [core-success]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#success +[core-union]: https://github.com/sovrin/unitas/blob/master/doc/api/core.md#union [api-terminals]: https://github.com/sovrin/unitas/blob/master/doc/api/terminals.md [terminals-char]: https://github.com/sovrin/unitas/blob/master/doc/api/terminals.md#char [terminals-charof]: https://github.com/sovrin/unitas/blob/master/doc/api/terminals.md#charof @@ -596,6 +691,62 @@ Plain helpers for `map` callbacks. **8** exports — [full reference →][api-ut [utils-shift]: https://github.com/sovrin/unitas/blob/master/doc/api/utils.md#shift [utils-spread]: https://github.com/sovrin/unitas/blob/master/doc/api/utils.md#spread +## Migrating from 0.2.x + +`0.3.0` changes how a parser reports where it got to. Parsers now take an offset into the input instead of receiving a sliced suffix, which is what makes line/column error reporting possible. + +**If you only compose the built-in parsers, nothing changes.** `run`, `grammar`, `map`, `choice` and everything built on them behave the same, and both new arguments are optional, so `myParser('input')` still works. + +Two things do change for everyone: + +```typescript +// reading a result +result.remaining; // 0.2.x +input.slice(result.index); // 0.3.0 — or just use result.index + +// a failed run +run(p, 'bad'); // 0.2.x: Error: Parsing failed: Unexpected error +run(p, 'bad'); // 0.3.0: ParseError, with .line, .column and .expected +``` + +If you write parsers by hand, the shape changed: + +```typescript +// 0.2.x +create((input) => { + if (input.startsWith('hello')) { + return success('hello', input.slice(5)); + } + + return failure('expected "hello"'); +}); + +// 0.3.0 +create((input, index = 0, ctx) => { + if (input.startsWith('hello', index)) { + return success('hello', index + 5); + } + + return failure(ctx, index, '"hello"'); +}); +``` + +The rest, in full: + +| 0.2.x | 0.3.0 | +| ---------------------------------- | --------------------------------------------------------------- | +| `Parser = (input) => Result` | `Parser = (input, index?, ctx?) => Result` | +| `success(value, remaining)` | `success(value, index)` | +| `failure()` / `failure(message)` | `failure(ctx, index, ...expected)` | +| `{ ok: true, value, remaining }` | `{ ok: true, value, index }` | +| `{ ok: false, error?: string }` | `{ ok: false, index, expected }` | +| `match`'s `failure: (error) => …` | `failure: (index, expected) => …` | +| `position` → remaining length | `position` → offset from the start | +| `label(p, 'x')` → `'expected x'` | `label(p, 'x')` → `expected: ['x']`, only when nothing consumed | +| `regex(/^\w+/)` | same, but sticky — a leading `^` is stripped unless multiline | + +New in this release: `parse` (non-throwing), `ParseError`, `locate`, `format`, and `context`/`record` for hand-written parsers. `memoize` is now keyed by offset rather than by the remaining input, so its cache entries are cheap and are dropped when a different input is parsed. Left-recursive `grammar` rules now throw a named error instead of overflowing the stack. + ## Contributing ```bash diff --git a/bin/generate-readme.mjs b/bin/generate-readme.mjs index 1e79e1e..0fb3b1f 100644 --- a/bin/generate-readme.mjs +++ b/bin/generate-readme.mjs @@ -13,10 +13,19 @@ const readmePath = path.join(rootDir, 'README.md'); const COMMENT_REGEX = /\/\*\*([\s\S]*?)\*\//g; +const DECLARATION_REGEX = + /^\s*export\s+(?:const|function|class|let|var)\s+([A-Za-z_$][\w$]*)/; + const CHEAT_SHEET_HEADING = '## Which function do I need?'; /** Covered by the "Core concepts" prose instead of the cheat sheet. */ -const CHEAT_SHEET_EXEMPT = new Set(['failure', 'parser', 'run', 'success']); +const CHEAT_SHEET_EXEMPT = new Set([ + 'create', + 'failure', + 'reject', + 'run', + 'success', +]); /** * Module metadata. Order here drives the order of the API index in the README @@ -27,44 +36,62 @@ const MODULES = [ key: 'core', title: 'Core', entry: 'unitas', + imports: 'run, parse, grammar', tagline: 'Types, constructors and the grammar runner.', - intro: - 'The core entry point holds everything a parser is built from: the `Parser` and `Result` types, the `success`/`failure` constructors, and the tools for running and wiring parsers together (`run`, `grammar`, `lazy`, `memoize`).', + intro: 'The core entry point holds everything a parser is built from: the `Parser` and `Result` types, the `success`/`failure` constructors, and the tools for running and wiring parsers together (`run`, `grammar`, `lazy`, `memoize`).', }, { key: 'terminals', title: 'Terminals', entry: 'unitas/terminals', + imports: 'char, string, regex', tagline: 'Factories that match the input string directly.', - intro: - 'Terminals are the leaves of a grammar. They do not take other parsers — they inspect the input string themselves. Each one is a factory: call it with what you want to match and it hands back a `Parser`.', + intro: 'Terminals are the leaves of a grammar. They do not take other parsers — they inspect the input string themselves. Each one is a factory: call it with what you want to match and it hands back a `Parser`.', }, { key: 'primitives', title: 'Primitives', entry: 'unitas/primitives', + imports: 'digit, letters, whitespace', tagline: 'Ready-made parsers for the usual suspects.', - intro: - 'Primitives are parser *instances*, not factories. Where a terminal needs an argument (`char("a")`), a primitive is already a parser and can be passed straight to a combinator (`many(digit)`).', + intro: 'Primitives are parser *instances*, not factories. Where a terminal needs an argument (`char("a")`), a primitive is already a parser and can be passed straight to a combinator (`many(digit)`).', }, { key: 'combinators', title: 'Combinators', entry: 'unitas/combinators', + imports: 'map, choice, sequence', tagline: 'Take parsers, return a new parser.', - intro: - 'Combinators are the glue. Every one of them takes one or more parsers and returns a new parser, which is what lets a grammar stay a set of small, independently testable pieces.', + intro: 'Combinators are the glue. Every one of them takes one or more parsers and returns a new parser, which is what lets a grammar stay a set of small, independently testable pieces.', }, { key: 'utils', title: 'Utils', entry: 'unitas/utils', + imports: 'pick, join, pipe', tagline: 'Plain helpers for `map` callbacks.', - intro: - 'Utils are not parsers. They are small curried helpers meant to be dropped into a `map` callback so reshaping a result stays a one-liner instead of an arrow function.', + intro: 'Utils are not parsers. They are small curried helpers meant to be dropped into a `map` callback so reshaping a result stays a one-liner instead of an arrow function.', }, ]; +/** + * The exported name a doc block belongs to. One file may document several + * exports, so the file name is not enough — it would collide in the index and + * point every entry at the same anchor. + */ +function declaredName(source) { + const [, name] = source.match(DECLARATION_REGEX) || []; + + return name; +} + +/** + * `{@link foo}` means nothing to a markdown reader, so it becomes code. + */ +function resolveLinks(description) { + return description.replace(/\{@link\s+([^}\s|]+)[^}]*\}/g, '`$1`'); +} + function extractDocs(filePath) { const content = fs.readFileSync(filePath, 'utf-8'); const docs = []; @@ -98,10 +125,13 @@ function extractDocs(filePath) { } if (exampleLines.length > 0) { + const after = match.index + match[0].length; docs.push({ - description: descriptionParts.join(' '), + description: resolveLinks(descriptionParts.join(' ')), example: exampleLines.join('\n'), - name: basename(filePath, '.ts'), + name: + declaredName(content.slice(after)) ?? + basename(filePath, '.ts'), }); } } @@ -204,6 +234,7 @@ function generateApiPage(module, items) { return template .replaceAll('<$title>', module.title) .replaceAll('<$entry>', module.entry) + .replaceAll('<$imports>', module.imports) .replaceAll('<$intro>', module.intro) .replaceAll('<$count>', String(items.length)) .replaceAll('<$nav>', nav) @@ -294,8 +325,7 @@ function warnAboutCheatSheetGaps(template, docsByNamespace) { (docsByNamespace[key] || []) .map(({ name }) => name) .filter( - (name) => - !mentioned.has(name) && !CHEAT_SHEET_EXEMPT.has(name), + (name) => !mentioned.has(name) && !CHEAT_SHEET_EXEMPT.has(name), ), ); diff --git a/doc/README.template.md b/doc/README.template.md index 846254d..ca8fe9f 100644 --- a/doc/README.template.md +++ b/doc/README.template.md @@ -35,6 +35,7 @@ run(csv.row, '"a,b",c'); // ['a,b', 'c'] - **Tree-shakeable** — five entry points, import only what you touch - **<$total> composable exports** — grouped by what they do, indexed below - **Mutual recursion out of the box** via `grammar`, no forward declarations +- **Errors that point at the problem** — line, column and what was expected > **Note:** This library is in active development. The API may change before v1.0.0. @@ -43,20 +44,22 @@ run(csv.row, '"a,b",c'); // ['a,b', 'c'] - [Entry points](#entry-points) - [Which function do I need?](#which-function-do-i-need) - [Core concepts](#core-concepts) +- [Error reporting](#error-reporting) - [More examples](#more-examples) - [API index](#api-index) +- [Migrating from 0.2.x](#migrating-from-02x) ## Entry points -Nothing is re-exported across entry points, so your bundler only ever sees what you import. +Nothing is re-exported across entry points, so your bundler only ever sees what you import. Code shared between them ships once, however many you import from. -| Import from | Contains | -| ---------------------- | --------------------------------------------------------------------- | -| `unitas` | Types, `success`/`failure`, `run`, `grammar`, `lazy`, `memoize` | -| `unitas/terminals` | Factories that match input directly — `char`, `string`, `regex`, … | -| `unitas/primitives` | Ready-made parsers — `digit`, `letters`, `whitespace`, … | -| `unitas/combinators` | Parsers that take parsers — `map`, `choice`, `sequence`, `many`, … | -| `unitas/utils` | Helpers for `map` callbacks — `pick`, `join`, `pipe`, … | +| Import from | Contains | +| -------------------- | ------------------------------------------------------------------ | +| `unitas` | Types, `success`/`failure`, `run`, `grammar`, `lazy`, `memoize` | +| `unitas/terminals` | Factories that match input directly — `char`, `string`, `regex`, … | +| `unitas/primitives` | Ready-made parsers — `digit`, `letters`, `whitespace`, … | +| `unitas/combinators` | Parsers that take parsers — `map`, `choice`, `sequence`, `many`, … | +| `unitas/utils` | Helpers for `map` callbacks — `pick`, `join`, `pipe`, … | ## Which function do I need? @@ -64,117 +67,124 @@ The library is broad on purpose: each function does one small thing, so grammars ### Matching a single thing -| I want to match… | Reach for | -| ------------------------------------------- | ---------------------------------------------------- | -| an exact character or string | `char`, `string` | -| one of several strings (longest wins) | `oneOf` | -| any character in / not in a set | `charOf`, `stringOf`, `noneOf` | -| a character passing a predicate | `satisfy` | -| a regular expression | `regex` | -| a fixed number of characters, or a run | `take`, `takeWhile` | -| a keyword not glued to more word characters | `word` | -| a token, eating trailing whitespace | `token`, `lexeme` | -| a value — no need to build it | `number`, `integer`, `float`, `bool`, `identifier`, `literal`, `line`, `rest` | +| I want to match… | Reach for | +| ------------------------------------------- | ------------------------------------------------------------------------------------------ | +| an exact character or string | `char`, `string` | +| one of several strings (longest wins) | `oneOf` | +| any character in / not in a set | `charOf`, `stringOf`, `noneOf` | +| a character passing a predicate | `satisfy` | +| a regular expression | `regex` | +| a fixed number of characters, or a run | `take`, `takeWhile` | +| a keyword not glued to more word characters | `word` | +| a token, eating trailing whitespace | `token`, `lexeme` | +| a value — no need to build it | `number`, `integer`, `float`, `bool`, `identifier`, `literal`, `line`, `rest` | | a character class | `letter`, `digit`, `alphaNum`, `hexDigit`, `octDigit`, `lowercase`, `uppercase`, `anyChar` | -| whitespace or a line break | `whitespace`, `space`, `tab`, `nl`, `crlf`, `eol`, `eof` | +| whitespace or a line break | `whitespace`, `space`, `tab`, `nl`, `crlf`, `eol`, `eof` | -Most primitives come in a singular and a plural form: `letter` matches one, `letters` matches a run of them. The same holds for `digit`/`digits`, `alphaNum`/`alphaNums`, `hexDigit`/`hexDigits`, `octDigit`/`octDigits`, `lowercase`/`lowercases`, `uppercase`/`uppercases`, `space`/`spaces` and `whitespace`/`whitespaces`. Need to know how much input is left? `position`. +Most primitives come in a singular and a plural form: `letter` matches one, `letters` matches a run of them. The same holds for `digit`/`digits`, `alphaNum`/`alphaNums`, `hexDigit`/`hexDigits`, `octDigit`/`octDigits`, `lowercase`/`lowercases`, `uppercase`/`uppercases`, `space`/`spaces` and `whitespace`/`whitespaces`. Need to know how far into the input you are? `position`. ### Repeating -| I want to… | Reach for | -| -------------------------------------------- | ---------------------------------------------------- | -| repeat zero or more / one or more times | `many`, `many1` | -| repeat a bounded number of times | `exactly`, `manyAtLeast`, `manyAtMost`, `manyBetween` | -| repeat with a separator between items | `separatedBy`, `separatedBy1`, `optionalSeparatedBy` | -| …and allow a trailing separator | `separatedEndBy`, `separatedEndBy1` | -| …where every item ends with the separator | `endBy`, `endBy1`, `interleaved` | -| repeat until something else shows up | `manyTill`, `until`, `separatedUntil` | -| repeat but throw the values away | `skip`, `skipMany`, `skipMany1` | -| reduce the repetitions as I go | `fold`, `fold1`, `foldRight`, `foldRight1` | +| I want to… | Reach for | +| ----------------------------------------- | ----------------------------------------------------- | +| repeat zero or more / one or more times | `many`, `many1` | +| repeat a bounded number of times | `exactly`, `manyAtLeast`, `manyAtMost`, `manyBetween` | +| repeat with a separator between items | `separatedBy`, `separatedBy1`, `optionalSeparatedBy` | +| …and allow a trailing separator | `separatedEndBy`, `separatedEndBy1` | +| …where every item ends with the separator | `endBy`, `endBy1`, `interleaved` | +| repeat until something else shows up | `manyTill`, `until`, `separatedUntil` | +| repeat but throw the values away | `skip`, `skipMany`, `skipMany1` | +| reduce the repetitions as I go | `fold`, `fold1`, `foldRight`, `foldRight1` | ### Choosing and branching -| I want to… | Reach for | -| -------------------------------------------- | ---------------------------------------------------- | -| take the first alternative that matches | `choice` | -| succeed either way | `optional`, `optionalConsume`, `recover`, `flag` | -| succeed only when something *doesn't* match | `not` | -| gate on a boolean I already have | `guard`, `unless`, `when` | -| look ahead without consuming input | `peek` | +| I want to… | Reach for | +| ------------------------------------------- | ------------------------------------------------ | +| take the first alternative that matches | `choice` | +| succeed either way | `optional`, `optionalConsume`, `recover`, `flag` | +| succeed only when something _doesn't_ match | `not` | +| gate on a boolean I already have | `guard`, `unless`, `when` | +| look ahead without consuming input | `peek` | ### Sequencing and pulling values out -| I want to… | Reach for | -| -------------------------------------------- | ---------------------------------------------------- | -| run parsers in order and keep every result | `sequence` | -| keep only one side of a pair | `left`, `right` | -| keep specific positions of a sequence | `first`, `last`, `nth`, `pick` | -| drop the delimiters, keep the middle | `inner`, `outer` | -| glue string results together | `concat`, `fuse`, `join` | -| match content inside delimiters | `surrounded`, `parenthesized`, `braced`, `bracketed`, `quoted`, `padded` | +| I want to… | Reach for | +| ------------------------------------------ | ------------------------------------------------------------------------ | +| run parsers in order and keep every result | `sequence` | +| keep only one side of a pair | `left`, `right` | +| keep specific positions of a sequence | `first`, `last`, `nth`, `pick` | +| drop the delimiters, keep the middle | `inner`, `outer` | +| glue string results together | `concat`, `fuse`, `join` | +| match content inside delimiters | `surrounded`, `parenthesized`, `braced`, `bracketed`, `quoted`, `padded` | ### Shaping the result -| I want to… | Reach for | -| -------------------------------------------- | ---------------------------------------------------- | -| transform the parsed value | `map`, `pipe` | -| replace it with a constant | `value`, `pure`, `consume` | -| build a typed AST node | `node` | -| reject a value after parsing it | `validate` | -| choose the next parser from the last value | `bind` | -| tidy up arrays inside a `map` | `pick`, `filter`, `flatten`, `shift`, `pop`, `spread` | +| I want to… | Reach for | +| ------------------------------------------ | ----------------------------------------------------- | +| transform the parsed value | `map`, `pipe` | +| replace it with a constant | `value`, `pure`, `consume` | +| build a typed AST node | `node` | +| reject a value after parsing it | `validate` | +| choose the next parser from the last value | `bind` | +| tidy up arrays inside a `map` | `pick`, `filter`, `flatten`, `shift`, `pop`, `spread` | ### Grammars, recursion and errors -| I want to… | Reach for | -| -------------------------------------------- | ---------------------------------------------------- | -| let rules reference each other by name | `grammar` | -| refer to a parser defined further down | `lazy` | -| cache results and avoid exponential blowup | `memoize` | -| attach a readable error message | `label` | +| I want to… | Reach for | +| -------------------------------------------- | ------------------------------------------------------ | +| let rules reference each other by name | `grammar` | +| refer to a parser defined further down | `lazy` | +| cache results and avoid exponential blowup | `memoize` | +| attach a readable error message | `label` | +| parse without throwing, with line and column | `parse` | +| turn an offset into a line and column | `locate` | +| render a failure with a caret under it | `format`, `ParseError` | +| record what a hand-written parser expected | `context`, `record`, `union` | | parse infix operators with precedence | `chainLeft`, `chainLeft1`, `chainRight`, `chainRight1` | -| parse prefix / postfix operators | `prefix`, `postfix` | -| handle both outcomes of a `Result` | `match` | +| parse prefix / postfix operators | `prefix`, `postfix` | +| handle both outcomes of a `Result` | `match` | ## Core concepts ### The `Parser` type -A `Parser` is a function from an input string to a `Result`. That is the whole abstraction — everything else in this library either produces one or wraps one. +A `Parser` reads `input` from an offset and returns a `Result`. That is the whole abstraction — everything else in this library either produces one or wraps one. Both extra arguments are optional, so a parser can still be called with just an input. ```typescript -type Parser = (input: string) => Result; +type Parser = (input: string, index?: number, ctx?: Context) => Result; ``` +Parsers never slice the input. They pass the same string down and move an integer instead, which is what lets an error know where in the original source it happened. + +`ctx` is the parse context, created by `run` and `parse`. It is where error information accumulates — see [Error reporting](#error-reporting). If you write your own combinator, pass it on to every parser you call. + ### The `Result` type ```typescript -type Success = { ok: true; value: T; remaining: string }; -type Failure = { ok: false; error?: string }; +type Success = { ok: true; value: T; index: number }; +type Failure = { ok: false; index: number; expected: readonly string[] }; type Result = Success | Failure; ``` ```typescript -{ ok: true, value: 'hello', remaining: ' world' } - │ │ │ - │ │ └── what is left to parse +{ ok: true, value: 'hello', index: 5 } + │ │ │ + │ │ └── how far this parser got │ └── the parsed value └── always true for success ``` -`remaining` is the important part: it is how input gets consumed and how parsers chain. A failure carries an optional message, which you can always supply later with `label`. +`index` is the important part: it is how input gets consumed and how parsers chain. A failure reports the same `index`, plus the set of things that would have matched there: ```typescript -{ ok: false } // generic failure -{ ok: false, error: 'expected a' } // failure with a message +{ ok: false, index: 6, expected: ['digit'] } ``` ### Backtracking is free -Note what a `Failure` does *not* carry: a position. There is nowhere to record how much input a failed parser got through, and combinators hand every alternative the same string they started with, so a branch that fails can never leave the cursor moved. `choice(a, b)` always offers `b` the full input, however far `a` got. +Combinators hand every alternative the same offset they started with, so a branch that fails can never leave the cursor moved. `choice(a, b)` always offers `b` the same starting offset, however far `a` got. -If you are coming from Parsec, this is why there is no `try`/`attempt` here — backtracking is unconditional, so there is nothing to opt into. +If you are coming from Parsec, this is why there is no `try`/`attempt` here — backtracking is unconditional, so there is nothing to opt into. What a failed branch _does_ leave behind is its expectation, recorded in the parse context at the offset it reached, so a later error can still mention it. ### Writing one by hand @@ -183,15 +193,30 @@ Most of the time you compose existing pieces, but nothing stops you from droppin ```typescript import { create, success, failure } from 'unitas'; -const parser = create((input) => { - if (input.startsWith('hello')) { - return success('hello', input.slice(5)); +const parser = create((input, index = 0, ctx) => { + if (input.startsWith('hello', index)) { + return success('hello', index + 5); } - return failure('expected "hello"'); + return failure(ctx, index, '"hello"'); }); ``` +Passing `ctx` to `failure` is what lets this parser's expectation appear in an error message even when a `choice` backtracks past it. Do the same for any parser you call: + +```typescript +const twice = (parser: Parser) => + create<[T, T]>((input, index = 0, ctx) => { + const first = parser(input, index, ctx); + if (!first.ok) return first; + + const second = parser(input, first.index, ctx); + if (!second.ok) return second; + + return success([first.value, second.value], second.index); + }); +``` + ### Grammars `grammar` hands every rule a proxy `p`, so rules can reference each other — including themselves — without forward declarations or `lazy` boilerplate. @@ -222,6 +247,60 @@ run(g.expr, '1+2+3'); // 6 run(g.expr, '(1+2)'); // 3 ``` +## Error reporting + +When a parse fails, `run` throws a `ParseError` that says where and what was expected: + +```typescript +import { run } from 'unitas'; +import { separatedBy, sequence } from 'unitas/combinators'; +import { char } from 'unitas/terminals'; +import { digits, letters } from 'unitas/primitives'; + +const pair = sequence(letters, char('='), digits); +const config = separatedBy(pair, char('\n')); + +run(config, 'host=1\nport=8080\ndebug=yes'); +// ParseError: 3:7 expected digit, found 'y' +// 3 | debug=yes +// | ^ +``` + +The position is the _furthest_ offset any branch reached, not wherever the parse happened to stop. That distinction is what makes the message useful: `separatedBy` succeeded with two pairs and left the rest unconsumed, but the error still points at the `y` that actually broke it. + +Alternatives are reported together, since `choice` unions the expectations of every branch it tried at the same offset: + +```typescript +run(sequence(choice(string('let'), string('const')), char(' ')), 'lot x'); +// ParseError: 1:1 expected 'const' or 'let', found 'l' +// 1 | lot x +// | ^ +``` + +Use `parse` when you would rather branch on the failure than catch it: + +```typescript +import { parse } from 'unitas'; + +const result = parse(config, 'host=1\nport=oops'); + +if (!result.ok) { + result.line; // 2 + result.column; // 6 + result.expected; // ['digit'] + result.message; // the formatted excerpt above +} +``` + +`label` replaces the expectations of a whole parser with one description, so errors can talk about your grammar instead of its characters: + +```typescript +import { label } from 'unitas'; + +run(label(pair, 'a key=value pair'), '!!!'); +// ParseError: 1:1 expected a key=value pair, found '!' +``` + ## More examples **JSON value** — a tagged union of scalars @@ -305,6 +384,62 @@ One line per export. Every name links to its full description and runnable examp <$index> +## Migrating from 0.2.x + +`0.3.0` changes how a parser reports where it got to. Parsers now take an offset into the input instead of receiving a sliced suffix, which is what makes line/column error reporting possible. + +**If you only compose the built-in parsers, nothing changes.** `run`, `grammar`, `map`, `choice` and everything built on them behave the same, and both new arguments are optional, so `myParser('input')` still works. + +Two things do change for everyone: + +```typescript +// reading a result +result.remaining; // 0.2.x +input.slice(result.index); // 0.3.0 — or just use result.index + +// a failed run +run(p, 'bad'); // 0.2.x: Error: Parsing failed: Unexpected error +run(p, 'bad'); // 0.3.0: ParseError, with .line, .column and .expected +``` + +If you write parsers by hand, the shape changed: + +```typescript +// 0.2.x +create((input) => { + if (input.startsWith('hello')) { + return success('hello', input.slice(5)); + } + + return failure('expected "hello"'); +}); + +// 0.3.0 +create((input, index = 0, ctx) => { + if (input.startsWith('hello', index)) { + return success('hello', index + 5); + } + + return failure(ctx, index, '"hello"'); +}); +``` + +The rest, in full: + +| 0.2.x | 0.3.0 | +| ---------------------------------- | --------------------------------------------------------------- | +| `Parser = (input) => Result` | `Parser = (input, index?, ctx?) => Result` | +| `success(value, remaining)` | `success(value, index)` | +| `failure()` / `failure(message)` | `failure(ctx, index, ...expected)` | +| `{ ok: true, value, remaining }` | `{ ok: true, value, index }` | +| `{ ok: false, error?: string }` | `{ ok: false, index, expected }` | +| `match`'s `failure: (error) => …` | `failure: (index, expected) => …` | +| `position` → remaining length | `position` → offset from the start | +| `label(p, 'x')` → `'expected x'` | `label(p, 'x')` → `expected: ['x']`, only when nothing consumed | +| `regex(/^\w+/)` | same, but sticky — a leading `^` is stripped unless multiline | + +New in this release: `parse` (non-throwing), `ParseError`, `locate`, `format`, and `context`/`record` for hand-written parsers. `memoize` is now keyed by offset rather than by the remaining input, so its cache entries are cheap and are dropped when a different input is parsed. Left-recursive `grammar` rules now throw a named error instead of overflowing the stack. + ## Contributing ```bash diff --git a/doc/api.template.md b/doc/api.template.md index 19e1ba6..18985ac 100644 --- a/doc/api.template.md +++ b/doc/api.template.md @@ -7,7 +7,7 @@ <$intro> ```typescript -import { /* … */ } from '<$entry>'; +import { <$imports> } from '<$entry>'; ``` ## Index diff --git a/doc/api/combinators.md b/doc/api/combinators.md index 0188ffc..3f73864 100644 --- a/doc/api/combinators.md +++ b/doc/api/combinators.md @@ -7,7 +7,7 @@ Combinators are the glue. Every one of them takes one or more parsers and returns a new parser, which is what lets a grammar stay a set of small, independently testable pieces. ```typescript -import {} from /* … */ 'unitas/combinators'; +import { map, choice, sequence } from 'unitas/combinators'; ``` ## Index @@ -21,7 +21,7 @@ import {} from /* … */ 'unitas/combinators'; Chain parsers where the second parser depends on the first result. ```typescript -bind(digits, (n) => take(n))('3abc'); // { ok: true, value: 'abc', remaining: '' } +bind(digits, (n) => take(n))('3abc'); // { ok: true, value: 'abc', index: 4 } ``` ### `braced` @@ -29,7 +29,7 @@ bind(digits, (n) => take(n))('3abc'); // { ok: true, value: 'abc', remaining: '' Parse content surrounded by braces. ```typescript -braced(string('hi'))('{hi}'); // { ok: true, value: 'hi', remaining: '' } +braced(string('hi'))('{hi}'); // { ok: true, value: 'hi', index: 4 } ``` ### `bracketed` @@ -37,7 +37,7 @@ braced(string('hi'))('{hi}'); // { ok: true, value: 'hi', remaining: '' } Parse content surrounded by brackets. ```typescript -bracketed(string('hi'))('[hi]'); // { ok: true, value: 'hi', remaining: '' } +bracketed(string('hi'))('[hi]'); // { ok: true, value: 'hi', index: 4 } ``` ### `chainLeft` @@ -45,8 +45,7 @@ bracketed(string('hi'))('[hi]'); // { ok: true, value: 'hi', remaining: '' } Chain left-associative operations (right-to-left for same precedence). ```typescript -chainLeft(digits, operation)('1+2+3'); // { ok: true, value: 6, remaining: '' } -chainLeft(digits, operation)('10-3+2'); // { ok: true, value: 9, remaining: '' } +chainLeft(digits, operation)('1+2+3'); // { ok: true, value: 6, index: 5 } ``` ### `chainLeft1` @@ -54,17 +53,16 @@ chainLeft(digits, operation)('10-3+2'); // { ok: true, value: 9, remaining: '' } Chain left-associative operations (fails on empty input). ```typescript -chainLeft1(digits, operation)('1+2+3'); // { ok: true, value: 6, remaining: '' } -chainLeft1(digits, operation)('8/2*3'); // { ok: true, value: 12, remaining: '' } +chainLeft1(digits, operation)('1+2+3'); // { ok: true, value: 6, index: 5 } +chainLeft1(digits, operation)('8/2*3'); // { ok: true, value: 12, index: 5 } ``` ### `chainRight` -Chain right-associative operations (right-to-left grouping). +Chain right-associative operations. ```typescript -chainRight(digits, operation)('2-1-1'); // { ok: true, value: 2, remaining: '' } -chainRight(digits, operation)('4/2/2'); // { ok: true, value: 4, remaining: '' } +chainRight(digits, operation)('1+2+3'); // { ok: true, value: 6, index: 5 } ``` ### `chainRight1` @@ -72,16 +70,15 @@ chainRight(digits, operation)('4/2/2'); // { ok: true, value: 4, remaining: '' } Chain right-associative operations (fails on empty input). ```typescript -chainRight1(digits, operation)('2-1-1'); // { ok: true, value: 2, remaining: '' } -chainRight1(digits, operation)('4/2/2'); // { ok: true, value: 4, remaining: '' } +chainRight1(digits, operation)('1+2+3'); // { ok: true, value: 6, index: 5 } ``` ### `choice` -Try each parser in order, return first success. +Try each parser in order, return first success. Alternatives all start from the same offset, so the expectations of those that failed there are unioned into the result. Anything they recorded deeper in the input is already in the parse context. ```typescript -choice(string('hello'), string('world'))('hello'); // { ok: true, value: 'hello', remaining: '' } +choice(string('hello'), string('world'))('hello'); // { ok: true, value: 'hello', index: 5 } ``` ### `concat` @@ -89,8 +86,8 @@ choice(string('hello'), string('world'))('hello'); // { ok: true, value: 'hello' Join string array parser result into a single string. ```typescript -concat(many(letter))('abc123'); // { ok: true, value: 'abc', remaining: '123' } -concat(many(letter), '-')('abc123'); // { ok: true, value: 'a-b-c', remaining: '123' } +concat(many(letter))('abc123'); // { ok: true, value: 'abc', index: 3 } +concat(many(letter), '-')('abc123'); // { ok: true, value: 'a-b-c', index: 3 } ``` ### `consume` @@ -98,7 +95,7 @@ concat(many(letter), '-')('abc123'); // { ok: true, value: 'a-b-c', remaining: ' Consume input but discard the result (return null). ```typescript -consume(string('hello'))('hello world'); // { ok: true, value: null, remaining: ' world' } +consume(string('hello'))('hello world'); // { ok: true, value: null, index: 5 } ``` ### `endBy` @@ -106,7 +103,7 @@ consume(string('hello'))('hello world'); // { ok: true, value: null, remaining: Zero or more items separated and ending with terminator. ```typescript -endBy(string('item'), char(';'))('item;item;item;'); // { ok: true, value: ['item', 'item', 'item'], remaining: '' } +endBy(string('item'), char(';'))('item;item;item;'); // { ok: true, value: ['item', 'item', 'item'], index: 15 } ``` ### `endBy1` @@ -114,7 +111,7 @@ endBy(string('item'), char(';'))('item;item;item;'); // { ok: true, value: ['ite One or more items separated and ending with terminator. ```typescript -endBy1(string('item'), char(';'))('item;item;item;'); // { ok: true, value: ['item', 'item', 'item'], remaining: '' } +endBy1(string('item'), char(';'))('item;item;item;'); // { ok: true, value: ['item', 'item', 'item'], index: 15 } ``` ### `exactly` @@ -122,7 +119,7 @@ endBy1(string('item'), char(';'))('item;item;item;'); // { ok: true, value: ['it Parse exactly n occurrences. ```typescript -exactly(char('a'), 3)('aaa'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +exactly(char('a'), 3)('aaa'); // { ok: true, value: ['a', 'a', 'a'], index: 3 } ``` ### `first` @@ -130,7 +127,7 @@ exactly(char('a'), 3)('aaa'); // { ok: true, value: ['a', 'a', 'a'], remaining: Extract the first element from a parser result array. ```typescript -first(sequence(char('a'), digit))('a1bc'); // { ok: true, value: 'a', remaining: 'bc' } +first(sequence(char('a'), digit))('a1bc'); // { ok: true, value: 'a', index: 2 } ``` ### `flag` @@ -138,40 +135,40 @@ first(sequence(char('a'), digit))('a1bc'); // { ok: true, value: 'a', remaining: Return true if parser succeeds, false otherwise. Always succeeds without consuming input on failure. ```typescript -flag(string('*'))('*abc'); // { ok: true, value: true, remaining: 'abc' } -flag(string('*'))('abc'); // { ok: true, value: false, remaining: 'abc' } +flag(string('*'))('*abc'); // { ok: true, value: true, index: 1 } +flag(string('*'))('abc'); // { ok: true, value: false, index: 0 } ``` ### `fold` -Parse zero or more and fold into a single value. +Fold zero or more occurrences into a single value. ```typescript -fold(digit, [], (acc, d) => [...acc, d])('123'); // { ok: true, value: [1, 2, 3], remaining: '' } +fold(digit, 0, (acc, d) => acc + d)('123'); // { ok: true, value: 6, index: 3 } ``` ### `fold1` -Parse one or more and fold into a single value. +Fold one or more occurrences into a single value. ```typescript -fold1(digit, 0, (acc, d) => acc + d)('123'); // { ok: true, value: 6, remaining: '' } +fold1(digit, 0, (acc, d) => acc + d)('123'); // { ok: true, value: 6, index: 3 } ``` ### `foldRight` -Parse zero or more and fold right-to-left. +Fold zero or more occurrences from the right into a single value. ```typescript -foldRight(digit, [], (acc, d) => [...acc, d])('123'); // { ok: true, value: [3, 2, 1], remaining: '' } +foldRight(digit, 0, (acc, d) => acc + d)('123'); // { ok: true, value: 6, index: 3 } ``` ### `foldRight1` -Parse one or more and fold right-to-left. +Fold one or more occurrences from the right into a single value. ```typescript -foldRight1(digit, [], (acc, d) => [...acc, d])('123'); // { ok: true, value: [3, 2, 1], remaining: '' } +foldRight1(digit, 0, (acc, d) => acc + d)('123'); // { ok: true, value: 6, index: 3 } ``` ### `fuse` @@ -179,8 +176,8 @@ foldRight1(digit, [], (acc, d) => [...acc, d])('123'); // { ok: true, value: [3, Fuse multiple string parsers into a single one. The fused parser concatenates all string results. ```typescript -fuse(char('a'), char('b'), char('c'))('abc'); // { ok: true, value: 'abc', remaining: '' } -fuse(string('hello'), char(' '), string('world'))('hello world'); // { ok: true, value: 'hello world', remaining: '' } +fuse(char('a'), char('b'), char('c'))('abc'); // { ok: true, value: 'abc', index: 3 } +fuse(string('hello'), char(' '), string('world'))('hello world'); // { ok: true, value: 'hello world', index: 11 } ``` ### `guard` @@ -188,8 +185,8 @@ fuse(string('hello'), char(' '), string('world'))('hello world'); // { ok: true, Conditionally apply parser based on a condition. ```typescript -guard(true, string('hello'))('hello'); // { ok: true, value: 'hello', remaining: '' } -guard(false, string('hello'))('hello'); // { ok: false } +guard(true, string('hello'))('hello'); // { ok: true, value: 'hello', index: 5 } +guard(false, string('hello'))('hello'); // { ok: false, index: 0, expected: [] } ``` ### `inner` @@ -197,15 +194,15 @@ guard(false, string('hello'))('hello'); // { ok: false } Extract inner value from surrounded content (like inner of braced). ```typescript -inner(char('('), string('hi'), char(')'))('(hi)'); // { ok: true, value: 'hi', remaining: '' } +inner(char('('), string('hi'), char(')'))('(hi)'); // { ok: true, value: 'hi', index: 4 } ``` ### `interleaved` -Parse items with interleaved separators. +Parse items separated by separators, keeping both in the result. ```typescript -interleaved(char('a'), char(','))('a,a,a'); // { ok: true, value: ['a', ',', 'a', ',', 'a'], remaining: '' } +interleaved(digits, char('+'))('1+2'); // { ok: true, value: [1, '+', 2], index: 3 } ``` ### `last` @@ -213,7 +210,7 @@ interleaved(char('a'), char(','))('a,a,a'); // { ok: true, value: ['a', ',', 'a' Extract the last element from a parser result array. ```typescript -last(sequence(char('a'), char('b')))('ab'); // { ok: true, value: 'b', remaining: '' } +last(sequence(char('a'), char('b')))('ab'); // { ok: true, value: 'b', index: 2 } ``` ### `left` @@ -221,7 +218,7 @@ last(sequence(char('a'), char('b')))('ab'); // { ok: true, value: 'b', remaining Keep only the left result from a sequence. ```typescript -left(string('hello'), string('world'))('helloworld'); // { ok: true, value: 'hello', remaining: '' } +left(string('hello'), string('world'))('helloworld'); // { ok: true, value: 'hello', index: 10 } ``` ### `lexeme` @@ -229,23 +226,23 @@ left(string('hello'), string('world'))('helloworld'); // { ok: true, value: 'hel Parser that consumes trailing whitespace. ```typescript -lexeme(string('hello'))('hello world'); // { ok: true, value: 'hello', remaining: 'world' } +lexeme(string('hello'))('hello world'); // { ok: true, value: 'hello', index: 8 } ``` ### `many` -Zero or more occurrences (never fails). +Zero or more occurrences (never fails). The failure that stopped the loop is kept as a trace, so a later error can still report what the repetition was expecting next. ```typescript -many(char('a'))('aaa'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +many(char('a'))('aaa'); // { ok: true, value: ['a', 'a', 'a'], index: 3 } ``` ### `many1` -One or more occurrences (fails if no matches). +One or more occurrences. ```typescript -many1(char('a'))('aaa'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +many1(char('a'))('aaa'); // { ok: true, value: ['a', 'a', 'a'], index: 3 } ``` ### `manyAtLeast` @@ -253,15 +250,15 @@ many1(char('a'))('aaa'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } Parse at least n occurrences. ```typescript -manyAtLeast(char('a'), 2)('aaa'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +manyAtLeast(char('a'), 2)('aaa'); // { ok: true, value: ['a', 'a', 'a'], index: 3 } ``` ### `manyAtMost` -Parse at most n occurrences. +Parse at most n occurrences (never fails). ```typescript -manyAtMost(char('a'), 2)('aaa'); // { ok: true, value: ['a', 'a'], remaining: 'a' } +manyAtMost(char('a'), 2)('aaa'); // { ok: true, value: ['a', 'a'], index: 2 } ``` ### `manyBetween` @@ -269,7 +266,7 @@ manyAtMost(char('a'), 2)('aaa'); // { ok: true, value: ['a', 'a'], remaining: 'a Parse between min and max occurrences. ```typescript -manyBetween(char('a'), 2, 3)('aaa'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +manyBetween(char('a'), 1, 2)('aaa'); // { ok: true, value: ['a', 'a'], index: 2 } ``` ### `manyTill` @@ -277,15 +274,15 @@ manyBetween(char('a'), 2, 3)('aaa'); // { ok: true, value: ['a', 'a', 'a'], rema Parse zero or more until terminator matches. ```typescript -manyTill(char('a'), char('b'))('aaab'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +manyTill(char('a'), char('b'))('aaab'); // { ok: true, value: ['a', 'a', 'a'], index: 4 } ``` ### `map` -Transform the parsed value. +Transform a parsed value through one or more functions. ```typescript -map(string('hello'), (v) => v.toUpperCase())('hello'); // { ok: true, value: 'HELLO', remaining: '' } +map(digits, (n) => n * 2)('21'); // { ok: true, value: 42, index: 2 } ``` ### `node` @@ -293,16 +290,16 @@ map(string('hello'), (v) => v.toUpperCase())('hello'); // { ok: true, value: 'HE Create a node from parser fields. ```typescript -node('binop', { left: digits, op: char('+'), right: digits })('1+2'); // { ok: true, value: { type: 'binop', left: 1, op: '+', right: 2 }, remaining: '' } -node('number', { value: digits })('123'); // { ok: true, value: { type: 'number', value: 123 }, remaining: '' } +node('binop', { left: digits, op: char('+'), right: digits })('1+2'); // { ok: true, value: { type: 'binop', left: 1, op: '+', right: 2 }, index: 3 } +node('number', { value: digits })('123'); // { ok: true, value: { type: 'number', value: 123 }, index: 3 } ``` ### `not` -Succeed if parser fails (without consuming input). +Succeed if parser fails (without consuming input). The inner parser runs against an isolated context: its failure is the expected outcome here, so reporting it would produce a misleading expectation in the final message. ```typescript -not(string('hello'))('world'); // { ok: true, value: null, remaining: 'world' } +not(string('hello'))('world'); // { ok: true, value: null, index: 0 } ``` ### `nth` @@ -310,7 +307,7 @@ not(string('hello'))('world'); // { ok: true, value: null, remaining: 'world' } Extract the nth element from a parser result array. ```typescript -nth(sequence(char('a'), char('b'), char('c')), 1)('abc'); // { ok: true, value: 'b', remaining: '' } +nth(sequence(char('a'), char('b'), char('c')), 1)('abc'); // { ok: true, value: 'b', index: 3 } ``` ### `optional` @@ -318,27 +315,24 @@ nth(sequence(char('a'), char('b'), char('c')), 1)('abc'); // { ok: true, value: Make parser optional (return null on failure, without consuming input). ```typescript -optional(string('hello'))('hello'); // { ok: true, value: 'hello', remaining: '' } -optional(string('hello'))('world'); // { ok: true, value: null, remaining: 'world' } +optional(string('hello'))('hello'); // { ok: true, value: 'hello', index: 5 } +optional(string('hello'))('world'); // { ok: true, value: null, index: 0 } ``` ### `optionalConsume` -Optionally consume input (always succeeds, returns void). +Consume input if the parser matches, discarding the result. ```typescript -optionalConsume(string('hello'))('hello world'); // { ok: true, value: undefined, remaining: ' world' } -optionalConsume(string('hello'))('world'); // { ok: true, value: undefined, remaining: 'world' } +optionalConsume(string('hi'))('hi there'); // { ok: true, index: 2 } ``` ### `optionalSeparatedBy` -Zero or more items separated by a separator, with optional null values. +Parse items separated by a separator, allowing empty slots. ```typescript -optionalSeparatedBy(digits, char(','))('1,2'); // { ok: true, value: [1, 2], remaining: '' } -optionalSeparatedBy(digits, char(','))(',1'); // { ok: true, value: [null, 1], remaining: '' } -optionalSeparatedBy(digits, char(','))('1,'); // { ok: true, value: [1], remaining: '' } +optionalSeparatedBy(digits, char(','))('1,2'); // { ok: true, value: [1, 2], index: 3 } ``` ### `outer` @@ -346,15 +340,15 @@ optionalSeparatedBy(digits, char(','))('1,'); // { ok: true, value: [1], remaini Extract outer values from a sequence of 3 parsers (skip middle). ```typescript -outer(char('('), string('hi'), char(')'))('(hi)'); // { ok: true, value: ['(', ')'], remaining: '' } +outer(char('('), string('hi'), char(')'))('(hi)'); // { ok: true, value: ['(', ')'], index: 4 } ``` ### `padded` -Parse content with optional whitespace on both sides. +Parse content surrounded by optional whitespace. ```typescript -padded(string('hi'))(' hi '); // { ok: true, value: 'hi', remaining: '' } +padded(string('hi'))(' hi '); // { ok: true, value: 'hi', index: 6 } ``` ### `parenthesized` @@ -362,46 +356,45 @@ padded(string('hi'))(' hi '); // { ok: true, value: 'hi', remaining: '' } Parse content surrounded by parentheses. ```typescript -parenthesized(string('hi'))('(hi)'); // { ok: true, value: 'hi', remaining: '' } +parenthesized(string('hi'))('(hi)'); // { ok: true, value: 'hi', index: 4 } ``` ### `peek` -Parse without consuming input. +Look ahead without consuming input. ```typescript -peek(string('hello'))('hello world'); // { ok: true, value: 'hello', remaining: 'hello world' } +peek(string('hello'))('hello world'); // { ok: true, value: 'hello', index: 0 } ``` ### `postfix` -Parse postfix operators (chains atom with operators that return functions). +Apply zero or more postfix operators to an atom. ```typescript postfix( - char('a'), - map(char('!'), () => (x) => x), -)('a!'); // { ok: true, value: 'a', remaining: '' } + digits, + map(char('!'), () => (n) => n * 2), +)('3!'); // { ok: true, value: 6, index: 2 } ``` ### `prefix` -Parse prefix operators (like - in -5). +Apply zero or more prefix operators to an atom. ```typescript prefix( - map(char('-'), () => (x) => -x), - digit, -)('-5'); // { ok: true, value: -5, remaining: '' } + map(char('-'), () => (n) => -n), + digits, +)('-3'); // { ok: true, value: -3, index: 2 } ``` ### `pure` -Always return a value without consuming input. +Always succeed with a value without consuming input. ```typescript -pure(42)('abc'); // { ok: true, value: 42, remaining: 'abc' } -pure('ok')(''); // { ok: true, value: 'ok', remaining: '' } +pure(42)('abc'); // { ok: true, value: 42, index: 0 } ``` ### `quoted` @@ -409,7 +402,7 @@ pure('ok')(''); // { ok: true, value: 'ok', remaining: '' } Parse content surrounded by single or double quotes. ```typescript -quoted(string('hello'))('"hello"'); // { ok: true, value: 'hello', remaining: '' } +quoted(string('hello'))('"hello"'); // { ok: true, value: 'hello', index: 7 } ``` ### `recover` @@ -417,7 +410,7 @@ quoted(string('hello'))('"hello"'); // { ok: true, value: 'hello', remaining: '' Use fallback value when parser fails. ```typescript -recover(string('hello'), 'default')('world'); // { ok: true, value: 'default', remaining: 'world' } +recover(string('hello'), 'default')('world'); // { ok: true, value: 'default', index: 0 } ``` ### `right` @@ -425,47 +418,47 @@ recover(string('hello'), 'default')('world'); // { ok: true, value: 'default', r Keep only the right result from a sequence. ```typescript -right(string('hello'), string('world'))('helloworld'); // { ok: true, value: 'world', remaining: '' } +right(string('hello'), string('world'))('helloworld'); // { ok: true, value: 'world', index: 10 } ``` ### `separatedBy` -Zero or more items separated by a separator. +Parse zero or more items separated by a separator. A trailing separator is not consumed: if the separator matches but the item after it does not, the list ends before the separator. ```typescript -separatedBy(char('a'), char(','))('a,a,a'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +separatedBy(digits, char(','))('1,2,3'); // { ok: true, value: [1, 2, 3], index: 5 } ``` ### `separatedBy1` -One or more items separated by a separator. +Parse one or more items separated by a separator. ```typescript -separatedBy1(char('a'), char(','))('a,a,a'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +separatedBy1(digits, char(','))('1,2,3'); // { ok: true, value: [1, 2, 3], index: 5 } ``` ### `separatedEndBy` -Zero or more items separated by and ending with a terminator. +Parse zero or more items separated by a separator, allowing a trailing one. ```typescript -separatedEndBy(char('a'), char(';'))('a;a;a;'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +separatedEndBy(digits, char(','))('1,2,3,'); // { ok: true, value: [1, 2, 3], index: 6 } ``` ### `separatedEndBy1` -One or more items separated by and ending with a terminator. +Parse one or more items separated by a separator, allowing a trailing one. ```typescript -separatedEndBy1(char('a'), char(';'))('a;a;a;'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +separatedEndBy1(digits, char(','))('1,2,3,'); // { ok: true, value: [1, 2, 3], index: 6 } ``` ### `separatedUntil` -Parse items separated by separator until terminator matches. +Parse items separated by a separator, up to a terminator. ```typescript -separatedUntil(char('a'), char(','), char(';'))('a,a,a;'); // { ok: true, value: ['a', 'a', 'a'], remaining: '' } +separatedUntil(digits, char(','), char(';'))('1,2,3;'); // { ok: true, value: [1, 2, 3], index: 6 } ``` ### `sequence` @@ -473,7 +466,7 @@ separatedUntil(char('a'), char(','), char(';'))('a,a,a;'); // { ok: true, value: Parse a sequence of parsers and return all results as an array. ```typescript -sequence(char('a'), char('b'), char('c'))('abc'); // { ok: true, value: ['a', 'b', 'c'], remaining: '' } +sequence(char('a'), char('b'), char('c'))('abc'); // { ok: true, value: ['a', 'b', 'c'], index: 3 } ``` ### `skip` @@ -481,7 +474,7 @@ sequence(char('a'), char('b'), char('c'))('abc'); // { ok: true, value: ['a', 'b Skip a parser n times. ```typescript -skip(char('a'), 2)('aabc'); // { ok: true, value: null, remaining: 'bc' } +skip(char('a'), 2)('aabc'); // { ok: true, value: null, index: 2 } ``` ### `skipMany` @@ -489,7 +482,7 @@ skip(char('a'), 2)('aabc'); // { ok: true, value: null, remaining: 'bc' } Skip zero or more occurrences (never fails, returns null). ```typescript -skipMany(char('a'))('aaabc'); // { ok: true, value: null, remaining: 'bc' } +skipMany(char('a'))('aaabc'); // { ok: true, value: null, index: 3 } ``` ### `skipMany1` @@ -497,7 +490,7 @@ skipMany(char('a'))('aaabc'); // { ok: true, value: null, remaining: 'bc' } Skip one or more occurrences (fails if no matches). ```typescript -skipMany1(char('a'))('aaabc'); // { ok: true, value: null, remaining: 'bc' } +skipMany1(char('a'))('aaabc'); // { ok: true, value: null, index: 3 } ``` ### `surrounded` @@ -505,8 +498,8 @@ skipMany1(char('a'))('aaabc'); // { ok: true, value: null, remaining: 'bc' } Parse content surrounded by delimiters. ```typescript -surrounded(char('['), string('hi'), char(']'))('[hi]'); // { ok: true, value: 'hi', remaining: '' } -surrounded(char('a'), char('b'), char('c'))('abc'); // { ok: true, value: 'b', remaining: '' } +surrounded(char('['), string('hi'), char(']'))('[hi]'); // { ok: true, value: 'hi', index: 4 } +surrounded(char('a'), char('b'), char('c'))('abc'); // { ok: true, value: 'b', index: 3 } ``` ### `unless` @@ -514,17 +507,16 @@ surrounded(char('a'), char('b'), char('c'))('abc'); // { ok: true, value: 'b', r Parse unless condition is true (inverse of guard). ```typescript -unless(false, string('hello'))('hello'); // { ok: true, value: 'hello', remaining: '' } -unless(true, string('hello'))('hello'); // { ok: true, value: null, remaining: 'hello' } +unless(false, string('hello'))('hello'); // { ok: true, value: 'hello', index: 5 } +unless(true, string('hello'))('hello'); // { ok: true, value: null, index: 0 } ``` ### `until` -Parse until terminator matches (fails if terminator never matches). +Parse zero or more until terminator matches, leaving the terminator unconsumed. ```typescript -until(char('a'), char('b'))('baaa'); // { ok: true, value: [], remaining: 'baaa' } -until(char('a'), char('b'))('aaba'); // { ok: true, value: ['a', 'a'], remaining: 'ba' } +until(char('a'), char('b'))('aaab'); // { ok: true, value: ['a', 'a', 'a'], index: 3 } ``` ### `validate` @@ -532,8 +524,8 @@ until(char('a'), char('b'))('aaba'); // { ok: true, value: ['a', 'a'], remaining Validate parsed value with a predicate. ```typescript -validate(digit, (n) => n > 5)('7'); // { ok: true, value: 7, remaining: '' } -validate(digit, (n) => n > 5)('3'); // { ok: false } +validate(digit, (n) => n > 5)('7'); // { ok: true, value: 7, index: 1 } +validate(digit, (n) => n > 5)('3'); // { ok: false, index: 0, expected: ['valid value'] } ``` ### `value` @@ -541,8 +533,8 @@ validate(digit, (n) => n > 5)('3'); // { ok: false } Replace parsed value with a constant. ```typescript -value(string('true'), true)('true'); // { ok: true, value: true, remaining: '' } -value(string('null'), null)('null'); // { ok: true, value: null, remaining: '' } +value(string('true'), true)('true'); // { ok: true, value: true, index: 4 } +value(string('null'), null)('null'); // { ok: true, value: null, index: 4 } ``` ### `when` @@ -550,6 +542,6 @@ value(string('null'), null)('null'); // { ok: true, value: null, remaining: '' } Branch on a boolean parser result. ```typescript -when(flag(char('*')), pure('many'), pure('one'))('*rest'); // { ok: true, value: 'many', remaining: 'rest' } -when(flag(char('*')), pure('many'), pure('one'))('abc'); // { ok: true, value: 'one', remaining: 'abc' } +when(flag(char('*')), pure('many'), pure('one'))('*rest'); // { ok: true, value: 'many', index: 1 } +when(flag(char('*')), pure('many'), pure('one'))('abc'); // { ok: true, value: 'one', index: 0 } ``` diff --git a/doc/api/core.md b/doc/api/core.md index b23d036..d5eebce 100644 --- a/doc/api/core.md +++ b/doc/api/core.md @@ -7,26 +7,50 @@ The core entry point holds everything a parser is built from: the `Parser` and `Result` types, the `success`/`failure` constructors, and the tools for running and wiring parsers together (`run`, `grammar`, `lazy`, `memoize`). ```typescript -import {} from /* … */ 'unitas'; +import { run, parse, grammar } from 'unitas'; ``` ## Index -9 exports: [`failure`](#failure) · [`grammar`](#grammar) · [`label`](#label) · [`lazy`](#lazy) · [`match`](#match) · [`memoize`](#memoize) · [`parser`](#parser) · [`run`](#run) · [`success`](#success) +17 exports: [`context`](#context) · [`create`](#create) · [`failure`](#failure) · [`format`](#format) · [`grammar`](#grammar) · [`label`](#label) · [`lazy`](#lazy) · [`locate`](#locate) · [`match`](#match) · [`memoize`](#memoize) · [`parse`](#parse) · [`ParseError`](#parseerror) · [`record`](#record) · [`reject`](#reject) · [`run`](#run) · [`success`](#success) · [`union`](#union) ## Reference +### `context` + +Creates a fresh context. `run` and `parse` make one per call. + +```typescript +context(); // { furthest: -1, expected: [] } +``` + +### `create` + +Creates a parser from a parser function. A parser reads `input` from `index` and never slices it. The third argument is the parse context — pass it on to any parser you call, so expectations recorded inside your parser reach the final error message. + +```typescript +create((input, index = 0) => success('parsed', index + 6))('hello world'); // { ok: true, value: 'parsed', index: 6 } +``` + ### `failure` -Creates a failed result with an optional error message. +Creates a failed result at an offset and notes it in the parse context. Expectations are phrased as nouns ('digit', "'{'"). Pass the context through from the parser so the message can name this position even if the branch is later backtracked over. ```typescript -failure('unexpected input'); // { ok: false, error: 'unexpected input' } +failure(undefined, 3, 'digit'); // { ok: false, index: 3, expected: ['digit'] } +``` + +### `format` + +Renders a parse failure as a source excerpt with a caret under the offset. + +```typescript +format('a=1,b=x', 6, ['digit']); // "1:7 expected digit, found 'x'\n 1 | a=1,b=x\n | ^" ``` ### `grammar` -Creates a recursive grammar where rules can reference each other. +Creates a recursive grammar where rules can reference each other. Rules are resolved lazily through a shared record, so they may refer to one another by name without forward declarations. Left-recursive rules are reported by name rather than overflowing the stack. ```typescript type Math = { @@ -54,10 +78,10 @@ run(g.expr, '(1+2)'); // 3 ### `label` -Labels a parser with a custom error message on failure. +Replaces the expectations of a parser with a single description. Only applies when the parser failed without consuming input: once a branch has committed, its own deeper error is more useful than the label. The labelled parser runs against an isolated context so that its internals can be suppressed rather than leaking into the message. ```typescript -label(char('x'), 'letter x')(''); // { ok: false, error: 'expected letter x' } +label(char('x'), 'letter x')(''); // { ok: false, index: 0, expected: ['letter x'] } ``` ### `lazy` @@ -65,7 +89,15 @@ label(char('x'), 'letter x')(''); // { ok: false, error: 'expected letter x' } Defers parser creation, useful for recursive grammars. ```typescript -lazy(() => char('a'))('abc'); // { ok: true, value: 'a', remaining: 'bc' } +lazy(() => char('a'))('abc'); // { ok: true, value: 'a', index: 1 } +``` + +### `locate` + +Converts a character offset into a 1-based line and column. + +```typescript +locate('a=1\nb=x', 5); // { index: 5, line: 2, column: 2 } ``` ### `match` @@ -73,29 +105,56 @@ lazy(() => char('a'))('abc'); // { ok: true, value: 'a', remaining: 'bc' } Pattern matching on a Result to handle success and failure cases. ```typescript -match(success('hello', ''), { success: (v) => v, failure: () => 'failed' }); // 'hello' +match(success('hello', 5), { success: (v) => v, failure: () => 'failed' }); // 'hello' ``` ### `memoize` -Memoizes a parser to cache results by input string. Useful for expensive parsers and recursive grammars to avoid exponential backtracking. +Memoizes a parser so each offset is parsed at most once (packrat caching). Keyed by offset rather than by the remaining input, so entries are cheap and the cache is dropped as soon as a different input is parsed. Each entry also remembers what the parse expected, so a cache hit still contributes to the error message rather than silently weakening it. ```typescript const memoDigits = memoize(digits); -memoDigits('123'); // { ok: true, value: 123, remaining: '' } +memoDigits('123'); // { ok: true, value: 123, index: 3 } ``` -### `parser` +### `parse` -Creates a parser from a parser function. +Runs a parser over the whole input without throwing. The non-throwing counterpart of `run`: on failure it reports the offset, line, column, expectations and a formatted message. ```typescript -create((input) => success('parsed', input.slice(6)))('hello world'); // { ok: true, value: 'parsed', remaining: 'world' } +parse(digits, '12x').ok; // false +``` + +### `ParseError` + +Error thrown by `run` when a parse fails, carrying the offset, the 1-based line/column and the set of expectations at that offset. + +```typescript +new ParseError('a=1,b=x', 6, ['digit']).line; // 1 +``` + +### `record` + +Notes what was expected at an offset, keeping only the furthest one seen. Replaces `expected` rather than mutating it, so a parser may pass the same array every time without it being altered underneath. + +```typescript +const ctx = context(); +record(ctx, 3, ['digit']); +ctx; // { furthest: 3, expected: ['digit'] } +``` + +### `reject` + +Creates a failed result from an expectation list that is already allocated, avoiding a fresh array on every failure. + +```typescript +const expected = ['digit']; +reject(undefined, 3, expected); // { ok: false, index: 3, expected: ['digit'] } ``` ### `run` -Runs a parser and returns the value, throws on failure or unconsumed input. +Runs a parser over the whole input and returns the value. Throws a `ParseError` — with line, column and the expectations at that point — if the parse fails or leaves input unconsumed. ```typescript run(string('hello'), 'hello'); // 'hello' @@ -103,8 +162,16 @@ run(string('hello'), 'hello'); // 'hello' ### `success` -Creates a successful result with a value and remaining input. +Creates a successful result with a value and the offset reached in the input. + +```typescript +success('hello', 5); // { ok: true, value: 'hello', index: 5 } +``` + +### `union` + +Adds the entries of `b` to `a`, returning `a` itself when it already covers them. Callers rely on that identity to skip allocating. ```typescript -success('hello', ' world'); // { ok: true, value: 'hello', remaining: ' world' } +union(['digit'], ['digit']); // ['digit'] ``` diff --git a/doc/api/primitives.md b/doc/api/primitives.md index 39d945f..f4cf0ca 100644 --- a/doc/api/primitives.md +++ b/doc/api/primitives.md @@ -7,7 +7,7 @@ Primitives are parser _instances_, not factories. Where a terminal needs an argument (`char("a")`), a primitive is already a parser and can be passed straight to a combinator (`many(digit)`). ```typescript -import {} from /* … */ 'unitas/primitives'; +import { digit, letters, whitespace } from 'unitas/primitives'; ``` ## Index @@ -21,16 +21,16 @@ import {} from /* … */ 'unitas/primitives'; Parse a single alphanumeric character. ```typescript -alphaNum('a1'); // { ok: true, value: 'a', remaining: '1' } -alphaNum('1a'); // { ok: true, value: '1', remaining: 'a' } +alphaNum('a1'); // { ok: true, value: 'a', index: 1 } +alphaNum('1a'); // { ok: true, value: '1', index: 1 } ``` ### `alphaNums` -Parse one or more alphanumeric characters. +Parse one or more alphanumeric characters as a string. ```typescript -alphaNums('abc123'); // { ok: true, value: 'abc123', remaining: '' } +alphaNums('placeholder'); // { ok: true, value: 'placeholder', index: 11 } ``` ### `anyChar` @@ -38,7 +38,7 @@ alphaNums('abc123'); // { ok: true, value: 'abc123', remaining: '' } Parse any single character. ```typescript -anyChar('abc'); // { ok: true, value: 'a', remaining: 'bc' } +anyChar('abc'); // { ok: true, value: 'a', index: 1 } ``` ### `bool` @@ -46,9 +46,9 @@ anyChar('abc'); // { ok: true, value: 'a', remaining: 'bc' } Parse a boolean literal. ```typescript -bool('true'); // { ok: true, value: true, remaining: '' } -bool('false'); // { ok: true, value: false, remaining: '' } -bool('trueABC'); // { ok: true, value: true, remaining: 'ABC' } +bool('true'); // { ok: true, value: true, index: 4 } +bool('false'); // { ok: true, value: false, index: 5 } +bool('trueABC'); // { ok: true, value: true, index: 4 } ``` ### `crlf` @@ -56,7 +56,7 @@ bool('trueABC'); // { ok: true, value: true, remaining: 'ABC' } Parse CRLF line ending. ```typescript -crlf('\r\nabc'); // { ok: true, value: '\r\n', remaining: 'abc' } +crlf('\r\nabc'); // { ok: true, value: '\r\n', index: 2 } ``` ### `digit` @@ -64,23 +64,23 @@ crlf('\r\nabc'); // { ok: true, value: '\r\n', remaining: 'abc' } Parse a single digit and return as number. ```typescript -digit('5abc'); // { ok: true, value: 5, remaining: 'abc' } +digit('5abc'); // { ok: true, value: 5, index: 1 } ``` ### `digits` -Parse one or more digits and return as number. +Parse one or more digits and return as a number. ```typescript -digits('123abc'); // { ok: true, value: 123, remaining: 'abc' } +digits('123'); // { ok: true, value: 123, index: 3 } ``` ### `eof` -Parse end of file (succeeds only on empty input). +Match the end of the input. ```typescript -eof(''); // { ok: true, value: null, remaining: '' } +eof(''); // { ok: true, value: null, index: 0 } ``` ### `eol` @@ -88,7 +88,7 @@ eof(''); // { ok: true, value: null, remaining: '' } Parse end of line (\\n, \\r\\n, or EOF). ```typescript -eol('\nabc'); // { ok: true, value: '\n', remaining: 'abc' } +eol('\nabc'); // { ok: true, value: '\n', index: 1 } ``` ### `float` @@ -96,9 +96,9 @@ eol('\nabc'); // { ok: true, value: '\n', remaining: 'abc' } Parse a floating point number. ```typescript -float('1.23'); // { ok: true, value: 1.23, remaining: '' } -float('-2.5'); // { ok: true, value: -2.5, remaining: '' } -float('1.23abc'); // { ok: true, value: 1.23, remaining: 'abc' } +float('1.23'); // { ok: true, value: 1.23, index: 4 } +float('-2.5'); // { ok: true, value: -2.5, index: 4 } +float('1.23abc'); // { ok: true, value: 1.23, index: 4 } ``` ### `hexDigit` @@ -106,15 +106,15 @@ float('1.23abc'); // { ok: true, value: 1.23, remaining: 'abc' } Parse a single hexadecimal digit. ```typescript -hexDigit('fF9'); // { ok: true, value: 'f', remaining: 'F9' } +hexDigit('fF9'); // { ok: true, value: 'f', index: 1 } ``` ### `hexDigits` -Parse one or more hexadecimal digits. +Parse one or more hex digits as a string. ```typescript -hexDigits('deadbeef'); // { ok: true, value: 'deadbeef', remaining: '' } +hexDigits('deadbeef'); // { ok: true, value: 'deadbeef', index: 8 } ``` ### `identifier` @@ -122,7 +122,7 @@ hexDigits('deadbeef'); // { ok: true, value: 'deadbeef', remaining: '' } Parse an identifier — starts with letter or underscore, no leading digit, no hyphen. ```typescript -identifier('variable_name'); // { ok: true, value: 'variable_name', remaining: '' } +identifier('variable_name'); // { ok: true, value: 'variable_name', index: 13 } ``` ### `integer` @@ -130,9 +130,9 @@ identifier('variable_name'); // { ok: true, value: 'variable_name', remaining: ' Parse a signed integer. ```typescript -integer('42'); // { ok: true, value: 42, remaining: '' } -integer('-7'); // { ok: true, value: -7, remaining: '' } -integer('123abc'); // { ok: true, value: 123, remaining: 'abc' } +integer('42'); // { ok: true, value: 42, index: 2 } +integer('-7'); // { ok: true, value: -7, index: 2 } +integer('123abc'); // { ok: true, value: 123, index: 3 } ``` ### `letter` @@ -140,15 +140,15 @@ integer('123abc'); // { ok: true, value: 123, remaining: 'abc' } Parse a single letter. ```typescript -letter('abc'); // { ok: true, value: 'a', remaining: 'bc' } +letter('abc'); // { ok: true, value: 'a', index: 1 } ``` ### `letters` -Parse one or more letters. +Parse one or more letters as a string. ```typescript -letters('abc123'); // { ok: true, value: 'abc', remaining: '123' } +letters('placeholder'); // { ok: true, value: 'placeholder', index: 11 } ``` ### `line` @@ -156,7 +156,7 @@ letters('abc123'); // { ok: true, value: 'abc', remaining: '123' } Parse until end of line. ```typescript -line('hello\nworld'); // { ok: true, value: 'hello', remaining: '\nworld' } +line('hello\nworld'); // { ok: true, value: 'hello', index: 5 } ``` ### `literal` @@ -164,8 +164,8 @@ line('hello\nworld'); // { ok: true, value: 'hello', remaining: '\nworld' } Parse a word-like value including hyphens. ```typescript -literal('foo-bar'); // { ok: true, value: 'foo-bar', remaining: '' } -literal('123abc'); // { ok: true, value: '123abc', remaining: '' } +literal('foo-bar'); // { ok: true, value: 'foo-bar', index: 7 } +literal('123abc'); // { ok: true, value: '123abc', index: 6 } ``` ### `lowercase` @@ -173,15 +173,15 @@ literal('123abc'); // { ok: true, value: '123abc', remaining: '' } Parse a single lowercase letter. ```typescript -lowercase('abc'); // { ok: true, value: 'a', remaining: 'bc' } +lowercase('abc'); // { ok: true, value: 'a', index: 1 } ``` ### `lowercases` -Parses one or more lowercase letters. +Parse one or more lowercase letters as a string. ```typescript -lowercases('abcDEF'); // { ok: true, value: 'abc', remaining: 'DEF' } +lowercases('placeholder'); // { ok: true, value: 'placeholder', index: 11 } ``` ### `nl` @@ -189,7 +189,7 @@ lowercases('abcDEF'); // { ok: true, value: 'abc', remaining: 'DEF' } Parse a newline character. ```typescript -nl('\ntext'); // { ok: true, value: '\n', remaining: 'text' } +nl('\ntext'); // { ok: true, value: '\n', index: 1 } ``` ### `number` @@ -197,10 +197,10 @@ nl('\ntext'); // { ok: true, value: '\n', remaining: 'text' } Parse an integer or float. ```typescript -number('42'); // { ok: true, value: 42, remaining: '' } -number('3.14'); // { ok: true, value: 3.14, remaining: '' } -number('-7'); // { ok: true, value: -7, remaining: '' } -number('-2.5'); // { ok: true, value: -2.5, remaining: '' } +number('42'); // { ok: true, value: 42, index: 2 } +number('3.14'); // { ok: true, value: 3.14, index: 4 } +number('-7'); // { ok: true, value: -7, index: 2 } +number('-2.5'); // { ok: true, value: -2.5, index: 4 } ``` ### `octDigit` @@ -208,31 +208,32 @@ number('-2.5'); // { ok: true, value: -2.5, remaining: '' } Parse a single octal digit. ```typescript -octDigit('7abc'); // { ok: true, value: '7', remaining: 'abc' } +octDigit('7abc'); // { ok: true, value: '7', index: 1 } ``` ### `octDigits` -Parse one or more octal digits. +Parse one or more octal digits as a string. ```typescript -octDigits('0777abc'); // { ok: true, value: '0777', remaining: 'abc' } +octDigits('755rest'); // { ok: true, value: '755', index: 3 } ``` ### `position` -Get current position (remaining input length). +Get the current offset into the input, without consuming anything. Counts forward from the start of the input, so it can be paired with `locate` to attach line/column information to a parsed node. ```typescript -position('abc'); // { ok: true, value: 3, remaining: 'abc' } +position('abc'); // { ok: true, value: 0, index: 0 } +right(string('ab'), position)('abc'); // { ok: true, value: 2, index: 2 } ``` ### `rest` -Parse the rest of the input. +Consume and return everything left in the input. ```typescript -rest('hello'); // { ok: true, value: 'hello', remaining: '' } +rest('abc'); // { ok: true, value: 'abc', index: 3 } ``` ### `space` @@ -240,15 +241,15 @@ rest('hello'); // { ok: true, value: 'hello', remaining: '' } Parse a single space character. ```typescript -space(' abc'); // { ok: true, value: ' ', remaining: 'abc' } +space(' abc'); // { ok: true, value: ' ', index: 1 } ``` ### `spaces` -Parse one or more space characters. +Parse one or more spaces as a string. ```typescript -spaces(' abc'); // { ok: true, value: ' ', remaining: 'abc' } +spaces(' abc'); // { ok: true, value: ' ', index: 3 } ``` ### `tab` @@ -256,7 +257,7 @@ spaces(' abc'); // { ok: true, value: ' ', remaining: 'abc' } Parse tab character. ```typescript -tab('\ttext'); // { ok: true, value: '\t', remaining: 'text' } +tab('\ttext'); // { ok: true, value: '\t', index: 1 } ``` ### `uppercase` @@ -264,15 +265,15 @@ tab('\ttext'); // { ok: true, value: '\t', remaining: 'text' } Parses a single uppercase letter. ```typescript -uppercase('ABC'); // { ok: true, value: 'A', remaining: 'BC' } +uppercase('ABC'); // { ok: true, value: 'A', index: 1 } ``` ### `uppercases` -Parses one or more uppercase letters. +Parse one or more uppercase letters as a string. ```typescript -uppercases('ABCdef'); // { ok: true, value: 'ABC', remaining: 'def' } +uppercases('ABCdef'); // { ok: true, value: 'ABC', index: 3 } ``` ### `whitespace` @@ -280,13 +281,13 @@ uppercases('ABCdef'); // { ok: true, value: 'ABC', remaining: 'def' } Parses a single whitespace character. ```typescript -whitespace(' abc'); // { ok: true, value: ' ', remaining: 'abc' } +whitespace(' abc'); // { ok: true, value: ' ', index: 1 } ``` ### `whitespaces` -Parses one or more whitespace characters. +Parse one or more whitespaces as a string. ```typescript -whitespaces(' abc'); // { ok: true, value: ' ', remaining: 'abc' } +whitespaces(' \t\nabc'); // { ok: true, value: ' \t\n', index: 3 } ``` diff --git a/doc/api/terminals.md b/doc/api/terminals.md index 424d9c8..d611fef 100644 --- a/doc/api/terminals.md +++ b/doc/api/terminals.md @@ -7,7 +7,7 @@ Terminals are the leaves of a grammar. They do not take other parsers — they inspect the input string themselves. Each one is a factory: call it with what you want to match and it hands back a `Parser`. ```typescript -import {} from /* … */ 'unitas/terminals'; +import { char, string, regex } from 'unitas/terminals'; ``` ## Index @@ -21,7 +21,7 @@ import {} from /* … */ 'unitas/terminals'; Parse a specific character. ```typescript -char('A')('ABC'); // { ok: true, value: 'A', remaining: 'BC' } +char('A')('ABC'); // { ok: true, value: 'A', index: 1 } ``` ### `charOf` @@ -29,7 +29,7 @@ char('A')('ABC'); // { ok: true, value: 'A', remaining: 'BC' } Parse any character from a set. ```typescript -charOf(['a', 'b', 'c'])('abc'); // { ok: true, value: 'a', remaining: 'bc' } +charOf(['a', 'b', 'c'])('abc'); // { ok: true, value: 'a', index: 1 } ``` ### `noneOf` @@ -37,7 +37,7 @@ charOf(['a', 'b', 'c'])('abc'); // { ok: true, value: 'a', remaining: 'bc' } Parse any character not in the set. ```typescript -noneOf(['a', 'b', 'c'])('xyz'); // { ok: true, value: 'x', remaining: 'yz' } +noneOf(['a', 'b', 'c'])('xyz'); // { ok: true, value: 'x', index: 1 } ``` ### `oneOf` @@ -45,15 +45,15 @@ noneOf(['a', 'b', 'c'])('xyz'); // { ok: true, value: 'x', remaining: 'yz' } Parse one string from a set of strings (longest match wins). ```typescript -oneOf(['hello', 'hell', 'help'])('helpful'); // { ok: true, value: 'help', remaining: 'ful' } +oneOf(['hello', 'hell', 'help'])('helpful'); // { ok: true, value: 'help', index: 4 } ``` ### `regex` -Parse with a regular expression. +Parse with a regular expression. The pattern is matched with the sticky flag at the current offset, so it is anchored by construction and never scans ahead. A leading `^` is redundant and is stripped — unless the pattern is multiline, where it asserts a line start and still carries meaning. ```typescript -regex(/^\w+/)('hello world'); // { ok: true, value: 'hello', remaining: ' world' } +regex(/^\w+/)('hello world'); // { ok: true, value: 'hello', index: 5 } ``` ### `satisfy` @@ -61,7 +61,7 @@ regex(/^\w+/)('hello world'); // { ok: true, value: 'hello', remaining: ' world' Parse a character satisfying a predicate. ```typescript -satisfy((c) => c === 'a')('abc'); // { ok: true, value: 'a', remaining: 'bc' } +satisfy((c) => c === 'a')('abc'); // { ok: true, value: 'a', index: 1 } ``` ### `string` @@ -69,7 +69,7 @@ satisfy((c) => c === 'a')('abc'); // { ok: true, value: 'a', remaining: 'bc' } Parse a specific string. ```typescript -string('hello')('hello world'); // { ok: true, value: 'hello', remaining: ' world' } +string('hello')('hello world'); // { ok: true, value: 'hello', index: 5 } ``` ### `stringOf` @@ -77,7 +77,7 @@ string('hello')('hello world'); // { ok: true, value: 'hello', remaining: ' worl Parse first character that exists in string (like charOf but for a string). ```typescript -stringOf('abc')('abcdef'); // { ok: true, value: 'a', remaining: 'bcdef' } +stringOf('abc')('abcdef'); // { ok: true, value: 'a', index: 1 } ``` ### `take` @@ -85,7 +85,7 @@ stringOf('abc')('abcdef'); // { ok: true, value: 'a', remaining: 'bcdef' } Take n characters. ```typescript -take(3)('abcdef'); // { ok: true, value: 'abc', remaining: 'def' } +take(3)('abcdef'); // { ok: true, value: 'abc', index: 3 } ``` ### `takeWhile` @@ -93,7 +93,7 @@ take(3)('abcdef'); // { ok: true, value: 'abc', remaining: 'def' } Takes characters while the predicate returns true. ```typescript -takeWhile((c) => c !== 'x')('abcx'); // { ok: true, value: 'abc', remaining: 'x' } +takeWhile((c) => c !== 'x')('abcx'); // { ok: true, value: 'abc', index: 3 } ``` ### `token` @@ -101,9 +101,9 @@ takeWhile((c) => c !== 'x')('abcx'); // { ok: true, value: 'abc', remaining: 'x' Parses a string as a token, skipping trailing whitespace. Unlike word, it does not enforce word boundaries - useful for parsing strings that might be followed by any character. ```typescript -token('let')('let x'); // { ok: true, value: 'let', remaining: 'x' } -token('let')('let1'); // { ok: true, value: 'let', remaining: '1' } -token('let')('let x'); // { ok: true, value: 'let', remaining: 'x' } +token('let')('let x'); // { ok: true, value: 'let', index: 4 } +token('let')('let1'); // { ok: true, value: 'let', index: 3 } +token('let')('let x'); // { ok: true, value: 'let', index: 5 } ``` ### `word` @@ -111,7 +111,7 @@ token('let')('let x'); // { ok: true, value: 'let', remaining: 'x' } Parses a specific word and ensures it is not followed by word characters. Use this when parsing keywords that should not be part of a longer identifier. ```typescript -word('let')('let x'); // { ok: true, value: 'let', remaining: 'x' } -word('let')('let1'); // { ok: false } -word('if')('if (x)'); // { ok: true, value: 'if', remaining: '(x)' } +word('let')('let x'); // { ok: true, value: 'let', index: 4 } +word('let')('let1'); // { ok: false, index: 3, expected: ['/(?!\\w)/'] } +word('if')('if (x)'); // { ok: true, value: 'if', index: 3 } ``` diff --git a/doc/api/utils.md b/doc/api/utils.md index 11e2e9d..11001ff 100644 --- a/doc/api/utils.md +++ b/doc/api/utils.md @@ -7,7 +7,7 @@ Utils are not parsers. They are small curried helpers meant to be dropped into a `map` callback so reshaping a result stays a one-liner instead of an arrow function. ```typescript -import {} from /* … */ 'unitas/utils'; +import { pick, join, pipe } from 'unitas/utils'; ``` ## Index @@ -57,7 +57,7 @@ pick(2, 4)(['a', 'b', 'c', 'd', 'e']); // ['c', 'e'] Pipe parser functions together. ```typescript -pipe(lexeme)(letters)('xyz abc'); // { ok: true, value: 'xyz', remaining: 'abc' } +pipe(lexeme)(letters)('xyz abc'); // { ok: true, value: 'xyz', index: 6 } ``` ### `pop` diff --git a/package.json b/package.json index 38f4fc3..c8ba8c9 100644 --- a/package.json +++ b/package.json @@ -1,22 +1,26 @@ { "name": "unitas", - "version": "0.2.5", + "version": "0.3.0", "description": "TypeScript-first parser combinator library — compose small typed parsers into whole grammars. Zero dependencies, ESM-only, tree-shakeable.", "keywords": [ + "combinator", + "esm", + "functional", + "grammar", + "lexer", "parser", "parser-combinator", - "combinator", "parsing", - "grammar", "peg", "recursive-descent", - "lexer", "tokenizer", - "functional", "typescript", - "esm", "zero-dependency" ], + "homepage": "https://github.com/sovrin/unitas#readme", + "bugs": { + "url": "https://github.com/sovrin/unitas/issues" + }, "license": "MIT", "author": { "name": "Oleg Kamlowski", @@ -26,10 +30,6 @@ "type": "git", "url": "https://github.com/sovrin/unitas.git" }, - "homepage": "https://github.com/sovrin/unitas#readme", - "bugs": { - "url": "https://github.com/sovrin/unitas/issues" - }, "files": [ "dist", "README.md" diff --git a/src/combinators/bind.test.ts b/src/combinators/bind.test.ts index 94d9655..9bbefb8 100644 --- a/src/combinators/bind.test.ts +++ b/src/combinators/bind.test.ts @@ -15,16 +15,20 @@ describe('bind', () => { const parser1 = createTestParser('A'); const parser2 = createTestParser('B'); const parser = bind(parser1, (a) => - bind(parser2, (b) => create(() => success(`${a}${b}`, 'CD'))), + bind(parser2, (b) => + create((_input, index = 0) => success(`${a}${b}`, index)), + ), ); const result = parser('ABCD'); - assertSuccess(result, 'AB', 'CD'); + assertSuccess(result, 'AB', 2); }); it('should fail if the first parser fails', () => { - const failingParser = create(() => failure()); + const failingParser = create((_input, index = 0) => + failure(undefined, index), + ); const parser = bind(failingParser, (_x) => createTestParser('B')); const result = parser('ABC'); @@ -34,7 +38,9 @@ describe('bind', () => { it('should fail if the second parser fails', () => { const parser1 = createTestParser('A'); - const failingSecond = create(() => failure()); + const failingSecond = create((_input, index = 0) => + failure(undefined, index), + ); const parser = bind(parser1, () => failingSecond); const result = parser('ABC'); @@ -43,34 +49,34 @@ describe('bind', () => { }); it('should allow the second parser to depend on the first result', () => { - const countParser = create((input) => { - const match = input.match(/^(\d+)/); - if (!match) return failure(); - return success(parseInt(match[1]), input.slice(match[1].length)); + const countParser = create((input, index = 0) => { + const match = input.slice(index).match(/^(\d+)/); + if (!match) return failure(undefined, index); + return success(parseInt(match[1]), index + match[1].length); }); const parser = bind(countParser, (count) => - create((input) => { - const taken = input.slice(0, count); - return success(taken, input.slice(count)); + create((input, index = 0) => { + const taken = input.slice(index, index + count); + return success(taken, index + count); }), ); const result = parser('3abc'); - assertSuccess(result, 'abc', ''); + assertSuccess(result, 'abc', 4); }); it('should handle nested bind chains', () => { const parser = bind(createTestParser('A'), (a) => bind(createTestParser('B'), (b) => bind(createTestParser('C'), (c) => - create(() => success(`${a}${b}${c}`, '')), + create((input) => success(`${a}${b}${c}`, input.length)), ), ), ); const result = parser('ABC'); - assertSuccess(result, 'ABC', ''); + assertSuccess(result, 'ABC', 3); }); }); diff --git a/src/combinators/bind.ts b/src/combinators/bind.ts index 5373e16..eab5dd6 100644 --- a/src/combinators/bind.ts +++ b/src/combinators/bind.ts @@ -1,4 +1,3 @@ -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { type Parser } from '../core/parser'; @@ -6,20 +5,18 @@ import { type Parser } from '../core/parser'; * Chain parsers where the second parser depends on the first result. * * @example - * bind(digits, (n) => take(n))('3abc') // { ok: true, value: 'abc', remaining: '' } + * bind(digits, (n) => take(n))('3abc') // { ok: true, value: 'abc', index: 4 } */ export const bind = ( parser: Parser, f: (a: A) => Parser, ): Parser => { - return create((input) => { - const result = parser(input); + return create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value, remaining } = result; - - return f(value)(remaining); + return f(result.value)(input, result.index, ctx); }); }; diff --git a/src/combinators/braced.test.ts b/src/combinators/braced.test.ts index e107fc2..51ad7ad 100644 --- a/src/combinators/braced.test.ts +++ b/src/combinators/braced.test.ts @@ -12,7 +12,7 @@ describe('braced', () => { const parser = braced(createTestParser('ABC')); const result = parser('{ABC}'); - assertSuccess<'ABC'>(result, 'ABC', ''); + assertSuccess<'ABC'>(result, 'ABC', 5); }); it('should fail with uneven braces', () => { @@ -26,13 +26,13 @@ describe('braced', () => { const parser = braced(createTestParser('ABC')); const result = parser('{ABC}rest'); - assertSuccess<'ABC'>(result, 'ABC', 'rest'); + assertSuccess<'ABC'>(result, 'ABC', 5); }); it('should handle empty braces input', () => { const parser = braced(createTestParser('')); const result = parser('{}'); - assertSuccess<''>(result, '', ''); + assertSuccess<''>(result, '', 2); }); }); diff --git a/src/combinators/braced.ts b/src/combinators/braced.ts index c829ff7..da6e6a3 100644 --- a/src/combinators/braced.ts +++ b/src/combinators/braced.ts @@ -7,7 +7,7 @@ import { surrounded } from './surrounded'; * Parse content surrounded by braces. * * @example - * braced(string('hi'))('{hi}') // { ok: true, value: 'hi', remaining: '' } + * braced(string('hi'))('{hi}') // { ok: true, value: 'hi', index: 4 } */ export const braced = (content: Parser) => { return surrounded(string('{'), content, string('}')); diff --git a/src/combinators/bracketed.test.ts b/src/combinators/bracketed.test.ts index d57688e..ea14800 100644 --- a/src/combinators/bracketed.test.ts +++ b/src/combinators/bracketed.test.ts @@ -12,7 +12,7 @@ describe('bracketed', () => { const parser = bracketed(createTestParser('ABC')); const result = parser('[ABC]'); - assertSuccess<'ABC'>(result, 'ABC', ''); + assertSuccess<'ABC'>(result, 'ABC', 5); }); it('should fail with uneven brackets', () => { @@ -26,13 +26,13 @@ describe('bracketed', () => { const parser = bracketed(createTestParser('ABC')); const result = parser('[ABC]rest'); - assertSuccess<'ABC'>(result, 'ABC', 'rest'); + assertSuccess<'ABC'>(result, 'ABC', 5); }); it('should handle empty brackets input', () => { const parser = bracketed(createTestParser('')); const result = parser('[]'); - assertSuccess<''>(result, '', ''); + assertSuccess<''>(result, '', 2); }); }); diff --git a/src/combinators/bracketed.ts b/src/combinators/bracketed.ts index 7f282c4..e0b868e 100644 --- a/src/combinators/bracketed.ts +++ b/src/combinators/bracketed.ts @@ -7,7 +7,7 @@ import { surrounded } from './surrounded'; * Parse content surrounded by brackets. * * @example - * bracketed(string('hi'))('[hi]') // { ok: true, value: 'hi', remaining: '' } + * bracketed(string('hi'))('[hi]') // { ok: true, value: 'hi', index: 4 } */ export const bracketed = (content: Parser) => { return surrounded(string('['), content, string(']')); diff --git a/src/combinators/chainLeft.test.ts b/src/combinators/chainLeft.test.ts index fa157c0..78d0165 100644 --- a/src/combinators/chainLeft.test.ts +++ b/src/combinators/chainLeft.test.ts @@ -16,14 +16,14 @@ describe('chainLeft', () => { const parser = chainLeft(digits, operation); const result = parser('1+2+3'); - assertSuccess(result, 6, ''); + assertSuccess(result, 6, 5); }); it('should return single value when successful', () => { const parser = chainLeft(digits, operation); const result = parser('42'); - assertSuccess(result, 42, ''); + assertSuccess(result, 42, 2); }); it('should handle empty input', () => { diff --git a/src/combinators/chainLeft.ts b/src/combinators/chainLeft.ts index 2e0f747..08cdda7 100644 --- a/src/combinators/chainLeft.ts +++ b/src/combinators/chainLeft.ts @@ -9,17 +9,20 @@ import { optional } from './optional'; * Chain left-associative operations (right-to-left for same precedence). * * @example - * chainLeft(digits, operation)('1+2+3') // { ok: true, value: 6, remaining: '' } - * chainLeft(digits, operation)('10-3+2') // { ok: true, value: 9, remaining: '' } + * chainLeft(digits, operation)('1+2+3') // { ok: true, value: 6, index: 5 } */ export const chainLeft = ( parser: Parser, operator: Parser<(a: T, b: T) => T>, ) => { - return create((input) => { - const result = optional(chainLeft1(parser, operator))(input); + return create((input, index = 0, ctx) => { + const result = optional(chainLeft1(parser, operator))( + input, + index, + ctx, + ); if (!result.ok || result.value === null) { - return failure(); + return failure(ctx, index); } return result; diff --git a/src/combinators/chainLeft1.test.ts b/src/combinators/chainLeft1.test.ts index c2b28ba..7b04b4f 100644 --- a/src/combinators/chainLeft1.test.ts +++ b/src/combinators/chainLeft1.test.ts @@ -9,35 +9,35 @@ describe('chainLeft1', () => { const parser = chainLeft1(digits, operation); const result = parser('1+1+1'); - assertSuccess(result, 3, ''); // ((1+1)+1) + assertSuccess(result, 3, 5); // ((1+1)+1) }); it('should handle single operand', () => { const parser = chainLeft1(digits, operation); const result = parser('42'); - assertSuccess(result, 42, ''); + assertSuccess(result, 42, 2); }); it('should parse left-associative operations', () => { const parser = chainLeft1(digits, operation); const result = parser('1+2+3'); - assertSuccess(result, 6, ''); // ((1+2)+3) + assertSuccess(result, 6, 5); // ((1+2)+3) }); it('should handle mixed operations with same precedence', () => { const parser = chainLeft1(digits, operation); const result = parser('10-3+2'); - assertSuccess(result, 9, ''); // ((10-3)+2) + assertSuccess(result, 9, 6); // ((10-3)+2) }); it('should stop when operator is not found', () => { const parser = chainLeft1(digits, operation); const result = parser('1+2*3^4'); - assertSuccess(result, 9, '^4'); // 1+2, then stops + assertSuccess(result, 9, 5); // 1+2, then stops }); it('should handle multiplication and division', () => { @@ -45,12 +45,12 @@ describe('chainLeft1', () => { { const result = parser('8/2*3'); - assertSuccess(result, 12, ''); // ((8/2)*3) + assertSuccess(result, 12, 5); // ((8/2)*3) } { const result = parser('24/3/2'); - assertSuccess(result, 4, ''); // ((24/3)/2) + assertSuccess(result, 4, 6); // ((24/3)/2) } }); @@ -65,6 +65,6 @@ describe('chainLeft1', () => { const parser = chainLeft1(digits, operation); const result = parser('1+2+'); - assertSuccess(result, 3, '+'); // Stops at incomplete operation + assertSuccess(result, 3, 3); // Stops at incomplete operation }); }); diff --git a/src/combinators/chainLeft1.ts b/src/combinators/chainLeft1.ts index f8f6d3d..eaf56e9 100644 --- a/src/combinators/chainLeft1.ts +++ b/src/combinators/chainLeft1.ts @@ -1,38 +1,44 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** * Chain left-associative operations (fails on empty input). * * @example - * chainLeft1(digits, operation)('1+2+3') // { ok: true, value: 6, remaining: '' } - * chainLeft1(digits, operation)('8/2*3') // { ok: true, value: 12, remaining: '' } + * chainLeft1(digits, operation)('1+2+3') // { ok: true, value: 6, index: 5 } + * chainLeft1(digits, operation)('8/2*3') // { ok: true, value: 12, index: 5 } */ export const chainLeft1 = ( term: Parser, operator: Parser<(left: T, right: T) => T>, ) => { - return create((input) => { - const firstResult = term(input); - if (!firstResult.ok) return failure(); + return create((input, index = 0, ctx) => { + const firstResult = term(input, index, ctx); + if (!firstResult.ok) { + return firstResult; + } let accumulator = firstResult.value; - let remaining = firstResult.remaining; + let at = firstResult.index; while (true) { - const opResult = operator(remaining); + const opResult: Result<(left: T, right: T) => T> = operator( + input, + at, + ctx, + ); if (!opResult.ok) break; - const nextResult = term(opResult.remaining); + const nextResult: Result = term(input, opResult.index, ctx); if (!nextResult.ok) break; accumulator = opResult.value(accumulator, nextResult.value); - remaining = nextResult.remaining; + at = nextResult.index; } - return success(accumulator, remaining); + return success(accumulator, at); }); }; diff --git a/src/combinators/chainRight.test.ts b/src/combinators/chainRight.test.ts index b787e0f..e4fa4e5 100644 --- a/src/combinators/chainRight.test.ts +++ b/src/combinators/chainRight.test.ts @@ -16,14 +16,14 @@ describe('chainRight', () => { const parser = chainRight(digits, operation); const result = parser('2**3'); - assertSuccess(result, 8, ''); + assertSuccess(result, 8, 4); }); it('should return single value when successful', () => { const parser = chainRight(digits, operation); const result = parser('5'); - assertSuccess(result, 5, ''); + assertSuccess(result, 5, 1); }); it('should handle empty input', () => { @@ -37,6 +37,6 @@ describe('chainRight', () => { const parser = chainRight(digits, operation); const result = parser('10-3-2'); - assertSuccess(result, 9, ''); // 10-(3-2) + assertSuccess(result, 9, 6); // 10-(3-2) }); }); diff --git a/src/combinators/chainRight.ts b/src/combinators/chainRight.ts index 46c4d64..dca8015 100644 --- a/src/combinators/chainRight.ts +++ b/src/combinators/chainRight.ts @@ -6,20 +6,23 @@ import { chainRight1 } from './chainRight1'; import { optional } from './optional'; /** - * Chain right-associative operations (right-to-left grouping). + * Chain right-associative operations. * * @example - * chainRight(digits, operation)('2-1-1') // { ok: true, value: 2, remaining: '' } - * chainRight(digits, operation)('4/2/2') // { ok: true, value: 4, remaining: '' } + * chainRight(digits, operation)('1+2+3') // { ok: true, value: 6, index: 5 } */ export const chainRight = ( parser: Parser, operator: Parser<(a: T, b: T) => T>, ) => { - return create((input) => { - const result = optional(chainRight1(parser, operator))(input); + return create((input, index = 0, ctx) => { + const result = optional(chainRight1(parser, operator))( + input, + index, + ctx, + ); if (!result.ok || result.value === null) { - return failure(); + return failure(ctx, index); } return result; diff --git a/src/combinators/chainRight1.test.ts b/src/combinators/chainRight1.test.ts index 5f20f2d..0c6f40b 100644 --- a/src/combinators/chainRight1.test.ts +++ b/src/combinators/chainRight1.test.ts @@ -9,35 +9,35 @@ describe('chainRight1', () => { const parser = chainRight1(digits, operation); const result = parser('2'); - assertSuccess(result, 2, ''); + assertSuccess(result, 2, 1); }); it('should parse right-associative operations', () => { const parser = chainRight1(digits, operation); const result = parser('2**3**2'); - assertSuccess(result, 512, ''); // 2**(3**2) = 2**9 = 512 + assertSuccess(result, 512, 7); // 2**(3**2) = 2**9 = 512 }); it('should be right-associative', () => { const parser = chainRight1(digits, operation); const result = parser('10-3-2'); - assertSuccess(result, 9, ''); // 10-(3-2) = 9 + assertSuccess(result, 9, 6); // 10-(3-2) = 9 }); it('should stop when operator is not found', () => { const parser = chainRight1(digits, operation); const result = parser('2**3^4'); - assertSuccess(result, 8, '^4'); // 2**3, then stops + assertSuccess(result, 8, 4); // 2**3, then stops }); it('should handle complex right-associative chains', () => { const parser = chainRight1(digits, operation); const result = parser('2**2**3'); - assertSuccess(result, 256, ''); // 2**(2**3) = 2**8 = 256 + assertSuccess(result, 256, 7); // 2**(2**3) = 2**8 = 256 }); it('should fail when first operand fails', () => { @@ -51,6 +51,6 @@ describe('chainRight1', () => { const parser = chainRight1(digits, operation); const result = parser('2**'); - assertSuccess(result, 2, '**'); // Stops at incomplete operation + assertSuccess(result, 2, 1); // Stops at incomplete operation }); }); diff --git a/src/combinators/chainRight1.ts b/src/combinators/chainRight1.ts index a0421fb..ba7cfc1 100644 --- a/src/combinators/chainRight1.ts +++ b/src/combinators/chainRight1.ts @@ -1,6 +1,5 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { type Result } from '../core/result'; import { success } from '../core/success'; @@ -9,28 +8,33 @@ import { success } from '../core/success'; * Chain right-associative operations (fails on empty input). * * @example - * chainRight1(digits, operation)('2-1-1') // { ok: true, value: 2, remaining: '' } - * chainRight1(digits, operation)('4/2/2') // { ok: true, value: 4, remaining: '' } + * chainRight1(digits, operation)('1+2+3') // { ok: true, value: 6, index: 5 } */ export const chainRight1 = ( term: Parser, operator: Parser<(left: T, right: T) => T>, ) => { - return create((input) => { - const leftResult = term(input); - if (!leftResult.ok) return failure(); + const parser: Parser = create((input, index = 0, ctx) => { + const leftResult = term(input, index, ctx); + if (!leftResult.ok) { + return leftResult; + } - const tryRightSide = (leftValue: T, remaining: string): Result => { - const opResult = operator(remaining); - if (!opResult.ok) return success(leftValue, remaining); + const opResult = operator(input, leftResult.index, ctx); + if (!opResult.ok) { + return success(leftResult.value, leftResult.index); + } - const rightResult = chainRight1(term, operator)(opResult.remaining); - if (!rightResult.ok) return success(leftValue, remaining); + const rightResult: Result = parser(input, opResult.index, ctx); + if (!rightResult.ok) { + return success(leftResult.value, leftResult.index); + } - const combinedValue = opResult.value(leftValue, rightResult.value); - return success(combinedValue, rightResult.remaining); - }; - - return tryRightSide(leftResult.value, leftResult.remaining); + return success( + opResult.value(leftResult.value, rightResult.value), + rightResult.index, + ); }); + + return parser; }; diff --git a/src/combinators/choice.test.ts b/src/combinators/choice.test.ts index fbf9211..280b8fd 100644 --- a/src/combinators/choice.test.ts +++ b/src/combinators/choice.test.ts @@ -12,30 +12,42 @@ import { choice } from './choice'; describe('choice', () => { it('should try parsers in order and return first success', () => { - const parserA = create<'A'>(() => failure()); - const parserB = create<'B'>((input) => success('B', input.slice(1))); - const parserC = create<'C'>((input) => success('C', input.slice(1))); + const parserA = create<'A'>((_input, index = 0) => + failure(undefined, index), + ); + const parserB = create<'B'>((input, index = 0) => + success('B', index + 1), + ); + const parserC = create<'C'>((input, index = 0) => + success('C', index + 1), + ); const parser = choice(parserA, parserB, parserC); const result = parser('ABC'); - assertSuccess<'A' | 'B' | 'C'>(result, 'B', 'BC'); + assertSuccess<'A' | 'B' | 'C'>(result, 'B', 1); }); it('should try all parsers if earlier ones fail', () => { - const parserA = create<'A'>(() => failure()); - const parserB = create<'B'>(() => failure()); - const parserC = create<'C'>((input) => success('C', input.slice(1))); + const parserA = create<'A'>((_input, index = 0) => + failure(undefined, index), + ); + const parserB = create<'B'>((_input, index = 0) => + failure(undefined, index), + ); + const parserC = create<'C'>((input, index = 0) => + success('C', index + 1), + ); const parser = choice(parserA, parserB, parserC); const result = parser('C D'); - assertSuccess<'A' | 'B' | 'C'>(result, 'C', ' D'); + assertSuccess<'A' | 'B' | 'C'>(result, 'C', 1); }); it('should fail if all parsers fail', () => { const parser = choice( - () => failure(), - () => failure(), - () => failure(), + (_input, index = 0) => failure(undefined, index), + (_input, index = 0) => failure(undefined, index), + (_input, index = 0) => failure(undefined, index), ); const result = parser('D'); @@ -43,14 +55,16 @@ describe('choice', () => { }); it('should handle different types', () => { - const parserA = create<'A'>((input) => success('A', input.slice(1))); - const parserB = create<'B'[]>((input) => - success(['B'], input.slice(1)), + const parserA = create<'A'>((input, index = 0) => + success('A', index + 1), + ); + const parserB = create<'B'[]>((input, index = 0) => + success(['B'], index + 1), ); const parser = choice(parserA, parserB); const result = parser('ABC'); - assertSuccess<'A' | 'B'[]>(result, 'A', 'BC'); + assertSuccess<'A' | 'B'[]>(result, 'A', 1); }); it('should handle single parser', () => { @@ -58,7 +72,7 @@ describe('choice', () => { const parser = choice(parser1); const result = parser('ABCD'); - assertSuccess<'A'>(result, 'A', 'BCD'); + assertSuccess<'A'>(result, 'A', 1); }); it('should handle empty choices', () => { diff --git a/src/combinators/choice.ts b/src/combinators/choice.ts index 6a3fa0c..24822b9 100644 --- a/src/combinators/choice.ts +++ b/src/combinators/choice.ts @@ -1,27 +1,40 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; +import { union } from '../core/context'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; type ParserOutput

= P extends Parser ? T : never; +const NONE: readonly string[] = []; + /** * Try each parser in order, return first success. * + * Alternatives all start from the same offset, so the expectations of those + * that failed there are unioned into the result. Anything they recorded deeper + * in the input is already in the parse context. + * * @example - * choice(string('hello'), string('world'))('hello') // { ok: true, value: 'hello', remaining: '' } + * choice(string('hello'), string('world'))('hello') // { ok: true, value: 'hello', index: 5 } */ export const choice = []>( ...parsers: P ): Parser> => { - return create>((input) => { + return create>((input, index = 0, ctx) => { + let expected: readonly string[] = NONE; + for (const parser of parsers) { - const result = parser(input); + const result: Result = parser(input, index, ctx); if (result.ok) { - return result; + return result as Result>; + } + + if (result.index === index) { + expected = union(expected, result.expected); } } - return failure(); + return { ok: false, index, expected }; }); }; diff --git a/src/combinators/concat.test.ts b/src/combinators/concat.test.ts index 25991e3..bec19e1 100644 --- a/src/combinators/concat.test.ts +++ b/src/combinators/concat.test.ts @@ -6,35 +6,35 @@ import { create } from '../core/parser'; import { success } from '../core/success'; import { concat } from './concat'; -const abcParser = create((input) => { - if (input.startsWith('abc')) { - return success(['a', 'b', 'c'], input.slice(3)); +const abcParser = create((input, index = 0) => { + if (input.startsWith('abc', index)) { + return success(['a', 'b', 'c'], index + 3); } - return failure(); + return failure(undefined, index); }); -const emptyParser = create((input) => success([], input)); +const emptyParser = create((input, index = 0) => success([], index)); describe('concat', () => { it('should join array results into string', () => { const parser = concat(abcParser); const result = parser('abc'); - assertSuccess(result, 'abc', ''); + assertSuccess(result, 'abc', 3); }); it('should join with separator', () => { const parser = concat(abcParser, '-'); const result = parser('abc'); - assertSuccess(result, 'a-b-c', ''); + assertSuccess(result, 'a-b-c', 3); }); it('should return empty string for empty array', () => { const parser = concat(emptyParser); const result = parser('xyz'); - assertSuccess(result, '', 'xyz'); + assertSuccess(result, '', 0); }); }); diff --git a/src/combinators/concat.ts b/src/combinators/concat.ts index 6151ccd..766c982 100644 --- a/src/combinators/concat.ts +++ b/src/combinators/concat.ts @@ -6,8 +6,8 @@ import { map } from './map'; * Join string array parser result into a single string. * * @example - * concat(many(letter))('abc123') // { ok: true, value: 'abc', remaining: '123' } - * concat(many(letter), '-')('abc123') // { ok: true, value: 'a-b-c', remaining: '123' } + * concat(many(letter))('abc123') // { ok: true, value: 'abc', index: 3 } + * concat(many(letter), '-')('abc123') // { ok: true, value: 'a-b-c', index: 3 } */ export const concat = ( parser: Parser, diff --git a/src/combinators/consume.test.ts b/src/combinators/consume.test.ts index 6dee132..4fdf01b 100644 --- a/src/combinators/consume.test.ts +++ b/src/combinators/consume.test.ts @@ -13,7 +13,7 @@ describe('consume', () => { const parser = consume(parser1); const result = parser('ABBB'); - assertSuccess(result, null, 'BBB'); + assertSuccess(result, null, 1); }); it('should fail when underlying parser fails', () => { diff --git a/src/combinators/consume.ts b/src/combinators/consume.ts index b2f6c8b..ca9fd39 100644 --- a/src/combinators/consume.ts +++ b/src/combinators/consume.ts @@ -7,7 +7,7 @@ import { map } from './map'; * Consume input but discard the result (return null). * * @example - * consume(string('hello'))('hello world') // { ok: true, value: null, remaining: ' world' } + * consume(string('hello'))('hello world') // { ok: true, value: null, index: 5 } */ export const consume = (parser: Parser) => { return create(map(parser, () => null)); diff --git a/src/combinators/endBy.test.ts b/src/combinators/endBy.test.ts index eabcc8b..5d64705 100644 --- a/src/combinators/endBy.test.ts +++ b/src/combinators/endBy.test.ts @@ -10,7 +10,7 @@ describe('endBy', () => { const parser = endBy(parser1, parser2); const result = parser('A,A,A,'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 6); }); it('should parse empty list when no elements', () => { @@ -19,7 +19,7 @@ describe('endBy', () => { const parser = endBy(parser1, parser2); const result = parser('CCC'); - assertSuccess<'A'[]>(result, [], 'CCC'); + assertSuccess<'A'[]>(result, [], 0); }); it('should require terminator after each element', () => { @@ -28,7 +28,7 @@ describe('endBy', () => { const parser = endBy(parser1, parser2); const result = parser('A,A,A'); - assertSuccess<'A'[]>(result, ['A', 'A'], 'A'); + assertSuccess<'A'[]>(result, ['A', 'A'], 4); }); it('should handle single element with terminator', () => { @@ -38,7 +38,7 @@ describe('endBy', () => { const parser = endBy(parser1, parser2); const result = parser('A,'); - assertSuccess<'A'[]>(result, ['A'], ''); + assertSuccess<'A'[]>(result, ['A'], 2); }); it('should handle empty input', () => { @@ -47,6 +47,6 @@ describe('endBy', () => { const parser = endBy(parser1, parser2); const result = parser(''); - assertSuccess<'A'[]>(result, [], ''); + assertSuccess<'A'[]>(result, [], 0); }); }); diff --git a/src/combinators/endBy.ts b/src/combinators/endBy.ts index 4af07d7..cf502dd 100644 --- a/src/combinators/endBy.ts +++ b/src/combinators/endBy.ts @@ -9,7 +9,7 @@ import { sequence } from './sequence'; * Zero or more items separated and ending with terminator. * * @example - * endBy(string('item'), char(';'))('item;item;item;') // { ok: true, value: ['item', 'item', 'item'], remaining: '' } + * endBy(string('item'), char(';'))('item;item;item;') // { ok: true, value: ['item', 'item', 'item'], index: 15 } */ export const endBy = (parser: Parser, terminator: Parser) => { return create( diff --git a/src/combinators/endBy1.test.ts b/src/combinators/endBy1.test.ts index d360764..a05d52f 100644 --- a/src/combinators/endBy1.test.ts +++ b/src/combinators/endBy1.test.ts @@ -23,7 +23,7 @@ describe('endBy1', () => { const parser = endBy1(parser1, parser2); const result = parser('A,'); - assertSuccess<'A'[]>(result, ['A'], ''); + assertSuccess<'A'[]>(result, ['A'], 2); }); it('should parse multiple elements each with terminator', () => { @@ -32,7 +32,7 @@ describe('endBy1', () => { const parser = endBy1(parser1, parser2); const result = parser('A,A,A,'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 6); }); it('should not match when element lacks terminator', () => { @@ -41,7 +41,7 @@ describe('endBy1', () => { const parser = endBy1(parser1, parser2); const result = parser('A,A,A'); - assertSuccess<'A'[]>(result, ['A', 'A'], 'A'); + assertSuccess<'A'[]>(result, ['A', 'A'], 4); }); it('should fail on empty input', () => { diff --git a/src/combinators/endBy1.ts b/src/combinators/endBy1.ts index fd783f4..a494944 100644 --- a/src/combinators/endBy1.ts +++ b/src/combinators/endBy1.ts @@ -9,7 +9,7 @@ import { sequence } from './sequence'; * One or more items separated and ending with terminator. * * @example - * endBy1(string('item'), char(';'))('item;item;item;') // { ok: true, value: ['item', 'item', 'item'], remaining: '' } + * endBy1(string('item'), char(';'))('item;item;item;') // { ok: true, value: ['item', 'item', 'item'], index: 15 } */ export const endBy1 = (parser: Parser, terminator: Parser) => { return create( diff --git a/src/combinators/exactly.test.ts b/src/combinators/exactly.test.ts index d63b52b..e9de6d4 100644 --- a/src/combinators/exactly.test.ts +++ b/src/combinators/exactly.test.ts @@ -14,7 +14,7 @@ describe('exactly', () => { const parser = exactly(parser1, 3); const result = parser('AAABCD'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 3); }); it('should fail if fewer than n occurrences', () => { @@ -28,14 +28,14 @@ describe('exactly', () => { const parser = exactly(parser1, 2); const result = parser('AAAAA'); - assertSuccess<'A'[]>(result, ['A', 'A'], 'AAA'); + assertSuccess<'A'[]>(result, ['A', 'A'], 2); }); it('should handle count of zero', () => { const parser = exactly(parser1, 0); const result = parser('AAABCD'); - assertSuccess<'A'[]>(result, [], 'AAABCD'); + assertSuccess<'A'[]>(result, [], 0); }); it('should fail on empty input when count > 0', () => { diff --git a/src/combinators/exactly.ts b/src/combinators/exactly.ts index ba3fea3..8bbb494 100644 --- a/src/combinators/exactly.ts +++ b/src/combinators/exactly.ts @@ -1,30 +1,30 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** * Parse exactly n occurrences. * * @example - * exactly(char('a'), 3)('aaa') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * exactly(char('a'), 3)('aaa') // { ok: true, value: ['a', 'a', 'a'], index: 3 } */ export const exactly = (parser: Parser, n: number) => { - return create((input) => { + return create((input, index = 0, ctx) => { const results: T[] = []; - let remaining = input; + let at = index; for (let i = 0; i < n; i++) { - const result = parser(remaining); + const result: Result = parser(input, at, ctx); if (!result.ok) { - return failure(); + return result; } results.push(result.value); - remaining = result.remaining; + at = result.index; } - return success(results, remaining); + return success(results, at); }); }; diff --git a/src/combinators/first.test.ts b/src/combinators/first.test.ts index 8629d9c..5fe4166 100644 --- a/src/combinators/first.test.ts +++ b/src/combinators/first.test.ts @@ -7,18 +7,20 @@ import { first } from './first'; describe('first', () => { it('should return first element of array parser result', () => { - const parser1 = create(() => success(['A', 'B', 'C'] as const, '')); + const parser1 = create((input) => + success(['A', 'B', 'C'] as const, input.length), + ); const parser = first(parser1); const result = parser('ABC'); - assertSuccess<'A'>(result, 'A', ''); + assertSuccess<'A'>(result, 'A', 3); }); it('should return undefined for empty array', () => { - const parser1 = create(() => success([] as const, 'ABC')); + const parser1 = create((_input, _index = 0) => success([] as const, 0)); const parser = first(parser1); const result = parser('ABC'); - assertSuccess(result, undefined, 'ABC'); + assertSuccess(result, undefined, 0); }); }); diff --git a/src/combinators/first.ts b/src/combinators/first.ts index c31c8df..6a369e8 100644 --- a/src/combinators/first.ts +++ b/src/combinators/first.ts @@ -16,7 +16,7 @@ export type First = T extends readonly [ * Extract the first element from a parser result array. * * @example - * first(sequence(char('a'), digit))('a1bc') // { ok: true, value: 'a', remaining: 'bc' } + * first(sequence(char('a'), digit))('a1bc') // { ok: true, value: 'a', index: 2 } */ export const first = ( parser: Parser, diff --git a/src/combinators/flag.test.ts b/src/combinators/flag.test.ts index a1b3784..1589c10 100644 --- a/src/combinators/flag.test.ts +++ b/src/combinators/flag.test.ts @@ -12,20 +12,22 @@ describe('flag', () => { const parser = flag(aParser); const result = parser('ABC'); - assertSuccess(result, true, 'BC'); + assertSuccess(result, true, 1); }); it('should return false when parser fails', () => { - const parser = flag(create<'A'>(() => failure())); + const parser = flag( + create<'A'>((_input, index = 0) => failure(undefined, index)), + ); const result = parser('ABC'); - assertSuccess(result, false, 'ABC'); + assertSuccess(result, false, 0); }); it('should not consume input on failure', () => { const parser = flag(aParser); const result = parser('XYZ'); - assertSuccess(result, false, 'XYZ'); + assertSuccess(result, false, 0); }); }); diff --git a/src/combinators/flag.ts b/src/combinators/flag.ts index d31dcef..0dda982 100644 --- a/src/combinators/flag.ts +++ b/src/combinators/flag.ts @@ -7,15 +7,13 @@ import { success } from '../core/success'; * Return true if parser succeeds, false otherwise. Always succeeds without consuming input on failure. * * @example - * flag(string('*'))('*abc') // { ok: true, value: true, remaining: 'abc' } - * flag(string('*'))('abc') // { ok: true, value: false, remaining: 'abc' } + * flag(string('*'))('*abc') // { ok: true, value: true, index: 1 } + * flag(string('*'))('abc') // { ok: true, value: false, index: 0 } */ export const flag = (parser: Parser): Parser => { - return create((input) => { - const result = parser(input); + return create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); - return result.ok - ? success(true, result.remaining) - : success(false, input); + return result.ok ? success(true, result.index) : success(false, index); }); }; diff --git a/src/combinators/fold.test.ts b/src/combinators/fold.test.ts index 74fb258..2211158 100644 --- a/src/combinators/fold.test.ts +++ b/src/combinators/fold.test.ts @@ -7,11 +7,11 @@ import { success } from '../core/success'; import { fold } from './fold'; describe('fold', () => { - const stringParser = create((input: string) => { - if (input.length === 0) { - return failure(); + const stringParser = create((input, index = 0) => { + if (index >= input.length) { + return failure(undefined, index); } - return success(input[0], input.slice(1)); + return success(input[index], index + 1); }); it('should fold left over parsed items', () => { @@ -22,29 +22,29 @@ describe('fold', () => { ); const result = parser('ABC'); - assertSuccess(result, '(((ZA)B)C)', ''); + assertSuccess(result, '(((ZA)B)C)', 3); }); it('should work with empty input (return initial value and not consume input)', () => { const parser = fold( - create(() => failure()), + create((_input, index = 0) => failure(undefined, index)), 'Z', (acc, item) => `(${acc}${item})`, ); const result = parser('ABC'); - assertSuccess(result, 'Z', 'ABC'); + assertSuccess(result, 'Z', 0); }); it('should work with empty input (return initial value)', () => { const parser = fold( - create(() => failure()), + create((_input, index = 0) => failure(undefined, index)), 'Z', (acc, item) => `(${acc}${item})`, ); const result = parser(''); - assertSuccess(result, 'Z', ''); + assertSuccess(result, 'Z', 0); }); it('should work with complex accumulator types', () => { @@ -61,7 +61,7 @@ describe('fold', () => { assertSuccess<{ label: string; count: number; - }>(result, { label: 'ABC', count: 3 }, ''); + }>(result, { label: 'ABC', count: 3 }, 3); }); it('should work with array building', () => { @@ -71,14 +71,16 @@ describe('fold', () => { ]); const result = parser('ABC'); - assertSuccess(result, ['AZ', 'BZ', 'CZ'], ''); + assertSuccess(result, ['AZ', 'BZ', 'CZ'], 3); }); it('should not fail and return the initial value and not consume', () => { - const failureParser = create(() => failure()); + const failureParser = create((_input, index = 0) => + failure(undefined, index), + ); const parser = fold(failureParser, 0, (acc) => acc + 1); const result = parser('ABC'); - assertSuccess(result, 0, 'ABC'); + assertSuccess(result, 0, 0); }); }); diff --git a/src/combinators/fold.ts b/src/combinators/fold.ts index 280884d..c9dcfe8 100644 --- a/src/combinators/fold.ts +++ b/src/combinators/fold.ts @@ -1,22 +1,23 @@ -import { create, type Parser } from '../core/parser'; -import { type Success, success } from '../core/success'; +import type { Parser } from '../core/parser'; + +import { create } from '../core/parser'; +import { success, type Success } from '../core/success'; import { many } from './many'; /** - * Parse zero or more and fold into a single value. + * Fold zero or more occurrences into a single value. * * @example - * fold(digit, [], (acc, d) => [...acc, d])('123') // { ok: true, value: [1, 2, 3], remaining: '' } + * fold(digit, 0, (acc, d) => acc + d)('123') // { ok: true, value: 6, index: 3 } */ export const fold = ( parser: Parser, initial: U, folder: (acc: U, item: T) => U, ): Parser => { - return create((input) => { - const result = many(parser)(input) as Success; - const { value, remaining } = result; + return create((input, index = 0, ctx) => { + const result = many(parser)(input, index, ctx) as Success; - return success(value.reduce(folder, initial), remaining); + return success(result.value.reduce(folder, initial), result.index); }); }; diff --git a/src/combinators/fold1.test.ts b/src/combinators/fold1.test.ts index 25a2707..6287889 100644 --- a/src/combinators/fold1.test.ts +++ b/src/combinators/fold1.test.ts @@ -8,21 +8,23 @@ import { fold1 } from './fold1'; describe('fold1', () => { it('should fold left over parsed items', () => { - const stringParser = create((input: string) => { - if (input.length === 0) { - return failure(); + const stringParser = create((input, index = 0) => { + if (index >= input.length) { + return failure(undefined, index); } - return success(input[0], input.slice(1)); + return success(input[index], index + 1); }); const parser = fold1(stringParser, '', (acc, value) => acc + value); const result = parser('ABC'); - assertSuccess(result, 'ABC', ''); + assertSuccess(result, 'ABC', 3); }); it('should return null, one or more successful parser returns are required', () => { - const failureParser = create(() => failure()); + const failureParser = create((_input, index = 0) => + failure(undefined, index), + ); const parser = fold1(failureParser, 42, (acc, digit) => acc + digit); const result = parser('ABC'); diff --git a/src/combinators/fold1.ts b/src/combinators/fold1.ts index 5009f37..874f725 100644 --- a/src/combinators/fold1.ts +++ b/src/combinators/fold1.ts @@ -1,35 +1,29 @@ -import { failure } from '../core/failure'; -import { create, type Parser } from '../core/parser'; -import { type Success, success } from '../core/success'; +import type { Parser } from '../core/parser'; + +import { create } from '../core/parser'; +import { success, type Success } from '../core/success'; import { many } from './many'; /** - * Parse one or more and fold into a single value. + * Fold one or more occurrences into a single value. * * @example - * fold1(digit, 0, (acc, d) => acc + d)('123') // { ok: true, value: 6, remaining: '' } + * fold1(digit, 0, (acc, d) => acc + d)('123') // { ok: true, value: 6, index: 3 } */ export const fold1 = ( parser: Parser, initial: U, folder: (acc: U, item: T) => U, ): Parser => { - return create((input) => { - const first = parser(input); + return create((input, index = 0, ctx) => { + const first = parser(input, index, ctx); if (!first.ok) { - return failure(); + return first; } - const { value: firstValue, remaining } = first; - - let acc = folder(initial, firstValue); - - const { value: items, remaining: finalRest } = many(parser)( - remaining, - ) as Success; - - acc = items.reduce(folder, acc); + const rest = many(parser)(input, first.index, ctx) as Success; + const acc = rest.value.reduce(folder, folder(initial, first.value)); - return success(acc, finalRest); + return success(acc, rest.index); }); }; diff --git a/src/combinators/foldRight.test.ts b/src/combinators/foldRight.test.ts index e6c13ee..056013b 100644 --- a/src/combinators/foldRight.test.ts +++ b/src/combinators/foldRight.test.ts @@ -7,12 +7,12 @@ import { success } from '../core/success'; import { foldRight } from './foldRight'; describe('foldRight', () => { - const stringParser = create((input: string) => { - if (input.length === 0) { - return failure(); + const stringParser = create((input, index = 0) => { + if (index >= input.length) { + return failure(undefined, index); } - return success(input[0], input.slice(1)); + return success(input[index], index + 1); }); it('should fold right over parsed items', () => { @@ -23,29 +23,29 @@ describe('foldRight', () => { ); const result = parser('ABC'); - assertSuccess(result, '(((ZC)B)A)', ''); + assertSuccess(result, '(((ZC)B)A)', 3); }); it('should work with empty input (return initial value and not consume input)', () => { const parser = foldRight( - create(() => failure()), + create((_input, index = 0) => failure(undefined, index)), 'Z', (acc, item) => `(${acc}${item})`, ); const result = parser('ABC'); - assertSuccess(result, 'Z', 'ABC'); + assertSuccess(result, 'Z', 0); }); it('should work with empty input (return initial value)', () => { const parser = foldRight( - create(() => failure()), + create((_input, index = 0) => failure(undefined, index)), 'Z', (acc, item) => `(${acc}${item})`, ); const result = parser(''); - assertSuccess(result, 'Z', ''); + assertSuccess(result, 'Z', 0); }); it('should work with complex accumulator types', () => { @@ -62,7 +62,7 @@ describe('foldRight', () => { assertSuccess<{ label: string; count: number; - }>(result, { label: 'ABC', count: 3 }, ''); + }>(result, { label: 'ABC', count: 3 }, 3); }); it('should work with array building', () => { @@ -72,14 +72,16 @@ describe('foldRight', () => { ]); const result = parser('ABC'); - assertSuccess(result, ['CZ', 'BZ', 'AZ'], ''); + assertSuccess(result, ['CZ', 'BZ', 'AZ'], 3); }); it('should not fail and return the initial value and not consume', () => { - const parserFail = create(() => failure()); + const parserFail = create((_input, index = 0) => + failure(undefined, index), + ); const parser = foldRight(parserFail, 0, (acc) => acc + 1); const result = parser('ABC'); - assertSuccess(result, 0, 'ABC'); + assertSuccess(result, 0, 0); }); }); diff --git a/src/combinators/foldRight.ts b/src/combinators/foldRight.ts index 7b85d08..44eb945 100644 --- a/src/combinators/foldRight.ts +++ b/src/combinators/foldRight.ts @@ -1,21 +1,23 @@ -import { create, type Parser } from '../core/parser'; -import { type Success, success } from '../core/success'; +import type { Parser } from '../core/parser'; + +import { create } from '../core/parser'; +import { success, type Success } from '../core/success'; import { many } from './many'; /** - * Parse zero or more and fold right-to-left. + * Fold zero or more occurrences from the right into a single value. * * @example - * foldRight(digit, [], (acc, d) => [...acc, d])('123') // { ok: true, value: [3, 2, 1], remaining: '' } + * foldRight(digit, 0, (acc, d) => acc + d)('123') // { ok: true, value: 6, index: 3 } */ export const foldRight = ( parser: Parser, initial: U, folder: (acc: U, item: T) => U, ): Parser => { - return create((input) => { - const { value, remaining } = many(parser)(input) as Success; + return create((input, index = 0, ctx) => { + const result = many(parser)(input, index, ctx) as Success; - return success(value.reduceRight(folder, initial), remaining); + return success(result.value.reduceRight(folder, initial), result.index); }); }; diff --git a/src/combinators/foldRight1.test.ts b/src/combinators/foldRight1.test.ts index 92e17c3..7d7f433 100644 --- a/src/combinators/foldRight1.test.ts +++ b/src/combinators/foldRight1.test.ts @@ -7,11 +7,11 @@ import { success } from '../core/success'; import { foldRight1 } from './foldRight1'; describe('foldRight1', () => { - const stringParser = create((input: string) => { - if (input.length === 0) { - return failure(); + const stringParser = create((input, index = 0) => { + if (index >= input.length) { + return failure(undefined, index); } - return success(input[0], input.slice(1)); + return success(input[index], index + 1); }); it('should fold right over parsed items', () => { @@ -22,11 +22,13 @@ describe('foldRight1', () => { ); const result = parser('ABC'); - assertSuccess(result, '(((ZC)B)A)', ''); + assertSuccess(result, '(((ZC)B)A)', 3); }); it('should return null, one or more successful parser returns are required', () => { - const failureParser = create(() => failure()); + const failureParser = create((_input, index = 0) => + failure(undefined, index), + ); const parser = foldRight1( failureParser, 42, diff --git a/src/combinators/foldRight1.ts b/src/combinators/foldRight1.ts index 24933d8..c59a77f 100644 --- a/src/combinators/foldRight1.ts +++ b/src/combinators/foldRight1.ts @@ -1,33 +1,29 @@ -import { failure } from '../core/failure'; -import { create, type Parser } from '../core/parser'; -import { type Success, success } from '../core/success'; +import type { Parser } from '../core/parser'; + +import { create } from '../core/parser'; +import { success, type Success } from '../core/success'; import { many } from './many'; /** - * Parse one or more and fold right-to-left. + * Fold one or more occurrences from the right into a single value. * * @example - * foldRight1(digit, [], (acc, d) => [...acc, d])('123') // { ok: true, value: [3, 2, 1], remaining: '' } + * foldRight1(digit, 0, (acc, d) => acc + d)('123') // { ok: true, value: 6, index: 3 } */ export const foldRight1 = ( parser: Parser, initial: U, folder: (acc: U, item: T) => U, ): Parser => { - return create((input) => { - const first = parser(input); + return create((input, index = 0, ctx) => { + const first = parser(input, index, ctx); if (!first.ok) { - return failure(); + return first; } - const { value: firstValue, remaining: rest } = first; - const { value: items, remaining: finalRest } = many(parser)( - rest, - ) as Success; - - const all = [firstValue, ...items]; - const folded = all.reduceRight(folder, initial); + const rest = many(parser)(input, first.index, ctx) as Success; + const all = [first.value, ...rest.value]; - return success(folded, finalRest); + return success(all.reduceRight(folder, initial), rest.index); }); }; diff --git a/src/combinators/fuse.test.ts b/src/combinators/fuse.test.ts index c888a6a..7735de5 100644 --- a/src/combinators/fuse.test.ts +++ b/src/combinators/fuse.test.ts @@ -16,7 +16,7 @@ describe('fuse', () => { ); const result = parser('abc'); - assertSuccess(result, 'abc', ''); + assertSuccess(result, 'abc', 3); }); it('should fail on non-matching input', () => { @@ -38,20 +38,20 @@ describe('fuse', () => { ); const result = parser('hello world'); - assertSuccess(result, 'hello world', ''); + assertSuccess(result, 'hello world', 11); }); it('should fuse two parsers into string', () => { const parser = fuse(createTestParser('a'), createTestParser('b')); const result = parser('ab'); - assertSuccess(result, 'ab', ''); + assertSuccess(result, 'ab', 2); }); it('should work with single parser', () => { const parser = fuse(createTestParser('a')); const result = parser('a'); - assertSuccess(result, 'a', ''); + assertSuccess(result, 'a', 1); }); }); diff --git a/src/combinators/fuse.ts b/src/combinators/fuse.ts index 0dfdbc0..9aeaac2 100644 --- a/src/combinators/fuse.ts +++ b/src/combinators/fuse.ts @@ -1,4 +1,7 @@ -import { create, type Parser } from '../core/parser'; +import type { Parser } from '../core/parser'; + +import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** @@ -6,23 +9,24 @@ import { success } from '../core/success'; * The fused parser concatenates all string results. * * @example - * fuse(char('a'), char('b'), char('c'))('abc') // { ok: true, value: 'abc', remaining: '' } - * fuse(string('hello'), char(' '), string('world'))('hello world') // { ok: true, value: 'hello world', remaining: '' } + * fuse(char('a'), char('b'), char('c'))('abc') // { ok: true, value: 'abc', index: 3 } + * fuse(string('hello'), char(' '), string('world'))('hello world') // { ok: true, value: 'hello world', index: 11 } */ export const fuse = []>(...parsers: T) => { - return create((input) => { - let result = ''; - let remaining = input; + return create((input, index = 0, ctx) => { + let value = ''; + let at = index; for (const parser of parsers) { - const parsed = parser(remaining); + const parsed: Result = parser(input, at, ctx); if (!parsed.ok) { - return { ok: false }; + return parsed; } - result += parsed.value; - remaining = parsed.remaining; + + value += parsed.value; + at = parsed.index; } - return success(result, remaining); + return success(value, at); }); }; diff --git a/src/combinators/guard.test.ts b/src/combinators/guard.test.ts index 4ecedbe..3a47be8 100644 --- a/src/combinators/guard.test.ts +++ b/src/combinators/guard.test.ts @@ -13,7 +13,7 @@ describe('guard', () => { const parser = guard(true, parser1); const result = parser('AAA'); - assertSuccess<'A' | null>(result, 'A', 'AA'); + assertSuccess<'A' | null>(result, 'A', 1); }); it('should fail when condition is false', () => { diff --git a/src/combinators/guard.ts b/src/combinators/guard.ts index 8c70fe0..0b22ed7 100644 --- a/src/combinators/guard.ts +++ b/src/combinators/guard.ts @@ -7,17 +7,15 @@ import { create } from '../core/parser'; * Conditionally apply parser based on a condition. * * @example - * guard(true, string('hello'))('hello') // { ok: true, value: 'hello', remaining: '' } - * guard(false, string('hello'))('hello') // { ok: false } + * guard(true, string('hello'))('hello') // { ok: true, value: 'hello', index: 5 } + * guard(false, string('hello'))('hello') // { ok: false, index: 0, expected: [] } */ export const guard = (condition: boolean, parser: Parser) => { - return create((input) => { + return create((input, index = 0, ctx) => { if (!condition) { - return failure(); + return failure(ctx, index); } - const result = parser(input); - - return result; + return parser(input, index, ctx); }); }; diff --git a/src/combinators/inner.test.ts b/src/combinators/inner.test.ts index 071f495..29460cc 100644 --- a/src/combinators/inner.test.ts +++ b/src/combinators/inner.test.ts @@ -16,7 +16,7 @@ describe('inner', () => { const parser = inner(parser1, parser2, parser3); const result = parser('ABC'); - assertSuccess<'B'>(result, 'B', ''); + assertSuccess<'B'>(result, 'B', 3); }); it('should fail if first parser fails', () => { diff --git a/src/combinators/inner.ts b/src/combinators/inner.ts index efb1990..2a7bed8 100644 --- a/src/combinators/inner.ts +++ b/src/combinators/inner.ts @@ -8,7 +8,7 @@ import { sequence } from './sequence'; * Extract inner value from surrounded content (like inner of braced). * * @example - * inner(char('('), string('hi'), char(')'))('(hi)') // { ok: true, value: 'hi', remaining: '' } + * inner(char('('), string('hi'), char(')'))('(hi)') // { ok: true, value: 'hi', index: 4 } */ export const inner = ( parserA: Parser, diff --git a/src/combinators/interleaved.test.ts b/src/combinators/interleaved.test.ts index 03540b5..d9c5945 100644 --- a/src/combinators/interleaved.test.ts +++ b/src/combinators/interleaved.test.ts @@ -14,7 +14,7 @@ describe('interleaved', () => { const parser = interleaved(parser1, parser2); const result = parser('ABABA'); - assertSuccess<('A' | 'B')[]>(result, ['A', 'B', 'A', 'B', 'A'], ''); + assertSuccess<('A' | 'B')[]>(result, ['A', 'B', 'A', 'B', 'A'], 5); }); it('should handle single item', () => { @@ -23,7 +23,7 @@ describe('interleaved', () => { const parser = interleaved(parser1, parser2); const result = parser('AC'); - assertSuccess<('A' | 'B')[]>(result, ['A'], 'C'); + assertSuccess<('A' | 'B')[]>(result, ['A'], 1); }); it('should fail when first item does not match', () => { @@ -41,6 +41,6 @@ describe('interleaved', () => { const parser = interleaved(parser1, parser2); const result = parser('ABABC'); - assertSuccess<('A' | 'B')[]>(result, ['A', 'B', 'A'], 'BC'); + assertSuccess<('A' | 'B')[]>(result, ['A', 'B', 'A'], 3); }); }); diff --git a/src/combinators/interleaved.ts b/src/combinators/interleaved.ts index bc5df84..cd1aa00 100644 --- a/src/combinators/interleaved.ts +++ b/src/combinators/interleaved.ts @@ -1,35 +1,37 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** - * Parse items with interleaved separators. + * Parse items separated by separators, keeping both in the result. * * @example - * interleaved(char('a'), char(','))('a,a,a') // { ok: true, value: ['a', ',', 'a', ',', 'a'], remaining: '' } + * interleaved(digits, char('+'))('1+2') // { ok: true, value: [1, '+', 2], index: 3 } */ export const interleaved = (item: Parser, separator: Parser) => { - return create>((input) => { - const firstResult = item(input); - if (!firstResult.ok) return failure(); + return create>((input, index = 0, ctx) => { + const firstResult = item(input, index, ctx); + if (!firstResult.ok) { + return firstResult; + } const results: Array = [firstResult.value]; - let remaining = firstResult.remaining; + let at = firstResult.index; while (true) { - const sepResult = separator(remaining); + const sepResult: Result = separator(input, at, ctx); if (!sepResult.ok) break; - const nextResult = item(sepResult.remaining); + const nextResult: Result = item(input, sepResult.index, ctx); if (!nextResult.ok) break; results.push(sepResult.value); results.push(nextResult.value); - remaining = nextResult.remaining; + at = nextResult.index; } - return success(results, remaining); + return success(results, at); }); }; diff --git a/src/combinators/last.test.ts b/src/combinators/last.test.ts index 3107211..74c7741 100644 --- a/src/combinators/last.test.ts +++ b/src/combinators/last.test.ts @@ -7,20 +7,22 @@ import { last } from './last'; describe('last', () => { it('should return last element of array parser result', () => { - const parser1 = create(() => success(['A', 'B', 'C'] as const, '')); + const parser1 = create((input) => + success(['A', 'B', 'C'] as const, input.length), + ); const parser = last(parser1); const result = parser('ABC'); - assertSuccess<'C'>(result, 'C', ''); + assertSuccess<'C'>(result, 'C', 3); }); it('should return undefined for empty array', () => { const parser1 = create(() => - success([] as unknown as [unknown, ...unknown[]], 'ABC'), + success([] as unknown as [unknown, ...unknown[]], 0), ); const parser = last(parser1); const result = parser('ABC'); - assertSuccess(result, undefined, 'ABC'); + assertSuccess(result, undefined, 0); }); }); diff --git a/src/combinators/last.ts b/src/combinators/last.ts index e538942..b2408de 100644 --- a/src/combinators/last.ts +++ b/src/combinators/last.ts @@ -16,7 +16,7 @@ export type Last = T extends readonly [ * Extract the last element from a parser result array. * * @example - * last(sequence(char('a'), char('b')))('ab') // { ok: true, value: 'b', remaining: '' } + * last(sequence(char('a'), char('b')))('ab') // { ok: true, value: 'b', index: 2 } */ export const last = (parser: Parser) => { return create>( diff --git a/src/combinators/left.test.ts b/src/combinators/left.test.ts index 949c8d6..4147c53 100644 --- a/src/combinators/left.test.ts +++ b/src/combinators/left.test.ts @@ -15,7 +15,7 @@ describe('left', () => { const parser = left(parser1, parser2); const result = parser('AB'); - assertSuccess<'A'>(result, 'A', ''); + assertSuccess<'A'>(result, 'A', 2); }); it('should fail if first parser fails', () => { diff --git a/src/combinators/left.ts b/src/combinators/left.ts index 204557d..0be510b 100644 --- a/src/combinators/left.ts +++ b/src/combinators/left.ts @@ -8,7 +8,7 @@ import { sequence } from './sequence'; * Keep only the left result from a sequence. * * @example - * left(string('hello'), string('world'))('helloworld') // { ok: true, value: 'hello', remaining: '' } + * left(string('hello'), string('world'))('helloworld') // { ok: true, value: 'hello', index: 10 } */ export const left = (parserA: Parser, parserB: Parser) => { return create(map(sequence(parserA, parserB), ([a]) => a)); diff --git a/src/combinators/lexeme.test.ts b/src/combinators/lexeme.test.ts index 6d7436c..deb2ba9 100644 --- a/src/combinators/lexeme.test.ts +++ b/src/combinators/lexeme.test.ts @@ -13,7 +13,7 @@ describe('lexeme', () => { const parser = lexeme(parser1); const result = parser('A B'); - assertSuccess<'A'>(result, 'A', 'B'); + assertSuccess<'A'>(result, 'A', 8); }); it('should parse a token with no trailing whitespace', () => { @@ -21,7 +21,7 @@ describe('lexeme', () => { const parser = lexeme(parser1); const result = parser('AB'); - assertSuccess<'A'>(result, 'A', 'B'); + assertSuccess<'A'>(result, 'A', 1); }); it('should consume various types of whitespace', () => { @@ -29,7 +29,7 @@ describe('lexeme', () => { const parser = lexeme(parser1); const result = parser('A \t\n\r B'); - assertSuccess<'A'>(result, 'A', 'B'); + assertSuccess<'A'>(result, 'A', 7); }); it('should fail when the underlying parser fails', () => { @@ -45,6 +45,6 @@ describe('lexeme', () => { const parser = lexeme(parser1); const result = parser('A '); - assertSuccess<'A'>(result, 'A', ''); + assertSuccess<'A'>(result, 'A', 4); }); }); diff --git a/src/combinators/lexeme.ts b/src/combinators/lexeme.ts index 6ac370e..c29b3b6 100644 --- a/src/combinators/lexeme.ts +++ b/src/combinators/lexeme.ts @@ -9,7 +9,7 @@ import { sequence } from './sequence'; * Parser that consumes trailing whitespace. * * @example - * lexeme(string('hello'))('hello world') // { ok: true, value: 'hello', remaining: 'world' } + * lexeme(string('hello'))('hello world') // { ok: true, value: 'hello', index: 8 } */ export const lexeme = (parser: Parser) => { return create(map(sequence(parser, regex(/^\s*/)), ([value]) => value)); diff --git a/src/combinators/many.test.ts b/src/combinators/many.test.ts index f12e373..a3f1082 100644 --- a/src/combinators/many.test.ts +++ b/src/combinators/many.test.ts @@ -8,11 +8,13 @@ import { many } from './many'; describe('many', () => { it('should parse zero occurrences', () => { - const failureParser = create(() => failure()); + const failureParser = create((_input, index = 0) => + failure(undefined, index), + ); const parser = many(failureParser); const result = parser('BCD'); - assertSuccess(result, [], 'BCD'); + assertSuccess(result, [], 0); }); it('should parse one occurrence', () => { @@ -20,7 +22,7 @@ describe('many', () => { const parser = many(parser1); const result = parser('ABCD'); - assertSuccess<'A'[]>(result, ['A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A'], 1); }); it('should parse multiple occurrences', () => { @@ -28,7 +30,7 @@ describe('many', () => { const parser = many(parser1); const result = parser('AAABCD'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 3); }); it('should handle empty input', () => { @@ -36,32 +38,34 @@ describe('many', () => { const parser = many(parser1); const result = parser(''); - assertSuccess<'A'[]>(result, [], ''); + assertSuccess<'A'[]>(result, [], 0); }); it('should prevent infinite loops with non-consuming parsers', () => { - const nonConsumingParser = () => ['', 'AB'] as [string, string]; - const parser = many(nonConsumingParser as any); + const nonConsumingParser = create((_input, index = 0) => + success('', index), + ); + const parser = many(nonConsumingParser); const result = parser('AB'); - assertSuccess(result, [], 'AB'); + assertSuccess(result, [], 0); }); it('should stop after partial progress when parser later stalls', () => { let callCount = 0; - const partiallyConsumingParser = create((input: string) => { + const partiallyConsumingParser = create((input, index = 0) => { callCount++; - if (input.startsWith('A')) { - return success('A', input.slice(1)); + if (input.startsWith('A', index)) { + return success('A', index + 1); } - return success('X', input); + return success('X', index); }); const parser = many(partiallyConsumingParser); const result = parser('ABCD'); - assertSuccess(result, ['A'], 'BCD'); + assertSuccess(result, ['A'], 1); expect(callCount).toBe(2); }); }); diff --git a/src/combinators/many.ts b/src/combinators/many.ts index e375579..525680b 100644 --- a/src/combinators/many.ts +++ b/src/combinators/many.ts @@ -1,32 +1,36 @@ import { create, type Parser } from '../core/parser'; +import { type Result } from '../core/result'; import { success, type Success } from '../core/success'; /** * Zero or more occurrences (never fails). * + * The failure that stopped the loop is kept as a trace, so a later error can + * still report what the repetition was expecting next. + * * @example - * many(char('a'))('aaa') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * many(char('a'))('aaa') // { ok: true, value: ['a', 'a', 'a'], index: 3 } */ export const many = (parser: Parser) => { - return create((input): Success => { + return create((input, index = 0, ctx): Success => { const results: T[] = []; - let remaining = input; + let at = index; while (true) { - const result = parser(remaining); + const result: Result = parser(input, at, ctx); if (!result.ok) { break; } // Prevent infinite loop: ensure progress is made - if (result.remaining === remaining) { + if (result.index === at) { break; } results.push(result.value); - remaining = result.remaining; + at = result.index; } - return success(results, remaining); + return success(results, at) as Success; }); }; diff --git a/src/combinators/many1.test.ts b/src/combinators/many1.test.ts index 0ec92bb..82f9bb0 100644 --- a/src/combinators/many1.test.ts +++ b/src/combinators/many1.test.ts @@ -22,14 +22,14 @@ describe('many1', () => { const parser = many1(parser1); const result = parser('ABCD'); - assertSuccess<'A'[]>(result, ['A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A'], 1); }); it('should parse multiple occurrences', () => { const parser = many1(parser1); const result = parser('AAABCD'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 3); }); it('should fail on empty input', () => { @@ -41,10 +41,10 @@ describe('many1', () => { it('should fail when many returns null after first success', () => { let callCount = 0; - const flakyParser = (input: string) => { + const flakyParser = (_input: string, index = 0) => { callCount++; if (callCount === 1) { - return success('A', input.slice(1)); + return success('A', index + 1); } // On subsequent calls, throw to make many() fail diff --git a/src/combinators/many1.ts b/src/combinators/many1.ts index ebe4c2d..7966184 100644 --- a/src/combinators/many1.ts +++ b/src/combinators/many1.ts @@ -1,25 +1,25 @@ -import { failure } from '../core/failure'; import { create, type Parser } from '../core/parser'; -import { type Success, success } from '../core/success'; +import { type Success } from '../core/success'; import { many } from './many'; /** - * One or more occurrences (fails if no matches). + * One or more occurrences. * * @example - * many1(char('a'))('aaa') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * many1(char('a'))('aaa') // { ok: true, value: ['a', 'a', 'a'], index: 3 } */ export const many1 = (parser: Parser) => { - return create((input) => { - const result = parser(input); + return create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value: manyValue, remaining: manyRemainder } = many(parser)( - result.remaining, - ) as Success; + const rest = many(parser)(input, result.index, ctx) as Success; - return success([result.value, ...manyValue], manyRemainder); + return { + ...rest, + value: [result.value, ...rest.value], + }; }); }; diff --git a/src/combinators/manyAtLeast.test.ts b/src/combinators/manyAtLeast.test.ts index a1b0e44..db049e5 100644 --- a/src/combinators/manyAtLeast.test.ts +++ b/src/combinators/manyAtLeast.test.ts @@ -14,14 +14,14 @@ describe('manyAtLeast', () => { const parser = manyAtLeast(parser1, 2); const result = parser('AAABCD'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 3); }); it('should parse exactly n occurrences', () => { const parser = manyAtLeast(parser1, 2); const result = parser('AABCD'); - assertSuccess<'A'[]>(result, ['A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A'], 2); }); it('should fail if fewer than n occurrences', () => { @@ -35,13 +35,13 @@ describe('manyAtLeast', () => { const parser = manyAtLeast(parser1, 0); const result = parser('BCD'); - assertSuccess<'A'[]>(result, [], 'BCD'); + assertSuccess<'A'[]>(result, [], 0); }); it('should parse many more than minimum', () => { const parser = manyAtLeast(parser1, 2); const result = parser('AAAAAA'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A', 'A', 'A'], 6); }); }); diff --git a/src/combinators/manyAtLeast.ts b/src/combinators/manyAtLeast.ts index 95e4c3d..439c912 100644 --- a/src/combinators/manyAtLeast.ts +++ b/src/combinators/manyAtLeast.ts @@ -1,6 +1,7 @@ -import { failure } from '../core/failure'; -import { create, type Parser } from '../core/parser'; -import { type Success, success } from '../core/success'; +import type { Parser } from '../core/parser'; + +import { create } from '../core/parser'; +import { type Success } from '../core/success'; import { exactly } from './exactly'; import { many } from './many'; @@ -8,19 +9,20 @@ import { many } from './many'; * Parse at least n occurrences. * * @example - * manyAtLeast(char('a'), 2)('aaa') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * manyAtLeast(char('a'), 2)('aaa') // { ok: true, value: ['a', 'a', 'a'], index: 3 } */ export const manyAtLeast = (parser: Parser, n: number) => { - return create((input) => { - const required = exactly(parser, n)(input); + return create((input, index = 0, ctx) => { + const required = exactly(parser, n)(input, index, ctx); if (!required.ok) { - return failure(); + return required; } - const { value: more, remaining: rest } = many(parser)( - required.remaining, - ) as Success; + const rest = many(parser)(input, required.index, ctx) as Success; - return success([...required.value, ...more], rest); + return { + ...rest, + value: [...required.value, ...rest.value], + }; }); }; diff --git a/src/combinators/manyAtMost.test.ts b/src/combinators/manyAtMost.test.ts index 59ef0ab..7a3528d 100644 --- a/src/combinators/manyAtMost.test.ts +++ b/src/combinators/manyAtMost.test.ts @@ -10,34 +10,34 @@ describe('manyAtMost', () => { const parser = manyAtMost(parser1, 3); const result = parser('AABCD'); - assertSuccess<'A'[]>(result, ['A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A'], 2); }); it('should parse exactly n occurrences when available', () => { const parser = manyAtMost(parser1, 3); const result = parser('AAABCD'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 3); }); it('should not parse more than n occurrences', () => { const parser = manyAtMost(parser1, 2); const result = parser('AAAAAA'); - assertSuccess<'A'[]>(result, ['A', 'A'], 'AAAA'); + assertSuccess<'A'[]>(result, ['A', 'A'], 2); }); it('should parse zero occurrences', () => { const parser = manyAtMost(parser1, 3); const result = parser('BCD'); - assertSuccess<'A'[]>(result, [], 'BCD'); + assertSuccess<'A'[]>(result, [], 0); }); it('should handle limit of zero', () => { const parser = manyAtMost(parser1, 0); const result = parser('AAABCD'); - assertSuccess<'A'[]>(result, [], 'AAABCD'); + assertSuccess<'A'[]>(result, [], 0); }); }); diff --git a/src/combinators/manyAtMost.ts b/src/combinators/manyAtMost.ts index 9139a10..2319c09 100644 --- a/src/combinators/manyAtMost.ts +++ b/src/combinators/manyAtMost.ts @@ -1,29 +1,30 @@ import type { Parser } from '../core/parser'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** - * Parse at most n occurrences. + * Parse at most n occurrences (never fails). * * @example - * manyAtMost(char('a'), 2)('aaa') // { ok: true, value: ['a', 'a'], remaining: 'a' } + * manyAtMost(char('a'), 2)('aaa') // { ok: true, value: ['a', 'a'], index: 2 } */ export const manyAtMost = (parser: Parser, n: number) => { - return create((input) => { + return create((input, index = 0, ctx) => { const results: T[] = []; - let remaining = input; + let at = index; for (let i = 0; i < n; i++) { - const result = parser(remaining); + const result: Result = parser(input, at, ctx); if (!result.ok) { break; } results.push(result.value); - remaining = result.remaining; + at = result.index; } - return success(results, remaining); + return success(results, at); }); }; diff --git a/src/combinators/manyBetween.test.ts b/src/combinators/manyBetween.test.ts index 573f7b7..219d597 100644 --- a/src/combinators/manyBetween.test.ts +++ b/src/combinators/manyBetween.test.ts @@ -14,28 +14,28 @@ describe('manyBetween', () => { const parser = manyBetween(parser1, 2, 4); const result = parser('AAABCD'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 3); }); it('should parse minimum required', () => { const parser = manyBetween(parser1, 2, 4); const result = parser('AABCD'); - assertSuccess<'A'[]>(result, ['A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A'], 2); }); it('should parse maximum allowed', () => { const parser = manyBetween(parser1, 2, 4); const result = parser('AAAABCD'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A'], 4); }); it('should not parse more than maximum', () => { const parser = manyBetween(parser1, 1, 3); const result = parser('AAAAAA'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 'AAA'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 3); }); it('should fail if fewer than minimum', () => { @@ -49,6 +49,6 @@ describe('manyBetween', () => { const parser = manyBetween(parser1, 3, 3); const result = parser('AAABCD'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 'BCD'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 3); }); }); diff --git a/src/combinators/manyBetween.ts b/src/combinators/manyBetween.ts index 56d4486..dce9ce9 100644 --- a/src/combinators/manyBetween.ts +++ b/src/combinators/manyBetween.ts @@ -1,6 +1,7 @@ -import { failure } from '../core/failure'; -import { create, type Parser } from '../core/parser'; -import { type Success, success } from '../core/success'; +import type { Parser } from '../core/parser'; + +import { create } from '../core/parser'; +import { type Success } from '../core/success'; import { exactly } from './exactly'; import { manyAtMost } from './manyAtMost'; @@ -8,20 +9,24 @@ import { manyAtMost } from './manyAtMost'; * Parse between min and max occurrences. * * @example - * manyBetween(char('a'), 2, 3)('aaa') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * manyBetween(char('a'), 1, 2)('aaa') // { ok: true, value: ['a', 'a'], index: 2 } */ export const manyBetween = (parser: Parser, min: number, max: number) => { - return create((input) => { - const required = exactly(parser, min)(input); + return create((input, index = 0, ctx) => { + const required = exactly(parser, min)(input, index, ctx); if (!required.ok) { - return failure(); + return required; } - const { value: more, remaining: rest } = manyAtMost( - parser, - max - min, - )(required.remaining) as Success; + const rest = manyAtMost(parser, max - min)( + input, + required.index, + ctx, + ) as Success; - return success([...required.value, ...more], rest); + return { + ...rest, + value: [...required.value, ...rest.value], + }; }); }; diff --git a/src/combinators/manyTill.test.ts b/src/combinators/manyTill.test.ts index 6732f8a..1e14dc2 100644 --- a/src/combinators/manyTill.test.ts +++ b/src/combinators/manyTill.test.ts @@ -14,7 +14,7 @@ describe('manyTill', () => { const parser = manyTill(parser1, parser2); const result = parser('AAAB'); - assertSuccess(result, ['A', 'A', 'A'], ''); + assertSuccess(result, ['A', 'A', 'A'], 4); }); it('should return empty array if terminator matches immediately', () => { @@ -23,7 +23,7 @@ describe('manyTill', () => { const parser = manyTill(parser1, parser2); const result = parser('B'); - assertSuccess(result, [], ''); + assertSuccess(result, [], 1); }); it('should fail if terminator never matches', () => { @@ -41,6 +41,6 @@ describe('manyTill', () => { const parser = manyTill(parser1, parser2); const result = parser('AABAB'); - assertSuccess(result, ['A', 'A'], 'AB'); + assertSuccess(result, ['A', 'A'], 3); }); }); diff --git a/src/combinators/manyTill.ts b/src/combinators/manyTill.ts index 29e047b..b044004 100644 --- a/src/combinators/manyTill.ts +++ b/src/combinators/manyTill.ts @@ -1,33 +1,32 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** * Parse zero or more until terminator matches. * * @example - * manyTill(char('a'), char('b'))('aaab') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * manyTill(char('a'), char('b'))('aaab') // { ok: true, value: ['a', 'a', 'a'], index: 4 } */ export const manyTill = (parser: Parser, terminator: Parser) => { - return create((input) => { + return create((input, index = 0, ctx) => { const results: T[] = []; - let remaining = input; + let at = index; while (true) { - const termResult = terminator(remaining); + const termResult: Result = terminator(input, at, ctx); if (termResult.ok) { - return success(results, termResult.remaining); + return success(results, termResult.index); } - - const parseResult = parser(remaining); + const parseResult: Result = parser(input, at, ctx); if (!parseResult.ok) { - return failure(); + return parseResult; } results.push(parseResult.value); - remaining = parseResult.remaining; + at = parseResult.index; } }); }; diff --git a/src/combinators/map.test.ts b/src/combinators/map.test.ts index 2565f28..a3963a4 100644 --- a/src/combinators/map.test.ts +++ b/src/combinators/map.test.ts @@ -8,15 +8,17 @@ import { map } from './map'; describe('map', () => { it('should transform parser result with single transform', () => { - const parser1 = create<'42'>(() => success('42', 'abc')); + const parser1 = create<'42'>((_input, index = 0) => + success('42', index + 2), + ); const parser = map(parser1, parseInt); const result = parser('42abc'); - assertSuccess(result, 42, 'abc'); + assertSuccess(result, 42, 2); }); it('should chain multiple transforms', () => { - const parser1 = create<'24'>(() => success('24', 'abc')); + const parser1 = create<'24'>((_input, _index = 0) => success('24', 2)); const parser = map( parser1, parseInt, @@ -25,11 +27,13 @@ describe('map', () => { ); const result = parser('21abc'); - assertSuccess(result, '48', 'abc'); + assertSuccess(result, '48', 2); }); it('should fail if underlying parser fails', () => { - const parser1 = create(() => failure()); + const parser1 = create((_input, _index = 0) => + failure(undefined, 0), + ); const parser = map(parser1, (s) => s.toUpperCase()); const result = parser('goodbye'); @@ -37,8 +41,8 @@ describe('map', () => { }); it('should handle complex transformations', () => { - const parser1 = create(() => - success(['count', '=', '5'] as const, ';'), + const parser1 = create((_input, index = 0) => + success(['count', '=', '5'] as const, index + 7), ); const parser = map( parser1 as never, @@ -49,14 +53,16 @@ describe('map', () => { const result = parser('count=5;'); // :O, surprising! - assertSuccess<{ count: number }>(result, { count: 5 }, ';'); + assertSuccess<{ count: number }>(result, { count: 5 }, 7); }); it('should maintain original input consumption', () => { - const parser1 = create<'test'>(() => success('test', 'ing')); + const parser1 = create<'test'>((_input, index = 0) => + success('test', index + 4), + ); const parser = map(parser1, (s) => s.length); const result = parser('testing'); - assertSuccess(result, 4, 'ing'); + assertSuccess(result, 4, 4); }); }); diff --git a/src/combinators/map.ts b/src/combinators/map.ts index 73e38b7..7480a1d 100644 --- a/src/combinators/map.ts +++ b/src/combinators/map.ts @@ -1,15 +1,8 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; -/** - * Transform the parsed value. - * - * @example - * map(string('hello'), (v) => v.toUpperCase())('hello') // { ok: true, value: 'HELLO', remaining: '' } - */ export function map( parser: Parser, transform: (value: A) => B, @@ -82,14 +75,21 @@ export function map( transform8: (value: H) => I, transform9: (value: I) => J, ): Parser; + +/** + * Transform a parsed value through one or more functions. + * + * @example + * map(digits, (n) => n * 2)('21') // { ok: true, value: 42, index: 2 } + */ export function map( parser: Parser, ...transforms: Array<(value: unknown) => unknown> ) { - return create((input) => { - const result = parser(input); + return create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } const finalValue = transforms.reduce( @@ -97,6 +97,6 @@ export function map( result.value as unknown, ); - return success(finalValue, result.remaining); + return success(finalValue, result.index); }); } diff --git a/src/combinators/node.test.ts b/src/combinators/node.test.ts index 7699e0e..4e0b9ce 100644 --- a/src/combinators/node.test.ts +++ b/src/combinators/node.test.ts @@ -11,7 +11,7 @@ describe('node', () => { const parser = node<{ value: 'A' }>('word', { value: aParser }); const result = parser('A'); - assertSuccess(result, { type: 'word', value: 'A' }, ''); + assertSuccess(result, { type: 'word', value: 'A' }, 1); }); it('should create node with multiple fields', () => { @@ -21,7 +21,7 @@ describe('node', () => { }); const result = parser('AB'); - assertSuccess(result, { type: 'pair', left: 'A', right: 'B' }, ''); + assertSuccess(result, { type: 'pair', left: 'A', right: 'B' }, 2); }); it('should handle nested nodes', () => { @@ -34,7 +34,7 @@ describe('node', () => { assertSuccess( result, { type: 'outer', child: { type: 'inner', value: 'A' } }, - '', + 1, ); }); @@ -42,15 +42,15 @@ describe('node', () => { const alpha = node<{ value: 'A' }>('alpha', { value: aParser }); const beta = node<{ value: 'B' }>('beta', { value: bParser }); - assertSuccess(alpha('A'), { type: 'alpha', value: 'A' }, ''); - assertSuccess(beta('B'), { type: 'beta', value: 'B' }, ''); + assertSuccess(alpha('A'), { type: 'alpha', value: 'A' }, 1); + assertSuccess(beta('B'), { type: 'beta', value: 'B' }, 1); }); it('should handle empty fields', () => { const parser = node>('empty', {}); const result = parser(''); - assertSuccess(result, { type: 'empty' }, ''); + assertSuccess(result, { type: 'empty' }, 0); }); it('should parse partial input and leave remaining', () => { @@ -60,6 +60,6 @@ describe('node', () => { }); const result = parser('AB rest'); - assertSuccess(result, { type: 'pair', left: 'A', right: 'B' }, ' rest'); + assertSuccess(result, { type: 'pair', left: 'A', right: 'B' }, 2); }); }); diff --git a/src/combinators/node.ts b/src/combinators/node.ts index 3ea408c..c7bb208 100644 --- a/src/combinators/node.ts +++ b/src/combinators/node.ts @@ -15,8 +15,8 @@ type Node> = { * Create a node from parser fields. * * @example - * node('binop', { left: digits, op: char('+'), right: digits })('1+2') // { ok: true, value: { type: 'binop', left: 1, op: '+', right: 2 }, remaining: '' } - * node('number', { value: digits })('123') // { ok: true, value: { type: 'number', value: 123 }, remaining: '' } + * node('binop', { left: digits, op: char('+'), right: digits })('1+2') // { ok: true, value: { type: 'binop', left: 1, op: '+', right: 2 }, index: 3 } + * node('number', { value: digits })('123') // { ok: true, value: { type: 'number', value: 123 }, index: 3 } */ export function node< U extends Record, diff --git a/src/combinators/not.test.ts b/src/combinators/not.test.ts index 1a581d7..bd5f51f 100644 --- a/src/combinators/not.test.ts +++ b/src/combinators/not.test.ts @@ -13,7 +13,7 @@ describe('not', () => { const parser = not(parser1); const result = parser('BCD'); - assertSuccess(result, null, 'BCD'); + assertSuccess(result, null, 0); }); it('should fail when parser succeeds', () => { diff --git a/src/combinators/not.ts b/src/combinators/not.ts index 9cebcfd..f5182e8 100644 --- a/src/combinators/not.ts +++ b/src/combinators/not.ts @@ -1,5 +1,6 @@ import type { Parser } from '../core/parser'; +import { context } from '../core/context'; import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; @@ -7,13 +8,17 @@ import { success } from '../core/success'; /** * Succeed if parser fails (without consuming input). * + * The inner parser runs against an isolated context: its failure is the + * expected outcome here, so reporting it would produce a misleading + * expectation in the final message. + * * @example - * not(string('hello'))('world') // { ok: true, value: null, remaining: 'world' } + * not(string('hello'))('world') // { ok: true, value: null, index: 0 } */ export const not = (parser: Parser) => { - return create((input) => { - const result = parser(input); + return create((input, index = 0, ctx) => { + const result = parser(input, index, context()); - return result.ok ? failure() : success(null, input); + return result.ok ? failure(ctx, index) : success(null, index); }); }; diff --git a/src/combinators/nth.test.ts b/src/combinators/nth.test.ts index 9d7c879..1c89cbf 100644 --- a/src/combinators/nth.test.ts +++ b/src/combinators/nth.test.ts @@ -7,51 +7,55 @@ import { nth } from './nth'; describe('nth', () => { it('should return element at specified index', () => { - const parser1 = create(() => success(['A', 'B', 'C'] as const, '')); + const parser1 = create((input) => + success(['A', 'B', 'C'] as const, input.length), + ); { const parser = nth(parser1, 0); const result = parser('ABC'); - assertSuccess<'A'>(result, 'A', ''); + assertSuccess<'A'>(result, 'A', 3); } { const parser = nth(parser1, 1); const result = parser('ABC'); - assertSuccess<'B'>(result, 'B', ''); + assertSuccess<'B'>(result, 'B', 3); } { const parser = nth(parser1, 2); const result = parser('ABC'); - assertSuccess<'C'>(result, 'C', ''); + assertSuccess<'C'>(result, 'C', 3); } }); it('should return undefined for out-of-bounds index', () => { - const parser1 = create(() => success(['A', 'B', 'C'] as const, '')); + const parser1 = create((_input, _index = 0) => + success(['A', 'B', 'C'] as const, 3), + ); { const parser = nth(parser1, 5); const result = parser('ABC'); - assertSuccess(result, undefined, ''); + assertSuccess(result, undefined, 3); } { const parser = nth(parser1, -1); const result = parser('ABC'); - assertSuccess(result, undefined, ''); + assertSuccess(result, undefined, 3); } }); it('should handle empty arrays', () => { - const parser1 = create(() => success([] as const, 'ABC')); + const parser1 = create((_input, _index = 0) => success([] as const, 0)); const parser = nth(parser1, 1); const result = parser(''); - assertSuccess(result, undefined, 'ABC'); + assertSuccess(result, undefined, 0); }); }); diff --git a/src/combinators/nth.ts b/src/combinators/nth.ts index bd5f31c..7a5c83d 100644 --- a/src/combinators/nth.ts +++ b/src/combinators/nth.ts @@ -29,7 +29,7 @@ export type Nth = * Extract the nth element from a parser result array. * * @example - * nth(sequence(char('a'), char('b'), char('c')), 1)('abc') // { ok: true, value: 'b', remaining: '' } + * nth(sequence(char('a'), char('b'), char('c')), 1)('abc') // { ok: true, value: 'b', index: 3 } */ export const nth = ( parser: Parser, diff --git a/src/combinators/optional.test.ts b/src/combinators/optional.test.ts index 1cb85e1..e362a6a 100644 --- a/src/combinators/optional.test.ts +++ b/src/combinators/optional.test.ts @@ -12,14 +12,16 @@ describe('optional', () => { const parser = optional(aParser); const result = parser('ABC'); - assertSuccess<'A' | null>(result, 'A', 'BC'); + assertSuccess<'A' | null>(result, 'A', 1); }); it('should return null when parser fails', () => { - const parser1 = create<'A'>(() => failure()); + const parser1 = create<'A'>((_input, index = 0) => + failure(undefined, index), + ); const parser = optional(parser1); const result = parser('ABC'); - assertSuccess<'A' | null>(result, null, 'ABC'); + assertSuccess<'A' | null>(result, null, 0); }); }); diff --git a/src/combinators/optional.ts b/src/combinators/optional.ts index cc8dab6..e8ec11f 100644 --- a/src/combinators/optional.ts +++ b/src/combinators/optional.ts @@ -7,13 +7,13 @@ import { success } from '../core/success'; * Make parser optional (return null on failure, without consuming input). * * @example - * optional(string('hello'))('hello') // { ok: true, value: 'hello', remaining: '' } - * optional(string('hello'))('world') // { ok: true, value: null, remaining: 'world' } + * optional(string('hello'))('hello') // { ok: true, value: 'hello', index: 5 } + * optional(string('hello'))('world') // { ok: true, value: null, index: 0 } */ export const optional = (parser: Parser) => { - return create((input) => { - const result = parser(input); + return create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); - return result.ok ? result : success(null, input); + return result.ok ? result : success(null, index); }); }; diff --git a/src/combinators/optionalConsume.test.ts b/src/combinators/optionalConsume.test.ts index c831d1c..35eb624 100644 --- a/src/combinators/optionalConsume.test.ts +++ b/src/combinators/optionalConsume.test.ts @@ -11,14 +11,16 @@ describe('optionalConsume', () => { const parser = optionalConsume(parser1); const result = parser('ABCD'); - assertSuccess(result, undefined, 'BCD'); + assertSuccess(result, undefined, 1); }); it('should not consume input on failure', () => { - const failureParser = create(() => failure()); + const failureParser = create((_input, index = 0) => + failure(undefined, index), + ); const parser = optionalConsume(failureParser); const result = parser('ABCD'); - assertSuccess(result, undefined, 'ABCD'); + assertSuccess(result, undefined, 0); }); }); diff --git a/src/combinators/optionalConsume.ts b/src/combinators/optionalConsume.ts index 4a6d1aa..22caffc 100644 --- a/src/combinators/optionalConsume.ts +++ b/src/combinators/optionalConsume.ts @@ -4,18 +4,17 @@ import { create } from '../core/parser'; import { success } from '../core/success'; /** - * Optionally consume input (always succeeds, returns void). + * Consume input if the parser matches, discarding the result. * * @example - * optionalConsume(string('hello'))('hello world') // { ok: true, value: undefined, remaining: ' world' } - * optionalConsume(string('hello'))('world') // { ok: true, value: undefined, remaining: 'world' } + * optionalConsume(string('hi'))('hi there') // { ok: true, index: 2 } */ export const optionalConsume = (parser: Parser) => { - return create((input) => { - const result = parser(input); + return create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); return result.ok - ? success(undefined, result.remaining) - : success(undefined, input); + ? success(undefined, result.index) + : success(undefined, index); }); }; diff --git a/src/combinators/optionalSeparatedBy.test.ts b/src/combinators/optionalSeparatedBy.test.ts index e6131c0..cd13d75 100644 --- a/src/combinators/optionalSeparatedBy.test.ts +++ b/src/combinators/optionalSeparatedBy.test.ts @@ -11,7 +11,7 @@ describe('optionalSeparatedBy', () => { const parser = optionalSeparatedBy(parser1, parser2); const result = parser('B'); - assertSuccess<(string | null)[]>(result, [], 'B'); + assertSuccess<(string | null)[]>(result, [], 0); }); it('should parse single element without separator', () => { @@ -21,7 +21,7 @@ describe('optionalSeparatedBy', () => { const parser = optionalSeparatedBy(parser1, parser2); const result = parser('AAACCC'); - assertSuccess<(string | null)[]>(result, ['A'], 'AACCC'); + assertSuccess<(string | null)[]>(result, ['A'], 1); }); it('should parse multiple elements separated by separator', () => { @@ -31,7 +31,7 @@ describe('optionalSeparatedBy', () => { const parser = optionalSeparatedBy(parser1, parser2); const result = parser('A,A,A'); - assertSuccess<(string | null)[]>(result, ['A', 'A', 'A'], ''); + assertSuccess<(string | null)[]>(result, ['A', 'A', 'A'], 5); }); it('should handle leading separator as null', () => { @@ -41,7 +41,7 @@ describe('optionalSeparatedBy', () => { const parser = optionalSeparatedBy(parser1, parser2); const result = parser(',A'); - assertSuccess<(string | null)[]>(result, [null, 'A'], ''); + assertSuccess<(string | null)[]>(result, [null, 'A'], 2); }); it('should handle trailing separator without adding null', () => { @@ -51,7 +51,7 @@ describe('optionalSeparatedBy', () => { const parser = optionalSeparatedBy(parser1, parser2); const result = parser('A,'); - assertSuccess<(string | null)[]>(result, ['A'], ''); + assertSuccess<(string | null)[]>(result, ['A'], 2); }); it('should handle empty input', () => { @@ -61,7 +61,7 @@ describe('optionalSeparatedBy', () => { const parser = optionalSeparatedBy(parser1, parser2); const result = parser(''); - assertSuccess<(string | null)[]>(result, [], ''); + assertSuccess<(string | null)[]>(result, [], 0); }); it('should handle single separator only as leading null', () => { @@ -71,6 +71,6 @@ describe('optionalSeparatedBy', () => { const parser = optionalSeparatedBy(parser1, parser2); const result = parser(','); - assertSuccess<(string | null)[]>(result, [null], ''); + assertSuccess<(string | null)[]>(result, [null], 1); }); }); diff --git a/src/combinators/optionalSeparatedBy.ts b/src/combinators/optionalSeparatedBy.ts index c197092..5c440f1 100644 --- a/src/combinators/optionalSeparatedBy.ts +++ b/src/combinators/optionalSeparatedBy.ts @@ -1,54 +1,54 @@ import type { Parser } from '../core/parser'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** - * Zero or more items separated by a separator, with optional null values. + * Parse items separated by a separator, allowing empty slots. * * @example - * optionalSeparatedBy(digits, char(','))('1,2') // { ok: true, value: [1, 2], remaining: '' } - * optionalSeparatedBy(digits, char(','))(',1') // { ok: true, value: [null, 1], remaining: '' } - * optionalSeparatedBy(digits, char(','))('1,') // { ok: true, value: [1], remaining: '' } + * optionalSeparatedBy(digits, char(','))('1,2') // { ok: true, value: [1, 2], index: 3 } */ export const optionalSeparatedBy = ( parser: Parser, separator: Parser, ) => { - return create<(T | null)[]>((input) => { + return create<(T | null)[]>((input, index = 0, ctx) => { const results: (T | null)[] = []; - let remaining = input; + let at = index; + + const leadingSep = separator(input, at, ctx); - const leadingSep = separator(remaining); if (leadingSep.ok) { results.push(null); - remaining = leadingSep.remaining; + at = leadingSep.index; } - const firstResult = parser(remaining); + const firstResult: Result = parser(input, at, ctx); if (firstResult.ok) { results.push(firstResult.value); - remaining = firstResult.remaining; + at = firstResult.index; } else if (!leadingSep.ok) { - return success([], input); + return success([], index); } while (true) { - const sepResult = separator(remaining); + const sepResult: Result = separator(input, at, ctx); if (!sepResult.ok) { break; } - remaining = sepResult.remaining; + at = sepResult.index; - const nextResult = parser(remaining); + const nextResult: Result = parser(input, at, ctx); if (!nextResult.ok) { break; } results.push(nextResult.value); - remaining = nextResult.remaining; + at = nextResult.index; } - return success(results, remaining); + return success(results, at); }); }; diff --git a/src/combinators/outer.test.ts b/src/combinators/outer.test.ts index a2bebd0..0ba5b78 100644 --- a/src/combinators/outer.test.ts +++ b/src/combinators/outer.test.ts @@ -16,7 +16,7 @@ describe('outer', () => { const parser = outer(parserA, parserB, parserC); const result = parser('ABC'); - assertSuccess<['A', 'C']>(result, ['A', 'C'], ''); + assertSuccess<['A', 'C']>(result, ['A', 'C'], 3); }); it('should fail if first parser fails', () => { diff --git a/src/combinators/outer.ts b/src/combinators/outer.ts index 214d546..9c7ba54 100644 --- a/src/combinators/outer.ts +++ b/src/combinators/outer.ts @@ -8,7 +8,7 @@ import { sequence } from './sequence'; * Extract outer values from a sequence of 3 parsers (skip middle). * * @example - * outer(char('('), string('hi'), char(')'))('(hi)') // { ok: true, value: ['(', ')'], remaining: '' } + * outer(char('('), string('hi'), char(')'))('(hi)') // { ok: true, value: ['(', ')'], index: 4 } */ export const outer = ( parserA: Parser, diff --git a/src/combinators/padded.test.ts b/src/combinators/padded.test.ts index d8c8234..da46ccd 100644 --- a/src/combinators/padded.test.ts +++ b/src/combinators/padded.test.ts @@ -12,28 +12,28 @@ describe('padded', () => { const parser = createTestParser('A'); const result = padded(parser)(' A '); - assertSuccess<'A'>(result, 'A', ''); + assertSuccess<'A'>(result, 'A', 5); }); it('should parse content with no padding', () => { const parser = createTestParser('A'); const result = padded(parser)('A'); - assertSuccess<'A'>(result, 'A', ''); + assertSuccess<'A'>(result, 'A', 1); }); it('should parse content with padding on left only', () => { const parser = createTestParser('A'); const result = padded(parser)(' A'); - assertSuccess<'A'>(result, 'A', ''); + assertSuccess<'A'>(result, 'A', 3); }); it('should parse content with padding on right only', () => { const parser = createTestParser('A'); const result = padded(parser)('A '); - assertSuccess<'A'>(result, 'A', ''); + assertSuccess<'A'>(result, 'A', 3); }); it('should fail if content fails', () => { @@ -47,13 +47,13 @@ describe('padded', () => { const parser = createTestParser('A'); const result = padded(parser)(' A B'); - assertSuccess<'A'>(result, 'A', 'B'); + assertSuccess<'A'>(result, 'A', 5); }); it('should handle multiple whitespace characters', () => { const parser = createTestParser('A'); const result = padded(parser)('\t\n A \r\n'); - assertSuccess<'A'>(result, 'A', ''); + assertSuccess<'A'>(result, 'A', 9); }); }); diff --git a/src/combinators/padded.ts b/src/combinators/padded.ts index 09df8a2..c82bbfe 100644 --- a/src/combinators/padded.ts +++ b/src/combinators/padded.ts @@ -7,10 +7,10 @@ import { map } from './map'; import { sequence } from './sequence'; /** - * Parse content with optional whitespace on both sides. + * Parse content surrounded by optional whitespace. * * @example - * padded(string('hi'))(' hi ') // { ok: true, value: 'hi', remaining: '' } + * padded(string('hi'))(' hi ') // { ok: true, value: 'hi', index: 6 } */ export const padded = (content: Parser) => { const parser = map( @@ -18,7 +18,7 @@ export const padded = (content: Parser) => { ([, value]) => value, ); - return create((input) => { - return parser(input); + return create((input, index = 0, ctx) => { + return parser(input, index, ctx); }); }; diff --git a/src/combinators/parenthesized.test.ts b/src/combinators/parenthesized.test.ts index d337843..adc58ce 100644 --- a/src/combinators/parenthesized.test.ts +++ b/src/combinators/parenthesized.test.ts @@ -12,7 +12,7 @@ describe('parenthesized', () => { const parser = parenthesized(createTestParser('ABC')); const result = parser('(ABC)'); - assertSuccess<'ABC'>(result, 'ABC', ''); + assertSuccess<'ABC'>(result, 'ABC', 5); }); it('should fail with uneven parentheses', () => { @@ -26,6 +26,6 @@ describe('parenthesized', () => { const parser = parenthesized(createTestParser('')); const result = parser('()'); - assertSuccess<''>(result, '', ''); + assertSuccess<''>(result, '', 2); }); }); diff --git a/src/combinators/parenthesized.ts b/src/combinators/parenthesized.ts index ba369d5..c566132 100644 --- a/src/combinators/parenthesized.ts +++ b/src/combinators/parenthesized.ts @@ -7,7 +7,7 @@ import { string } from '../terminals/string'; * Parse content surrounded by parentheses. * * @example - * parenthesized(string('hi'))('(hi)') // { ok: true, value: 'hi', remaining: '' } + * parenthesized(string('hi'))('(hi)') // { ok: true, value: 'hi', index: 4 } */ export const parenthesized = (content: Parser) => { return surrounded(string('('), content, string(')')); diff --git a/src/combinators/peek.test.ts b/src/combinators/peek.test.ts index d4f9a85..96681c5 100644 --- a/src/combinators/peek.test.ts +++ b/src/combinators/peek.test.ts @@ -13,7 +13,7 @@ describe('peek', () => { const parser = peek(parser1); const result = parser('AB'); - assertSuccess<'A'>(result, 'A', 'AB'); + assertSuccess<'A'>(result, 'A', 0); }); it('should fail when peek parser does not match', () => { diff --git a/src/combinators/peek.ts b/src/combinators/peek.ts index c50c426..35a27a5 100644 --- a/src/combinators/peek.ts +++ b/src/combinators/peek.ts @@ -1,19 +1,18 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; /** - * Parse without consuming input. + * Look ahead without consuming input. * * @example - * peek(string('hello'))('hello world') // { ok: true, value: 'hello', remaining: 'hello world' } + * peek(string('hello'))('hello world') // { ok: true, value: 'hello', index: 0 } */ export const peek = (parser: Parser) => { - return create((input) => { - const result = parser(input); + return create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); - return result.ok ? success(result.value, input) : failure(); + return result.ok ? success(result.value, index) : result; }); }; diff --git a/src/combinators/postfix.test.ts b/src/combinators/postfix.test.ts index 3bd487d..7feea05 100644 --- a/src/combinators/postfix.test.ts +++ b/src/combinators/postfix.test.ts @@ -9,8 +9,11 @@ import { success } from '../core/success'; import { postfix } from './postfix'; describe('postfix', () => { - const postfixOps: Parser<(value: number) => number> = (input) => { - if (input.startsWith('!')) { + const postfixOps: Parser<(value: number) => number> = ( + input, + index = 0, + ) => { + if (input.startsWith('!', index)) { return success((value) => { let result = 1; for (let i = 2; i <= value; i += 1) { @@ -18,19 +21,19 @@ describe('postfix', () => { } return result; - }, input.slice(1)); + }, index + 1); } - if (input.startsWith('²')) { - return success((value) => value * value, input.slice(1)); + if (input.startsWith('²', index)) { + return success((value) => value * value, index + 1); } - return failure(); + return failure(undefined, index); }; it('should handle atom without postfix operators', () => { const parser = postfix(digits, postfixOps); const result = parser('5'); - assertSuccess(result, 5, ''); + assertSuccess(result, 5, 1); }); it('should apply single postfix operator', () => { @@ -38,12 +41,12 @@ describe('postfix', () => { { const result = parser('5!'); - assertSuccess(result, 120, ''); // 5! = 120 + assertSuccess(result, 120, 2); // 5! = 120 } { const result = parser('3²'); - assertSuccess(result, 9, ''); // 3² = 9 + assertSuccess(result, 9, 2); // 3² = 9 } }); @@ -51,7 +54,7 @@ describe('postfix', () => { const parser = postfix(digits, postfixOps); const result = parser('3²!'); - assertSuccess(result, 362880, ''); // (3²)! = 9! = 362880 + assertSuccess(result, 362880, 3); // (3²)! = 9! = 362880 }); it('should fail when atom parser fails', () => { @@ -65,13 +68,13 @@ describe('postfix', () => { const parser = postfix(digits, postfixOps); const result = parser('2²²'); - assertSuccess(result, 16, ''); // (2²)² = 4² = 16 + assertSuccess(result, 16, 3); // (2²)² = 4² = 16 }); it('should stop when no more operators match', () => { const parser = postfix(digits, postfixOps); const result = parser('3!+'); - assertSuccess(result, 6, '+'); // 3! = 6, stops at + + assertSuccess(result, 6, 2); // 3! = 6, stops at + }); }); diff --git a/src/combinators/postfix.ts b/src/combinators/postfix.ts index b7b5b60..b30d3e4 100644 --- a/src/combinators/postfix.ts +++ b/src/combinators/postfix.ts @@ -1,32 +1,36 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** - * Parse postfix operators (chains atom with operators that return functions). + * Apply zero or more postfix operators to an atom. * * @example - * postfix(char('a'), map(char('!'), () => (x) => x))('a!') // { ok: true, value: 'a', remaining: '' } + * postfix(digits, map(char('!'), () => (n) => n * 2))('3!') // { ok: true, value: 6, index: 2 } */ export const postfix = ( atom: Parser, operator: Parser<(value: T) => T>, ) => { - return create((input) => { - const atomResult = atom(input); - if (!atomResult.ok) return failure(); + return create((input, index = 0, ctx) => { + const atomResult = atom(input, index, ctx); + if (!atomResult.ok) { + return atomResult; + } - let { value, remaining } = atomResult; + let value = atomResult.value; + let at = atomResult.index; while (true) { - const opResult = operator(remaining); + const opResult: Result<(value: T) => T> = operator(input, at, ctx); if (!opResult.ok) break; + value = opResult.value(value); - remaining = opResult.remaining; + at = opResult.index; } - return success(value, remaining); + return success(value, at); }); }; diff --git a/src/combinators/prefix.test.ts b/src/combinators/prefix.test.ts index cb8abfc..71964be 100644 --- a/src/combinators/prefix.test.ts +++ b/src/combinators/prefix.test.ts @@ -9,21 +9,21 @@ import { success } from '../core/success'; import { prefix } from './prefix'; describe('prefix', () => { - const unaryOps: Parser<(value: number) => number> = (input) => { - if (input.startsWith('-')) { - return success((value) => -value, input.slice(1)); + const unaryOps: Parser<(value: number) => number> = (input, index = 0) => { + if (input.startsWith('-', index)) { + return success((value) => -value, index + 1); } - if (input.startsWith('+')) { - return success((value) => Math.abs(value), input.slice(1)); + if (input.startsWith('+', index)) { + return success((value) => Math.abs(value), index + 1); } - return failure(); + return failure(undefined, index); }; it('should handle atom without prefix operators', () => { const parser = prefix(unaryOps, digits); const result = parser('42'); - assertSuccess(result, 42, ''); + assertSuccess(result, 42, 2); }); it('should apply single prefix operator', () => { @@ -31,12 +31,12 @@ describe('prefix', () => { { const result = parser('-5'); - assertSuccess(result, -5, ''); + assertSuccess(result, -5, 2); } { const result = parser('+5'); - assertSuccess(result, 5, ''); // abs(5) = 5 + assertSuccess(result, 5, 2); // abs(5) = 5 } }); @@ -45,12 +45,12 @@ describe('prefix', () => { { const result = parser('--5'); - assertSuccess(result, 5, ''); // -(-5) = 5 + assertSuccess(result, 5, 3); // -(-5) = 5 } { const result = parser('+-5'); - assertSuccess(result, 5, ''); // +(-5) = abs(-5) = 5 + assertSuccess(result, 5, 3); // +(-5) = abs(-5) = 5 } }); @@ -65,13 +65,13 @@ describe('prefix', () => { const parser = prefix(unaryOps, digits); const result = parser('---5'); - assertSuccess(result, -5, ''); // -(-(- 5)) = -5 + assertSuccess(result, -5, 4); // -(-(- 5)) = -5 }); it('should handle operators that consume no input when none match', () => { const parser = prefix(unaryOps, digits); const result = parser('123*'); - assertSuccess(result, 123, '*'); + assertSuccess(result, 123, 3); }); }); diff --git a/src/combinators/prefix.ts b/src/combinators/prefix.ts index c6e298f..dc8f602 100644 --- a/src/combinators/prefix.ts +++ b/src/combinators/prefix.ts @@ -1,38 +1,41 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** - * Parse prefix operators (like - in -5). + * Apply zero or more prefix operators to an atom. * * @example - * prefix(map(char('-'), () => (x) => -x), digit)('-5') // { ok: true, value: -5, remaining: '' } + * prefix(map(char('-'), () => (n) => -n), digits)('-3') // { ok: true, value: -3, index: 2 } */ export const prefix = ( operator: Parser<(value: T) => T>, atom: Parser, ) => { - return create((input) => { + return create((input, index = 0, ctx) => { const operators: Array<(value: T) => T> = []; - let remaining = input; + let at = index; while (true) { - const opResult = operator(remaining); + const opResult: Result<(value: T) => T> = operator(input, at, ctx); if (!opResult.ok) break; + operators.push(opResult.value); - remaining = opResult.remaining; + at = opResult.index; } - const atomResult = atom(remaining); - if (!atomResult.ok) return failure(); + const atomResult = atom(input, at, ctx); + if (!atomResult.ok) { + return atomResult; + } const finalValue = operators.reduceRight( (value, op) => op(value), atomResult.value, ); - return success(finalValue, atomResult.remaining); + return success(finalValue, atomResult.index); }); }; diff --git a/src/combinators/pure.test.ts b/src/combinators/pure.test.ts index 21fda1c..1f44510 100644 --- a/src/combinators/pure.test.ts +++ b/src/combinators/pure.test.ts @@ -8,20 +8,20 @@ describe('pure', () => { const parser = pure(42); const result = parser('abc'); - assertSuccess(result, 42, 'abc'); + assertSuccess(result, 42, 0); }); it('should work on empty input', () => { const parser = pure('ok'); const result = parser(''); - assertSuccess(result, 'ok', ''); + assertSuccess(result, 'ok', 0); }); it('should work with any type', () => { const parser = pure({ x: 1 }); const result = parser('xyz'); - assertSuccess<{ x: number }>(result, { x: 1 }, 'xyz'); + assertSuccess<{ x: number }>(result, { x: 1 }, 0); }); }); diff --git a/src/combinators/pure.ts b/src/combinators/pure.ts index 6e790c9..22403d7 100644 --- a/src/combinators/pure.ts +++ b/src/combinators/pure.ts @@ -4,12 +4,11 @@ import { create } from '../core/parser'; import { success } from '../core/success'; /** - * Always return a value without consuming input. + * Always succeed with a value without consuming input. * * @example - * pure(42)('abc') // { ok: true, value: 42, remaining: 'abc' } - * pure('ok')('') // { ok: true, value: 'ok', remaining: '' } + * pure(42)('abc') // { ok: true, value: 42, index: 0 } */ export const pure = (value: T): Parser => { - return create((input) => success(value, input)); + return create((_input, index = 0) => success(value, index)); }; diff --git a/src/combinators/quoted.test.ts b/src/combinators/quoted.test.ts index b9edd91..62fb547 100644 --- a/src/combinators/quoted.test.ts +++ b/src/combinators/quoted.test.ts @@ -12,14 +12,14 @@ describe('quoted', () => { const parser = quoted(createTestParser('ABC')); const result = parser('"ABC"'); - assertSuccess<'ABC'>(result, 'ABC', ''); + assertSuccess<'ABC'>(result, 'ABC', 5); }); it('should parse single-quoted content', () => { const parser = quoted(createTestParser('ABC')); const result = parser("'ABC'"); - assertSuccess<'ABC'>(result, 'ABC', ''); + assertSuccess<'ABC'>(result, 'ABC', 5); }); it('should fail with mismatched quotes', () => { @@ -33,13 +33,13 @@ describe('quoted', () => { const parser = quoted(createTestParser('ABC')); const result = parser('"ABC"rest'); - assertSuccess<'ABC'>(result, 'ABC', 'rest'); + assertSuccess<'ABC'>(result, 'ABC', 5); }); it('should handle empty quotes input', () => { const parser = quoted(createTestParser('')); const result = parser('""'); - assertSuccess<''>(result, '', ''); + assertSuccess<''>(result, '', 2); }); }); diff --git a/src/combinators/quoted.ts b/src/combinators/quoted.ts index 9af7539..c8001f8 100644 --- a/src/combinators/quoted.ts +++ b/src/combinators/quoted.ts @@ -8,7 +8,7 @@ import { string } from '../terminals/string'; * Parse content surrounded by single or double quotes. * * @example - * quoted(string('hello'))('"hello"') // { ok: true, value: 'hello', remaining: '' } + * quoted(string('hello'))('"hello"') // { ok: true, value: 'hello', index: 7 } */ export const quoted = (content: Parser) => { return choice( diff --git a/src/combinators/recover.test.ts b/src/combinators/recover.test.ts index 72990fa..c40bf4b 100644 --- a/src/combinators/recover.test.ts +++ b/src/combinators/recover.test.ts @@ -12,26 +12,30 @@ describe('recover', () => { const parser = recover(parserA, 'default'); const result = parser('BCD'); - assertSuccess<'A' | 'default'>(result, 'default', 'BCD'); + assertSuccess<'A' | 'default'>(result, 'default', 0); }); it('should not consume input when parser fails', () => { - const parserFail = create<'A'>(() => failure()); + const parserFail = create<'A'>((_input, index = 0) => + failure(undefined, index), + ); const parser = recover(parserFail, 'world'); const result = parser('goodbye'); - assertSuccess<'A' | 'world'>(result, 'world', 'goodbye'); + assertSuccess<'A' | 'world'>(result, 'world', 0); }); it('should handle empty input', () => { const parser = recover(parserA, 'empty'); const result = parser(''); - assertSuccess<'A' | 'empty'>(result, 'empty', ''); + assertSuccess<'A' | 'empty'>(result, 'empty', 0); }); it('should work with complex default values', () => { - const parser1 = create(() => failure()); + const parser1 = create((_input, index = 0) => + failure(undefined, index), + ); const parser = recover<{ default: boolean; value: number } | string>( parser1, { @@ -44,7 +48,7 @@ describe('recover', () => { assertSuccess<{ default: boolean; value: number } | string>( result, { default: true, value: 42 }, - 'y', + 0, ); }); @@ -52,6 +56,6 @@ describe('recover', () => { const parser = recover(parserA, 'default'); const result = parser('ABCD'); - assertSuccess<'A' | 'default'>(result, 'A', 'BCD'); + assertSuccess<'A' | 'default'>(result, 'A', 1); }); }); diff --git a/src/combinators/recover.ts b/src/combinators/recover.ts index ed3829c..497fcce 100644 --- a/src/combinators/recover.ts +++ b/src/combinators/recover.ts @@ -7,12 +7,12 @@ import { success } from '../core/success'; * Use fallback value when parser fails. * * @example - * recover(string('hello'), 'default')('world') // { ok: true, value: 'default', remaining: 'world' } + * recover(string('hello'), 'default')('world') // { ok: true, value: 'default', index: 0 } */ export const recover = (parser: Parser, fallback: T) => { - return create((input) => { - const result = parser(input); + return create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); - return result.ok ? result : success(fallback, input); + return result.ok ? result : success(fallback, index); }); }; diff --git a/src/combinators/right.test.ts b/src/combinators/right.test.ts index d62868f..5db5961 100644 --- a/src/combinators/right.test.ts +++ b/src/combinators/right.test.ts @@ -15,7 +15,7 @@ describe('right', () => { const parser = right(parser1, parser2); const result = parser('AB'); - assertSuccess<'B'>(result, 'B', ''); + assertSuccess<'B'>(result, 'B', 2); }); it('should fail if first parser fails', () => { diff --git a/src/combinators/right.ts b/src/combinators/right.ts index 464a6ef..53416a2 100644 --- a/src/combinators/right.ts +++ b/src/combinators/right.ts @@ -8,7 +8,7 @@ import { sequence } from './sequence'; * Keep only the right result from a sequence. * * @example - * right(string('hello'), string('world'))('helloworld') // { ok: true, value: 'world', remaining: '' } + * right(string('hello'), string('world'))('helloworld') // { ok: true, value: 'world', index: 10 } */ export const right = (parserA: Parser, parserB: Parser) => { return create(map(sequence(parserA, parserB), ([, b]) => b)); diff --git a/src/combinators/separatedBy.test.ts b/src/combinators/separatedBy.test.ts index 6b330fc..d4c2da0 100644 --- a/src/combinators/separatedBy.test.ts +++ b/src/combinators/separatedBy.test.ts @@ -11,7 +11,7 @@ describe('separatedBy', () => { const parser = separatedBy(parser1, parser2); const result = parser('C'); - assertSuccess<'A'[]>(result, [], 'C'); + assertSuccess<'A'[]>(result, [], 0); }); it('should parse single element without separator', () => { @@ -21,7 +21,7 @@ describe('separatedBy', () => { const parser = separatedBy(parser1, parser2); const result = parser('AAACCC'); - assertSuccess<'A'[]>(result, ['A'], 'AACCC'); + assertSuccess<'A'[]>(result, ['A'], 1); }); it('should parse multiple elements separated by separator', () => { @@ -31,7 +31,7 @@ describe('separatedBy', () => { const parser = separatedBy(parser1, parser2); const result = parser('A,A,A'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 5); }); it('should handle trailing separator by not consuming it', () => { @@ -41,7 +41,7 @@ describe('separatedBy', () => { const parser = separatedBy(parser1, parser2); const result = parser('A,A,A,'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], ','); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 5); }); it('should handle separator without following element', () => { @@ -51,6 +51,6 @@ describe('separatedBy', () => { const parser = separatedBy(parser1, parser2); const result = parser('A,A,C'); - assertSuccess<'A'[]>(result, ['A', 'A'], ',C'); + assertSuccess<'A'[]>(result, ['A', 'A'], 3); }); }); diff --git a/src/combinators/separatedBy.ts b/src/combinators/separatedBy.ts index 475fe49..f6c8990 100644 --- a/src/combinators/separatedBy.ts +++ b/src/combinators/separatedBy.ts @@ -1,39 +1,41 @@ import type { Parser } from '../core/parser'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** - * Zero or more items separated by a separator. + * Parse zero or more items separated by a separator. + * + * A trailing separator is not consumed: if the separator matches but the item + * after it does not, the list ends before the separator. * * @example - * separatedBy(char('a'), char(','))('a,a,a') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * separatedBy(digits, char(','))('1,2,3') // { ok: true, value: [1, 2, 3], index: 5 } */ export const separatedBy = (parser: Parser, separator: Parser) => { - return create((input) => { - const firstResult = parser(input); + return create((input, index = 0, ctx) => { + const firstResult = parser(input, index, ctx); if (!firstResult.ok) { - return success([], input); + return success([], index); } const results = [firstResult.value]; - let remaining = firstResult.remaining; + let at = firstResult.index; while (true) { - const sepResult = separator(remaining); + const sepResult: Result = separator(input, at, ctx); if (!sepResult.ok) break; - const nextResult = parser(sepResult.remaining); - if (!nextResult.ok) { - // If separator matched but parser failed, backtrack - // Don't consume the separator - break; - } + const nextResult: Result = parser(input, sepResult.index, ctx); + // If separator matched but parser failed, backtrack + // Don't consume the separator + if (!nextResult.ok) break; results.push(nextResult.value); - remaining = nextResult.remaining; + at = nextResult.index; } - return success(results, remaining); + return success(results, at); }); }; diff --git a/src/combinators/separatedBy1.test.ts b/src/combinators/separatedBy1.test.ts index fb94b1a..327a13b 100644 --- a/src/combinators/separatedBy1.test.ts +++ b/src/combinators/separatedBy1.test.ts @@ -25,7 +25,7 @@ describe('separatedBy1', () => { const parser = separatedBy1(parser1, parser2); const result = parser('ABC'); - assertSuccess<'A'[]>(result, ['A'], 'BC'); + assertSuccess<'A'[]>(result, ['A'], 1); }); it('should parse multiple elements', () => { @@ -35,7 +35,7 @@ describe('separatedBy1', () => { const parser = separatedBy1(parser1, parser2); const result = parser('A,A,A,A'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A'], 7); }); it('should handle trailing separator', () => { @@ -45,7 +45,7 @@ describe('separatedBy1', () => { const parser = separatedBy1(parser1, parser2); const result = parser('A,A,A,A,'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A'], ','); + assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A'], 7); }); it('should fail on empty input', () => { diff --git a/src/combinators/separatedBy1.ts b/src/combinators/separatedBy1.ts index 5e4580c..9ad494c 100644 --- a/src/combinators/separatedBy1.ts +++ b/src/combinators/separatedBy1.ts @@ -1,21 +1,28 @@ +import type { Parser } from '../core/parser'; + import { failure } from '../core/failure'; -import { create, type Parser } from '../core/parser'; -import { type Success, success } from '../core/success'; +import { create } from '../core/parser'; +import { type Success } from '../core/success'; import { separatedBy } from './separatedBy'; /** - * One or more items separated by a separator. + * Parse one or more items separated by a separator. * * @example - * separatedBy1(char('a'), char(','))('a,a,a') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * separatedBy1(digits, char(','))('1,2,3') // { ok: true, value: [1, 2, 3], index: 5 } */ export const separatedBy1 = (parser: Parser, separator: Parser) => { - return create((input) => { - const result = separatedBy(parser, separator)(input); + return create((input, index = 0, ctx) => { + const result = separatedBy(parser, separator)( + input, + index, + ctx, + ) as Success; - const { value: values, remaining } = result as Success; - if (values.length === 0) return failure(); + if (result.value.length === 0) { + return failure(ctx, index); + } - return success(values, remaining); + return result; }); }; diff --git a/src/combinators/separatedEndBy.test.ts b/src/combinators/separatedEndBy.test.ts index 6fefdce..262f3f7 100644 --- a/src/combinators/separatedEndBy.test.ts +++ b/src/combinators/separatedEndBy.test.ts @@ -11,7 +11,7 @@ describe('separatedEndBy', () => { const parser = separatedEndBy(parser1, parser2); const result = parser('CCC'); - assertSuccess<'A'[]>(result, [], 'CCC'); + assertSuccess<'A'[]>(result, [], 0); }); it('should parse elements without trailing separator', () => { @@ -21,7 +21,7 @@ describe('separatedEndBy', () => { const parser = separatedEndBy(parser1, parser2); const result = parser('A,A,A'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 5); }); it('should parse elements with trailing separator', () => { @@ -31,7 +31,7 @@ describe('separatedEndBy', () => { const parser = separatedEndBy(parser1, parser2); const result = parser('A,A,A,'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 6); }); it('should handle single element with separator', () => { @@ -41,7 +41,7 @@ describe('separatedEndBy', () => { const parser = separatedEndBy(parser1, parser2); const result = parser('A,'); - assertSuccess<'A'[]>(result, ['A'], ''); + assertSuccess<'A'[]>(result, ['A'], 2); }); it('should handle single element without separator', () => { @@ -51,6 +51,6 @@ describe('separatedEndBy', () => { const parser = separatedEndBy(parser1, parser2); const result = parser('A'); - assertSuccess<'A'[]>(result, ['A'], ''); + assertSuccess<'A'[]>(result, ['A'], 1); }); }); diff --git a/src/combinators/separatedEndBy.ts b/src/combinators/separatedEndBy.ts index 663fc07..ff66720 100644 --- a/src/combinators/separatedEndBy.ts +++ b/src/combinators/separatedEndBy.ts @@ -1,20 +1,27 @@ -import { create, type Parser } from '../core/parser'; -import { type Success, success } from '../core/success'; +import type { Parser } from '../core/parser'; + +import { create } from '../core/parser'; +import { success, type Success } from '../core/success'; import { separatedBy } from './separatedBy'; /** - * Zero or more items separated by and ending with a terminator. + * Parse zero or more items separated by a separator, allowing a trailing one. * * @example - * separatedEndBy(char('a'), char(';'))('a;a;a;') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * separatedEndBy(digits, char(','))('1,2,3,') // { ok: true, value: [1, 2, 3], index: 6 } */ export const separatedEndBy = (parser: Parser, separator: Parser) => { - return create((input) => { - const result = separatedBy(parser, separator)(input); - - const { value: values, remaining } = result as Success; - const sepResult = separator(remaining); + return create((input, index = 0, ctx) => { + const result = separatedBy(parser, separator)( + input, + index, + ctx, + ) as Success; + const sepResult = separator(input, result.index, ctx); - return success(values, sepResult.ok ? sepResult.remaining : remaining); + return success( + result.value, + sepResult.ok ? sepResult.index : result.index, + ); }); }; diff --git a/src/combinators/separatedEndBy1.test.ts b/src/combinators/separatedEndBy1.test.ts index c306e55..3a85886 100644 --- a/src/combinators/separatedEndBy1.test.ts +++ b/src/combinators/separatedEndBy1.test.ts @@ -25,7 +25,7 @@ describe('separatedEndBy1', () => { const parser = separatedEndBy1(parser1, parser2); const result = parser('ACC'); - assertSuccess<'A'[]>(result, ['A'], 'CC'); + assertSuccess<'A'[]>(result, ['A'], 1); }); it('should parse single element with separator', () => { @@ -35,7 +35,7 @@ describe('separatedEndBy1', () => { const parser = separatedEndBy1(parser1, parser2); const result = parser('A,'); - assertSuccess<'A'[]>(result, ['A'], ''); + assertSuccess<'A'[]>(result, ['A'], 2); }); it('should parse multiple elements with trailing separator', () => { @@ -45,7 +45,7 @@ describe('separatedEndBy1', () => { const parser = separatedEndBy1(parser1, parser2); const result = parser('A,A,A,'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 6); }); it('should parse multiple elements without trailing separator', () => { @@ -55,7 +55,7 @@ describe('separatedEndBy1', () => { const parser = separatedEndBy1(parser1, parser2); const result = parser('A,A,A'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 5); }); it('should fail on empty input', () => { diff --git a/src/combinators/separatedEndBy1.ts b/src/combinators/separatedEndBy1.ts index 3901105..3f836d8 100644 --- a/src/combinators/separatedEndBy1.ts +++ b/src/combinators/separatedEndBy1.ts @@ -1,24 +1,27 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { separatedBy1 } from './separatedBy1'; /** - * One or more items separated by and ending with a terminator. + * Parse one or more items separated by a separator, allowing a trailing one. * * @example - * separatedEndBy1(char('a'), char(';'))('a;a;a;') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * separatedEndBy1(digits, char(','))('1,2,3,') // { ok: true, value: [1, 2, 3], index: 6 } */ export const separatedEndBy1 = (parser: Parser, separator: Parser) => { - return create((input) => { - const result = separatedBy1(parser, separator)(input); - if (!result.ok) return failure(); + return create((input, index = 0, ctx) => { + const result = separatedBy1(parser, separator)(input, index, ctx); + if (!result.ok) { + return result; + } - const { value: values, remaining } = result; - const sepResult = separator(remaining); + const sepResult = separator(input, result.index, ctx); - return success(values, sepResult.ok ? sepResult.remaining : remaining); + return success( + result.value, + sepResult.ok ? sepResult.index : result.index, + ); }); }; diff --git a/src/combinators/separatedUntil.test.ts b/src/combinators/separatedUntil.test.ts index 13273f0..3ecdb68 100644 --- a/src/combinators/separatedUntil.test.ts +++ b/src/combinators/separatedUntil.test.ts @@ -16,7 +16,7 @@ describe('separatedUntil', () => { const parser = separatedUntil(parser1, parser2, parser3); const result = parser('A,A,A;'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A'], ''); + assertSuccess<'A'[]>(result, ['A', 'A', 'A'], 6); }); it('should handle single element with terminator', () => { @@ -27,7 +27,7 @@ describe('separatedUntil', () => { const parser = separatedUntil(parser1, parser2, parser3); const result = parser('A;'); - assertSuccess<'A'[]>(result, ['A'], ''); + assertSuccess<'A'[]>(result, ['A'], 2); }); it('should handle empty list with terminator', () => { @@ -38,7 +38,7 @@ describe('separatedUntil', () => { const parser = separatedUntil(parser1, parser2, parser3); const result = parser(';'); - assertSuccess<'A'[]>(result, [], ''); + assertSuccess<'A'[]>(result, [], 1); }); it('should fail without terminator', () => { diff --git a/src/combinators/separatedUntil.ts b/src/combinators/separatedUntil.ts index e4bfb6f..c767dec 100644 --- a/src/combinators/separatedUntil.ts +++ b/src/combinators/separatedUntil.ts @@ -1,27 +1,28 @@ -import { failure } from '../core/failure'; -import { create, type Parser } from '../core/parser'; +import type { Parser } from '../core/parser'; + +import { create } from '../core/parser'; import { success, type Success } from '../core/success'; import { separatedBy } from './separatedBy'; /** - * Parse items separated by separator until terminator matches. + * Parse items separated by a separator, up to a terminator. * * @example - * separatedUntil(char('a'), char(','), char(';'))('a,a,a;') // { ok: true, value: ['a', 'a', 'a'], remaining: '' } + * separatedUntil(digits, char(','), char(';'))('1,2,3;') // { ok: true, value: [1, 2, 3], index: 6 } */ export const separatedUntil = ( parser: Parser, separator: Parser, terminator: Parser, ) => { - return create((input) => { - const { value: values, remaining } = separatedBy( - parser, - separator, - )(input) as Success; - - const result = terminator(remaining); + return create((input, index = 0, ctx) => { + const items = separatedBy(parser, separator)( + input, + index, + ctx, + ) as Success; + const result = terminator(input, items.index, ctx); - return result.ok ? success(values, result.remaining) : failure(); + return result.ok ? success(items.value, result.index) : result; }); }; diff --git a/src/combinators/sequence.test.ts b/src/combinators/sequence.test.ts index 93c233f..9a13f84 100644 --- a/src/combinators/sequence.test.ts +++ b/src/combinators/sequence.test.ts @@ -18,7 +18,7 @@ describe('sequence', () => { const parser = sequence(parser1, parser2, parser3); const result = parser('ABCDE'); - assertSuccess<['A', 'B', 'C'] | null>(result, ['A', 'B', 'C'], 'DE'); + assertSuccess<['A', 'B', 'C'] | null>(result, ['A', 'B', 'C'], 3); }); it('should thread remaining input through parsers', () => { @@ -27,11 +27,13 @@ describe('sequence', () => { const parser = sequence(parser1, parser2); const result = parser('ABC'); - assertSuccess<['A', 'B'] | null>(result, ['A', 'B'], 'C'); + assertSuccess<['A', 'B'] | null>(result, ['A', 'B'], 2); }); it('should fail if first parser fails', () => { - const failureParser = create(() => failure()); + const failureParser = create((_input, index = 0) => + failure(undefined, index), + ); const parser2 = createTestParser('B'); const parser = sequence(failureParser, parser2); const result = parser('xxx'); @@ -41,7 +43,9 @@ describe('sequence', () => { it('should fail if middle parser fails', () => { const parser1 = createTestParser('A'); - const failureParser = create(() => failure()); + const failureParser = create((_input, index = 0) => + failure(undefined, index), + ); const parser3 = createTestParser('C'); const parser = sequence(parser1, failureParser, parser3); const result = parser('xxx'); @@ -53,15 +57,17 @@ describe('sequence', () => { const parser = sequence(); const result = parser('anything'); - assertSuccess<[] | null>(result, [], 'anything'); + assertSuccess<[] | null>(result, [], 0); }); it('should preserve parser result types', () => { - const strParser = create<'text'>(() => success('text', '')); - const numParser = create<42>(() => success(42, '')); + const strParser = create<'text'>((input) => + success('text', input.length), + ); + const numParser = create<42>((input) => success(42, input.length)); const parser = sequence(strParser, numParser); const result = parser('xx'); - assertSuccess<[string, number]>(result, ['text', 42], ''); + assertSuccess<[string, number]>(result, ['text', 42], 2); }); }); diff --git a/src/combinators/sequence.ts b/src/combinators/sequence.ts index d2148d2..f9d9d48 100644 --- a/src/combinators/sequence.ts +++ b/src/combinators/sequence.ts @@ -1,32 +1,32 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** * Parse a sequence of parsers and return all results as an array. * * @example - * sequence(char('a'), char('b'), char('c'))('abc') // { ok: true, value: ['a', 'b', 'c'], remaining: '' } + * sequence(char('a'), char('b'), char('c'))('abc') // { ok: true, value: ['a', 'b', 'c'], index: 3 } */ export const sequence = ( ...parsers: { [K in keyof T]: Parser } ) => { - return create((input) => { + return create((input, index = 0, ctx) => { const results: unknown[] = []; - let remaining = input; + let at = index; for (const parser of parsers) { - const result = parser(remaining); + const result: Result = parser(input, at, ctx); if (!result.ok) { - return failure(); + return result; } results.push(result.value); - remaining = result.remaining; + at = result.index; } - return success(results as unknown as T, remaining); + return success(results as unknown as T, at); }); }; diff --git a/src/combinators/skip.test.ts b/src/combinators/skip.test.ts index c83bce2..81a80d4 100644 --- a/src/combinators/skip.test.ts +++ b/src/combinators/skip.test.ts @@ -13,7 +13,7 @@ describe('skip', () => { const parser = skip(parser1, 3); const result = parser('AAAB'); - assertSuccess(result, null, 'B'); + assertSuccess(result, null, 3); }); it('should fail if not enough occurrences', () => { @@ -29,6 +29,6 @@ describe('skip', () => { const parser = skip(parser1, 0); const result = parser('ABC'); - assertSuccess(result, null, 'ABC'); + assertSuccess(result, null, 0); }); }); diff --git a/src/combinators/skip.ts b/src/combinators/skip.ts index 8f2797d..2ce6446 100644 --- a/src/combinators/skip.ts +++ b/src/combinators/skip.ts @@ -1,27 +1,28 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** * Skip a parser n times. * * @example - * skip(char('a'), 2)('aabc') // { ok: true, value: null, remaining: 'bc' } + * skip(char('a'), 2)('aabc') // { ok: true, value: null, index: 2 } */ export const skip = (parser: Parser, count: number) => { - return create((input) => { - let remaining = input; + return create((input, index = 0, ctx) => { + let at = index; for (let i = 0; i < count; i++) { - const result = parser(remaining); + const result: Result = parser(input, at, ctx); if (!result.ok) { - return failure(); + return result; } - remaining = result.remaining; + + at = result.index; } - return success(null, remaining); + return success(null, at); }); }; diff --git a/src/combinators/skipMany.test.ts b/src/combinators/skipMany.test.ts index a3505ff..47655cb 100644 --- a/src/combinators/skipMany.test.ts +++ b/src/combinators/skipMany.test.ts @@ -9,7 +9,7 @@ describe('skipMany', () => { const parser = skipMany(parser1); const result = parser('AAABBB'); - assertSuccess(result, null, 'BBB'); + assertSuccess(result, null, 3); }); it('should return null even when no matches found', () => { @@ -17,7 +17,7 @@ describe('skipMany', () => { const parser = skipMany(parser1); const result = parser('BBB'); - assertSuccess(result, null, 'BBB'); + assertSuccess(result, null, 0); }); it('should handle empty input', () => { @@ -25,6 +25,6 @@ describe('skipMany', () => { const parser = skipMany(parser1); const result = parser(''); - assertSuccess(result, null, ''); + assertSuccess(result, null, 0); }); }); diff --git a/src/combinators/skipMany.ts b/src/combinators/skipMany.ts index 1926642..7b238e4 100644 --- a/src/combinators/skipMany.ts +++ b/src/combinators/skipMany.ts @@ -8,7 +8,7 @@ import { map } from './map'; * Skip zero or more occurrences (never fails, returns null). * * @example - * skipMany(char('a'))('aaabc') // { ok: true, value: null, remaining: 'bc' } + * skipMany(char('a'))('aaabc') // { ok: true, value: null, index: 3 } */ export const skipMany = (parser: Parser) => { return create(map(many(parser), () => null)); diff --git a/src/combinators/skipMany1.test.ts b/src/combinators/skipMany1.test.ts index 2920cd9..1e86cae 100644 --- a/src/combinators/skipMany1.test.ts +++ b/src/combinators/skipMany1.test.ts @@ -14,7 +14,7 @@ describe('skipMany1', () => { const parser = skipMany1(parser1); const result = parser('AAABBB'); - assertSuccess(result, null, 'BBB'); + assertSuccess(result, null, 3); }); it('should fail when no matches found', () => { @@ -32,7 +32,7 @@ describe('skipMany1', () => { const parser = skipMany1(parser1); const result = parser('ABBB'); - assertSuccess(result, null, 'BBB'); + assertSuccess(result, null, 1); }); it('should require at least one match', () => { diff --git a/src/combinators/skipMany1.ts b/src/combinators/skipMany1.ts index 3bff0c3..4aff71e 100644 --- a/src/combinators/skipMany1.ts +++ b/src/combinators/skipMany1.ts @@ -8,7 +8,7 @@ import { map } from './map'; * Skip one or more occurrences (fails if no matches). * * @example - * skipMany1(char('a'))('aaabc') // { ok: true, value: null, remaining: 'bc' } + * skipMany1(char('a'))('aaabc') // { ok: true, value: null, index: 3 } */ export const skipMany1 = (parser: Parser) => { return create(map(many1(parser), () => null)); diff --git a/src/combinators/surrounded.test.ts b/src/combinators/surrounded.test.ts index d471030..f1c59ac 100644 --- a/src/combinators/surrounded.test.ts +++ b/src/combinators/surrounded.test.ts @@ -15,7 +15,7 @@ describe('surrounded', () => { const parser = surrounded(parser1, parser2); const result = parser('ABA'); - assertSuccess<'B'>(result, 'B', ''); + assertSuccess<'B'>(result, 'B', 3); }); it('should fail if opening delimiter fails', () => { @@ -56,7 +56,7 @@ describe('surrounded', () => { const parser = surrounded(parser1, parser2, parser3); const result = parser('ABCAA'); - assertSuccess<'B'>(result, 'B', 'AA'); + assertSuccess<'B'>(result, 'B', 3); }); it('should leave remaining input', () => { @@ -66,6 +66,6 @@ describe('surrounded', () => { const parser = surrounded(parser1, parser2); const result = parser('ABAAA'); - assertSuccess<'B'>(result, 'B', 'AA'); + assertSuccess<'B'>(result, 'B', 3); }); }); diff --git a/src/combinators/surrounded.ts b/src/combinators/surrounded.ts index 573030f..d0914b8 100644 --- a/src/combinators/surrounded.ts +++ b/src/combinators/surrounded.ts @@ -7,8 +7,8 @@ import { inner } from './inner'; * Parse content surrounded by delimiters. * * @example - * surrounded(char('['), string('hi'), char(']'))('[hi]') // { ok: true, value: 'hi', remaining: '' } - * surrounded(char('a'), char('b'), char('c'))('abc') // { ok: true, value: 'b', remaining: '' } + * surrounded(char('['), string('hi'), char(']'))('[hi]') // { ok: true, value: 'hi', index: 4 } + * surrounded(char('a'), char('b'), char('c'))('abc') // { ok: true, value: 'b', index: 3 } */ export const surrounded = ( first: Parser, diff --git a/src/combinators/unless.test.ts b/src/combinators/unless.test.ts index cad5fe6..b570098 100644 --- a/src/combinators/unless.test.ts +++ b/src/combinators/unless.test.ts @@ -12,14 +12,14 @@ describe('unless', () => { const parser = unless(false, createTestParser('A')); const result = parser('AAA'); - assertSuccess<'A' | null>(result, 'A', 'AA'); + assertSuccess<'A' | null>(result, 'A', 1); }); it('should return null and consume no input when condition is true', () => { const parser = unless(true, createTestParser('A')); const result = parser('AAA'); - assertSuccess<'A' | null>(result, null, 'AAA'); + assertSuccess<'A' | null>(result, null, 0); }); it('should fail when condition is false but parser fails', () => { diff --git a/src/combinators/unless.ts b/src/combinators/unless.ts index 1c3fe46..87e4155 100644 --- a/src/combinators/unless.ts +++ b/src/combinators/unless.ts @@ -7,11 +7,11 @@ import { success } from '../core/success'; * Parse unless condition is true (inverse of guard). * * @example - * unless(false, string('hello'))('hello') // { ok: true, value: 'hello', remaining: '' } - * unless(true, string('hello'))('hello') // { ok: true, value: null, remaining: 'hello' } + * unless(false, string('hello'))('hello') // { ok: true, value: 'hello', index: 5 } + * unless(true, string('hello'))('hello') // { ok: true, value: null, index: 0 } */ export const unless = (condition: boolean, parser: Parser) => { - return create((input) => { - return !condition ? parser(input) : success(null, input); + return create((input, index = 0, ctx) => { + return !condition ? parser(input, index, ctx) : success(null, index); }); }; diff --git a/src/combinators/until.test.ts b/src/combinators/until.test.ts index f8a9644..0189150 100644 --- a/src/combinators/until.test.ts +++ b/src/combinators/until.test.ts @@ -17,15 +17,17 @@ describe('until', () => { const parser = until(aParser, bParser); const result = parser('AAAABAAAA'); - assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A'], 'BAAAA'); + assertSuccess<'A'[]>(result, ['A', 'A', 'A', 'A'], 4); }); it('should return empty array when terminator is at start', () => { - const failureParser = create(() => failure()); + const failureParser = create((_input, index = 0) => + failure(undefined, index), + ); const parser = until(failureParser, bParser); const result = parser('BAAAA'); - assertSuccess(result, [], 'BAAAA'); + assertSuccess(result, [], 0); }); it('should fail when terminator is never found and parser fails', () => { diff --git a/src/combinators/until.ts b/src/combinators/until.ts index c1032fc..a5f4efc 100644 --- a/src/combinators/until.ts +++ b/src/combinators/until.ts @@ -1,36 +1,34 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; +import { type Result } from '../core/result'; import { success } from '../core/success'; /** - * Parse until terminator matches (fails if terminator never matches). + * Parse zero or more until terminator matches, leaving the terminator unconsumed. * * @example - * until(char('a'), char('b'))('baaa') // { ok: true, value: [], remaining: 'baaa' } - * until(char('a'), char('b'))('aaba') // { ok: true, value: ['a', 'a'], remaining: 'ba' } + * until(char('a'), char('b'))('aaab') // { ok: true, value: ['a', 'a', 'a'], index: 3 } */ export const until = (parser: Parser, terminator: Parser) => { - return create((input) => { + return create((input, index = 0, ctx) => { const results: T[] = []; - let remaining = input; + let at = index; while (true) { - const termResult = terminator(remaining); + const termResult: Result = terminator(input, at, ctx); if (termResult.ok) { break; } - - const parseResult = parser(remaining); + const parseResult: Result = parser(input, at, ctx); if (!parseResult.ok) { - return failure(); + return parseResult; } results.push(parseResult.value); - remaining = parseResult.remaining; + at = parseResult.index; } - return success(results, remaining); + return success(results, at); }); }; diff --git a/src/combinators/validate.test.ts b/src/combinators/validate.test.ts index 76fb009..f413ac0 100644 --- a/src/combinators/validate.test.ts +++ b/src/combinators/validate.test.ts @@ -13,7 +13,7 @@ describe('validate', () => { const parser = validate(parser1, (value) => value === 'A'); const result = parser('AAA'); - assertSuccess<'A'>(result, 'A', 'AA'); + assertSuccess<'A'>(result, 'A', 1); }); it('should fail when parser succeeds but predicate returns false', () => { diff --git a/src/combinators/validate.ts b/src/combinators/validate.ts index 3c2ebd3..002c026 100644 --- a/src/combinators/validate.ts +++ b/src/combinators/validate.ts @@ -7,17 +7,18 @@ import { create } from '../core/parser'; * Validate parsed value with a predicate. * * @example - * validate(digit, (n) => n > 5)('7') // { ok: true, value: 7, remaining: '' } - * validate(digit, (n) => n > 5)('3') // { ok: false } + * validate(digit, (n) => n > 5)('7') // { ok: true, value: 7, index: 1 } + * validate(digit, (n) => n > 5)('3') // { ok: false, index: 0, expected: ['valid value'] } */ export const validate = ( parser: Parser, predicate: (value: T) => boolean, + expected = 'valid value', ) => { - return create((input) => { - const result = parser(input); - if (!result.ok) return failure(); + return create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); + if (!result.ok) return result; - return predicate(result.value) ? result : failure(); + return predicate(result.value) ? result : failure(ctx, index, expected); }); }; diff --git a/src/combinators/value.test.ts b/src/combinators/value.test.ts index 534f770..565c2d3 100644 --- a/src/combinators/value.test.ts +++ b/src/combinators/value.test.ts @@ -14,21 +14,21 @@ describe('value', () => { const parser = value(aParser, 42); const result = parser('ABC'); - assertSuccess(result, 42, 'BC'); + assertSuccess(result, 42, 1); }); it('should work with boolean constant', () => { const parser = value(aParser, true); const result = parser('ABC'); - assertSuccess(result, true, 'BC'); + assertSuccess(result, true, 1); }); it('should work with null constant', () => { const parser = value(aParser, null); const result = parser('ABC'); - assertSuccess(result, null, 'BC'); + assertSuccess(result, null, 1); }); it('should fail when underlying parser fails', () => { diff --git a/src/combinators/value.ts b/src/combinators/value.ts index 69e8f6c..87c1cc4 100644 --- a/src/combinators/value.ts +++ b/src/combinators/value.ts @@ -6,8 +6,8 @@ import { map } from './map'; * Replace parsed value with a constant. * * @example - * value(string('true'), true)('true') // { ok: true, value: true, remaining: '' } - * value(string('null'), null)('null') // { ok: true, value: null, remaining: '' } + * value(string('true'), true)('true') // { ok: true, value: true, index: 4 } + * value(string('null'), null)('null') // { ok: true, value: null, index: 4 } */ export const value = (parser: Parser, constant: U): Parser => { return map(parser, () => constant); diff --git a/src/combinators/when.test.ts b/src/combinators/when.test.ts index 2496c47..706392d 100644 --- a/src/combinators/when.test.ts +++ b/src/combinators/when.test.ts @@ -10,15 +10,17 @@ import { create } from '../core/parser'; import { success } from '../core/success'; import { when } from './when'; -const boolParser = create((input) => { - if (input.startsWith('*')) { - return success(true, input.slice(1)); +const boolParser = create((input, index = 0) => { + if (input.startsWith('*', index)) { + return success(true, index + 1); } - return success(false, input); + return success(false, index); }); -const failingCondition = create(() => failure()); +const failingCondition = create((_input, index = 0) => + failure(undefined, index), +); describe('when', () => { const thenP = createTestParser('yes'); @@ -28,14 +30,14 @@ describe('when', () => { const parser = when(boolParser, thenP, elseP); const result = parser('*yes'); - assertSuccess<'yes' | 'no'>(result, 'yes', ''); + assertSuccess<'yes' | 'no'>(result, 'yes', 4); }); it('should run elseParser when condition is false', () => { const parser = when(boolParser, thenP, elseP); const result = parser('no'); - assertSuccess<'yes' | 'no'>(result, 'no', ''); + assertSuccess<'yes' | 'no'>(result, 'no', 2); }); it('should fail when condition parser fails', () => { diff --git a/src/combinators/when.ts b/src/combinators/when.ts index 2906799..1e9115a 100644 --- a/src/combinators/when.ts +++ b/src/combinators/when.ts @@ -1,29 +1,28 @@ import type { Parser } from '../core/parser'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; /** * Branch on a boolean parser result. * * @example - * when(flag(char('*')), pure('many'), pure('one'))('*rest') // { ok: true, value: 'many', remaining: 'rest' } - * when(flag(char('*')), pure('many'), pure('one'))('abc') // { ok: true, value: 'one', remaining: 'abc' } + * when(flag(char('*')), pure('many'), pure('one'))('*rest') // { ok: true, value: 'many', index: 1 } + * when(flag(char('*')), pure('many'), pure('one'))('abc') // { ok: true, value: 'one', index: 0 } */ export const when = ( condition: Parser, thenParser: Parser, elseParser: Parser, ): Parser => { - return create((input) => { - const result = condition(input); + return create((input, index = 0, ctx) => { + const result = condition(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - return result.value - ? thenParser(result.remaining) - : elseParser(result.remaining); + const branch = result.value ? thenParser : elseParser; + + return branch(input, result.index, ctx); }); }; diff --git a/src/core/context.test.ts b/src/core/context.test.ts new file mode 100644 index 0000000..716064c --- /dev/null +++ b/src/core/context.test.ts @@ -0,0 +1,61 @@ +import { describe, expect, it } from 'vitest'; + +import { context, record, union } from './context'; + +describe('context', () => { + it('should start with nothing recorded', () => { + expect(context()).toEqual({ furthest: -1, expected: [] }); + }); +}); + +describe('record', () => { + it('should keep the furthest failure', () => { + const ctx = context(); + record(ctx, 3, ['digit']); + record(ctx, 7, ['letter']); + record(ctx, 5, ['comma']); + + expect(ctx).toEqual({ furthest: 7, expected: ['letter'] }); + }); + + it('should union expectations recorded at the same offset', () => { + const ctx = context(); + record(ctx, 4, ['digit']); + record(ctx, 4, ['letter']); + + expect(ctx.expected).toEqual(['digit', 'letter']); + }); + + it('should not repeat an expectation', () => { + const ctx = context(); + record(ctx, 4, ['digit']); + record(ctx, 4, ['digit']); + + expect(ctx.expected).toEqual(['digit']); + }); + + it('should leave the caller array untouched when unioning', () => { + const ctx = context(); + const first = ['digit']; + record(ctx, 4, first); + record(ctx, 4, ['letter']); + + expect(first).toEqual(['digit']); + }); + + it('should do nothing without a context', () => { + expect(() => record(undefined, 4, ['digit'])).not.toThrow(); + }); +}); + +describe('union', () => { + it('should return the first array when it already covers the second', () => { + const a = ['digit', 'letter']; + + expect(union(a, ['digit'])).toBe(a); + }); + + it('should combine disjoint sets', () => { + expect(union(['digit'], ['letter'])).toEqual(['digit', 'letter']); + }); +}); diff --git a/src/core/context.ts b/src/core/context.ts new file mode 100644 index 0000000..4eb00c0 --- /dev/null +++ b/src/core/context.ts @@ -0,0 +1,78 @@ +const NONE: readonly string[] = []; + +/** + * Per-parse bookkeeping for error messages: the furthest offset any branch + * reached and what was wanted there. + * + * A parse can succeed while leaving input unconsumed, and a branch that failed + * deep in the input can be backtracked over entirely. The context is what lets + * the final error point at the real problem instead of wherever parsing + * happened to stop. It only ever moves forward, so backtracking cannot lose it. + */ +export type Context = { furthest: number; expected: readonly string[] }; + +/** + * Creates a fresh context. {@link run} and {@link parse} make one per call. + * + * @example + * context() // { furthest: -1, expected: [] } + */ +export const context = (): Context => ({ furthest: -1, expected: NONE }); + +/** + * Adds the entries of `b` to `a`, returning `a` itself when it already covers + * them. Callers rely on that identity to skip allocating. + * + * @example + * union(['digit'], ['digit']) // ['digit'] + */ +export const union = ( + a: readonly string[], + b: readonly string[], +): readonly string[] => { + if (b.length === 0 || a === b) return a; + if (a.length === 0) return b; + + let extra: string[] | null = null; + for (const item of b) { + if (!a.includes(item)) { + (extra ??= []).push(item); + } + } + + return extra ? [...a, ...extra] : a; +}; + +/** + * Notes what was expected at an offset, keeping only the furthest one seen. + * + * Replaces `expected` rather than mutating it, so a parser may pass the same + * array every time without it being altered underneath. + * + * @example + * const ctx = context(); + * record(ctx, 3, ['digit']); + * ctx // { furthest: 3, expected: ['digit'] } + */ +export const record = ( + ctx: Context | undefined, + index: number, + expected: readonly string[], +): void => { + if (ctx === undefined) { + return; + } + + if (index > ctx.furthest) { + ctx.furthest = index; + ctx.expected = expected; + + return; + } + + if (index < ctx.furthest || ctx.expected === expected) { + return; + } + + ctx.expected = union(ctx.expected, expected); +}; diff --git a/src/core/error.ts b/src/core/error.ts new file mode 100644 index 0000000..ee32c74 --- /dev/null +++ b/src/core/error.ts @@ -0,0 +1,28 @@ +import { format } from './format'; +import { locate } from './locate'; + +/** + * Error thrown by {@link run} when a parse fails, carrying the offset, the + * 1-based line/column and the set of expectations at that offset. + * + * @example + * new ParseError('a=1,b=x', 6, ['digit']).line // 1 + */ +export class ParseError extends Error { + readonly index: number; + readonly line: number; + readonly column: number; + readonly expected: readonly string[]; + + constructor(input: string, index: number, expected: readonly string[]) { + super(format(input, index, expected)); + + const { line, column } = locate(input, index); + + this.name = 'ParseError'; + this.index = index; + this.line = line; + this.column = column; + this.expected = expected; + } +} diff --git a/src/core/failure.test.ts b/src/core/failure.test.ts index 4c81d56..b755bef 100644 --- a/src/core/failure.test.ts +++ b/src/core/failure.test.ts @@ -1,12 +1,25 @@ -import { describe, it } from 'vitest'; +import { describe, expect, it } from 'vitest'; -import { assertFailure } from '../../test/utils'; +import { context } from './context'; import { failure } from './failure'; describe('failure', () => { - it('should create failed parse result', () => { - const result = failure(); + it('should create failed parse result at an offset', () => { + const result = failure(undefined, 3); - assertFailure(result); + expect(result).toEqual({ ok: false, index: 3, expected: [] }); + }); + + it('should record what was expected at that offset', () => { + const result = failure(undefined, 3, 'digit', 'letter'); + + expect(result.expected).toEqual(['digit', 'letter']); + }); + + it('should note the failure in the parse context', () => { + const ctx = context(); + failure(ctx, 3, 'digit'); + + expect(ctx).toEqual({ furthest: 3, expected: ['digit'] }); }); }); diff --git a/src/core/failure.ts b/src/core/failure.ts index 1f48646..1adf72b 100644 --- a/src/core/failure.ts +++ b/src/core/failure.ts @@ -1,12 +1,41 @@ -export type Failure = { ok: false; error?: string }; +import { type Context, record } from './context'; + +export type Failure = { ok: false; index: number; expected: readonly string[] }; /** - * Creates a failed result with an optional error message. + * Creates a failed result at an offset and notes it in the parse context. + * + * Expectations are phrased as nouns ('digit', "'{'"). Pass the context through + * from the parser so the message can name this position even if the branch is + * later backtracked over. * * @example - * failure('unexpected input') // { ok: false, error: 'unexpected input' } + * failure(undefined, 3, 'digit') // { ok: false, index: 3, expected: ['digit'] } */ -export const failure = (error?: string): Failure => ({ - ok: false, - error, -}); +export const failure = ( + ctx: Context | undefined, + index: number, + ...expected: string[] +): Failure => { + record(ctx, index, expected); + + return { ok: false, index, expected }; +}; + +/** + * Creates a failed result from an expectation list that is already allocated, + * avoiding a fresh array on every failure. + * + * @example + * const expected = ['digit']; + * reject(undefined, 3, expected) // { ok: false, index: 3, expected: ['digit'] } + */ +export const reject = ( + ctx: Context | undefined, + index: number, + expected: readonly string[], +): Failure => { + record(ctx, index, expected); + + return { ok: false, index, expected }; +}; diff --git a/src/core/format.test.ts b/src/core/format.test.ts new file mode 100644 index 0000000..88b82eb --- /dev/null +++ b/src/core/format.test.ts @@ -0,0 +1,43 @@ +import { describe, expect, it } from 'vitest'; + +import { format } from './format'; + +describe('format', () => { + it('should point a caret at the offset', () => { + expect(format('a=1,b=x', 6, ['digit'])).toBe( + ["1:7 expected digit, found 'x'", ' 1 | a=1,b=x', ' | ^'].join('\n'), + ); + }); + + it('should report the line the offset falls on', () => { + expect(format('one\ntwo\nthree', 8, ['digit'])).toBe( + ["3:1 expected digit, found 't'", ' 3 | three', ' | ^'].join('\n'), + ); + }); + + it('should say end of input when there is nothing at the offset', () => { + expect(format('ab', 2, ['digit'])).toContain('found end of input'); + }); + + it('should escape a control character rather than print it raw', () => { + expect(format('a\nb', 1, ['digit'])).toContain("found '\\n'"); + }); + + it('should list several expectations', () => { + expect(format('x', 0, ['digit', 'letter', 'comma'])).toContain( + 'expected comma, digit or letter', + ); + }); + + it('should fall back when nothing was expected', () => { + expect(format('x', 0, [])).toContain('expected something else'); + }); + + it('should widen the gutter for larger line numbers', () => { + const input = Array.from({ length: 10 }, () => 'x').join('\n'); + + expect(format(input, 18, ['digit'])).toBe( + ["10:1 expected digit, found 'x'", ' 10 | x', ' | ^'].join('\n'), + ); + }); +}); diff --git a/src/core/format.ts b/src/core/format.ts new file mode 100644 index 0000000..2049997 --- /dev/null +++ b/src/core/format.ts @@ -0,0 +1,45 @@ +import { locate } from './locate'; + +const describeFound = (input: string, index: number): string => { + if (index >= input.length) return 'end of input'; + + const char = input[index]; + const escaped = JSON.stringify(char).slice(1, -1); + + return `'${escaped}'`; +}; + +const describeExpected = (expected: readonly string[]): string => { + if (expected.length === 0) return 'expected something else'; + if (expected.length === 1) return `expected ${expected[0]}`; + + const sorted = [...expected].sort(); + const last = sorted.pop(); + + return `expected ${sorted.join(', ')} or ${last}`; +}; + +/** + * Renders a parse failure as a source excerpt with a caret under the offset. + * + * @example + * format('a=1,b=x', 6, ['digit']) // "1:7 expected digit, found 'x'\n 1 | a=1,b=x\n | ^" + */ +export const format = ( + input: string, + index: number, + expected: readonly string[], +): string => { + const { line, column } = locate(input, index); + const lines = input.split('\n'); + const source = lines[line - 1]; + const gutter = String(line); + const pad = ' '.repeat(gutter.length); + const headline = `${line}:${column} ${describeExpected(expected)}, found ${describeFound(input, index)}`; + + return [ + headline, + ` ${gutter} | ${source}`, + ` ${pad} | ${' '.repeat(column - 1)}^`, + ].join('\n'); +}; diff --git a/src/core/grammar.test.ts b/src/core/grammar.test.ts index ccd674c..04c0959 100644 --- a/src/core/grammar.test.ts +++ b/src/core/grammar.test.ts @@ -5,6 +5,7 @@ import { choice } from '../combinators/choice'; import { map } from '../combinators/map'; import { sequence } from '../combinators/sequence'; import { surrounded } from '../combinators/surrounded'; +import { digits } from '../primitives/digits'; import { char } from '../terminals/char'; import { regex } from '../terminals/regex'; import { string } from '../terminals/string'; @@ -44,12 +45,12 @@ describe('grammar', () => { { const result = a('abc'); - assertSuccess<'a'>(result, 'a', 'bc'); + assertSuccess<'a'>(result, 'a', 1); } { const result = b('bcd'); - assertSuccess<'b'>(result, 'b', 'cd'); + assertSuccess<'b'>(result, 'b', 1); } }); @@ -73,22 +74,36 @@ describe('grammar', () => { { const result = add('2+3'); - assertSuccess(result, 5, ''); + assertSuccess(result, 5, 3); } { const result = number('5'); - assertSuccess(result, 5, ''); + assertSuccess(result, 5, 1); } { const result = term('(2+3)'); - assertSuccess(result, 5, ''); + assertSuccess(result, 5, 5); } { const result = expr('1+(2+3)'); // [6, ''] - assertSuccess(result, 6, ''); + assertSuccess(result, 6, 7); } }); + + it('should name a left-recursive rule instead of overflowing the stack', () => { + const g = grammar<{ expr: number }>({ + expr: (p) => + choice( + map(sequence(p.expr, char('+'), digits), ([l, , r]) => l + r), + digits, + ), + }); + + expect(() => g.expr('1+2')).toThrowError( + 'Left recursion detected in grammar rule "expr" at offset 0', + ); + }); }); diff --git a/src/core/grammar.ts b/src/core/grammar.ts index 57c5d43..8be3801 100644 --- a/src/core/grammar.ts +++ b/src/core/grammar.ts @@ -1,14 +1,44 @@ -import type { Parser } from '../core/parser'; - -import { lazy } from './lazy'; +import type { Parser } from './parser'; export type Grammar> = { [K in keyof T]: (parsers: { [P in keyof T]: Parser }) => Parser; }; +/** + * Wraps a rule so re-entering it at an offset it is already parsing throws a + * named error instead of overflowing the stack. + */ +const guardLeftRecursion = ( + name: string, + thunk: () => Parser, +): Parser => { + const active: number[] = []; + + return (input, index = 0, ctx) => { + if (active.includes(index)) { + throw new Error( + `Left recursion detected in grammar rule "${name}" at offset ${index}. ` + + `Rewrite the rule to consume input before recursing — e.g. with chainLeft1 or many.`, + ); + } + + active.push(index); + + try { + return thunk()(input, index, ctx); + } finally { + active.pop(); + } + }; +}; + /** * Creates a recursive grammar where rules can reference each other. * + * Rules are resolved lazily through a shared record, so they may refer to one + * another by name without forward declarations. Left-recursive rules are + * reported by name rather than overflowing the stack. + * * @example * type Math = { * expr: number; @@ -30,7 +60,7 @@ export const grammar = >( const parsers = {} as { [K in keyof T]: Parser }; for (const key in definitions) { - parsers[key] = lazy(() => definitions[key](parsers)); + parsers[key] = guardLeftRecursion(key, () => definitions[key](parsers)); } return parsers; diff --git a/src/core/index.ts b/src/core/index.ts index e93843e..cdee1c5 100644 --- a/src/core/index.ts +++ b/src/core/index.ts @@ -1,9 +1,14 @@ +export * from './context'; +export * from './error'; export * from './failure'; +export * from './format'; export * from './grammar'; export * from './label'; export * from './lazy'; +export * from './locate'; export * from './match'; export * from './memoize'; +export * from './parse'; export * from './parser'; export * from './result'; export * from './run'; diff --git a/src/core/label.test.ts b/src/core/label.test.ts index d2fb825..0799a3b 100644 --- a/src/core/label.test.ts +++ b/src/core/label.test.ts @@ -5,6 +5,9 @@ import { assertSuccess, createTestParser, } from '../../test/utils'; +import { sequence } from '../combinators/sequence'; +import { char } from '../terminals/char'; +import { string } from '../terminals/string'; import { label } from './label'; describe('label', () => { @@ -13,7 +16,7 @@ describe('label', () => { const labeled = label(parser, 'letter'); const result = labeled('ABC'); - assertSuccess(result, 'A', 'BC'); + assertSuccess(result, 'A', 1); }); it('should include label in error message on failure', () => { @@ -21,6 +24,13 @@ describe('label', () => { const labeled = label(parser, 'letter'); const result = labeled('BCD'); - assertFailure(result, 'expected letter'); + assertFailure(result, 0, ['letter']); + }); + + it('should keep the inner error once the parser has consumed input', () => { + const parser = label(sequence(string('aa'), char('!')), 'a bang'); + const result = parser('aab'); + + assertFailure(result, 2, ["'!'"]); }); }); diff --git a/src/core/label.ts b/src/core/label.ts index fb5f7f4..5f38299 100644 --- a/src/core/label.ts +++ b/src/core/label.ts @@ -1,21 +1,35 @@ -import type { Parser } from '../core/parser'; +import type { Parser } from './parser'; -import { failure } from './failure'; +import { context, record } from './context'; import { create } from './parser'; /** - * Labels a parser with a custom error message on failure. + * Replaces the expectations of a parser with a single description. + * + * Only applies when the parser failed without consuming input: once a branch + * has committed, its own deeper error is more useful than the label. The + * labelled parser runs against an isolated context so that its internals can + * be suppressed rather than leaking into the message. * * @example - * label(char('x'), 'letter x')('') // { ok: false, error: 'expected letter x' } + * label(char('x'), 'letter x')('') // { ok: false, index: 0, expected: ['letter x'] } */ export const label = (parser: Parser, expected: string): Parser => { - return create((input) => { - const result = parser(input); - if (!result.ok) { - return failure(`expected ${expected}`); + return create((input, index = 0, ctx) => { + const inner = context(); + const result = parser(input, index, inner); + + if (result.ok || result.index > index) { + if (inner.furthest >= 0) { + record(ctx, inner.furthest, inner.expected); + } + + return result; } - return result; + const described = [expected]; + record(ctx, index, described); + + return { ok: false, index, expected: described }; }); }; diff --git a/src/core/lazy.test.ts b/src/core/lazy.test.ts index b3bf242..0eb11db 100644 --- a/src/core/lazy.test.ts +++ b/src/core/lazy.test.ts @@ -25,75 +25,76 @@ describe('lazy', () => { expect(called).toBe(false); const result = parser('ABC'); - assertSuccess<'A'>(result, 'A', 'BC'); + assertSuccess<'A'>(result, 'A', 1); expect(called).toBe(true); }); it('should enable recursive parsers', () => { const charParser = (expected: string) => { - return create((input) => { - if (input.length > 0 && input[0] === expected) { - return success(expected, input.slice(1)); + return create((input, index = 0) => { + if (input[index] === expected) { + return success(expected, index + 1); } - return failure(); + return failure(undefined, index, expected); }); }; const parent: Parser = lazy(() => { const baseCase = charParser('x'); - const recursiveCase = create((input) => { - if (input.length === 0 || input[0] !== '(') { - return failure(); + const recursiveCase = create((input, index = 0) => { + if (input[index] !== '(') { + return failure(undefined, index, "'('"); } - const innerResult = parent(input.slice(1)); + const innerResult = parent(input, index + 1); if (!innerResult.ok) { - return failure(); + return innerResult; } - const { value: innerValue, remaining: afterInner } = - innerResult; - if (afterInner.length === 0 || afterInner[0] !== ')') { - return failure(); + const after = innerResult.index; + if (input[after] !== ')') { + return failure(undefined, after, "')'"); } - return success(innerValue, afterInner.slice(1)); + return success(innerResult.value, after + 1); }); // Try the recursive case first, then base case - return create((input) => { - const recursiveResult = recursiveCase(input); + return create((input, index = 0) => { + const recursiveResult = recursiveCase(input, index); if (recursiveResult.ok) return recursiveResult; - return baseCase(input); + return baseCase(input, index); }); }); { const result = parent('x'); - assertSuccess(result, 'x', ''); + assertSuccess(result, 'x', 1); } { const result = parent('(x)'); - assertSuccess(result, 'x', ''); + assertSuccess(result, 'x', 3); } { const result = parent('((x))'); - assertSuccess(result, 'x', ''); + assertSuccess(result, 'x', 5); } { const result = parent('(((x)))'); - assertSuccess(result, 'x', ''); + assertSuccess(result, 'x', 7); } }); it('should handle parser that fails', () => { - const failureParser = create(() => failure()); + const failureParser = create((_input, index = 0) => + failure(undefined, index), + ); const result = failureParser('goodbye'); assertFailure(result); diff --git a/src/core/lazy.ts b/src/core/lazy.ts index 73cfb21..6b0e9d2 100644 --- a/src/core/lazy.ts +++ b/src/core/lazy.ts @@ -1,4 +1,4 @@ -import type { Parser } from '../core/parser'; +import type { Parser } from './parser'; import { create } from './parser'; @@ -6,8 +6,8 @@ import { create } from './parser'; * Defers parser creation, useful for recursive grammars. * * @example - * lazy(() => char('a'))('abc') // { ok: true, value: 'a', remaining: 'bc' } + * lazy(() => char('a'))('abc') // { ok: true, value: 'a', index: 1 } */ export const lazy = (thunk: () => Parser) => { - return create((input) => thunk()(input)); + return create((input, index = 0, ctx) => thunk()(input, index, ctx)); }; diff --git a/src/core/locate.ts b/src/core/locate.ts new file mode 100644 index 0000000..714eace --- /dev/null +++ b/src/core/locate.ts @@ -0,0 +1,23 @@ +export type Location = { index: number; line: number; column: number }; + +/** + * Converts a character offset into a 1-based line and column. + * + * @example + * locate('a=1\nb=x', 5) // { index: 5, line: 2, column: 2 } + */ +export const locate = (input: string, index: number): Location => { + const clamped = Math.max(0, Math.min(index, input.length)); + + let line = 1; + let lineStart = 0; + + for (let i = 0; i < clamped; i++) { + if (input.charCodeAt(i) === 10) { + line++; + lineStart = i + 1; + } + } + + return { index: clamped, line, column: clamped - lineStart + 1 }; +}; diff --git a/src/core/match.test.ts b/src/core/match.test.ts index 9304cd7..2dd62cf 100644 --- a/src/core/match.test.ts +++ b/src/core/match.test.ts @@ -5,48 +5,51 @@ import { match } from './match'; import { success } from './success'; describe('match', () => { - it('should call success branch with value and remaining', () => { - const result = success('test', 'rest'); + it('should call success branch with value and offset', () => { + const result = success('test', 4); let capturedValue = ''; - let capturedRemaining = ''; + let capturedIndex = -1; match(result, { - success: (value, remaining) => { + success: (value, index) => { capturedValue = value; - capturedRemaining = remaining; + capturedIndex = index; }, failure: () => {}, }); expect(capturedValue).toBe('test'); - expect(capturedRemaining).toBe('rest'); + expect(capturedIndex).toBe(4); }); - it('should call failure branch with error', () => { - const result = failure('test error'); - let capturedError = 'not called'; + it('should call failure branch with offset and expectations', () => { + const result = failure(undefined, 2, 'digit'); + let capturedIndex = -1; + let capturedExpected: readonly string[] = []; match(result, { success: () => {}, - failure: (error) => { - capturedError = error ?? 'no error'; + failure: (index, expected) => { + capturedIndex = index; + capturedExpected = expected; }, }); - expect(capturedError).toBe('test error'); + expect(capturedIndex).toBe(2); + expect(capturedExpected).toEqual(['digit']); }); - it('should call failure branch with undefined when no error', () => { - const result = failure(); - let capturedError = 'not called'; + it('should call failure branch with no expectations when none were recorded', () => { + const result = failure(undefined, 0); + let capturedExpected: readonly string[] = ['not called']; match(result, { success: () => {}, - failure: (error) => { - capturedError = error ?? 'no error'; + failure: (_index, expected) => { + capturedExpected = expected; }, }); - expect(capturedError).toBe('no error'); + expect(capturedExpected).toEqual([]); }); }); diff --git a/src/core/match.ts b/src/core/match.ts index 4fd4920..607a99e 100644 --- a/src/core/match.ts +++ b/src/core/match.ts @@ -4,16 +4,16 @@ import { type Result } from './result'; * Pattern matching on a Result to handle success and failure cases. * * @example - * match(success('hello', ''), { success: (v) => v, failure: () => 'failed' }) // 'hello' + * match(success('hello', 5), { success: (v) => v, failure: () => 'failed' }) // 'hello' */ export const match = ( result: Result, branches: { - success: (value: T, remaining: string) => U; - failure: (error?: string) => U; + success: (value: T, index: number) => U; + failure: (index: number, expected: readonly string[]) => U; }, ): U => { return result.ok - ? branches.success(result.value, result.remaining) - : branches.failure(result.error); + ? branches.success(result.value, result.index) + : branches.failure(result.index, result.expected); }; diff --git a/src/core/memoize.test.ts b/src/core/memoize.test.ts index 9b9f05c..317461f 100644 --- a/src/core/memoize.test.ts +++ b/src/core/memoize.test.ts @@ -36,8 +36,8 @@ describe('memoize', () => { const result1 = parser('123'); const result2 = parser('456'); - assertSuccess(result1, 123, ''); - assertSuccess(result2, 456, ''); + assertSuccess(result1, 123, 3); + assertSuccess(result2, 456, 3); }); it('should work with char parser', () => { @@ -46,7 +46,7 @@ describe('memoize', () => { const result1 = parser('xyz'); const result2 = parser('xyz'); - assertSuccess(result1, 'x', 'yz'); + assertSuccess(result1, 'x', 1); expect(result1).toBe(result2); }); }); diff --git a/src/core/memoize.ts b/src/core/memoize.ts index a4ece12..6590fb9 100644 --- a/src/core/memoize.ts +++ b/src/core/memoize.ts @@ -1,26 +1,56 @@ -import { type Parser } from '../core/parser'; +import { type Context, context, record } from './context'; +import { type Parser } from './parser'; import { type Result } from './result'; +type Entry = { + result: Result; + furthest: number; + expected: readonly string[]; +}; + /** - * Memoizes a parser to cache results by input string. - * Useful for expensive parsers and recursive grammars to avoid - * exponential backtracking. + * Memoizes a parser so each offset is parsed at most once (packrat caching). + * + * Keyed by offset rather than by the remaining input, so entries are cheap and + * the cache is dropped as soon as a different input is parsed. Each entry also + * remembers what the parse expected, so a cache hit still contributes to the + * error message rather than silently weakening it. * * @example * const memoDigits = memoize(digits); - * memoDigits('123') // { ok: true, value: 123, remaining: '' } + * memoDigits('123') // { ok: true, value: 123, index: 3 } */ export const memoize = (parser: Parser): Parser => { - const cache = new Map>(); + let source: string | null = null; + let cache = new Map>(); + + return (input, index = 0, ctx?: Context) => { + if (input !== source) { + source = input; + cache = new Map(); + } - return (input) => { - const cached = cache.get(input); + const cached = cache.get(index); if (cached !== undefined) { - return cached; + if (cached.furthest >= 0) { + record(ctx, cached.furthest, cached.expected); + } + + return cached.result; + } + + const probe = context(); + const result = parser(input, index, probe); + cache.set(index, { + result, + furthest: probe.furthest, + expected: probe.expected, + }); + + if (probe.furthest >= 0) { + record(ctx, probe.furthest, probe.expected); } - const result = parser(input); - cache.set(input, result); return result; }; }; diff --git a/src/core/parse.test.ts b/src/core/parse.test.ts new file mode 100644 index 0000000..0a8fbf8 --- /dev/null +++ b/src/core/parse.test.ts @@ -0,0 +1,34 @@ +import { describe, expect, it } from 'vitest'; + +import { many } from '../combinators/many'; +import { sequence } from '../combinators/sequence'; +import { digits } from '../primitives/digits'; +import { letters } from '../primitives/letters'; +import { char } from '../terminals/char'; +import { parse } from './parse'; + +describe('parse', () => { + it('should return the value when the whole input is consumed', () => { + expect(parse(digits, '123')).toEqual({ ok: true, value: 123 }); + }); + + it('should report where a failure happened instead of throwing', () => { + const result = parse(sequence(letters, char('='), digits), 'port=x'); + + expect(result.ok).toBe(false); + expect(result).toMatchObject({ index: 5, line: 1, column: 6 }); + expect(result.ok === false && result.expected).toEqual(['digit']); + }); + + it('should report the line and column on a later line', () => { + const result = parse(many(sequence(letters, char('\n'))), 'ab\ncd\n1'); + + expect(result).toMatchObject({ line: 3, column: 1 }); + }); + + it('should include a formatted excerpt', () => { + const result = parse(digits, '12x'); + + expect(result.ok === false && result.message).toContain(' 1 | 12x'); + }); +}); diff --git a/src/core/parse.ts b/src/core/parse.ts new file mode 100644 index 0000000..8f0ab24 --- /dev/null +++ b/src/core/parse.ts @@ -0,0 +1,48 @@ +import type { Parser } from './parser'; + +import { context } from './context'; +import { format } from './format'; +import { locate } from './locate'; +import { diagnose } from './run'; + +export type ParseResult = + | { ok: true; value: T } + | { + ok: false; + index: number; + line: number; + column: number; + expected: readonly string[]; + message: string; + }; + +/** + * Runs a parser over the whole input without throwing. + * + * The non-throwing counterpart of {@link run}: on failure it reports the + * offset, line, column, expectations and a formatted message. + * + * @example + * parse(digits, '12x').ok // false + */ +export const parse = (parser: Parser, input: string): ParseResult => { + const ctx = context(); + const result = parser(input, 0, ctx); + const problem = diagnose(input, result, ctx); + + if (!problem) { + return { ok: true, value: (result as { value: T }).value }; + } + + const { index, expected } = problem; + const { line, column } = locate(input, index); + + return { + ok: false, + index, + line, + column, + expected, + message: format(input, index, expected), + }; +}; diff --git a/src/core/parser.test.ts b/src/core/parser.test.ts index 5e9b45b..d5bcfdf 100644 --- a/src/core/parser.test.ts +++ b/src/core/parser.test.ts @@ -7,23 +7,23 @@ import { success } from './success'; describe('create', () => { it('should return the result of the parser function on success', () => { - const parser = create((input) => success(input, '')); + const parser = create((input) => success(input, input.length)); const result = parser('hello'); - assertSuccess(result, 'hello', ''); + assertSuccess(result, 'hello', 5); }); it('should return null when the parser function fails', () => { - const parser = create(() => failure()); + const parser = create((_input, index = 0) => failure(undefined, index)); const result = parser('hello'); assertFailure(result); }); it('should pass the input through to the parser function', () => { - const parser = create((input) => success(input.toUpperCase(), input)); + const parser = create((input) => success(input.toUpperCase(), 0)); const result = parser('hello'); - assertSuccess(result, 'HELLO', 'hello'); + assertSuccess(result, 'HELLO', 0); }); }); diff --git a/src/core/parser.ts b/src/core/parser.ts index e069033..6104146 100644 --- a/src/core/parser.ts +++ b/src/core/parser.ts @@ -1,12 +1,21 @@ +import { type Context } from './context'; import { type Result } from './result'; -export type Parser = (input: string) => Result; +export type Parser = ( + input: string, + index?: number, + ctx?: Context, +) => Result; /** * Creates a parser from a parser function. * + * A parser reads `input` from `index` and never slices it. The third argument + * is the parse context — pass it on to any parser you call, so expectations + * recorded inside your parser reach the final error message. + * * @example - * create((input) => success('parsed', input.slice(6)))('hello world') // { ok: true, value: 'parsed', remaining: 'world' } + * create((input, index = 0) => success('parsed', index + 6))('hello world') // { ok: true, value: 'parsed', index: 6 } */ export const create = (parserFn: Parser): Parser => { return parserFn; diff --git a/src/core/propagation.test.ts b/src/core/propagation.test.ts new file mode 100644 index 0000000..58fecf5 --- /dev/null +++ b/src/core/propagation.test.ts @@ -0,0 +1,109 @@ +import fs from 'node:fs'; +import path from 'node:path'; +import { describe, expect, it } from 'vitest'; + +import { choice } from '../combinators/choice'; +import { many } from '../combinators/many'; +import { not } from '../combinators/not'; +import { optional } from '../combinators/optional'; +import { separatedBy } from '../combinators/separatedBy'; +import { sequence } from '../combinators/sequence'; +import { char } from '../terminals/char'; +import { string } from '../terminals/string'; +import { context } from './context'; +import { label } from './label'; +import { memoize } from './memoize'; +import { type Parser } from './parser'; + +/** + * A combinator that calls a parser without passing the context on will + * silently lose expectations: the parse still succeeds or fails the same way, + * only the error message gets worse. Nothing else would catch that, so the + * call sites are checked directly. + */ +describe('context is forwarded at every parser call site', () => { + const roots = ['combinators', 'primitives', 'terminals']; + const files = roots.flatMap((dir) => { + const full = path.join(process.cwd(), 'src', dir); + + return fs + .readdirSync(full) + .filter( + (f) => + f.endsWith('.ts') && + !f.endsWith('.test.ts') && + f !== 'index.ts', + ) + .map((f) => path.join('src', dir, f)); + }); + + // `foo(input, bar)` / `foo(\n input,\n bar,\n)` — a parser being invoked. + const CALL = /\(\s*input,\s*[^()]*?\)/gs; + + it.each(files)('%s', (file) => { + const source = fs + .readFileSync(path.join(process.cwd(), file), 'utf-8') + .replace(/\/\*[\s\S]*?\*\//g, '') + .replace(/\/\/.*/g, ''); + + const offenders = [...source.matchAll(CALL)] + .map((m) => m[0]) + .filter((call) => !/\bctx\b/.test(call)) + // parser *declarations* legitimately omit it when unused + .filter((call) => !/index = 0/.test(call)); + + expect(offenders, `parser call without ctx in ${file}`).toEqual([]); + }); +}); + +describe('expectations survive backtracking', () => { + // Succeeds on 'aa', then wants '!' at offset 2 — deeper than any caller reaches. + const deep = sequence(string('aa'), char('!')); + const furthestOf = (parser: Parser, input: string) => { + const ctx = context(); + parser(input, 0, ctx); + + return ctx; + }; + + const cases: [string, Parser][] = [ + ['optional', optional(deep)], + ['many', many(deep)], + ['choice', choice(deep, string('zz'))], + ['separatedBy', separatedBy(deep, char(','))], + ['memoize', memoize(deep)], + ]; + + it.each(cases)( + '%s reports the furthest failure it backtracked over', + (_name, parser) => { + const ctx = furthestOf(parser, 'aaX'); + + expect(ctx.furthest).toBe(2); + expect(ctx.expected).toContain("'!'"); + }, + ); + + it('memoize replays its trace on a cache hit', () => { + const memo = memoize(deep); + memo('aaX', 0, context()); // prime the cache + + const ctx = context(); + memo('aaX', 0, ctx); + + expect(ctx.furthest).toBe(2); + expect(ctx.expected).toContain("'!'"); + }); + + it('label hides the internals it replaces', () => { + const ctx = furthestOf(label(string('hello'), 'a greeting'), 'x'); + + expect(ctx.expected).toEqual(['a greeting']); + }); + + it('not does not report the failure it wanted', () => { + const ctx = furthestOf(not(string('hello')), 'world'); + + expect(ctx.furthest).toBe(-1); + }); +}); diff --git a/src/core/run.test.ts b/src/core/run.test.ts index 2f7d207..0a8e605 100644 --- a/src/core/run.test.ts +++ b/src/core/run.test.ts @@ -1,5 +1,13 @@ import { describe, expect, it } from 'vitest'; +import { choice } from '../combinators/choice'; +import { many } from '../combinators/many'; +import { separatedBy } from '../combinators/separatedBy'; +import { sequence } from '../combinators/sequence'; +import { digits } from '../primitives/digits'; +import { letters } from '../primitives/letters'; +import { char } from '../terminals/char'; +import { string } from '../terminals/string'; import { createTestParser } from '../../test/utils'; import { failure } from './failure'; import { create } from './parser'; @@ -14,24 +22,68 @@ describe('run', () => { expect(result).toBe('A'); }); - it('should return null result', () => { + it('should report the expectation and position on failure', () => { expect(() => { run(parser, 'B'); - }).toThrowError('Parsing failed: Unexpected error'); + }).toThrowError("1:1 expected A, found 'B'"); }); - it('should throw error for not remaining input', () => { + it('should throw error for unconsumed input', () => { expect(() => { run(parser, 'AB'); - }).toThrowError('Not all input consumed'); + }).toThrowError("1:2 expected end of input, found 'B'"); }); - it('should throw error with message', () => { + it('should throw error with the expectation the parser recorded', () => { expect(() => { run( - create(() => failure('Something weird happened')), + create((_input, index = 0) => + failure(undefined, index, 'something weird'), + ), 'AB', ); - }).toThrowError('Something weird happened'); + }).toThrowError('expected something weird'); + }); + + /** Records 'A' the way a real terminal does, then discards the failure. */ + const swallowing = (...expected: string[]) => { + return create((_input, index = 0, ctx) => { + failure(ctx, index, 'A'); + + return failure(ctx, index, ...expected); + }); + }; + + it('should fall back to the context when the failure names nothing', () => { + expect(() => { + run(swallowing(), 'B'); + }).toThrowError("1:1 expected A, found 'B'"); + }); + + it('should combine the context and the failure at the same offset', () => { + expect(() => { + run(swallowing('B'), 'C'); + }).toThrowError("1:1 expected A or B, found 'C'"); + }); + + it('should report a branch that reached further than the failure', () => { + // choice fails at 0, but the first alternative got to offset 2. + const alternatives = choice(sequence(string('aa'), char('!')), char('z')); + + expect(() => run(alternatives, 'aab')).toThrowError("1:3 expected '!'"); + }); + + it('should report a branch that reached further than a partial success', () => { + // separatedBy succeeds with one pair and stops, but the failure that + // actually matters is inside the pair it backtracked over. + const pairs = separatedBy(sequence(letters, char('='), digits), char(',')); + + expect(() => run(pairs, 'a=1,b=x')).toThrowError('1:7 expected digit'); + }); + + it('should ask for end of input when nothing reached further', () => { + expect(() => run(many(char('a')), 'aab')).toThrowError( + "1:3 expected 'a' or end of input", + ); }); }); diff --git a/src/core/run.ts b/src/core/run.ts index c765686..8d4c692 100644 --- a/src/core/run.ts +++ b/src/core/run.ts @@ -1,25 +1,79 @@ -import type { Parser } from '../core/parser'; +import type { Parser } from './parser'; + +import { type Context, context, union } from './context'; +import { ParseError } from './error'; +import { type Result } from './result'; + +/** + * Folds in whatever the context recorded at the same offset, so expectations + * from branches that were backtracked over are not lost. + */ +const merge = ( + ctx: Context, + index: number, + expected: readonly string[], +): readonly string[] => { + return ctx.furthest === index ? union(ctx.expected, expected) : expected; +}; /** - * Runs a parser and returns the value, throws on failure or unconsumed input. + * Resolves where a parse went wrong. * - * @example - * run(string('hello'), 'hello') // 'hello' + * A parser can succeed without consuming everything — the interesting position + * is then the furthest offset any branch reached, not where the parse stopped. + * + * A result reaching the same offset as the context carries no more authority + * than the context does, so the two expectation sets are merged. Combinators + * that fail without an expectation of their own — `chainLeft`, `guard`, a + * hand-written `failure(ctx, index)` — would otherwise report nothing at all. */ -export const run = (parser: Parser, input: string): T => { - const result = parser(input); +export const diagnose = ( + input: string, + result: Result, + ctx: Context, +): { index: number; expected: readonly string[] } | null => { if (!result.ok) { - if (result.error === undefined) { - throw new Error('Parsing failed: Unexpected error'); + if (ctx.furthest > result.index) { + return { index: ctx.furthest, expected: ctx.expected }; } - throw new Error(`Parsing failed: ${result.error}`); + return { + index: result.index, + expected: merge(ctx, result.index, result.expected), + }; } - const { value, remaining } = result; - if (remaining !== '') { - throw new Error(`Not all input consumed: "${remaining}"`); + if (result.index === input.length) { + return null; + } + + if (ctx.furthest > result.index) { + return { index: ctx.furthest, expected: ctx.expected }; + } + + return { + index: result.index, + expected: merge(ctx, result.index, ['end of input']), + }; +}; + +/** + * Runs a parser over the whole input and returns the value. + * + * Throws a {@link ParseError} — with line, column and the expectations at that + * point — if the parse fails or leaves input unconsumed. + * + * @example + * run(string('hello'), 'hello') // 'hello' + */ +export const run = (parser: Parser, input: string): T => { + const ctx = context(); + const result = parser(input, 0, ctx); + const problem = diagnose(input, result, ctx); + + if (problem) { + throw new ParseError(input, problem.index, problem.expected); } - return value; + return (result as { value: T }).value; }; diff --git a/src/core/success.test.ts b/src/core/success.test.ts index e47cc21..f7cdd2c 100644 --- a/src/core/success.test.ts +++ b/src/core/success.test.ts @@ -5,8 +5,8 @@ import { success } from './success'; describe('success', () => { it('should create successful parse result', () => { - const result = success('test', 'remaining'); + const result = success('test', 4); - assertSuccess(result, 'test', 'remaining'); + assertSuccess(result, 'test', 4); }); }); diff --git a/src/core/success.ts b/src/core/success.ts index 0e33cad..32d4dbe 100644 --- a/src/core/success.ts +++ b/src/core/success.ts @@ -1,13 +1,13 @@ -export type Success = { ok: true; value: T; remaining: string }; +export type Success = { ok: true; value: T; index: number }; /** - * Creates a successful result with a value and remaining input. + * Creates a successful result with a value and the offset reached in the input. * * @example - * success('hello', ' world') // { ok: true, value: 'hello', remaining: ' world' } + * success('hello', 5) // { ok: true, value: 'hello', index: 5 } */ -export const success = (value: T, remaining: string): Success => ({ +export const success = (value: T, index: number): Success => ({ ok: true, value, - remaining, + index, }); diff --git a/src/primitives/alphaNum.test.ts b/src/primitives/alphaNum.test.ts index 6d6519c..c3fab47 100644 --- a/src/primitives/alphaNum.test.ts +++ b/src/primitives/alphaNum.test.ts @@ -7,13 +7,13 @@ describe('alphaNum', () => { it('should parse letter', () => { const result = alphaNum('a1'); - assertSuccess(result, 'a', '1'); + assertSuccess(result, 'a', 1); }); it('should parse digit', () => { const result = alphaNum('1a'); - assertSuccess(result, '1', 'a'); + assertSuccess(result, '1', 1); }); it('should fail on non-alphanumeric', () => { @@ -31,6 +31,6 @@ describe('alphaNum', () => { it('should narrow result to first character type for const string', () => { const result = alphaNum('a1' as const); - assertSuccess<'a'>(result, 'a', '1'); + assertSuccess<'a'>(result, 'a', 1); }); }); diff --git a/src/primitives/alphaNum.ts b/src/primitives/alphaNum.ts index 5ec66a5..7ac3b42 100644 --- a/src/primitives/alphaNum.ts +++ b/src/primitives/alphaNum.ts @@ -1,3 +1,4 @@ +import { type Context } from '../core/context'; import { create } from '../core/parser'; import { type Result } from '../core/result'; import { satisfy } from '../terminals/satisfy'; @@ -7,19 +8,28 @@ import { type Letter } from './letter'; type Head = S extends `${infer C}${string}` ? C : never; export type AlphaNum = Letter | Digit; -const parser = satisfy((c) => /[a-zA-Z0-9]/.test(c)); +const parser = satisfy( + (c) => /[a-zA-Z0-9]/.test(c), + 'alphanumeric character', +); /** * Parse a single alphanumeric character. * * @example - * alphaNum('a1') // { ok: true, value: 'a', remaining: '1' } - * alphaNum('1a') // { ok: true, value: '1', remaining: 'a' } + * alphaNum('a1') // { ok: true, value: 'a', index: 1 } + * alphaNum('1a') // { ok: true, value: '1', index: 1 } */ export function alphaNum( input: S, + index?: number, + ctx?: Context, ): Result & AlphaNum>; -export function alphaNum(input: string): Result; -export function alphaNum(input: string) { - return create(parser)(input); +export function alphaNum( + input: string, + index?: number, + ctx?: Context, +): Result; +export function alphaNum(input: string, index = 0, ctx?: Context) { + return create(parser)(input, index, ctx); } diff --git a/src/primitives/alphaNums.test.ts b/src/primitives/alphaNums.test.ts index d5ceda1..5b18db7 100644 --- a/src/primitives/alphaNums.test.ts +++ b/src/primitives/alphaNums.test.ts @@ -8,17 +8,17 @@ describe('alphaNums', () => { { const result = alphaNums('abc123 rest'); - assertSuccess(result, 'abc123', ' rest'); + assertSuccess(result, 'abc123', 6); } { const result = alphaNums('Hello2World!'); - assertSuccess(result, 'Hello2World', '!'); + assertSuccess(result, 'Hello2World', 11); } { const result = alphaNums('abc123'); - assertSuccess(result, 'abc123', ''); + assertSuccess(result, 'abc123', 6); } }); @@ -43,12 +43,12 @@ describe('alphaNums', () => { it('should handle single alphanumeric character', () => { const result = alphaNums('a!'); - assertSuccess(result, 'a', '!'); + assertSuccess(result, 'a', 1); }); it('should stop at first non-alphanumeric character', () => { const result = alphaNums('abc_def'); - assertSuccess(result, 'abc', '_def'); + assertSuccess(result, 'abc', 3); }); }); diff --git a/src/primitives/alphaNums.ts b/src/primitives/alphaNums.ts index 1484ac2..6f553f9 100644 --- a/src/primitives/alphaNums.ts +++ b/src/primitives/alphaNums.ts @@ -1,5 +1,4 @@ import { many1 } from '../combinators/many1'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { alphaNum } from './alphaNum'; @@ -7,18 +6,16 @@ import { alphaNum } from './alphaNum'; const parser = many1(alphaNum); /** - * Parse one or more alphanumeric characters. + * Parse one or more alphanumeric characters as a string. * * @example - * alphaNums('abc123') // { ok: true, value: 'abc123', remaining: '' } + * alphaNums('placeholder') // { ok: true, value: 'placeholder', index: 11 } */ -export const alphaNums = create((input) => { - const result = parser(input); +export const alphaNums = create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value, remaining } = result; - - return success(value.join(''), remaining); + return success(result.value.join(''), result.index); }); diff --git a/src/primitives/anyChar.test.ts b/src/primitives/anyChar.test.ts index 48a6f29..7f94170 100644 --- a/src/primitives/anyChar.test.ts +++ b/src/primitives/anyChar.test.ts @@ -7,13 +7,13 @@ describe('anyChar', () => { it('should match any single character', () => { const result = anyChar('abc'); - assertSuccess(result, 'a', 'bc'); + assertSuccess(result, 'a', 1); }); it('should match special characters', () => { const result = anyChar('@#$'); - assertSuccess(result, '@', '#$'); + assertSuccess(result, '@', 1); }); it('should fail on empty input', () => { diff --git a/src/primitives/anyChar.ts b/src/primitives/anyChar.ts index d961df4..61dbd96 100644 --- a/src/primitives/anyChar.ts +++ b/src/primitives/anyChar.ts @@ -6,8 +6,10 @@ import { success } from '../core/success'; * Parse any single character. * * @example - * anyChar('abc') // { ok: true, value: 'a', remaining: 'bc' } + * anyChar('abc') // { ok: true, value: 'a', index: 1 } */ -export const anyChar = create((input) => { - return input.length > 0 ? success(input[0], input.slice(1)) : failure(); +export const anyChar = create((input, index = 0, ctx) => { + return index < input.length + ? success(input[index], index + 1) + : failure(ctx, index, 'any character'); }); diff --git a/src/primitives/bool.test.ts b/src/primitives/bool.test.ts index 0da3b16..f8ec6dd 100644 --- a/src/primitives/bool.test.ts +++ b/src/primitives/bool.test.ts @@ -8,12 +8,12 @@ describe('bool', () => { { const result = bool('true'); - assertSuccess(result, true, ''); + assertSuccess(result, true, 4); } { const result = bool('trueabc'); - assertSuccess(result, true, 'abc'); + assertSuccess(result, true, 4); } }); @@ -21,12 +21,12 @@ describe('bool', () => { { const result = bool('false'); - assertSuccess(result, false, ''); + assertSuccess(result, false, 5); } { const result = bool('falseabc'); - assertSuccess(result, false, 'abc'); + assertSuccess(result, false, 5); } }); diff --git a/src/primitives/bool.ts b/src/primitives/bool.ts index 95a385f..1899ca2 100644 --- a/src/primitives/bool.ts +++ b/src/primitives/bool.ts @@ -12,8 +12,8 @@ const parser = choice( * Parse a boolean literal. * * @example - * bool('true') // { ok: true, value: true, remaining: '' } - * bool('false') // { ok: true, value: false, remaining: '' } - * bool('trueABC') // { ok: true, value: true, remaining: 'ABC' } + * bool('true') // { ok: true, value: true, index: 4 } + * bool('false') // { ok: true, value: false, index: 5 } + * bool('trueABC') // { ok: true, value: true, index: 4 } */ export const bool = create(parser); diff --git a/src/primitives/crlf.test.ts b/src/primitives/crlf.test.ts index 7e837d9..8e55e35 100644 --- a/src/primitives/crlf.test.ts +++ b/src/primitives/crlf.test.ts @@ -7,7 +7,7 @@ describe('crlf', () => { it('should parse CRLF sequence', () => { const result = crlf('\r\nabc'); - assertSuccess(result, '\r\n', 'abc'); + assertSuccess(result, '\r\n', 2); }); it('should fail on single CR or LF', () => { diff --git a/src/primitives/crlf.ts b/src/primitives/crlf.ts index ba477b0..17a7a1d 100644 --- a/src/primitives/crlf.ts +++ b/src/primitives/crlf.ts @@ -7,6 +7,6 @@ const parser = string('\r\n'); * Parse CRLF line ending. * * @example - * crlf('\r\nabc') // { ok: true, value: '\r\n', remaining: 'abc' } + * crlf('\r\nabc') // { ok: true, value: '\r\n', index: 2 } */ export const crlf = create(parser); diff --git a/src/primitives/digit.test.ts b/src/primitives/digit.test.ts index 03f3fc7..9c3a4d5 100644 --- a/src/primitives/digit.test.ts +++ b/src/primitives/digit.test.ts @@ -8,17 +8,17 @@ describe('digit', () => { { const result = digit('5abc'); - assertSuccess(result, 5, 'abc'); + assertSuccess(result, 5, 1); } { const result = digit('0xyz'); - assertSuccess(result, 0, 'xyz'); + assertSuccess(result, 0, 1); } { const result = digit('9'); - assertSuccess(result, 9, ''); + assertSuccess(result, 9, 1); } }); @@ -43,6 +43,6 @@ describe('digit', () => { it('should only parse first digit', () => { const result = digit('123'); - assertSuccess(result, 1, '23'); + assertSuccess(result, 1, 1); }); }); diff --git a/src/primitives/digit.ts b/src/primitives/digit.ts index 0dfad0d..539601e 100644 --- a/src/primitives/digit.ts +++ b/src/primitives/digit.ts @@ -5,7 +5,7 @@ import { satisfy } from '../terminals/satisfy'; export type Digit = '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9'; const parser = map( - satisfy((c) => /^[0-9]$/.test(c)), + satisfy((c) => /^[0-9]$/.test(c), 'digit'), (c) => parseInt(c, 10), ); @@ -13,6 +13,6 @@ const parser = map( * Parse a single digit and return as number. * * @example - * digit('5abc') // { ok: true, value: 5, remaining: 'abc' } + * digit('5abc') // { ok: true, value: 5, index: 1 } */ export const digit = create(parser); diff --git a/src/primitives/digits.test.ts b/src/primitives/digits.test.ts index 1c229b7..59aa865 100644 --- a/src/primitives/digits.test.ts +++ b/src/primitives/digits.test.ts @@ -8,17 +8,17 @@ describe('digits', () => { { const result = digits('123abc'); - assertSuccess(result, 123, 'abc'); + assertSuccess(result, 123, 3); } { const result = digits('42'); - assertSuccess(result, 42, ''); + assertSuccess(result, 42, 2); } { const result = digits('007xyz'); - assertSuccess(result, 7, 'xyz'); + assertSuccess(result, 7, 3); } }); @@ -38,6 +38,6 @@ describe('digits', () => { it('should handle single digit', () => { const result = digits('5abc'); - assertSuccess(result, 5, 'abc'); + assertSuccess(result, 5, 1); }); }); diff --git a/src/primitives/digits.ts b/src/primitives/digits.ts index 88a1d7f..3f5eea4 100644 --- a/src/primitives/digits.ts +++ b/src/primitives/digits.ts @@ -1,5 +1,4 @@ import { many1 } from '../combinators/many1'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { digit } from './digit'; @@ -7,19 +6,18 @@ import { digit } from './digit'; const parser = many1(digit); /** - * Parse one or more digits and return as number. + * Parse one or more digits and return as a number. * * @example - * digits('123abc') // { ok: true, value: 123, remaining: 'abc' } + * digits('123') // { ok: true, value: 123, index: 3 } */ -export const digits = create((input) => { - const result = parser(input); +export const digits = create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value: list, remaining: rest } = result; - const value = list.reduce((acc, d) => acc * 10 + d, 0); + const value = result.value.reduce((acc, d) => acc * 10 + d, 0); - return success(value, rest); + return success(value, result.index); }); diff --git a/src/primitives/eof.test.ts b/src/primitives/eof.test.ts index d211862..de513be 100644 --- a/src/primitives/eof.test.ts +++ b/src/primitives/eof.test.ts @@ -7,7 +7,7 @@ describe('eof', () => { it('should succeed on empty input', () => { const result = eof(''); - assertSuccess(result, null, ''); + assertSuccess(result, null, 0); }); it('should fail on non-empty input', () => { diff --git a/src/primitives/eof.ts b/src/primitives/eof.ts index b8c5641..d0a9639 100644 --- a/src/primitives/eof.ts +++ b/src/primitives/eof.ts @@ -3,11 +3,13 @@ import { create } from '../core/parser'; import { success } from '../core/success'; /** - * Parse end of file (succeeds only on empty input). + * Match the end of the input. * * @example - * eof('') // { ok: true, value: null, remaining: '' } + * eof('') // { ok: true, value: null, index: 0 } */ -export const eof = create((input) => { - return input.length === 0 ? success(null, input) : failure(); +export const eof = create((input, index = 0, ctx) => { + return index >= input.length + ? success(null, index) + : failure(ctx, index, 'end of input'); }); diff --git a/src/primitives/eol.test.ts b/src/primitives/eol.test.ts index 16b7a59..58ba8aa 100644 --- a/src/primitives/eol.test.ts +++ b/src/primitives/eol.test.ts @@ -8,24 +8,24 @@ describe('eol', () => { { const result = eol('\nabc'); - assertSuccess(result, '\n', 'abc'); + assertSuccess(result, '\n', 1); } { const result = eol('\r\nabc'); - assertSuccess(result, '\r\n', 'abc'); + assertSuccess(result, '\r\n', 2); } }); it('should handle end of file', () => { const result = eol(''); - assertSuccess(result, '', ''); + assertSuccess(result, '', 0); }); it('should prefer CRLF over individual characters', () => { const result = eol('\r\n'); - assertSuccess(result, '\r\n', ''); + assertSuccess(result, '\r\n', 2); }); }); diff --git a/src/primitives/eol.ts b/src/primitives/eol.ts index 615c2fd..647d7de 100644 --- a/src/primitives/eol.ts +++ b/src/primitives/eol.ts @@ -15,6 +15,6 @@ const parser = choice( * Parse end of line (\\n, \\r\\n, or EOF). * * @example - * eol('\nabc') // { ok: true, value: '\n', remaining: 'abc' } + * eol('\nabc') // { ok: true, value: '\n', index: 1 } */ export const eol = create(parser); diff --git a/src/primitives/float.test.ts b/src/primitives/float.test.ts index da0719c..b13269a 100644 --- a/src/primitives/float.test.ts +++ b/src/primitives/float.test.ts @@ -7,19 +7,19 @@ describe('float', () => { it('should parse a positive float', () => { const result = float('1.23'); - assertSuccess(result, 1.23, ''); + assertSuccess(result, 1.23, 4); }); it('should parse a negative float', () => { const result = float('-2.5'); - assertSuccess(result, -2.5, ''); + assertSuccess(result, -2.5, 4); }); it('should parse float with many decimal places', () => { const result = float('123.456789'); - assertSuccess(result, 123.456789, ''); + assertSuccess(result, 123.456789, 10); }); it('should fail on integer input', () => { @@ -43,6 +43,6 @@ describe('float', () => { it('should stop at non-numeric character', () => { const result = float('1.23abc'); - assertSuccess(result, 1.23, 'abc'); + assertSuccess(result, 1.23, 4); }); }); diff --git a/src/primitives/float.ts b/src/primitives/float.ts index 8bacdfe..aa22ed1 100644 --- a/src/primitives/float.ts +++ b/src/primitives/float.ts @@ -8,8 +8,8 @@ const parser = map(regex(/^-?\d+\.\d+/), (match: string) => parseFloat(match)); * Parse a floating point number. * * @example - * float('1.23') // { ok: true, value: 1.23, remaining: '' } - * float('-2.5') // { ok: true, value: -2.5, remaining: '' } - * float('1.23abc') // { ok: true, value: 1.23, remaining: 'abc' } + * float('1.23') // { ok: true, value: 1.23, index: 4 } + * float('-2.5') // { ok: true, value: -2.5, index: 4 } + * float('1.23abc') // { ok: true, value: 1.23, index: 4 } */ export const float = create(parser); diff --git a/src/primitives/hexDigit.test.ts b/src/primitives/hexDigit.test.ts index 6f3d865..b09a58f 100644 --- a/src/primitives/hexDigit.test.ts +++ b/src/primitives/hexDigit.test.ts @@ -6,7 +6,7 @@ import { hexDigit, type HexDigit } from './hexDigit'; describe('hexDigit', () => { it('should parse hexadecimal digits', () => { const result = hexDigit('fF9'); - assertSuccess(result, 'f', 'F9'); + assertSuccess(result, 'f', 1); }); it('should fail on non-hexadecimal', () => { @@ -21,11 +21,11 @@ describe('hexDigit', () => { it('should narrow result to first character type for const string', () => { const result = hexDigit('f1a' as const); - assertSuccess<'f'>(result, 'f', '1a'); + assertSuccess<'f'>(result, 'f', 1); }); it('should narrow result for uppercase hex const string', () => { const result = hexDigit('A2B' as const); - assertSuccess<'A'>(result, 'A', '2B'); + assertSuccess<'A'>(result, 'A', 1); }); }); diff --git a/src/primitives/hexDigit.ts b/src/primitives/hexDigit.ts index 2e16f59..11f0a30 100644 --- a/src/primitives/hexDigit.ts +++ b/src/primitives/hexDigit.ts @@ -1,3 +1,4 @@ +import { type Context } from '../core/context'; import { create } from '../core/parser'; import { type Result } from '../core/result'; import { satisfy } from '../terminals/satisfy'; @@ -19,18 +20,24 @@ export type HexDigit = | 'E' | 'F'; -const parser = satisfy((c) => /[0-9a-fA-F]/.test(c)); +const parser = satisfy((c) => /[0-9a-fA-F]/.test(c), 'hex digit'); /** * Parse a single hexadecimal digit. * * @example - * hexDigit('fF9') // { ok: true, value: 'f', remaining: 'F9' } + * hexDigit('fF9') // { ok: true, value: 'f', index: 1 } */ export function hexDigit( input: S, + index?: number, + ctx?: Context, ): Result & HexDigit>; -export function hexDigit(input: string): Result; -export function hexDigit(input: string) { - return create(parser)(input); +export function hexDigit( + input: string, + index?: number, + ctx?: Context, +): Result; +export function hexDigit(input: string, index = 0, ctx?: Context) { + return create(parser)(input, index, ctx); } diff --git a/src/primitives/hexDigits.test.ts b/src/primitives/hexDigits.test.ts index bc6c801..57772cb 100644 --- a/src/primitives/hexDigits.test.ts +++ b/src/primitives/hexDigits.test.ts @@ -8,17 +8,17 @@ describe('hexDigits', () => { { const result = hexDigits('1a2bXY'); - assertSuccess(result, '1a2b', 'XY'); + assertSuccess(result, '1a2b', 4); } { const result = hexDigits('DEADBEEF'); - assertSuccess(result, 'DEADBEEF', ''); + assertSuccess(result, 'DEADBEEF', 8); } { const result = hexDigits('ff00ff rest'); - assertSuccess(result, 'ff00ff', ' rest'); + assertSuccess(result, 'ff00ff', 6); } }); @@ -43,12 +43,12 @@ describe('hexDigits', () => { it('should handle single hex digit', () => { const result = hexDigits('fXY'); - assertSuccess(result, 'f', 'XY'); + assertSuccess(result, 'f', 1); }); it('should stop at first non-hex character', () => { const result = hexDigits('a1g9'); - assertSuccess(result, 'a1', 'g9'); + assertSuccess(result, 'a1', 2); }); }); diff --git a/src/primitives/hexDigits.ts b/src/primitives/hexDigits.ts index cf02e74..1890e69 100644 --- a/src/primitives/hexDigits.ts +++ b/src/primitives/hexDigits.ts @@ -1,5 +1,4 @@ import { many1 } from '../combinators/many1'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { hexDigit } from './hexDigit'; @@ -7,18 +6,16 @@ import { hexDigit } from './hexDigit'; const parser = many1(hexDigit); /** - * Parse one or more hexadecimal digits. + * Parse one or more hex digits as a string. * * @example - * hexDigits('deadbeef') // { ok: true, value: 'deadbeef', remaining: '' } + * hexDigits('deadbeef') // { ok: true, value: 'deadbeef', index: 8 } */ -export const hexDigits = create((input) => { - const result = parser(input); +export const hexDigits = create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value, remaining } = result; - - return success(value.join(''), remaining); + return success(result.value.join(''), result.index); }); diff --git a/src/primitives/identifier.test.ts b/src/primitives/identifier.test.ts index db5f3b9..2ce0797 100644 --- a/src/primitives/identifier.test.ts +++ b/src/primitives/identifier.test.ts @@ -8,42 +8,42 @@ describe('identifier', () => { { const result = identifier('variable_name'); - assertSuccess(result, 'variable_name', ''); + assertSuccess(result, 'variable_name', 13); } { const result = identifier('_private'); - assertSuccess(result, '_private', ''); + assertSuccess(result, '_private', 8); } { const result = identifier('camelCase'); - assertSuccess(result, 'camelCase', ''); + assertSuccess(result, 'camelCase', 9); } { const result = identifier('PascalCase'); - assertSuccess(result, 'PascalCase', ''); + assertSuccess(result, 'PascalCase', 10); } { const result = identifier('name123'); - assertSuccess(result, 'name123', ''); + assertSuccess(result, 'name123', 7); } { const result = identifier('some-name'); - assertSuccess(result, 'some', '-name'); + assertSuccess(result, 'some', 4); } { const result = identifier('some.name'); - assertSuccess(result, 'some', '.name'); + assertSuccess(result, 'some', 4); } { const result = identifier('some_name!'); - assertSuccess(result, 'some_name', '!'); + assertSuccess(result, 'some_name', 9); } }); @@ -65,12 +65,12 @@ describe('identifier', () => { { const result = identifier('a'); - assertSuccess(result, 'a', ''); + assertSuccess(result, 'a', 1); } { const result = identifier('_'); - assertSuccess(result, '_', ''); + assertSuccess(result, '_', 1); } }); }); diff --git a/src/primitives/identifier.ts b/src/primitives/identifier.ts index e789ff4..248fbf7 100644 --- a/src/primitives/identifier.ts +++ b/src/primitives/identifier.ts @@ -7,6 +7,6 @@ const parser = regex(/^[a-zA-Z_][a-zA-Z0-9_]*/); * Parse an identifier — starts with letter or underscore, no leading digit, no hyphen. * * @example - * identifier('variable_name') // { ok: true, value: 'variable_name', remaining: '' } + * identifier('variable_name') // { ok: true, value: 'variable_name', index: 13 } */ export const identifier = create(parser); diff --git a/src/primitives/integer.test.ts b/src/primitives/integer.test.ts index 8308071..7877cb2 100644 --- a/src/primitives/integer.test.ts +++ b/src/primitives/integer.test.ts @@ -7,25 +7,25 @@ describe('integer', () => { it('should parse a positive integer', () => { const result = integer('42'); - assertSuccess(result, 42, ''); + assertSuccess(result, 42, 2); }); it('should parse a negative integer', () => { const result = integer('-7'); - assertSuccess(result, -7, ''); + assertSuccess(result, -7, 2); }); it('should parse multi-digit negative integer', () => { const result = integer('-123'); - assertSuccess(result, -123, ''); + assertSuccess(result, -123, 4); }); it('should stop at non-numeric character', () => { const result = integer('123abc'); - assertSuccess(result, 123, 'abc'); + assertSuccess(result, 123, 3); }); it('should fail on non-numeric input', () => { diff --git a/src/primitives/integer.ts b/src/primitives/integer.ts index 684cbb9..6db6c43 100644 --- a/src/primitives/integer.ts +++ b/src/primitives/integer.ts @@ -8,8 +8,8 @@ const parser = map(regex(/^-?\d+/), (match: string) => parseInt(match, 10)); * Parse a signed integer. * * @example - * integer('42') // { ok: true, value: 42, remaining: '' } - * integer('-7') // { ok: true, value: -7, remaining: '' } - * integer('123abc') // { ok: true, value: 123, remaining: 'abc' } + * integer('42') // { ok: true, value: 42, index: 2 } + * integer('-7') // { ok: true, value: -7, index: 2 } + * integer('123abc') // { ok: true, value: 123, index: 3 } */ export const integer = create(parser); diff --git a/src/primitives/letter.test.ts b/src/primitives/letter.test.ts index d1d8eb0..3930f89 100644 --- a/src/primitives/letter.test.ts +++ b/src/primitives/letter.test.ts @@ -7,7 +7,7 @@ describe('letter', () => { it('should parse alphabetic characters', () => { const result = letter('A' as const); - assertSuccess(result, 'A', ''); + assertSuccess(result, 'A', 1); }); it('should fail on non-letter characters', () => { @@ -19,24 +19,24 @@ describe('letter', () => { it('should only parse first character', () => { const result = letter('hello' as never); - assertSuccess(result, 'h', 'ello'); + assertSuccess(result, 'h', 1); }); it('should match the expected type', () => { const result = letter('A'); - assertSuccess(result, 'A', ''); + assertSuccess(result, 'A', 1); }); it('should narrow result to first character type for const string', () => { const result = letter('abc' as const); - assertSuccess<'a'>(result, 'a', 'bc'); + assertSuccess<'a'>(result, 'a', 1); }); it('should narrow result type for uppercase const string', () => { const result = letter('XYZ' as const); - assertSuccess<'X'>(result, 'X', 'YZ'); + assertSuccess<'X'>(result, 'X', 1); }); }); diff --git a/src/primitives/letter.ts b/src/primitives/letter.ts index cac1e02..940754b 100644 --- a/src/primitives/letter.ts +++ b/src/primitives/letter.ts @@ -1,3 +1,4 @@ +import { type Context } from '../core/context'; import { create } from '../core/parser'; import { type Result } from '../core/result'; import { satisfy } from '../terminals/satisfy'; @@ -7,18 +8,24 @@ import { type UppercaseLetter } from './uppercase'; type Head = S extends `${infer C}${string}` ? C : never; export type Letter = LowercaseLetter | UppercaseLetter; -const parser = satisfy((c) => /[a-zA-Z]/.test(c)); +const parser = satisfy((c) => /[a-zA-Z]/.test(c), 'letter'); /** * Parse a single letter. * * @example - * letter('abc') // { ok: true, value: 'a', remaining: 'bc' } + * letter('abc') // { ok: true, value: 'a', index: 1 } */ export function letter( input: S, + index?: number, + ctx?: Context, ): Result & Letter>; -export function letter(input: string): Result; -export function letter(input: string) { - return create(parser)(input); +export function letter( + input: string, + index?: number, + ctx?: Context, +): Result; +export function letter(input: string, index = 0, ctx?: Context) { + return create(parser)(input, index, ctx); } diff --git a/src/primitives/letters.test.ts b/src/primitives/letters.test.ts index 0f8164a..9aa0d66 100644 --- a/src/primitives/letters.test.ts +++ b/src/primitives/letters.test.ts @@ -8,17 +8,17 @@ describe('letters', () => { { const result = letters('abc123'); - assertSuccess(result, 'abc', '123'); + assertSuccess(result, 'abc', 3); } { const result = letters('Hello World'); - assertSuccess(result, 'Hello', ' World'); + assertSuccess(result, 'Hello', 5); } { const result = letters('ABC'); - assertSuccess(result, 'ABC', ''); + assertSuccess(result, 'ABC', 3); } }); @@ -43,6 +43,6 @@ describe('letters', () => { it('should handle single letter', () => { const result = letters('a1b2'); - assertSuccess(result, 'a', '1b2'); + assertSuccess(result, 'a', 1); }); }); diff --git a/src/primitives/letters.ts b/src/primitives/letters.ts index c8baeda..a442654 100644 --- a/src/primitives/letters.ts +++ b/src/primitives/letters.ts @@ -1,5 +1,4 @@ import { many1 } from '../combinators/many1'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { letter } from './letter'; @@ -7,18 +6,16 @@ import { letter } from './letter'; const parser = many1(letter); /** - * Parse one or more letters. + * Parse one or more letters as a string. * * @example - * letters('abc123') // { ok: true, value: 'abc', remaining: '123' } + * letters('placeholder') // { ok: true, value: 'placeholder', index: 11 } */ -export const letters = create((input) => { - const result = parser(input); +export const letters = create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value, remaining } = result; - - return success(value.join(''), remaining); + return success(result.value.join(''), result.index); }); diff --git a/src/primitives/line.test.ts b/src/primitives/line.test.ts index 1f87a1a..8911710 100644 --- a/src/primitives/line.test.ts +++ b/src/primitives/line.test.ts @@ -7,37 +7,37 @@ describe('line', () => { it('should parse until newline', () => { const result = line('hello\nworld'); - assertSuccess(result, 'hello', '\nworld'); + assertSuccess(result, 'hello', 5); }); it('should parse until carriage return', () => { const result = line('hello\rworld'); - assertSuccess(result, 'hello', '\rworld'); + assertSuccess(result, 'hello', 5); }); it('should stop before CRLF', () => { const result = line('hello\r\nworld'); - assertSuccess(result, 'hello', '\r\nworld'); + assertSuccess(result, 'hello', 5); }); it('should allow empty line', () => { { const result = line('\nrest'); - assertSuccess(result, '', '\nrest'); + assertSuccess(result, '', 0); } { const result = line('\rrest'); - assertSuccess(result, '', '\rrest'); + assertSuccess(result, '', 0); } }); it('should consume entire input when no line breaks', () => { const result = line('justtext'); - assertSuccess(result, 'justtext', ''); + assertSuccess(result, 'justtext', 8); }); }); diff --git a/src/primitives/line.ts b/src/primitives/line.ts index ebc6b7e..06b71e4 100644 --- a/src/primitives/line.ts +++ b/src/primitives/line.ts @@ -7,6 +7,6 @@ const parser = takeWhile((c) => c !== '\n' && c !== '\r'); * Parse until end of line. * * @example - * line('hello\nworld') // { ok: true, value: 'hello', remaining: '\nworld' } + * line('hello\nworld') // { ok: true, value: 'hello', index: 5 } */ export const line = create(parser); diff --git a/src/primitives/literal.test.ts b/src/primitives/literal.test.ts index e74acf3..27f5d05 100644 --- a/src/primitives/literal.test.ts +++ b/src/primitives/literal.test.ts @@ -8,42 +8,42 @@ describe('literal', () => { { const result = literal('foo'); - assertSuccess(result, 'foo', ''); + assertSuccess(result, 'foo', 3); } { const result = literal('123f'); - assertSuccess(result, '123f', ''); + assertSuccess(result, '123f', 4); } { const result = literal('foo-bar'); - assertSuccess(result, 'foo-bar', ''); + assertSuccess(result, 'foo-bar', 7); } { const result = literal('_private'); - assertSuccess(result, '_private', ''); + assertSuccess(result, '_private', 8); } { const result = literal('a'); - assertSuccess(result, 'a', ''); + assertSuccess(result, 'a', 1); } { const result = literal('_'); - assertSuccess(result, '_', ''); + assertSuccess(result, '_', 1); } { const result = literal('foo_bar'); - assertSuccess(result, 'foo_bar', ''); + assertSuccess(result, 'foo_bar', 7); } { const result = literal('foo-bar_baz'); - assertSuccess(result, 'foo-bar_baz', ''); + assertSuccess(result, 'foo-bar_baz', 11); } }); diff --git a/src/primitives/literal.ts b/src/primitives/literal.ts index 28ea8ca..d43c5bf 100644 --- a/src/primitives/literal.ts +++ b/src/primitives/literal.ts @@ -7,7 +7,7 @@ const parser = regex(/^[a-zA-Z0-9_-]+/); * Parse a word-like value including hyphens. * * @example - * literal('foo-bar'); // { ok: true, value: 'foo-bar', remaining: '' } - * literal('123abc'); // { ok: true, value: '123abc', remaining: '' } + * literal('foo-bar'); // { ok: true, value: 'foo-bar', index: 7 } + * literal('123abc'); // { ok: true, value: '123abc', index: 6 } */ export const literal = create(parser); diff --git a/src/primitives/lowercase.test.ts b/src/primitives/lowercase.test.ts index 77e124a..20053c5 100644 --- a/src/primitives/lowercase.test.ts +++ b/src/primitives/lowercase.test.ts @@ -7,7 +7,7 @@ describe('lowercase', () => { it('should parse lowercase letter', () => { const result = lowercase('abc'); - assertSuccess(result, 'a', 'bc'); + assertSuccess(result, 'a', 1); }); it('should fail on uppercase', () => { @@ -25,6 +25,6 @@ describe('lowercase', () => { it('should narrow result to first character type for const string', () => { const result = lowercase('abc' as const); - assertSuccess<'a'>(result, 'a', 'bc'); + assertSuccess<'a'>(result, 'a', 1); }); }); diff --git a/src/primitives/lowercase.ts b/src/primitives/lowercase.ts index 238abac..de83bc5 100644 --- a/src/primitives/lowercase.ts +++ b/src/primitives/lowercase.ts @@ -1,3 +1,4 @@ +import { type Context } from '../core/context'; import { create } from '../core/parser'; import { type Result } from '../core/result'; import { satisfy } from '../terminals/satisfy'; @@ -31,18 +32,27 @@ export type LowercaseLetter = | 'y' | 'z'; -const parser = satisfy((c) => /[a-z]/.test(c)); +const parser = satisfy( + (c) => /[a-z]/.test(c), + 'lowercase letter', +); /** * Parse a single lowercase letter. * * @example - * lowercase('abc') // { ok: true, value: 'a', remaining: 'bc' } + * lowercase('abc') // { ok: true, value: 'a', index: 1 } */ export function lowercase( input: S, + index?: number, + ctx?: Context, ): Result & LowercaseLetter>; -export function lowercase(input: string): Result; -export function lowercase(input: string) { - return create(parser)(input); +export function lowercase( + input: string, + index?: number, + ctx?: Context, +): Result; +export function lowercase(input: string, index = 0, ctx?: Context) { + return create(parser)(input, index, ctx); } diff --git a/src/primitives/lowercases.test.ts b/src/primitives/lowercases.test.ts index d812ec6..1e92fca 100644 --- a/src/primitives/lowercases.test.ts +++ b/src/primitives/lowercases.test.ts @@ -8,17 +8,17 @@ describe('lowercases', () => { { const result = lowercases('abcDEF'); - assertSuccess(result, 'abc', 'DEF'); + assertSuccess(result, 'abc', 3); } { const result = lowercases('hello world'); - assertSuccess(result, 'hello', ' world'); + assertSuccess(result, 'hello', 5); } { const result = lowercases('abc'); - assertSuccess(result, 'abc', ''); + assertSuccess(result, 'abc', 3); } }); @@ -43,6 +43,6 @@ describe('lowercases', () => { it('should handle single lowercase letter', () => { const result = lowercases('aBC'); - assertSuccess(result, 'a', 'BC'); + assertSuccess(result, 'a', 1); }); }); diff --git a/src/primitives/lowercases.ts b/src/primitives/lowercases.ts index 60b1933..42c3a8b 100644 --- a/src/primitives/lowercases.ts +++ b/src/primitives/lowercases.ts @@ -1,5 +1,4 @@ import { many1 } from '../combinators/many1'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { lowercase } from './lowercase'; @@ -7,18 +6,16 @@ import { lowercase } from './lowercase'; const parser = many1(lowercase); /** - * Parses one or more lowercase letters. + * Parse one or more lowercase letters as a string. * * @example - * lowercases('abcDEF') // { ok: true, value: 'abc', remaining: 'DEF' } + * lowercases('placeholder') // { ok: true, value: 'placeholder', index: 11 } */ -export const lowercases = create((input) => { - const result = parser(input); +export const lowercases = create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value, remaining } = result; - - return success(value.join(''), remaining); + return success(result.value.join(''), result.index); }); diff --git a/src/primitives/nl.test.ts b/src/primitives/nl.test.ts index 90f93dc..db6a05c 100644 --- a/src/primitives/nl.test.ts +++ b/src/primitives/nl.test.ts @@ -7,7 +7,7 @@ describe('nl', () => { it('should parse newline character', () => { const result = nl('\nabc'); - assertSuccess<'\n'>(result, '\n', 'abc'); + assertSuccess<'\n'>(result, '\n', 1); }); it('should fail on non-newline characters', () => { diff --git a/src/primitives/nl.ts b/src/primitives/nl.ts index 2dd5b4f..10e2a7e 100644 --- a/src/primitives/nl.ts +++ b/src/primitives/nl.ts @@ -1,12 +1,12 @@ import { create } from '../core/parser'; import { satisfy } from '../terminals/satisfy'; -const parser = satisfy<'\n'>((c) => /\n/.test(c)); +const parser = satisfy<'\n'>((c) => /\n/.test(c), 'newline'); /** * Parse a newline character. * * @example - * nl('\ntext') // { ok: true, value: '\n', remaining: 'text' } + * nl('\ntext') // { ok: true, value: '\n', index: 1 } */ export const nl = create<'\n'>(parser); diff --git a/src/primitives/number.test.ts b/src/primitives/number.test.ts index 7e04d17..d9066bc 100644 --- a/src/primitives/number.test.ts +++ b/src/primitives/number.test.ts @@ -7,31 +7,31 @@ describe('number', () => { it('should parse integer', () => { const result = number('42'); - assertSuccess(result, 42, ''); + assertSuccess(result, 42, 2); }); it('should parse float', () => { const result = number('3.14'); - assertSuccess(result, 3.14, ''); + assertSuccess(result, 3.14, 4); }); it('should parse negative integer', () => { const result = number('-7'); - assertSuccess(result, -7, ''); + assertSuccess(result, -7, 2); }); it('should parse negative float', () => { const result = number('-2.5'); - assertSuccess(result, -2.5, ''); + assertSuccess(result, -2.5, 4); }); it('should stop at non-numeric', () => { const result = number('42abc'); - assertSuccess(result, 42, 'abc'); + assertSuccess(result, 42, 2); }); it('should fail on non-numeric input', () => { diff --git a/src/primitives/number.ts b/src/primitives/number.ts index bfe2cfc..92172c7 100644 --- a/src/primitives/number.ts +++ b/src/primitives/number.ts @@ -9,9 +9,9 @@ const parser = choice(float, integer); * Parse an integer or float. * * @example - * number('42') // { ok: true, value: 42, remaining: '' } - * number('3.14') // { ok: true, value: 3.14, remaining: '' } - * number('-7') // { ok: true, value: -7, remaining: '' } - * number('-2.5') // { ok: true, value: -2.5, remaining: '' } + * number('42') // { ok: true, value: 42, index: 2 } + * number('3.14') // { ok: true, value: 3.14, index: 4 } + * number('-7') // { ok: true, value: -7, index: 2 } + * number('-2.5') // { ok: true, value: -2.5, index: 4 } */ export const number = create(parser); diff --git a/src/primitives/octDigit.test.ts b/src/primitives/octDigit.test.ts index ede74a5..1ca60bc 100644 --- a/src/primitives/octDigit.test.ts +++ b/src/primitives/octDigit.test.ts @@ -6,7 +6,7 @@ import { octDigit, type OctDigit } from './octDigit'; describe('octDigit', () => { it('should parse octal digits', () => { const result = octDigit('712'); - assertSuccess(result, '7', '12'); + assertSuccess(result, '7', 1); }); it('should fail on non-octal', () => { @@ -21,6 +21,6 @@ describe('octDigit', () => { it('should narrow result to first character type for const string', () => { const result = octDigit('37x' as const); - assertSuccess<'3'>(result, '3', '7x'); + assertSuccess<'3'>(result, '3', 1); }); }); diff --git a/src/primitives/octDigit.ts b/src/primitives/octDigit.ts index 15c2bb5..7ec90a8 100644 --- a/src/primitives/octDigit.ts +++ b/src/primitives/octDigit.ts @@ -1,3 +1,4 @@ +import { type Context } from '../core/context'; import { create } from '../core/parser'; import { type Result } from '../core/result'; import { satisfy } from '../terminals/satisfy'; @@ -5,18 +6,24 @@ import { satisfy } from '../terminals/satisfy'; type Head = S extends `${infer C}${string}` ? C : never; export type OctDigit = '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7'; -const parser = satisfy((c) => /[0-7]/.test(c)); +const parser = satisfy((c) => /[0-7]/.test(c), 'octal digit'); /** * Parse a single octal digit. * * @example - * octDigit('7abc') // { ok: true, value: '7', remaining: 'abc' } + * octDigit('7abc') // { ok: true, value: '7', index: 1 } */ export function octDigit( input: S, + index?: number, + ctx?: Context, ): Result & OctDigit>; -export function octDigit(input: string): Result; -export function octDigit(input: string) { - return create(parser)(input); +export function octDigit( + input: string, + index?: number, + ctx?: Context, +): Result; +export function octDigit(input: string, index = 0, ctx?: Context) { + return create(parser)(input, index, ctx); } diff --git a/src/primitives/octDigits.test.ts b/src/primitives/octDigits.test.ts index 6186a9f..0c58caf 100644 --- a/src/primitives/octDigits.test.ts +++ b/src/primitives/octDigits.test.ts @@ -8,17 +8,17 @@ describe('octDigits', () => { { const result = octDigits('0777rest'); - assertSuccess(result, '0777', 'rest'); + assertSuccess(result, '0777', 4); } { const result = octDigits('1234567'); - assertSuccess(result, '1234567', ''); + assertSuccess(result, '1234567', 7); } { const result = octDigits('042 end'); - assertSuccess(result, '042', ' end'); + assertSuccess(result, '042', 3); } }); @@ -43,12 +43,12 @@ describe('octDigits', () => { it('should handle single octal digit', () => { const result = octDigits('7rest'); - assertSuccess(result, '7', 'rest'); + assertSuccess(result, '7', 1); }); it('should stop at first non-octal character', () => { const result = octDigits('0128'); - assertSuccess(result, '012', '8'); + assertSuccess(result, '012', 3); }); }); diff --git a/src/primitives/octDigits.ts b/src/primitives/octDigits.ts index a8a2b81..5054abc 100644 --- a/src/primitives/octDigits.ts +++ b/src/primitives/octDigits.ts @@ -1,5 +1,4 @@ import { many1 } from '../combinators/many1'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { octDigit } from './octDigit'; @@ -7,18 +6,16 @@ import { octDigit } from './octDigit'; const parser = many1(octDigit); /** - * Parse one or more octal digits. + * Parse one or more octal digits as a string. * * @example - * octDigits('0777abc') // { ok: true, value: '0777', remaining: 'abc' } + * octDigits('755rest') // { ok: true, value: '755', index: 3 } */ -export const octDigits = create((input) => { - const result = parser(input); +export const octDigits = create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value, remaining } = result; - - return success(value.join(''), remaining); + return success(result.value.join(''), result.index); }); diff --git a/src/primitives/position.test.ts b/src/primitives/position.test.ts index 87d4c18..603dae1 100644 --- a/src/primitives/position.test.ts +++ b/src/primitives/position.test.ts @@ -4,15 +4,15 @@ import { assertSuccess } from '../../test/utils'; import { position } from './position'; describe('position', () => { - it('should return current position (input length)', () => { + it('should return the current offset', () => { const result = position('hello'); - assertSuccess(result, 5, 'hello'); + assertSuccess(result, 0, 0); }); it('should return 0 for empty input', () => { const result = position(''); - assertSuccess(result, 0, ''); + assertSuccess(result, 0, 0); }); }); diff --git a/src/primitives/position.ts b/src/primitives/position.ts index ce239f8..c4f9dc8 100644 --- a/src/primitives/position.ts +++ b/src/primitives/position.ts @@ -2,9 +2,15 @@ import { create } from '../core/parser'; import { success } from '../core/success'; /** - * Get current position (remaining input length). + * Get the current offset into the input, without consuming anything. + * + * Counts forward from the start of the input, so it can be paired with + * {@link locate} to attach line/column information to a parsed node. * * @example - * position('abc') // { ok: true, value: 3, remaining: 'abc' } + * position('abc') // { ok: true, value: 0, index: 0 } + * right(string('ab'), position)('abc') // { ok: true, value: 2, index: 2 } */ -export const position = create((input) => success(input.length, input)); +export const position = create((_input, index = 0) => + success(index, index), +); diff --git a/src/primitives/rest.test.ts b/src/primitives/rest.test.ts index 4ff752b..07a1229 100644 --- a/src/primitives/rest.test.ts +++ b/src/primitives/rest.test.ts @@ -7,12 +7,12 @@ describe('rest', () => { it('should return all remaining input', () => { const result = rest('hello world'); - assertSuccess(result, 'hello world', ''); + assertSuccess(result, 'hello world', 11); }); it('should handle empty input', () => { const result = rest(''); - assertSuccess(result, '', ''); + assertSuccess(result, '', 0); }); }); diff --git a/src/primitives/rest.ts b/src/primitives/rest.ts index e03945f..7476e4a 100644 --- a/src/primitives/rest.ts +++ b/src/primitives/rest.ts @@ -2,9 +2,11 @@ import { create } from '../core/parser'; import { success } from '../core/success'; /** - * Parse the rest of the input. + * Consume and return everything left in the input. * * @example - * rest('hello') // { ok: true, value: 'hello', remaining: '' } + * rest('abc') // { ok: true, value: 'abc', index: 3 } */ -export const rest = create((input) => success(input, '')); +export const rest = create((input, index = 0) => + success(input.slice(index), input.length), +); diff --git a/src/primitives/space.test.ts b/src/primitives/space.test.ts index 7673e2e..791a641 100644 --- a/src/primitives/space.test.ts +++ b/src/primitives/space.test.ts @@ -7,7 +7,7 @@ describe('space', () => { it('should parse a single space', () => { const result = space(' abc'); - assertSuccess(result, ' ', 'abc'); + assertSuccess(result, ' ', 1); }); it('should fail on tab', () => { diff --git a/src/primitives/space.ts b/src/primitives/space.ts index f2b3de3..8d3ef41 100644 --- a/src/primitives/space.ts +++ b/src/primitives/space.ts @@ -1,12 +1,12 @@ import { create } from '../core/parser'; import { satisfy } from '../terminals/satisfy'; -const parser = satisfy((c) => c === ' '); +const parser = satisfy((c) => c === ' ', 'space'); /** * Parse a single space character. * * @example - * space(' abc') // { ok: true, value: ' ', remaining: 'abc' } + * space(' abc') // { ok: true, value: ' ', index: 1 } */ export const space = create(parser); diff --git a/src/primitives/spaces.test.ts b/src/primitives/spaces.test.ts index 55b8b4e..3893cee 100644 --- a/src/primitives/spaces.test.ts +++ b/src/primitives/spaces.test.ts @@ -7,13 +7,13 @@ describe('spaces', () => { it('should parse one or more spaces', () => { const result = spaces(' abc'); - assertSuccess(result, ' ', 'abc'); + assertSuccess(result, ' ', 3); }); it('should parse single space', () => { const result = spaces(' abc'); - assertSuccess(result, ' ', 'abc'); + assertSuccess(result, ' ', 1); }); it('should fail on tab', () => { diff --git a/src/primitives/spaces.ts b/src/primitives/spaces.ts index 5244184..4f8a182 100644 --- a/src/primitives/spaces.ts +++ b/src/primitives/spaces.ts @@ -1,5 +1,4 @@ import { many1 } from '../combinators/many1'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { space } from './space'; @@ -7,17 +6,16 @@ import { space } from './space'; const parser = many1(space); /** - * Parse one or more space characters. + * Parse one or more spaces as a string. * * @example - * spaces(' abc') // { ok: true, value: ' ', remaining: 'abc' } + * spaces(' abc') // { ok: true, value: ' ', index: 3 } */ -export const spaces = create((input) => { - const result = parser(input); - +export const spaces = create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - return success(result.value.join(''), result.remaining); + return success(result.value.join(''), result.index); }); diff --git a/src/primitives/tab.test.ts b/src/primitives/tab.test.ts index a72ec31..b1231a9 100644 --- a/src/primitives/tab.test.ts +++ b/src/primitives/tab.test.ts @@ -7,7 +7,7 @@ describe('tab', () => { it('should parse tab character', () => { const result = tab('\tabc'); - assertSuccess(result, '\t', 'abc'); + assertSuccess(result, '\t', 1); }); it('should fail on non-tab characters', () => { diff --git a/src/primitives/tab.ts b/src/primitives/tab.ts index 02f6d0a..a1e4a40 100644 --- a/src/primitives/tab.ts +++ b/src/primitives/tab.ts @@ -7,6 +7,6 @@ const parser = char('\t'); * Parse tab character. * * @example - * tab('\ttext') // { ok: true, value: '\t', remaining: 'text' } + * tab('\ttext') // { ok: true, value: '\t', index: 1 } */ export const tab = create(parser); diff --git a/src/primitives/uppercase.test.ts b/src/primitives/uppercase.test.ts index 4b51794..5f23738 100644 --- a/src/primitives/uppercase.test.ts +++ b/src/primitives/uppercase.test.ts @@ -7,7 +7,7 @@ describe('uppercase', () => { it('should parse uppercase letter', () => { const result = uppercase('ABC'); - assertSuccess(result, 'A', 'BC'); + assertSuccess(result, 'A', 1); }); it('should fail on lowercase', () => { @@ -25,6 +25,6 @@ describe('uppercase', () => { it('should narrow result to first character type for const string', () => { const result = uppercase('ABC' as const); - assertSuccess<'A'>(result, 'A', 'BC'); + assertSuccess<'A'>(result, 'A', 1); }); }); diff --git a/src/primitives/uppercase.ts b/src/primitives/uppercase.ts index 7339536..bc92e9c 100644 --- a/src/primitives/uppercase.ts +++ b/src/primitives/uppercase.ts @@ -1,3 +1,4 @@ +import { type Context } from '../core/context'; import { create } from '../core/parser'; import { type Result } from '../core/result'; import { satisfy } from '../terminals/satisfy'; @@ -31,18 +32,27 @@ export type UppercaseLetter = | 'Y' | 'Z'; -const parser = satisfy((c) => /[A-Z]/.test(c)); +const parser = satisfy( + (c) => /[A-Z]/.test(c), + 'uppercase letter', +); /** * Parses a single uppercase letter. * * @example - * uppercase('ABC') // { ok: true, value: 'A', remaining: 'BC' } + * uppercase('ABC') // { ok: true, value: 'A', index: 1 } */ export function uppercase( input: S, + index?: number, + ctx?: Context, ): Result & UppercaseLetter>; -export function uppercase(input: string): Result; -export function uppercase(input: string) { - return create(parser)(input); +export function uppercase( + input: string, + index?: number, + ctx?: Context, +): Result; +export function uppercase(input: string, index = 0, ctx?: Context) { + return create(parser)(input, index, ctx); } diff --git a/src/primitives/uppercases.test.ts b/src/primitives/uppercases.test.ts index 792c812..43f1dd1 100644 --- a/src/primitives/uppercases.test.ts +++ b/src/primitives/uppercases.test.ts @@ -8,17 +8,17 @@ describe('uppercases', () => { { const result = uppercases('ABCdef'); - assertSuccess(result, 'ABC', 'def'); + assertSuccess(result, 'ABC', 3); } { const result = uppercases('HELLO world'); - assertSuccess(result, 'HELLO', ' world'); + assertSuccess(result, 'HELLO', 5); } { const result = uppercases('ABC'); - assertSuccess(result, 'ABC', ''); + assertSuccess(result, 'ABC', 3); } }); @@ -43,6 +43,6 @@ describe('uppercases', () => { it('should handle single uppercase letter', () => { const result = uppercases('Abc'); - assertSuccess(result, 'A', 'bc'); + assertSuccess(result, 'A', 1); }); }); diff --git a/src/primitives/uppercases.ts b/src/primitives/uppercases.ts index 60641dc..4bc78ba 100644 --- a/src/primitives/uppercases.ts +++ b/src/primitives/uppercases.ts @@ -1,5 +1,4 @@ import { many1 } from '../combinators/many1'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { uppercase } from './uppercase'; @@ -7,18 +6,16 @@ import { uppercase } from './uppercase'; const parser = many1(uppercase); /** - * Parses one or more uppercase letters. + * Parse one or more uppercase letters as a string. * * @example - * uppercases('ABCdef') // { ok: true, value: 'ABC', remaining: 'def' } + * uppercases('ABCdef') // { ok: true, value: 'ABC', index: 3 } */ -export const uppercases = create((input) => { - const result = parser(input); +export const uppercases = create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value, remaining } = result; - - return success(value.join(''), remaining); + return success(result.value.join(''), result.index); }); diff --git a/src/primitives/whitespace.test.ts b/src/primitives/whitespace.test.ts index b2f9ca5..5876653 100644 --- a/src/primitives/whitespace.test.ts +++ b/src/primitives/whitespace.test.ts @@ -7,13 +7,13 @@ describe('whitespace', () => { it('should parse whitespace', () => { const result = whitespace(' '); - assertSuccess(result, ' ', ''); + assertSuccess(result, ' ', 1); }); it('should parse only one whitespace', () => { const result = whitespace(' '); - assertSuccess(result, ' ', ' '); + assertSuccess(result, ' ', 1); }); it('should fail on non-whitespace', () => { diff --git a/src/primitives/whitespace.ts b/src/primitives/whitespace.ts index 52dfe0b..86cccf2 100644 --- a/src/primitives/whitespace.ts +++ b/src/primitives/whitespace.ts @@ -1,12 +1,12 @@ import { create } from '../core/parser'; import { satisfy } from '../terminals/satisfy'; -const parser = satisfy((c) => /\s/.test(c)); +const parser = satisfy((c) => /\s/.test(c), 'whitespace'); /** * Parses a single whitespace character. * * @example - * whitespace(' abc') //{ ok: true, value: ' ', remaining: 'abc' } + * whitespace(' abc') // { ok: true, value: ' ', index: 1 } */ export const whitespace = create(parser); diff --git a/src/primitives/whitespaces.test.ts b/src/primitives/whitespaces.test.ts index a8f8943..4f030e3 100644 --- a/src/primitives/whitespaces.test.ts +++ b/src/primitives/whitespaces.test.ts @@ -8,17 +8,17 @@ describe('whitespaces', () => { { const result = whitespaces(' abc'); - assertSuccess(result, ' ', 'abc'); + assertSuccess(result, ' ', 3); } { const result = whitespaces(' \t\n end'); - assertSuccess(result, ' \t\n ', 'end'); + assertSuccess(result, ' \t\n ', 4); } { const result = whitespaces(' '); - assertSuccess(result, ' ', ''); + assertSuccess(result, ' ', 3); } }); @@ -38,6 +38,6 @@ describe('whitespaces', () => { it('should handle single whitespace character', () => { const result = whitespaces(' abc'); - assertSuccess(result, ' ', 'abc'); + assertSuccess(result, ' ', 1); }); }); diff --git a/src/primitives/whitespaces.ts b/src/primitives/whitespaces.ts index 2f679f2..744507a 100644 --- a/src/primitives/whitespaces.ts +++ b/src/primitives/whitespaces.ts @@ -1,5 +1,4 @@ import { many1 } from '../combinators/many1'; -import { failure } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { whitespace } from './whitespace'; @@ -7,18 +6,16 @@ import { whitespace } from './whitespace'; const parser = many1(whitespace); /** - * Parses one or more whitespace characters. + * Parse one or more whitespaces as a string. * * @example - * whitespaces(' abc') // { ok: true, value: ' ', remaining: 'abc' } + * whitespaces(' \t\nabc') // { ok: true, value: ' \t\n', index: 3 } */ -export const whitespaces = create((input) => { - const result = parser(input); +export const whitespaces = create((input, index = 0, ctx) => { + const result = parser(input, index, ctx); if (!result.ok) { - return failure(); + return result; } - const { value, remaining } = result; - - return success(value.join(''), remaining); + return success(result.value.join(''), result.index); }); diff --git a/src/terminals/char.test.ts b/src/terminals/char.test.ts index ff0b696..842888b 100644 --- a/src/terminals/char.test.ts +++ b/src/terminals/char.test.ts @@ -8,7 +8,7 @@ describe('char', () => { const parser = char('A'); const result = parser('ABC'); - assertSuccess<'A' | null>(result, 'A', 'BC'); + assertSuccess<'A' | null>(result, 'A', 1); }); it('should fail when character does not match', () => { diff --git a/src/terminals/char.ts b/src/terminals/char.ts index 346cad5..b761884 100644 --- a/src/terminals/char.ts +++ b/src/terminals/char.ts @@ -1,4 +1,4 @@ -import { failure } from '../core/failure'; +import { reject } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; @@ -9,7 +9,7 @@ export type Char = * Parse a specific character. * * @example - * char('A')('ABC') // { ok: true, value: 'A', remaining: 'BC' } + * char('A')('ABC') // { ok: true, value: 'A', index: 1 } */ export const char = (expected: Char) => { if ((expected as string).length !== 1) { @@ -18,9 +18,11 @@ export const char = (expected: Char) => { ); } - return create>((input) => - input.length > 0 && input[0] === expected - ? success(expected, input.slice(1)) - : failure(), + const described = [`'${expected}'`]; + + return create>((input, index = 0, ctx) => + input[index] === expected + ? success(expected, index + 1) + : reject(ctx, index, described), ); }; diff --git a/src/terminals/charOf.test.ts b/src/terminals/charOf.test.ts index 4f9e7be..ba55e5d 100644 --- a/src/terminals/charOf.test.ts +++ b/src/terminals/charOf.test.ts @@ -8,7 +8,7 @@ describe('charOf', () => { const parser = charOf(['A', 'B', 'C'] as const); const result = parser('ABCDEFG'); - assertSuccess<'A' | 'B' | 'C'>(result, 'A', 'BCDEFG'); + assertSuccess<'A' | 'B' | 'C'>(result, 'A', 1); }); it('should fail when character is not in set', () => { diff --git a/src/terminals/charOf.ts b/src/terminals/charOf.ts index a7dd092..2ed66e0 100644 --- a/src/terminals/charOf.ts +++ b/src/terminals/charOf.ts @@ -1,23 +1,25 @@ -import { failure } from '../core/failure'; +import { reject } from '../core/failure'; import { create } from '../core/parser'; -import { type Char, char } from './char'; +import { success } from '../core/success'; +import { type Char, type char } from './char'; /** * Parse any character from a set. * * @example - * charOf(['a', 'b', 'c'])('abc') // { ok: true, value: 'a', remaining: 'bc' } + * charOf(['a', 'b', 'c'])('abc') // { ok: true, value: 'a', index: 1 } */ export function charOf( chars: readonly Char[], ): ReturnType>>; export function charOf(chars: readonly string[]) { - return create((input) => { - if (input.length === 0) { - return failure(); - } + const described = chars.map((c) => `'${c}'`); - const next = input[0]; - return chars.includes(next) ? char(next as Char)(input) : failure(); + return create((input, index = 0, ctx) => { + const next = input[index]; + + return index < input.length && chars.includes(next) + ? success(next, index + 1) + : reject(ctx, index, described); }); } diff --git a/src/terminals/noneOf.test.ts b/src/terminals/noneOf.test.ts index 3f355e9..6471a6d 100644 --- a/src/terminals/noneOf.test.ts +++ b/src/terminals/noneOf.test.ts @@ -8,7 +8,7 @@ describe('noneOf', () => { const parser = noneOf(['X', 'Y', 'Z']); const result = parser('ABC'); - assertSuccess(result, 'A', 'BC'); + assertSuccess(result, 'A', 1); }); it('should fail when character is in forbidden set', () => { @@ -22,6 +22,6 @@ describe('noneOf', () => { const parser = noneOf([]); const result = parser('ABC'); - assertSuccess(result, 'A', 'BC'); + assertSuccess(result, 'A', 1); }); }); diff --git a/src/terminals/noneOf.ts b/src/terminals/noneOf.ts index da5bca2..e32b227 100644 --- a/src/terminals/noneOf.ts +++ b/src/terminals/noneOf.ts @@ -1,4 +1,4 @@ -import { failure } from '../core/failure'; +import { reject } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; @@ -6,12 +6,16 @@ import { success } from '../core/success'; * Parse any character not in the set. * * @example - * noneOf(['a', 'b', 'c'])('xyz') // { ok: true, value: 'x', remaining: 'yz' } + * noneOf(['a', 'b', 'c'])('xyz') // { ok: true, value: 'x', index: 1 } */ export function noneOf(chars: readonly string[]) { - return create((input) => { - return input.length > 0 && !chars.includes(input[0]) - ? success(input[0], input.slice(1)) - : failure(); + const described = [ + `character other than ${chars.map((c) => `'${c}'`).join(', ')}`, + ]; + + return create((input, index = 0, ctx) => { + return index < input.length && !chars.includes(input[index]) + ? success(input[index], index + 1) + : reject(ctx, index, described); }); } diff --git a/src/terminals/oneOf.test.ts b/src/terminals/oneOf.test.ts index 341ab42..59fe556 100644 --- a/src/terminals/oneOf.test.ts +++ b/src/terminals/oneOf.test.ts @@ -8,14 +8,14 @@ describe('oneOf', () => { const parser = oneOf(['foo', 'bar', 'baz'] as const); const result = parser('foorest'); - assertSuccess<'foo' | 'bar' | 'baz'>(result, 'foo', 'rest'); + assertSuccess<'foo' | 'bar' | 'baz'>(result, 'foo', 3); }); it('should match a later string in the set', () => { const parser = oneOf(['foo', 'bar', 'baz'] as const); const result = parser('barbarian'); - assertSuccess<'foo' | 'bar' | 'baz'>(result, 'bar', 'barian'); + assertSuccess<'foo' | 'bar' | 'baz'>(result, 'bar', 3); }); it('should fail when no string matches', () => { @@ -29,7 +29,7 @@ describe('oneOf', () => { const parser = oneOf(['foo', 'bar'] as const); const result = parser('foo'); - assertSuccess<'foo' | 'bar'>(result, 'foo', ''); + assertSuccess<'foo' | 'bar'>(result, 'foo', 3); }); it('should fail on empty input', () => { @@ -43,21 +43,21 @@ describe('oneOf', () => { const parser = oneOf(['hello'] as const); const result = parser('hello world'); - assertSuccess<'hello'>(result, 'hello', ' world'); + assertSuccess<'hello'>(result, 'hello', 5); }); it('should match strings with multi-character tokens', () => { const parser = oneOf(['+=', '+', '='] as const); const result = parser('+=1'); - assertSuccess<'+=' | '+' | '='>(result, '+=', '1'); + assertSuccess<'+=' | '+' | '='>(result, '+=', 2); }); it('should match the first candidate even when a later one is also a prefix', () => { const parser = oneOf(['+', '+='] as const); const result = parser('+=1'); - assertSuccess<'+' | '+='>(result, '+', '=1'); + assertSuccess<'+' | '+='>(result, '+', 1); }); it('should fail when input is a prefix of a candidate but not a full match', () => { diff --git a/src/terminals/oneOf.ts b/src/terminals/oneOf.ts index db8a0e9..d840730 100644 --- a/src/terminals/oneOf.ts +++ b/src/terminals/oneOf.ts @@ -1,4 +1,4 @@ -import { failure } from '../core/failure'; +import { reject } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; @@ -6,16 +6,18 @@ import { success } from '../core/success'; * Parse one string from a set of strings (longest match wins). * * @example - * oneOf(['hello', 'hell', 'help'])('helpful') // { ok: true, value: 'help', remaining: 'ful' } + * oneOf(['hello', 'hell', 'help'])('helpful') // { ok: true, value: 'help', index: 4 } */ export const oneOf = (strings: readonly [S, ...S[]]) => { - return create((input) => { + const described = strings.map((s) => `'${s}'`); + + return create((input, index = 0, ctx) => { for (const str of strings) { - if (input.startsWith(str)) { - return success(str, input.slice(str.length)); + if (input.startsWith(str, index)) { + return success(str, index + str.length); } } - return failure(); + return reject(ctx, index, described); }); }; diff --git a/src/terminals/regex.test.ts b/src/terminals/regex.test.ts index 295bc01..afea62e 100644 --- a/src/terminals/regex.test.ts +++ b/src/terminals/regex.test.ts @@ -8,7 +8,7 @@ describe('regex', () => { const parser = regex(/\d+/); const result = parser('123abc'); - assertSuccess(result, '123', 'abc'); + assertSuccess(result, '123', 3); }); it('should fail when pattern does not match at beginning', () => { @@ -22,14 +22,22 @@ describe('regex', () => { const parser = regex(/^[a-z]+/); const result = parser('hello123'); - assertSuccess(result, 'hello', '123'); + assertSuccess(result, 'hello', 5); + }); + + it('should keep a leading caret as a line anchor when multiline', () => { + const parser = regex(/^# /m); + + assertSuccess(parser('# hi'), '# ', 2); + assertSuccess(parser('a\n# hi', 2), '# ', 4); + assertFailure(parser('a# hi', 1), 1); }); it('should handle empty matches', () => { const parser = regex(/\d*/); const result = parser('abc'); - assertSuccess(result, '', 'abc'); + assertSuccess(result, '', 0); }); it('should throw error if a regex with global flag is being used', () => { diff --git a/src/terminals/regex.ts b/src/terminals/regex.ts index 40a892f..ebcb0aa 100644 --- a/src/terminals/regex.ts +++ b/src/terminals/regex.ts @@ -1,23 +1,37 @@ -import { failure } from '../core/failure'; +import { reject } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; /** * Parse with a regular expression. * + * The pattern is matched with the sticky flag at the current offset, so it is + * anchored by construction and never scans ahead. A leading `^` is redundant + * and is stripped — unless the pattern is multiline, where it asserts a line + * start and still carries meaning. + * * @example - * regex(/^\w+/)('hello world') // { ok: true, value: 'hello', remaining: ' world' } + * regex(/^\w+/)('hello world') // { ok: true, value: 'hello', index: 5 } */ -export const regex = (pattern: RegExp) => { +export const regex = (pattern: RegExp, expected?: string) => { if (pattern.global) { throw new Error('Global flag is not supported in regex parsers'); } - return create((input) => { - const match = input.match(pattern); + const source = + pattern.source.startsWith('^') && !pattern.multiline + ? pattern.source.slice(1) + : pattern.source; + const flags = pattern.flags.replace('y', '') + 'y'; + const sticky = new RegExp(source, flags); + const described = [expected ?? `/${source}/`]; + + return create((input, index = 0, ctx) => { + sticky.lastIndex = index; + const match = sticky.exec(input); - return match && match.index === 0 - ? success(match[0] as T, input.slice(match[0].length)) - : failure(); + return match + ? success(match[0] as T, index + match[0].length) + : reject(ctx, index, described); }); }; diff --git a/src/terminals/satisfy.test.ts b/src/terminals/satisfy.test.ts index 435b75b..89dca1e 100644 --- a/src/terminals/satisfy.test.ts +++ b/src/terminals/satisfy.test.ts @@ -8,7 +8,7 @@ describe('satisfy', () => { const parser = satisfy((c: string) => c >= '0' && c <= '9'); const result = parser('5abc'); - assertSuccess(result, '5', 'abc'); + assertSuccess(result, '5', 1); }); it('should fail when character does not satisfy predicate', () => { diff --git a/src/terminals/satisfy.ts b/src/terminals/satisfy.ts index a506a04..b05329d 100644 --- a/src/terminals/satisfy.ts +++ b/src/terminals/satisfy.ts @@ -1,4 +1,4 @@ -import { failure } from '../core/failure'; +import { reject } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; import { type Char } from './char'; @@ -7,14 +7,19 @@ import { type Char } from './char'; * Parse a character satisfying a predicate. * * @example - * satisfy((c) => c === 'a')('abc') // { ok: true, value: 'a', remaining: 'bc' } + * satisfy((c) => c === 'a')('abc') // { ok: true, value: 'a', index: 1 } */ -export function satisfy(predicate: (c: Char) => boolean) { - return create((input) => { - const c = input[0] as Char; +export function satisfy( + predicate: (c: Char) => boolean, + expected = 'matching character', +) { + const described = [expected]; - return input.length > 0 && predicate(c) - ? success(c as T, input.slice(1)) - : failure(); + return create((input, index = 0, ctx) => { + const c = input[index] as Char; + + return index < input.length && predicate(c) + ? success(c as T, index + 1) + : reject(ctx, index, described); }); } diff --git a/src/terminals/string.test.ts b/src/terminals/string.test.ts index a4229b7..0e6b4e8 100644 --- a/src/terminals/string.test.ts +++ b/src/terminals/string.test.ts @@ -8,7 +8,7 @@ describe('string', () => { const parser = string('test'); const result = parser('testing'); - assertSuccess<'test'>(result, 'test', 'ing'); + assertSuccess<'test'>(result, 'test', 4); }); it('should fail when string does not match', () => { @@ -22,13 +22,13 @@ describe('string', () => { const parser = string('hello'); const result = parser('hello'); - assertSuccess<'hello'>(result, 'hello', ''); + assertSuccess<'hello'>(result, 'hello', 5); }); it('should handle empty string', () => { const parser = string(''); const result = parser('anything'); - assertSuccess<''>(result, '', 'anything'); + assertSuccess<''>(result, '', 0); }); }); diff --git a/src/terminals/string.ts b/src/terminals/string.ts index 8a5c71f..9c60d2e 100644 --- a/src/terminals/string.ts +++ b/src/terminals/string.ts @@ -1,4 +1,4 @@ -import { failure } from '../core/failure'; +import { reject } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; @@ -6,12 +6,15 @@ import { success } from '../core/success'; * Parse a specific string. * * @example - * string('hello')('hello world') // { ok: true, value: 'hello', remaining: ' world' } + * string('hello')('hello world') // { ok: true, value: 'hello', index: 5 } */ export const string = (str: S) => { - return create((input) => { - return input.startsWith(str) - ? success(str, input.slice(str.length)) - : failure(); + const described = [`'${str}'`]; + const { length } = str; + + return create((input, index = 0, ctx) => { + return input.startsWith(str, index) + ? success(str, index + length) + : reject(ctx, index, described); }); }; diff --git a/src/terminals/stringOf.test.ts b/src/terminals/stringOf.test.ts index 8993274..3cf1f6e 100644 --- a/src/terminals/stringOf.test.ts +++ b/src/terminals/stringOf.test.ts @@ -8,7 +8,7 @@ describe('stringOf', () => { it('should parse a character from the set', () => { const result = parser('abc'); - assertSuccess(result, 'a', 'bc'); + assertSuccess(result, 'a', 1); }); it('should fail on character not in set', () => { diff --git a/src/terminals/stringOf.ts b/src/terminals/stringOf.ts index 2438ac4..67060f8 100644 --- a/src/terminals/stringOf.ts +++ b/src/terminals/stringOf.ts @@ -1,4 +1,4 @@ -import { failure } from '../core/failure'; +import { reject } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; @@ -6,12 +6,14 @@ import { success } from '../core/success'; * Parse first character that exists in string (like charOf but for a string). * * @example - * stringOf('abc')('abcdef') // { ok: true, value: 'a', remaining: 'bcdef' } + * stringOf('abc')('abcdef') // { ok: true, value: 'a', index: 1 } */ export const stringOf = (chars: string) => { - return create((input) => - input.length > 0 && chars.includes(input[0]) - ? success(input[0], input.slice(1)) - : failure(), + const described = [`one of '${chars}'`]; + + return create((input, index = 0, ctx) => + index < input.length && chars.includes(input[index]) + ? success(input[index], index + 1) + : reject(ctx, index, described), ); }; diff --git a/src/terminals/take.test.ts b/src/terminals/take.test.ts index 001cb8c..b7cc8f0 100644 --- a/src/terminals/take.test.ts +++ b/src/terminals/take.test.ts @@ -8,14 +8,14 @@ describe('take', () => { const parser = take(3); const result = parser('abcdef'); - assertSuccess(result, 'abc', 'def'); + assertSuccess(result, 'abc', 3); }); it('should take all characters when count equals input length', () => { const parser = take(3); const result = parser('abc'); - assertSuccess(result, 'abc', ''); + assertSuccess(result, 'abc', 3); }); it('should fail when input is shorter than count', () => { @@ -29,6 +29,18 @@ describe('take', () => { const parser = take(0); const result = parser('abc'); - assertSuccess(result, '', 'abc'); + assertSuccess(result, '', 0); + }); + + it('should describe a single character in the singular', () => { + const result = take(1)(''); + + assertFailure(result, 0, ['1 more character']); + }); + + it('should describe several characters in the plural', () => { + const result = take(3)('ab'); + + assertFailure(result, 0, ['3 more characters']); }); }); diff --git a/src/terminals/take.ts b/src/terminals/take.ts index 1173299..41d7ee6 100644 --- a/src/terminals/take.ts +++ b/src/terminals/take.ts @@ -1,4 +1,4 @@ -import { failure } from '../core/failure'; +import { reject } from '../core/failure'; import { create } from '../core/parser'; import { success } from '../core/success'; @@ -6,12 +6,14 @@ import { success } from '../core/success'; * Take n characters. * * @example - * take(3)('abcdef') // { ok: true, value: 'abc', remaining: 'def' } + * take(3)('abcdef') // { ok: true, value: 'abc', index: 3 } */ export const take = (count: number) => { - return create((input) => - input.length >= count - ? success(input.slice(0, count), input.slice(count)) - : failure(), + const described = [`${count} more character${count === 1 ? '' : 's'}`]; + + return create((input, index = 0, ctx) => + input.length - index >= count + ? success(input.slice(index, index + count), index + count) + : reject(ctx, index, described), ); }; diff --git a/src/terminals/takeWhile.test.ts b/src/terminals/takeWhile.test.ts index 807e20f..155859e 100644 --- a/src/terminals/takeWhile.test.ts +++ b/src/terminals/takeWhile.test.ts @@ -8,27 +8,27 @@ describe('takeWhile', () => { const parser = takeWhile((c) => c >= '0' && c <= '9'); const result = parser('123ABC'); - assertSuccess(result, '123', 'ABC'); + assertSuccess(result, '123', 3); }); it('should return empty string when first character fails predicate', () => { const parser = takeWhile((c) => c >= '0' && c <= '9'); const result = parser('ABC123'); - assertSuccess(result, '', 'ABC123'); + assertSuccess(result, '', 0); }); it('should take all characters when all satisfy predicate', () => { const parser = takeWhile((c) => c >= '0' && c <= '9'); const result = parser('123'); - assertSuccess(result, '123', ''); + assertSuccess(result, '123', 3); }); it('should handle empty input', () => { const parser = takeWhile(() => true); const result = parser(''); - assertSuccess(result, '', ''); + assertSuccess(result, '', 0); }); }); diff --git a/src/terminals/takeWhile.ts b/src/terminals/takeWhile.ts index ed2e12a..083c996 100644 --- a/src/terminals/takeWhile.ts +++ b/src/terminals/takeWhile.ts @@ -5,15 +5,15 @@ import { success } from '../core/success'; * Takes characters while the predicate returns true. * * @example - * takeWhile((c) => c !== 'x')('abcx') // { ok: true, value: 'abc', remaining: 'x' } + * takeWhile((c) => c !== 'x')('abcx') // { ok: true, value: 'abc', index: 3 } */ export const takeWhile = (predicate: (char: string) => boolean) => { - return create((input) => { - let index = 0; - while (index < input.length && predicate(input[index])) { - index++; + return create((input, index = 0) => { + let end = index; + while (end < input.length && predicate(input[end])) { + end++; } - return success(input.slice(0, index), input.slice(index)); + return success(input.slice(index, end), end); }); }; diff --git a/src/terminals/token.test.ts b/src/terminals/token.test.ts index 18e806b..7331dd3 100644 --- a/src/terminals/token.test.ts +++ b/src/terminals/token.test.ts @@ -8,21 +8,21 @@ describe('token', () => { const parser = token('if'); const result = parser('if (condition)'); - assertSuccess<'if'>(result, 'if', '(condition)'); + assertSuccess<'if'>(result, 'if', 5); }); it('should parse operators with whitespace', () => { const parser = token('=='); const result = parser('== value'); - assertSuccess<'=='>(result, '==', 'value'); + assertSuccess<'=='>(result, '==', 4); }); it('should parse punctuation symbols', () => { const parser = token('('); const result = parser('( )'); - assertSuccess<'('>(result, '(', ')'); + assertSuccess<'('>(result, '(', 3); }); it('should fail when symbol does not match', () => { @@ -36,20 +36,20 @@ describe('token', () => { const parser = token(';'); const result = parser(';next'); - assertSuccess<';'>(result, ';', 'next'); + assertSuccess<';'>(result, ';', 1); }); it('should handle multi-character symbols', () => { const parser = token('<='); const result = parser('<= 100'); - assertSuccess<'<='>(result, '<=', '100'); + assertSuccess<'<='>(result, '<=', 4); }); it('should work with empty string symbol', () => { const parser = token(''); const result = parser(' anything'); - assertSuccess<''>(result, '', 'anything'); + assertSuccess<''>(result, '', 3); }); }); diff --git a/src/terminals/token.ts b/src/terminals/token.ts index 071ac26..76cae85 100644 --- a/src/terminals/token.ts +++ b/src/terminals/token.ts @@ -8,9 +8,9 @@ import { string } from './string'; * strings that might be followed by any character. * * @example - * token('let')('let x') // { ok: true, value: 'let', remaining: 'x' } - * token('let')('let1') // { ok: true, value: 'let', remaining: '1' } - * token('let')('let x') // { ok: true, value: 'let', remaining: 'x' } + * token('let')('let x') // { ok: true, value: 'let', index: 4 } + * token('let')('let1') // { ok: true, value: 'let', index: 3 } + * token('let')('let x') // { ok: true, value: 'let', index: 5 } */ export const token = (input: T) => { return create(lexeme(string(input))); diff --git a/src/terminals/word.test.ts b/src/terminals/word.test.ts index 29b3061..7034b06 100644 --- a/src/terminals/word.test.ts +++ b/src/terminals/word.test.ts @@ -9,13 +9,13 @@ describe('word', () => { const parser = word('test'); const result = parser('test abc'); - assertSuccess(result, 'test', 'abc'); + assertSuccess(result, 'test', 5); } { const parser = word('hello'); const result = parser('hello world'); - assertSuccess(result, 'hello', 'world'); + assertSuccess(result, 'hello', 6); } }); @@ -39,13 +39,13 @@ describe('word', () => { const parser = word('test'); const result = parser('test!'); - assertSuccess(result, 'test', '!'); + assertSuccess(result, 'test', 4); } { const parser = word('word'); const result = parser('word.'); - assertSuccess(result, 'word', '.'); + assertSuccess(result, 'word', 4); } }); }); diff --git a/src/terminals/word.ts b/src/terminals/word.ts index 2ce951d..558b694 100644 --- a/src/terminals/word.ts +++ b/src/terminals/word.ts @@ -10,9 +10,9 @@ import { string } from './string'; * Use this when parsing keywords that should not be part of a longer identifier. * * @example - * word('let')('let x') // { ok: true, value: 'let', remaining: 'x' } - * word('let')('let1') // { ok: false } - * word('if')('if (x)') // { ok: true, value: 'if', remaining: '(x)' } + * word('let')('let x') // { ok: true, value: 'let', index: 4 } + * word('let')('let1') // { ok: false, index: 3, expected: ['/(?!\\w)/'] } + * word('if')('if (x)') // { ok: true, value: 'if', index: 3 } */ export const word = (targetWord: string) => { return create( diff --git a/src/utils/join.ts b/src/utils/join.ts index 6f95dc9..66ebe4a 100644 --- a/src/utils/join.ts +++ b/src/utils/join.ts @@ -2,8 +2,8 @@ * Join array elements into a string. * * @example - * join()([1, 2, 3]) // '123' - * join('-')([1, 2, 3]) // '1-2-3' + * join()([1, 2, 3]) // '123' + * join('-')([1, 2, 3]) // '1-2-3' */ export const join = (separator: string = '') => { return >(haystack: T): string => { diff --git a/src/utils/pipe.test.ts b/src/utils/pipe.test.ts index 0621e8a..e6dedbd 100644 --- a/src/utils/pipe.test.ts +++ b/src/utils/pipe.test.ts @@ -11,7 +11,7 @@ describe('pipe', () => { (n: number) => n * 2, ); - const parser = create(() => success(5, 'abc')); + const parser = create((_input, index = 0) => success(5, index)); const result = transform.call(parser, 5); expect(result).toBe(12); @@ -24,7 +24,7 @@ describe('pipe', () => { (n: number) => n - 3, ); - const parser = create(() => success(5, 'abc')); + const parser = create((_input, index = 0) => success(5, index)); const result = transform.call(parser, 5); expect(result).toBe(9); @@ -33,7 +33,7 @@ describe('pipe', () => { it('should work with a single function', () => { const transform = pipe((n: number) => n * 2); - const parser = create(() => success(5, 'abc')); + const parser = create((_input, index = 0) => success(5, index)); const result = transform.call(parser, 5); expect(result).toBe(10); @@ -46,14 +46,18 @@ describe('pipe', () => { (s: string) => s + '!', ); - const parser = create(() => success(' hello ', '!')); + const parser = create((_input, index = 0) => + success(' hello ', index), + ); const result = transform.call(parser, ' hello '); expect(result).toBe('HELLO!'); }); it('should allow accessing parser state via this', () => { - const parser = create(() => success(100, 'abc')); + const parser = create((_input, index = 0) => + success(100, index), + ); const transform = pipe(function (n: number) { return n + (this ? 1 : 0); diff --git a/src/utils/pipe.ts b/src/utils/pipe.ts index 343e3c7..48e312c 100644 --- a/src/utils/pipe.ts +++ b/src/utils/pipe.ts @@ -4,7 +4,7 @@ import { type Parser } from '../core/parser'; * Pipe parser functions together. * * @example - * pipe(lexeme)(letters)('xyz abc') // { ok: true, value: 'xyz', remaining: 'abc' } + * pipe(lexeme)(letters)('xyz abc') // { ok: true, value: 'xyz', index: 6 } */ type Pipe = (this: Parser, arg: A) => B; diff --git a/test/examples.test.ts b/test/examples.test.ts index f50acfa..5b1ff28 100644 --- a/test/examples.test.ts +++ b/test/examples.test.ts @@ -22,78 +22,56 @@ describe('examples from source', () => { describe('combinators', () => { it('bind: Chain parsers where the second parser depends on the first result.', () => { const result0 = bind(digits, (n) => take(n))('3abc'); - expect(result0).toEqual({ ok: true, value: 'abc', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 'abc', index: 4 }); }); it('braced: Parse content surrounded by braces.', () => { const result0 = braced(string('hi'))('{hi}'); - expect(result0).toEqual({ ok: true, value: 'hi', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 'hi', index: 4 }); }); it('bracketed: Parse content surrounded by brackets.', () => { const result0 = bracketed(string('hi'))('[hi]'); - expect(result0).toEqual({ ok: true, value: 'hi', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 'hi', index: 4 }); }); it('chainLeft: Chain left-associative operations (right-to-left for same precedence).', () => { const result0 = chainLeft(digits, operation)('1+2+3'); - expect(result0).toEqual({ ok: true, value: 6, remaining: '' }); - const result1 = chainLeft(digits, operation)('10-3+2'); - expect(result1).toEqual({ ok: true, value: 9, remaining: '' }); + expect(result0).toEqual({ ok: true, value: 6, index: 5 }); }); it('chainLeft1: Chain left-associative operations (fails on empty input).', () => { const result0 = chainLeft1(digits, operation)('1+2+3'); - expect(result0).toEqual({ ok: true, value: 6, remaining: '' }); + expect(result0).toEqual({ ok: true, value: 6, index: 5 }); const result1 = chainLeft1(digits, operation)('8/2*3'); - expect(result1).toEqual({ ok: true, value: 12, remaining: '' }); + expect(result1).toEqual({ ok: true, value: 12, index: 5 }); }); - it('chainRight: Chain right-associative operations (right-to-left grouping).', () => { - const result0 = chainRight(digits, operation)('2-1-1'); - expect(result0).toEqual({ ok: true, value: 2, remaining: '' }); - const result1 = chainRight(digits, operation)('4/2/2'); - expect(result1).toEqual({ ok: true, value: 4, remaining: '' }); + it('chainRight: Chain right-associative operations.', () => { + const result0 = chainRight(digits, operation)('1+2+3'); + expect(result0).toEqual({ ok: true, value: 6, index: 5 }); }); it('chainRight1: Chain right-associative operations (fails on empty input).', () => { - const result0 = chainRight1(digits, operation)('2-1-1'); - expect(result0).toEqual({ ok: true, value: 2, remaining: '' }); - const result1 = chainRight1(digits, operation)('4/2/2'); - expect(result1).toEqual({ ok: true, value: 4, remaining: '' }); + const result0 = chainRight1(digits, operation)('1+2+3'); + expect(result0).toEqual({ ok: true, value: 6, index: 5 }); }); - it('choice: Try each parser in order, return first success.', () => { + it('choice: in the input is already in the parse context.', () => { const result0 = choice(string('hello'), string('world'))('hello'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 5 }); }); it('concat: Join string array parser result into a single string.', () => { const result0 = concat(many(letter))('abc123'); - expect(result0).toEqual({ - ok: true, - value: 'abc', - remaining: '123', - }); + expect(result0).toEqual({ ok: true, value: 'abc', index: 3 }); const result1 = concat(many(letter), '-')('abc123'); - expect(result1).toEqual({ - ok: true, - value: 'a-b-c', - remaining: '123', - }); + expect(result1).toEqual({ ok: true, value: 'a-b-c', index: 3 }); }); it('consume: Consume input but discard the result (return null).', () => { const result0 = consume(string('hello'))('hello world'); - expect(result0).toEqual({ - ok: true, - value: null, - remaining: ' world', - }); + expect(result0).toEqual({ ok: true, value: null, index: 5 }); }); it('endBy: Zero or more items separated and ending with terminator.', () => { @@ -101,7 +79,7 @@ describe('examples from source', () => { expect(result0).toEqual({ ok: true, value: ['item', 'item', 'item'], - remaining: '', + index: 15, }); }); @@ -113,7 +91,7 @@ describe('examples from source', () => { expect(result0).toEqual({ ok: true, value: ['item', 'item', 'item'], - remaining: '', + index: 15, }); }); @@ -122,69 +100,45 @@ describe('examples from source', () => { expect(result0).toEqual({ ok: true, value: ['a', 'a', 'a'], - remaining: '', + index: 3, }); }); it('first: Extract the first element from a parser result array.', () => { const result0 = first(sequence(char('a'), digit))('a1bc'); - expect(result0).toEqual({ ok: true, value: 'a', remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: 'a', index: 2 }); }); it('flag: Return true if parser succeeds, false otherwise. Always succeeds without consuming input on failure.', () => { const result0 = flag(string('*'))('*abc'); - expect(result0).toEqual({ - ok: true, - value: true, - remaining: 'abc', - }); + expect(result0).toEqual({ ok: true, value: true, index: 1 }); const result1 = flag(string('*'))('abc'); - expect(result1).toEqual({ - ok: true, - value: false, - remaining: 'abc', - }); + expect(result1).toEqual({ ok: true, value: false, index: 0 }); }); - it('fold: Parse zero or more and fold into a single value.', () => { - const result0 = fold(digit, [], (acc, d) => [...acc, d])('123'); - expect(result0).toEqual({ - ok: true, - value: [1, 2, 3], - remaining: '', - }); + it('fold: Fold zero or more occurrences into a single value.', () => { + const result0 = fold(digit, 0, (acc, d) => acc + d)('123'); + expect(result0).toEqual({ ok: true, value: 6, index: 3 }); }); - it('fold1: Parse one or more and fold into a single value.', () => { + it('fold1: Fold one or more occurrences into a single value.', () => { const result0 = fold1(digit, 0, (acc, d) => acc + d)('123'); - expect(result0).toEqual({ ok: true, value: 6, remaining: '' }); + expect(result0).toEqual({ ok: true, value: 6, index: 3 }); }); - it('foldRight: Parse zero or more and fold right-to-left.', () => { - const result0 = foldRight(digit, [], (acc, d) => [...acc, d])( - '123', - ); - expect(result0).toEqual({ - ok: true, - value: [3, 2, 1], - remaining: '', - }); + it('foldRight: Fold zero or more occurrences from the right into a single value.', () => { + const result0 = foldRight(digit, 0, (acc, d) => acc + d)('123'); + expect(result0).toEqual({ ok: true, value: 6, index: 3 }); }); - it('foldRight1: Parse one or more and fold right-to-left.', () => { - const result0 = foldRight1(digit, [], (acc, d) => [...acc, d])( - '123', - ); - expect(result0).toEqual({ - ok: true, - value: [3, 2, 1], - remaining: '', - }); + it('foldRight1: Fold one or more occurrences from the right into a single value.', () => { + const result0 = foldRight1(digit, 0, (acc, d) => acc + d)('123'); + expect(result0).toEqual({ ok: true, value: 6, index: 3 }); }); it('fuse: The fused parser concatenates all string results.', () => { const result0 = fuse(char('a'), char('b'), char('c'))('abc'); - expect(result0).toEqual({ ok: true, value: 'abc', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 'abc', index: 3 }); const result1 = fuse( string('hello'), char(' '), @@ -193,38 +147,30 @@ describe('examples from source', () => { expect(result1).toEqual({ ok: true, value: 'hello world', - remaining: '', + index: 11, }); }); it('guard: Conditionally apply parser based on a condition.', () => { const result0 = guard(true, string('hello'))('hello'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 5 }); const result1 = guard(false, string('hello'))('hello'); - expect(result1).toEqual({ ok: false }); + expect(result1).toEqual({ ok: false, index: 0, expected: [] }); }); it('inner: Extract inner value from surrounded content (like inner of braced).', () => { const result0 = inner(char('('), string('hi'), char(')'))('(hi)'); - expect(result0).toEqual({ ok: true, value: 'hi', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 'hi', index: 4 }); }); - it('interleaved: Parse items with interleaved separators.', () => { - const result0 = interleaved(char('a'), char(','))('a,a,a'); - expect(result0).toEqual({ - ok: true, - value: ['a', ',', 'a', ',', 'a'], - remaining: '', - }); + it('interleaved: Parse items separated by separators, keeping both in the result.', () => { + const result0 = interleaved(digits, char('+'))('1+2'); + expect(result0).toEqual({ ok: true, value: [1, '+', 2], index: 3 }); }); it('last: Extract the last element from a parser result array.', () => { const result0 = last(sequence(char('a'), char('b')))('ab'); - expect(result0).toEqual({ ok: true, value: 'b', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 'b', index: 2 }); }); it('left: Keep only the left result from a sequence.', () => { @@ -232,37 +178,29 @@ describe('examples from source', () => { string('hello'), string('world'), )('helloworld'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 10 }); }); it('lexeme: Parser that consumes trailing whitespace.', () => { const result0 = lexeme(string('hello'))('hello world'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: 'world', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 8 }); }); - it('many: Zero or more occurrences (never fails).', () => { + it('many: still report what the repetition was expecting next.', () => { const result0 = many(char('a'))('aaa'); expect(result0).toEqual({ ok: true, value: ['a', 'a', 'a'], - remaining: '', + index: 3, }); }); - it('many1: One or more occurrences (fails if no matches).', () => { + it('many1: One or more occurrences.', () => { const result0 = many1(char('a'))('aaa'); expect(result0).toEqual({ ok: true, value: ['a', 'a', 'a'], - remaining: '', + index: 3, }); }); @@ -271,26 +209,18 @@ describe('examples from source', () => { expect(result0).toEqual({ ok: true, value: ['a', 'a', 'a'], - remaining: '', + index: 3, }); }); - it('manyAtMost: Parse at most n occurrences.', () => { + it('manyAtMost: Parse at most n occurrences (never fails).', () => { const result0 = manyAtMost(char('a'), 2)('aaa'); - expect(result0).toEqual({ - ok: true, - value: ['a', 'a'], - remaining: 'a', - }); + expect(result0).toEqual({ ok: true, value: ['a', 'a'], index: 2 }); }); it('manyBetween: Parse between min and max occurrences.', () => { - const result0 = manyBetween(char('a'), 2, 3)('aaa'); - expect(result0).toEqual({ - ok: true, - value: ['a', 'a', 'a'], - remaining: '', - }); + const result0 = manyBetween(char('a'), 1, 2)('aaa'); + expect(result0).toEqual({ ok: true, value: ['a', 'a'], index: 2 }); }); it('manyTill: Parse zero or more until terminator matches.', () => { @@ -298,19 +228,13 @@ describe('examples from source', () => { expect(result0).toEqual({ ok: true, value: ['a', 'a', 'a'], - remaining: '', + index: 4, }); }); - it('map: Transform the parsed value.', () => { - const result0 = map(string('hello'), (v) => v.toUpperCase())( - 'hello', - ); - expect(result0).toEqual({ - ok: true, - value: 'HELLO', - remaining: '', - }); + it('map: Transform a parsed value through one or more functions.', () => { + const result0 = map(digits, (n) => n * 2)('21'); + expect(result0).toEqual({ ok: true, value: 42, index: 2 }); }); it('node: Create a node from parser fields.', () => { @@ -322,23 +246,19 @@ describe('examples from source', () => { expect(result0).toEqual({ ok: true, value: { type: 'binop', left: 1, op: '+', right: 2 }, - remaining: '', + index: 3, }); const result1 = node('number', { value: digits })('123'); expect(result1).toEqual({ ok: true, value: { type: 'number', value: 123 }, - remaining: '', + index: 3, }); }); - it('not: Succeed if parser fails (without consuming input).', () => { + it('not: expectation in the final message.', () => { const result0 = not(string('hello'))('world'); - expect(result0).toEqual({ - ok: true, - value: null, - remaining: 'world', - }); + expect(result0).toEqual({ ok: true, value: null, index: 0 }); }); it('nth: Extract the nth element from a parser result array.', () => { @@ -346,119 +266,75 @@ describe('examples from source', () => { sequence(char('a'), char('b'), char('c')), 1, )('abc'); - expect(result0).toEqual({ ok: true, value: 'b', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 'b', index: 3 }); }); it('optional: Make parser optional (return null on failure, without consuming input).', () => { const result0 = optional(string('hello'))('hello'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 5 }); const result1 = optional(string('hello'))('world'); - expect(result1).toEqual({ - ok: true, - value: null, - remaining: 'world', - }); + expect(result1).toEqual({ ok: true, value: null, index: 0 }); }); - it('optionalConsume: Optionally consume input (always succeeds, returns void).', () => { - const result0 = optionalConsume(string('hello'))('hello world'); - expect(result0).toEqual({ - ok: true, - value: undefined, - remaining: ' world', - }); - const result1 = optionalConsume(string('hello'))('world'); - expect(result1).toEqual({ - ok: true, - value: undefined, - remaining: 'world', - }); + it('optionalConsume: Consume input if the parser matches, discarding the result.', () => { + const result0 = optionalConsume(string('hi'))('hi there'); + expect(result0).toEqual({ ok: true, index: 2 }); }); - it('optionalSeparatedBy: Zero or more items separated by a separator, with optional null values.', () => { + it('optionalSeparatedBy: Parse items separated by a separator, allowing empty slots.', () => { const result0 = optionalSeparatedBy(digits, char(','))('1,2'); - expect(result0).toEqual({ ok: true, value: [1, 2], remaining: '' }); - const result1 = optionalSeparatedBy(digits, char(','))(',1'); - expect(result1).toEqual({ - ok: true, - value: [null, 1], - remaining: '', - }); - const result2 = optionalSeparatedBy(digits, char(','))('1,'); - expect(result2).toEqual({ ok: true, value: [1], remaining: '' }); + expect(result0).toEqual({ ok: true, value: [1, 2], index: 3 }); }); it('outer: Extract outer values from a sequence of 3 parsers (skip middle).', () => { const result0 = outer(char('('), string('hi'), char(')'))('(hi)'); - expect(result0).toEqual({ - ok: true, - value: ['(', ')'], - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: ['(', ')'], index: 4 }); }); - it('padded: Parse content with optional whitespace on both sides.', () => { - const result0 = padded(string('hi'))(' hi '); - expect(result0).toEqual({ ok: true, value: 'hi', remaining: '' }); + it('padded: Parse content surrounded by optional whitespace.', () => { + const result0 = padded(string('hi'))(' hi '); + expect(result0).toEqual({ ok: true, value: 'hi', index: 6 }); }); it('parenthesized: Parse content surrounded by parentheses.', () => { const result0 = parenthesized(string('hi'))('(hi)'); - expect(result0).toEqual({ ok: true, value: 'hi', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 'hi', index: 4 }); }); - it('peek: Parse without consuming input.', () => { + it('peek: Look ahead without consuming input.', () => { const result0 = peek(string('hello'))('hello world'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: 'hello world', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 0 }); }); - it('postfix: Parse postfix operators (chains atom with operators that return functions).', () => { + it('postfix: Apply zero or more postfix operators to an atom.', () => { const result0 = postfix( - char('a'), - map(char('!'), () => (x) => x), - )('a!'); - expect(result0).toEqual({ ok: true, value: 'a', remaining: '' }); + digits, + map(char('!'), () => (n) => n * 2), + )('3!'); + expect(result0).toEqual({ ok: true, value: 6, index: 2 }); }); - it('prefix: Parse prefix operators (like - in -5).', () => { + it('prefix: Apply zero or more prefix operators to an atom.', () => { const result0 = prefix( - map(char('-'), () => (x) => -x), - digit, - )('-5'); - expect(result0).toEqual({ ok: true, value: -5, remaining: '' }); + map(char('-'), () => (n) => -n), + digits, + )('-3'); + expect(result0).toEqual({ ok: true, value: -3, index: 2 }); }); - it('pure: Always return a value without consuming input.', () => { + it('pure: Always succeed with a value without consuming input.', () => { const result0 = pure(42)('abc'); - expect(result0).toEqual({ ok: true, value: 42, remaining: 'abc' }); - const result1 = pure('ok')(''); - expect(result1).toEqual({ ok: true, value: 'ok', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 42, index: 0 }); }); it('quoted: Parse content surrounded by single or double quotes.', () => { const result0 = quoted(string('hello'))('"hello"'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 7 }); }); it('recover: Use fallback value when parser fails.', () => { const result0 = recover(string('hello'), 'default')('world'); - expect(result0).toEqual({ - ok: true, - value: 'default', - remaining: 'world', - }); + expect(result0).toEqual({ ok: true, value: 'default', index: 0 }); }); it('right: Keep only the right result from a sequence.', () => { @@ -466,60 +342,36 @@ describe('examples from source', () => { string('hello'), string('world'), )('helloworld'); - expect(result0).toEqual({ - ok: true, - value: 'world', - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: 'world', index: 10 }); }); - it('separatedBy: Zero or more items separated by a separator.', () => { - const result0 = separatedBy(char('a'), char(','))('a,a,a'); - expect(result0).toEqual({ - ok: true, - value: ['a', 'a', 'a'], - remaining: '', - }); + it('separatedBy: after it does not, the list ends before the separator.', () => { + const result0 = separatedBy(digits, char(','))('1,2,3'); + expect(result0).toEqual({ ok: true, value: [1, 2, 3], index: 5 }); }); - it('separatedBy1: One or more items separated by a separator.', () => { - const result0 = separatedBy1(char('a'), char(','))('a,a,a'); - expect(result0).toEqual({ - ok: true, - value: ['a', 'a', 'a'], - remaining: '', - }); + it('separatedBy1: Parse one or more items separated by a separator.', () => { + const result0 = separatedBy1(digits, char(','))('1,2,3'); + expect(result0).toEqual({ ok: true, value: [1, 2, 3], index: 5 }); }); - it('separatedEndBy: Zero or more items separated by and ending with a terminator.', () => { - const result0 = separatedEndBy(char('a'), char(';'))('a;a;a;'); - expect(result0).toEqual({ - ok: true, - value: ['a', 'a', 'a'], - remaining: '', - }); + it('separatedEndBy: Parse zero or more items separated by a separator, allowing a trailing one.', () => { + const result0 = separatedEndBy(digits, char(','))('1,2,3,'); + expect(result0).toEqual({ ok: true, value: [1, 2, 3], index: 6 }); }); - it('separatedEndBy1: One or more items separated by and ending with a terminator.', () => { - const result0 = separatedEndBy1(char('a'), char(';'))('a;a;a;'); - expect(result0).toEqual({ - ok: true, - value: ['a', 'a', 'a'], - remaining: '', - }); + it('separatedEndBy1: Parse one or more items separated by a separator, allowing a trailing one.', () => { + const result0 = separatedEndBy1(digits, char(','))('1,2,3,'); + expect(result0).toEqual({ ok: true, value: [1, 2, 3], index: 6 }); }); - it('separatedUntil: Parse items separated by separator until terminator matches.', () => { + it('separatedUntil: Parse items separated by a separator, up to a terminator.', () => { const result0 = separatedUntil( - char('a'), + digits, char(','), char(';'), - )('a,a,a;'); - expect(result0).toEqual({ - ok: true, - value: ['a', 'a', 'a'], - remaining: '', - }); + )('1,2,3;'); + expect(result0).toEqual({ ok: true, value: [1, 2, 3], index: 6 }); }); it('sequence: Parse a sequence of parsers and return all results as an array.', () => { @@ -527,23 +379,23 @@ describe('examples from source', () => { expect(result0).toEqual({ ok: true, value: ['a', 'b', 'c'], - remaining: '', + index: 3, }); }); it('skip: Skip a parser n times.', () => { const result0 = skip(char('a'), 2)('aabc'); - expect(result0).toEqual({ ok: true, value: null, remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: null, index: 2 }); }); it('skipMany: Skip zero or more occurrences (never fails, returns null).', () => { const result0 = skipMany(char('a'))('aaabc'); - expect(result0).toEqual({ ok: true, value: null, remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: null, index: 3 }); }); it('skipMany1: Skip one or more occurrences (fails if no matches).', () => { const result0 = skipMany1(char('a'))('aaabc'); - expect(result0).toEqual({ ok: true, value: null, remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: null, index: 3 }); }); it('surrounded: Parse content surrounded by delimiters.', () => { @@ -552,49 +404,43 @@ describe('examples from source', () => { string('hi'), char(']'), )('[hi]'); - expect(result0).toEqual({ ok: true, value: 'hi', remaining: '' }); + expect(result0).toEqual({ ok: true, value: 'hi', index: 4 }); const result1 = surrounded(char('a'), char('b'), char('c'))('abc'); - expect(result1).toEqual({ ok: true, value: 'b', remaining: '' }); + expect(result1).toEqual({ ok: true, value: 'b', index: 3 }); }); it('unless: Parse unless condition is true (inverse of guard).', () => { const result0 = unless(false, string('hello'))('hello'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 5 }); const result1 = unless(true, string('hello'))('hello'); - expect(result1).toEqual({ - ok: true, - value: null, - remaining: 'hello', - }); + expect(result1).toEqual({ ok: true, value: null, index: 0 }); }); - it('until: Parse until terminator matches (fails if terminator never matches).', () => { - const result0 = until(char('a'), char('b'))('baaa'); - expect(result0).toEqual({ ok: true, value: [], remaining: 'baaa' }); - const result1 = until(char('a'), char('b'))('aaba'); - expect(result1).toEqual({ + it('until: Parse zero or more until terminator matches, leaving the terminator unconsumed.', () => { + const result0 = until(char('a'), char('b'))('aaab'); + expect(result0).toEqual({ ok: true, - value: ['a', 'a'], - remaining: 'ba', + value: ['a', 'a', 'a'], + index: 3, }); }); it('validate: Validate parsed value with a predicate.', () => { const result0 = validate(digit, (n) => n > 5)('7'); - expect(result0).toEqual({ ok: true, value: 7, remaining: '' }); + expect(result0).toEqual({ ok: true, value: 7, index: 1 }); const result1 = validate(digit, (n) => n > 5)('3'); - expect(result1).toEqual({ ok: false }); + expect(result1).toEqual({ + ok: false, + index: 0, + expected: ['valid value'], + }); }); it('value: Replace parsed value with a constant.', () => { const result0 = value(string('true'), true)('true'); - expect(result0).toEqual({ ok: true, value: true, remaining: '' }); + expect(result0).toEqual({ ok: true, value: true, index: 4 }); const result1 = value(string('null'), null)('null'); - expect(result1).toEqual({ ok: true, value: null, remaining: '' }); + expect(result1).toEqual({ ok: true, value: null, index: 4 }); }); it('when: Branch on a boolean parser result.', () => { @@ -603,31 +449,66 @@ describe('examples from source', () => { pure('many'), pure('one'), )('*rest'); - expect(result0).toEqual({ - ok: true, - value: 'many', - remaining: 'rest', - }); + expect(result0).toEqual({ ok: true, value: 'many', index: 1 }); const result1 = when( flag(char('*')), pure('many'), pure('one'), )('abc'); - expect(result1).toEqual({ - ok: true, - value: 'one', - remaining: 'abc', - }); + expect(result1).toEqual({ ok: true, value: 'one', index: 0 }); }); }); describe('core', () => { - it('failure: Creates a failed result with an optional error message.', () => { - const result0 = failure('unexpected input'); - expect(result0).toEqual({ ok: false, error: 'unexpected input' }); + it('context: Creates a fresh context. {@link run} and {@link parse} make one per call.', () => { + const result0 = context(); + expect(result0).toEqual({ furthest: -1, expected: [] }); + }); + + it('context: them. Callers rely on that identity to skip allocating.', () => { + const result0 = union(['digit'], ['digit']); + expect(result0).toEqual(['digit']); + }); + + it('context: array every time without it being altered underneath.', () => { + const ctx = context(); + record(ctx, 3, ['digit']); + const result0 = ctx; + expect(result0).toEqual({ furthest: 3, expected: ['digit'] }); + }); + + it('error: 1-based line/column and the set of expectations at that offset.', () => { + const result0 = new ParseError('a=1,b=x', 6, ['digit']).line; + expect(result0).toEqual(1); + }); + + it('failure: later backtracked over.', () => { + const result0 = failure(undefined, 3, 'digit'); + expect(result0).toEqual({ + ok: false, + index: 3, + expected: ['digit'], + }); + }); + + it('failure: avoiding a fresh array on every failure.', () => { + const expected = ['digit']; + const result0 = reject(undefined, 3, expected); + expect(result0).toEqual({ + ok: false, + index: 3, + expected: ['digit'], + }); + }); + + it('format: Renders a parse failure as a source excerpt with a caret under the offset.', () => { + const result0 = format('a=1,b=x', 6, ['digit']); + expect(result0).toEqual( + "1:7 expected digit, found 'x'\n 1 | a=1,b=x\n | ^", + ); }); - it('grammar: Creates a recursive grammar where rules can reference each other.', () => { + it('grammar: reported by name rather than overflowing the stack.', () => { type Math = { expr: number; term: number; @@ -657,149 +538,135 @@ describe('examples from source', () => { expect(result2).toEqual(3); }); - it('label: Labels a parser with a custom error message on failure.', () => { + it('label: be suppressed rather than leaking into the message.', () => { const result0 = label(char('x'), 'letter x')(''); - expect(result0).toEqual({ ok: false, error: 'expected letter x' }); + expect(result0).toEqual({ + ok: false, + index: 0, + expected: ['letter x'], + }); }); it('lazy: Defers parser creation, useful for recursive grammars.', () => { const result0 = lazy(() => char('a'))('abc'); - expect(result0).toEqual({ ok: true, value: 'a', remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: 'a', index: 1 }); + }); + + it('locate: Converts a character offset into a 1-based line and column.', () => { + const result0 = locate('a=1\nb=x', 5); + expect(result0).toEqual({ index: 5, line: 2, column: 2 }); }); it('match: Pattern matching on a Result to handle success and failure cases.', () => { - const result0 = match(success('hello', ''), { + const result0 = match(success('hello', 5), { success: (v) => v, failure: () => 'failed', }); expect(result0).toEqual('hello'); }); - it('memoize: exponential backtracking.', () => { + it('memoize: error message rather than silently weakening it.', () => { const memoDigits = memoize(digits); const result0 = memoDigits('123'); - expect(result0).toEqual({ ok: true, value: 123, remaining: '' }); + expect(result0).toEqual({ ok: true, value: 123, index: 3 }); + }); + + it('parse: offset, line, column, expectations and a formatted message.', () => { + const result0 = parse(digits, '12x').ok; + expect(result0).toEqual(false); }); - it('parser: Creates a parser from a parser function.', () => { - const result0 = create((input) => - success('parsed', input.slice(6)), + it('parser: recorded inside your parser reach the final error message.', () => { + const result0 = create((input, index = 0) => + success('parsed', index + 6), )('hello world'); - expect(result0).toEqual({ - ok: true, - value: 'parsed', - remaining: 'world', - }); + expect(result0).toEqual({ ok: true, value: 'parsed', index: 6 }); }); - it('run: Runs a parser and returns the value, throws on failure or unconsumed input.', () => { + it('run: point — if the parse fails or leaves input unconsumed.', () => { const result0 = run(string('hello'), 'hello'); expect(result0).toEqual('hello'); }); - it('success: Creates a successful result with a value and remaining input.', () => { - const result0 = success('hello', ' world'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: ' world', - }); + it('success: Creates a successful result with a value and the offset reached in the input.', () => { + const result0 = success('hello', 5); + expect(result0).toEqual({ ok: true, value: 'hello', index: 5 }); }); }); describe('primitives', () => { it('alphaNum: Parse a single alphanumeric character.', () => { const result0 = alphaNum('a1'); - expect(result0).toEqual({ ok: true, value: 'a', remaining: '1' }); + expect(result0).toEqual({ ok: true, value: 'a', index: 1 }); const result1 = alphaNum('1a'); - expect(result1).toEqual({ ok: true, value: '1', remaining: 'a' }); + expect(result1).toEqual({ ok: true, value: '1', index: 1 }); }); - it('alphaNums: Parse one or more alphanumeric characters.', () => { - const result0 = alphaNums('abc123'); + it('alphaNums: Parse one or more alphanumeric characters as a string.', () => { + const result0 = alphaNums('placeholder'); expect(result0).toEqual({ ok: true, - value: 'abc123', - remaining: '', + value: 'placeholder', + index: 11, }); }); it('anyChar: Parse any single character.', () => { const result0 = anyChar('abc'); - expect(result0).toEqual({ ok: true, value: 'a', remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: 'a', index: 1 }); }); it('bool: Parse a boolean literal.', () => { const result0 = bool('true'); - expect(result0).toEqual({ ok: true, value: true, remaining: '' }); + expect(result0).toEqual({ ok: true, value: true, index: 4 }); const result1 = bool('false'); - expect(result1).toEqual({ ok: true, value: false, remaining: '' }); + expect(result1).toEqual({ ok: true, value: false, index: 5 }); const result2 = bool('trueABC'); - expect(result2).toEqual({ - ok: true, - value: true, - remaining: 'ABC', - }); + expect(result2).toEqual({ ok: true, value: true, index: 4 }); }); it('crlf: Parse CRLF line ending.', () => { const result0 = crlf('\r\nabc'); - expect(result0).toEqual({ - ok: true, - value: '\r\n', - remaining: 'abc', - }); + expect(result0).toEqual({ ok: true, value: '\r\n', index: 2 }); }); it('digit: Parse a single digit and return as number.', () => { const result0 = digit('5abc'); - expect(result0).toEqual({ ok: true, value: 5, remaining: 'abc' }); + expect(result0).toEqual({ ok: true, value: 5, index: 1 }); }); - it('digits: Parse one or more digits and return as number.', () => { - const result0 = digits('123abc'); - expect(result0).toEqual({ ok: true, value: 123, remaining: 'abc' }); + it('digits: Parse one or more digits and return as a number.', () => { + const result0 = digits('123'); + expect(result0).toEqual({ ok: true, value: 123, index: 3 }); }); - it('eof: Parse end of file (succeeds only on empty input).', () => { + it('eof: Match the end of the input.', () => { const result0 = eof(''); - expect(result0).toEqual({ ok: true, value: null, remaining: '' }); + expect(result0).toEqual({ ok: true, value: null, index: 0 }); }); it('eol: Parse end of line (\\n, \\r\\n, or EOF).', () => { const result0 = eol('\nabc'); - expect(result0).toEqual({ - ok: true, - value: '\n', - remaining: 'abc', - }); + expect(result0).toEqual({ ok: true, value: '\n', index: 1 }); }); it('float: Parse a floating point number.', () => { const result0 = float('1.23'); - expect(result0).toEqual({ ok: true, value: 1.23, remaining: '' }); + expect(result0).toEqual({ ok: true, value: 1.23, index: 4 }); const result1 = float('-2.5'); - expect(result1).toEqual({ ok: true, value: -2.5, remaining: '' }); + expect(result1).toEqual({ ok: true, value: -2.5, index: 4 }); const result2 = float('1.23abc'); - expect(result2).toEqual({ - ok: true, - value: 1.23, - remaining: 'abc', - }); + expect(result2).toEqual({ ok: true, value: 1.23, index: 4 }); }); it('hexDigit: Parse a single hexadecimal digit.', () => { const result0 = hexDigit('fF9'); - expect(result0).toEqual({ ok: true, value: 'f', remaining: 'F9' }); + expect(result0).toEqual({ ok: true, value: 'f', index: 1 }); }); - it('hexDigits: Parse one or more hexadecimal digits.', () => { + it('hexDigits: Parse one or more hex digits as a string.', () => { const result0 = hexDigits('deadbeef'); - expect(result0).toEqual({ - ok: true, - value: 'deadbeef', - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: 'deadbeef', index: 8 }); }); it('identifier: Parse an identifier — starts with letter or underscore, no leading digit, no hyphen.', () => { @@ -807,262 +674,204 @@ describe('examples from source', () => { expect(result0).toEqual({ ok: true, value: 'variable_name', - remaining: '', + index: 13, }); }); it('integer: Parse a signed integer.', () => { const result0 = integer('42'); - expect(result0).toEqual({ ok: true, value: 42, remaining: '' }); + expect(result0).toEqual({ ok: true, value: 42, index: 2 }); const result1 = integer('-7'); - expect(result1).toEqual({ ok: true, value: -7, remaining: '' }); + expect(result1).toEqual({ ok: true, value: -7, index: 2 }); const result2 = integer('123abc'); - expect(result2).toEqual({ ok: true, value: 123, remaining: 'abc' }); + expect(result2).toEqual({ ok: true, value: 123, index: 3 }); }); it('letter: Parse a single letter.', () => { const result0 = letter('abc'); - expect(result0).toEqual({ ok: true, value: 'a', remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: 'a', index: 1 }); }); - it('letters: Parse one or more letters.', () => { - const result0 = letters('abc123'); + it('letters: Parse one or more letters as a string.', () => { + const result0 = letters('placeholder'); expect(result0).toEqual({ ok: true, - value: 'abc', - remaining: '123', + value: 'placeholder', + index: 11, }); }); it('line: Parse until end of line.', () => { const result0 = line('hello\nworld'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: '\nworld', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 5 }); }); it('literal: Parse a word-like value including hyphens.', () => { const result0 = literal('foo-bar'); - expect(result0).toEqual({ - ok: true, - value: 'foo-bar', - remaining: '', - }); + expect(result0).toEqual({ ok: true, value: 'foo-bar', index: 7 }); const result1 = literal('123abc'); - expect(result1).toEqual({ - ok: true, - value: '123abc', - remaining: '', - }); + expect(result1).toEqual({ ok: true, value: '123abc', index: 6 }); }); it('lowercase: Parse a single lowercase letter.', () => { const result0 = lowercase('abc'); - expect(result0).toEqual({ ok: true, value: 'a', remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: 'a', index: 1 }); }); - it('lowercases: Parses one or more lowercase letters.', () => { - const result0 = lowercases('abcDEF'); + it('lowercases: Parse one or more lowercase letters as a string.', () => { + const result0 = lowercases('placeholder'); expect(result0).toEqual({ ok: true, - value: 'abc', - remaining: 'DEF', + value: 'placeholder', + index: 11, }); }); it('nl: Parse a newline character.', () => { const result0 = nl('\ntext'); - expect(result0).toEqual({ - ok: true, - value: '\n', - remaining: 'text', - }); + expect(result0).toEqual({ ok: true, value: '\n', index: 1 }); }); it('number: Parse an integer or float.', () => { const result0 = number('42'); - expect(result0).toEqual({ ok: true, value: 42, remaining: '' }); + expect(result0).toEqual({ ok: true, value: 42, index: 2 }); const result1 = number('3.14'); - expect(result1).toEqual({ ok: true, value: 3.14, remaining: '' }); + expect(result1).toEqual({ ok: true, value: 3.14, index: 4 }); const result2 = number('-7'); - expect(result2).toEqual({ ok: true, value: -7, remaining: '' }); + expect(result2).toEqual({ ok: true, value: -7, index: 2 }); const result3 = number('-2.5'); - expect(result3).toEqual({ ok: true, value: -2.5, remaining: '' }); + expect(result3).toEqual({ ok: true, value: -2.5, index: 4 }); }); it('octDigit: Parse a single octal digit.', () => { const result0 = octDigit('7abc'); - expect(result0).toEqual({ ok: true, value: '7', remaining: 'abc' }); + expect(result0).toEqual({ ok: true, value: '7', index: 1 }); }); - it('octDigits: Parse one or more octal digits.', () => { - const result0 = octDigits('0777abc'); - expect(result0).toEqual({ - ok: true, - value: '0777', - remaining: 'abc', - }); + it('octDigits: Parse one or more octal digits as a string.', () => { + const result0 = octDigits('755rest'); + expect(result0).toEqual({ ok: true, value: '755', index: 3 }); }); - it('position: Get current position (remaining input length).', () => { + it('position: {@link locate} to attach line/column information to a parsed node.', () => { const result0 = position('abc'); - expect(result0).toEqual({ ok: true, value: 3, remaining: 'abc' }); + expect(result0).toEqual({ ok: true, value: 0, index: 0 }); + const result1 = right(string('ab'), position)('abc'); + expect(result1).toEqual({ ok: true, value: 2, index: 2 }); }); - it('rest: Parse the rest of the input.', () => { - const result0 = rest('hello'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: '', - }); + it('rest: Consume and return everything left in the input.', () => { + const result0 = rest('abc'); + expect(result0).toEqual({ ok: true, value: 'abc', index: 3 }); }); it('space: Parse a single space character.', () => { const result0 = space(' abc'); - expect(result0).toEqual({ ok: true, value: ' ', remaining: 'abc' }); + expect(result0).toEqual({ ok: true, value: ' ', index: 1 }); }); - it('spaces: Parse one or more space characters.', () => { + it('spaces: Parse one or more spaces as a string.', () => { const result0 = spaces(' abc'); - expect(result0).toEqual({ - ok: true, - value: ' ', - remaining: 'abc', - }); + expect(result0).toEqual({ ok: true, value: ' ', index: 3 }); }); it('tab: Parse tab character.', () => { const result0 = tab('\ttext'); - expect(result0).toEqual({ - ok: true, - value: '\t', - remaining: 'text', - }); + expect(result0).toEqual({ ok: true, value: '\t', index: 1 }); }); it('uppercase: Parses a single uppercase letter.', () => { const result0 = uppercase('ABC'); - expect(result0).toEqual({ ok: true, value: 'A', remaining: 'BC' }); + expect(result0).toEqual({ ok: true, value: 'A', index: 1 }); }); - it('uppercases: Parses one or more uppercase letters.', () => { + it('uppercases: Parse one or more uppercase letters as a string.', () => { const result0 = uppercases('ABCdef'); - expect(result0).toEqual({ - ok: true, - value: 'ABC', - remaining: 'def', - }); + expect(result0).toEqual({ ok: true, value: 'ABC', index: 3 }); }); it('whitespace: Parses a single whitespace character.', () => { const result0 = whitespace(' abc'); - expect(result0).toEqual({ ok: true, value: ' ', remaining: 'abc' }); + expect(result0).toEqual({ ok: true, value: ' ', index: 1 }); }); - it('whitespaces: Parses one or more whitespace characters.', () => { - const result0 = whitespaces(' abc'); - expect(result0).toEqual({ - ok: true, - value: ' ', - remaining: 'abc', - }); + it('whitespaces: Parse one or more whitespaces as a string.', () => { + const result0 = whitespaces(' \t\nabc'); + expect(result0).toEqual({ ok: true, value: ' \t\n', index: 3 }); }); }); describe('terminals', () => { it('char: Parse a specific character.', () => { const result0 = char('A')('ABC'); - expect(result0).toEqual({ ok: true, value: 'A', remaining: 'BC' }); + expect(result0).toEqual({ ok: true, value: 'A', index: 1 }); }); it('charOf: Parse any character from a set.', () => { const result0 = charOf(['a', 'b', 'c'])('abc'); - expect(result0).toEqual({ ok: true, value: 'a', remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: 'a', index: 1 }); }); it('noneOf: Parse any character not in the set.', () => { const result0 = noneOf(['a', 'b', 'c'])('xyz'); - expect(result0).toEqual({ ok: true, value: 'x', remaining: 'yz' }); + expect(result0).toEqual({ ok: true, value: 'x', index: 1 }); }); it('oneOf: Parse one string from a set of strings (longest match wins).', () => { const result0 = oneOf(['hello', 'hell', 'help'])('helpful'); - expect(result0).toEqual({ - ok: true, - value: 'help', - remaining: 'ful', - }); + expect(result0).toEqual({ ok: true, value: 'help', index: 4 }); }); - it('regex: Parse with a regular expression.', () => { + it('regex: start and still carries meaning.', () => { const result0 = regex(/^\w+/)('hello world'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: ' world', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 5 }); }); it('satisfy: Parse a character satisfying a predicate.', () => { const result0 = satisfy((c) => c === 'a')('abc'); - expect(result0).toEqual({ ok: true, value: 'a', remaining: 'bc' }); + expect(result0).toEqual({ ok: true, value: 'a', index: 1 }); }); it('string: Parse a specific string.', () => { const result0 = string('hello')('hello world'); - expect(result0).toEqual({ - ok: true, - value: 'hello', - remaining: ' world', - }); + expect(result0).toEqual({ ok: true, value: 'hello', index: 5 }); }); it('stringOf: Parse first character that exists in string (like charOf but for a string).', () => { const result0 = stringOf('abc')('abcdef'); - expect(result0).toEqual({ - ok: true, - value: 'a', - remaining: 'bcdef', - }); + expect(result0).toEqual({ ok: true, value: 'a', index: 1 }); }); it('take: Take n characters.', () => { const result0 = take(3)('abcdef'); - expect(result0).toEqual({ - ok: true, - value: 'abc', - remaining: 'def', - }); + expect(result0).toEqual({ ok: true, value: 'abc', index: 3 }); }); it('takeWhile: Takes characters while the predicate returns true.', () => { const result0 = takeWhile((c) => c !== 'x')('abcx'); - expect(result0).toEqual({ ok: true, value: 'abc', remaining: 'x' }); + expect(result0).toEqual({ ok: true, value: 'abc', index: 3 }); }); it('token: strings that might be followed by any character.', () => { const result0 = token('let')('let x'); - expect(result0).toEqual({ ok: true, value: 'let', remaining: 'x' }); + expect(result0).toEqual({ ok: true, value: 'let', index: 4 }); const result1 = token('let')('let1'); - expect(result1).toEqual({ ok: true, value: 'let', remaining: '1' }); + expect(result1).toEqual({ ok: true, value: 'let', index: 3 }); const result2 = token('let')('let x'); - expect(result2).toEqual({ ok: true, value: 'let', remaining: 'x' }); + expect(result2).toEqual({ ok: true, value: 'let', index: 5 }); }); it('word: Use this when parsing keywords that should not be part of a longer identifier.', () => { const result0 = word('let')('let x'); - expect(result0).toEqual({ ok: true, value: 'let', remaining: 'x' }); + expect(result0).toEqual({ ok: true, value: 'let', index: 4 }); const result1 = word('let')('let1'); - expect(result1).toEqual({ ok: false }); - const result2 = word('if')('if (x)'); - expect(result2).toEqual({ - ok: true, - value: 'if', - remaining: '(x)', + expect(result1).toEqual({ + ok: false, + index: 3, + expected: ['/(?!\\w)/'], }); + const result2 = word('if')('if (x)'); + expect(result2).toEqual({ ok: true, value: 'if', index: 3 }); }); }); @@ -1097,11 +906,7 @@ describe('examples from source', () => { it('pipe: Pipe parser functions together.', () => { const result0 = pipe(lexeme)(letters)('xyz abc'); - expect(result0).toEqual({ - ok: true, - value: 'xyz', - remaining: 'abc', - }); + expect(result0).toEqual({ ok: true, value: 'xyz', index: 6 }); }); it('pop: Get the last element of an array.', () => { diff --git a/test/helpers.ts b/test/helpers.ts index 52e66a3..f52cb7c 100644 --- a/test/helpers.ts +++ b/test/helpers.ts @@ -1,25 +1,29 @@ import { create, failure, success } from '../src'; -export const operation = create((input) => { - const ops: Record number> = { - '+': (left, right) => left + right, - '-': (left, right) => left - right, - '*': (left, right) => left * right, - '/': (left, right) => left / right, - '**': (left, right) => Math.pow(left, right), - }; +const OPERATORS: Record number> = { + '+': (left, right) => left + right, + '-': (left, right) => left - right, + '*': (left, right) => left * right, + '/': (left, right) => left / right, + '**': (left, right) => Math.pow(left, right), +}; - const [operator] = input.match(/\*\*|[+\-*/]/) || []; - if (!operator) return failure(); +const OPERATOR = /\*\*|[+\-*/]/y; - const op = ops[operator]; +export const operation = create((input, index = 0) => { + OPERATOR.lastIndex = index; + const [operator] = OPERATOR.exec(input) || []; + if (!operator) return failure(undefined, index, 'operator'); - return success(op, input.slice(operator.length)); + return success(OPERATORS[operator], index + operator.length); }); -export const digits = create((value: string) => { - const [, match, rest] = value.match(/^(\d+)(.*)/) || ['0']; - if (!match) return failure(); +const DIGITS = /\d+/y; - return success(parseInt(match), rest); +export const digits = create((input, index = 0) => { + DIGITS.lastIndex = index; + const [match] = DIGITS.exec(input) || []; + if (!match) return failure(undefined, index, 'digit'); + + return success(parseInt(match), index + match.length); }); diff --git a/test/utils.ts b/test/utils.ts index d158420..25b4bf9 100644 --- a/test/utils.ts +++ b/test/utils.ts @@ -6,35 +6,54 @@ import { type Result } from '../src/core/result'; import { success } from '../src/core/success'; export const createTestParser = (tester: T) => { - return create((input) => { - const stringTester = String(tester); - if (input.startsWith(stringTester)) { - return success(tester, input.slice(stringTester.length)); + const stringTester = String(tester); + + return create((input, index = 0) => { + if (input.startsWith(stringTester, index)) { + return success(tester, index + stringTester.length); } - return failure(); + return failure(undefined, index, stringTester); }); }; +/** + * Asserts a parser succeeded with a value, having consumed up to `index`. + * + * The furthest-failure trace is deliberately not asserted: it is bookkeeping + * for error messages, not part of a parser's contract. + */ export const assertSuccess = ( result: Result, value: T, - remaining: string, + index: number, ) => { assertType>(result); - expect(result).toEqual({ - ok: true, - value, - remaining, - }); + expect(result.ok).toBe(true); + expect((result as { value: T }).value).toEqual(value); + expect(result.index).toBe(index); }; -export const assertFailure = (result: Result, error?: string) => { +/** + * Asserts a parser failed, optionally at a given offset and with given + * expectations. + */ +export const assertFailure = ( + result: Result, + index?: number, + expected?: readonly string[], +) => { assertType>(result); - expect(result).toEqual({ - ok: false, - error, - }); + expect(result.ok).toBe(false); + + if (index !== undefined) { + expect(result.index).toBe(index); + } + + if (expected !== undefined) { + const actual = (result as { expected: readonly string[] }).expected; + expect([...actual].sort()).toEqual([...expected].sort()); + } }; diff --git a/tsdown.config.ts b/tsdown.config.ts index c54b2dc..8ac7720 100644 --- a/tsdown.config.ts +++ b/tsdown.config.ts @@ -1,34 +1,22 @@ import { defineConfig } from 'tsdown'; -export default defineConfig([ - { - entry: { index: './src/index.ts' }, - format: ['esm'], - minify: true, - dts: true, +/** + * One build with five entries rather than five separate builds. + * + * Every entry point shares the core types and constructors. Built separately, + * each bundle inlines its own copy, so importing from more than one entry — + * which is the normal way to use this library — ships that code several times + * over. A single build emits it once as a shared chunk instead. + */ +export default defineConfig({ + entry: { + index: './src/index.ts', + combinators: './src/combinators/index.ts', + terminals: './src/terminals/index.ts', + primitives: './src/primitives/index.ts', + utils: './src/utils/index.ts', }, - { - entry: { combinators: './src/combinators/index.ts' }, - format: ['esm'], - minify: true, - dts: true, - }, - { - entry: { terminals: './src/terminals/index.ts' }, - format: ['esm'], - minify: true, - dts: true, - }, - { - entry: { primitives: './src/primitives/index.ts' }, - format: ['esm'], - minify: true, - dts: true, - }, - { - entry: { utils: './src/utils/index.ts' }, - format: ['esm'], - minify: true, - dts: true, - }, -]); + format: ['esm'], + minify: true, + dts: true, +});