The book is the source of the program. Edit its explanations and code fragments together, then regenerate the TypeScript and HTML. VS Code is useful for browsing fragments, but it is not required to build or verify this repository.
Use Node.js 22 LTS and the Yarn version pinned in package.json. Enable Corepack
if necessary, then install the locked dependencies:
yarn install --immutable
yarn build:cli
The checked-in generated TypeScript is the bootstrap: build:cli bundles only
the CLI's local modules into out/cli-entry.js, using installed packages at runtime.
It does not type-check or compile
the extension, so an extension error does not prevent generation. No global
TypeScript compiler or published Literate extension is required. .gitattributes
keeps text checkouts in LF form, matching the engine's deterministic output even
on Windows. Output names follow a portable path policy: case-only collisions,
Windows device names, and ambiguous trailing dots or spaces are rejected.
The shared engine is explained in Headless generation engine, the
filesystem adapter in Command-line generation, and the editor adapter
in Literate Programming. Change those .literate chapters, not
src/engine.ts, src/cli.ts, src/self-hosting.ts, or src/extension.ts
directly. The grabber is also generated. Tests, package configuration, templates, and styles remain ordinary
source files.
yarn literate generate
yarn test
yarn esbuild
yarn literate bundles the currently checked-in/generated CLI before running it. generate writes source and HTML only when their bytes change.
yarn test compiles the new implementation, lints it, runs headless regression
tests, and checks that generation agrees with all the files on disk.
yarn esbuild bundles the extension's actual entry point, out/main.js.
Review the generated diff as well as the chapter changes. Prose-only edits should change HTML but not TypeScript. A reorganization intended to preserve the program should not alter its generated source.
yarn literate check
The check command regenerates in memory and reports missing or stale files. It
never updates the book or generated outputs. This is also available as
yarn check:generated. The optional final argument selects another project root:
yarn literate generate /path/to/a/test-project
yarn literate check /path/to/a/test-project
Both source and HTML outputs are checked. Orphaned output files are not detected or deleted: removing a root fragment or renaming an output requires an explicit review of the old file.
A generator change needs two generations. First the previous executable produces the new source. Rebuild it and verify that the new executable reproduces that source and its documentation:
yarn literate generate
yarn literate self-check
yarn test
self-check requires current outputs, type-checks freshly generated TypeScript
through an in-memory overlay, then builds and runs a fresh engine in a temporary
child process. Its output paths and bytes must match the current generation plan.
It does not write project files or silently repair stale outputs. This is an
explicit engine fixed-point check, not a replacement for CLI runtime tests.
Use it only on trusted code; the child is not a security sandbox.
If the new generator deliberately changes formatting, the check will report the
old output. Run yarn literate generate again, review the differences, then repeat
the check. Keep generated TypeScript committed so the next checkout can bootstrap.
If newly generated CLI code fails to build, keep using a known-working executable
with node out/cli-entry.js generate to regenerate after fixing the literate source.
Do not repair generated TypeScript by hand. Keep a separate known-good CLI bundle
before risky generator syntax changes; out/cli-entry.js can also be overwritten by
compilation, so it is not a permanent backup.
For syntax changes, retain compatibility long enough to generate the new parser before migrating the book. During this initial extraction the generated engine was bootstrapped once; subsequent changes use the same literate-first workflow.
yarn literate inspect --json
yarn literate check --json
yarn literate self-check --json
Inspection lists ordered input documents, referenced templates, and each fragment's
output, definitions, and dependencies. Locations refer to the literate sources,
not generated files. JSON responses use schema version 1 and zero-based positions;
usage errors and I/O failures are also structured. Inspection and checks never
write project output. The yarn literate wrapper does rebuild out/cli-entry.js; invoke
node out/cli-entry.js directly when the whole invocation must avoid project writes.
A missing fragment, a cycle, an invalid declaration, or an unsafe output path is an error. The engine returns diagnostics and no output files, preserving the last successful generated program. Fragments are validated even when unused. Output paths must remain within the project, and outputs cannot overwrite inputs or collide with one another.
A literal opening delimiter inside a code fragment is written as \<<. It becomes
<< in generated source and documentation, without triggering expansion. This
allows the generator to describe and implement its own fragment syntax without
special-case exceptions. Unescaped fragment references are expanded even inside
strings or comments; this is a text generator, not a programming-language parser.
The CLI reads saved files. The VS Code adapter uses open editor buffers, including empty buffers, and automatically generates on literate content changes. It shares the engine with the CLI; completion refreshes do not write files. Template-only editor edits also regenerate projects referencing that source or HTML template, including templates with custom filenames. Saved literate inputs and referenced template creation, changes, and deletion trigger a per-workspace debounced refresh; open buffers still take precedence. Generated outputs and vendor directories do not trigger regeneration. Do not run CLI generation concurrently with editor generation against the same project or leave unsaved literate buffers open while checking disk-based results.
Both adapters preflight every output in a project before any output directory
creation or writes. The CLI uses lstat; VS Code uses workspace.fs.stat and,
when a component appears missing, its parent's readDirectory entries to detect
dangling links. Below the trusted workspace root, symbolic-link leaves and
ancestors, non-directory parents, and non-file targets (including unknown types)
are rejected. VS Code checks all outputs before reading their contents, including
outputs that would otherwise be unchanged. Read-only editor refreshes do not
preflight or write outputs. Permission errors propagate rather than being treated
as missing files. Remote workspaces use their filesystem provider, not Node's
local filesystem.
Both adapters now use the shared output transaction coordinator. It stages every changed file before installation, backs up originals, and attempts reverse-order rollback on failure. Successful rollback restores original bytes and removes new outputs and owned empty directories. Local permission bits are preserved; remote providers do not expose portable permission metadata. A failed rollback preserves recovery material and reports its paths. A post-commit cleanup failure explicitly reports that the batch was committed; do not mistake it for a rolled-back batch.
This safety depends on the provider reporting symbolic links in file types;
VS Code has no portable no-follow write API. Concurrent filesystem changes can
race preflight (TOCTOU). The transaction has no durable journal, crash recovery,
concurrency lock, or simultaneous atomic visibility across all files. A power loss
can interrupt a batch; preserve remaining .literate-txn-* directories for manual
inspection. The CLI also skips symlink inputs; output preflight does not establish
an equivalent input policy for the VS Code provider. See the
output transaction chapter for exact failure semantics.
Headless tests exercise the engine, CLI, and a mocked VS Code adapter without a GUI or downloads. They do not replace testing the real extension host.
Disable the published extension in this repository: older generators do not understand the explicit literal-delimiter escapes used by the new chapters. Bootstrap with the CLI, then debug the newly built extension.
For interactive debugging, select Run Extension and press F5. The default
build task generates the book sources, compiles TypeScript, and builds both the
extension and browser preview bundles before launch. It terminates normally; no
separate watcher is required. Use a disposable test workspace and associate
*.literate with Markdown for editor providers.
yarn test:extension builds the extension and invokes the existing VS Code test
runner. That runner may download VS Code and requires an appropriate GUI/display
environment. Its original sample test is not a substitute for the headless
regression suite or an interactive smoke test.