Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "lest"
version = "0.5.0"
version = "0.6.0-canary.2"
edition = "2021"
description = "lest: a testing toolchain for Luau"
license = "MIT"
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ neither knows nor cares where a test ran.
| `lute` | A spawned `lute run` process | Scripts using `@lute/*`, code transforms, tooling | — | ✅ |
| `cloud` | A real Roblox place via Open Cloud | Instances, services, the DataModel — in CI | — | — |
| `studio` | A launched Roblox Studio | The same engine code, locally, zero clicks | — | — |
| `gargantuan` | A spawned headless [Gargantuan](https://github.com/teamfireworks/gargantuan) engine | Gargantuan engine code (experimental) | — | — |

```toml
[suites.unit]
Expand Down
38 changes: 38 additions & 0 deletions docs/backends.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ CI output — neither knows nor cares where a test ran.
| [`lune`](#lune--lute) | A spawned `lune run` process | — | ✅ |
| [`lute`](#lune--lute) | A spawned `lute run` process | — | ✅ |
| [`cloud`](#cloud) | A real Roblox place via Open Cloud | — | — |
| [`studio`](#studio) | A launched Roblox Studio | — | — |
| [`gargantuan`](#gargantuan-experimental) | A spawned headless Gargantuan engine | — | — |

**No backend fakes an environment.** Nothing mocks Instances, and nothing
reimplements a runtime's standard library. If a test needs an environment, Lest
Expand Down Expand Up @@ -250,6 +252,42 @@ matters the moment a spec asks `IsServer()` or touches DataStores
refuses to run under `$CI`, and watch mode does not include it. Details and
troubleshooting: **[Studio](studio.md)**.

## gargantuan (experimental)

[Gargantuan](https://github.com/teamfireworks/gargantuan) is an independent,
Roblox-shaped game engine scripted with Luau — a `game` DataModel,
Instances, Signals, a `task` library. The gargantuan backend runs specs
inside the real engine, headless: Lest bundles the suite (the same bundler
as cloud and studio), spawns `gargantuan --script <bundle> --headless`, and
decodes sentinel-framed events from its stdout with the lune/lute decoder (in bursts rather than live — the engine never flushes `print`).
No emulation, per the usual rule — specs get the engine's actual Instances
because they genuinely run in it.

```toml
[suites.engine-gg]
include = ["tests/gargantuan/**/*.spec.luau"]
backend = "gargantuan"
default = false

[gargantuan]
binary = "vendor/gargantuan/build/gargantuan"
```

**Experimental, stated plainly.** The engine is pre-release: it has no
tagged releases (build it from source and point `[gargantuan] binary` at
the result — with no `binary` configured, Lest looks for `gargantuan` on
`PATH`), and an API surface that is still filling in, so specs will find
`not yet implemented` edges. Those are engine facts, not test failures.
Excluded from watch mode and from `$CI` auto-enable; run it by naming the
suite explicitly.

How a run ends depends on the engine build: on engines with
`ProcessService`, Lest's generated entrypoint exits the engine cleanly
(`ExitAsync(0)`) once the suite completes; on builds that predate the
service (or whose `ExitAsync` fails), Lest kills the engine a few
seconds after the suite's completion marker arrives — deliberate, not an
error.

## Overriding a backend

`--backend` forces every selected suite onto one backend for a single run:
Expand Down
40 changes: 21 additions & 19 deletions docs/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ than accepted and ignored.
#### `--backend <BACKEND>`

Force every selected suite onto one backend for this run: `native`, `lune`,
`lute`, `cloud`, or `studio`. A debugging override — see
`lute`, `cloud`, `studio`, or `gargantuan`. A debugging override — see
[Backends](backends.md#overriding-a-backend).

### Watch mode
Expand All @@ -86,8 +86,8 @@ $ lest run unit --watch
Watch mode is graph-driven, not glob-driven. Saving a file invalidates its
content hash; the inverted require graph then yields exactly the specs whose
transitive requires touched it, and only those re-run. Watching is debounced,
hidden files are ignored (except `.luaurc`, which affects resolution), and
`cloud` and `studio` suites are always excluded — the fast loop never waits on a network round-trip or a Studio boot.
hidden files are ignored (except `.luaurc` and `.config.luau`, which affect resolution), and
`cloud`, `studio`, and `gargantuan` suites are always excluded — the fast loop never waits on a network round-trip or an engine boot.

At startup and after every pass, a dim banner marks the loop as alive:

Expand All @@ -111,11 +111,11 @@ $ lest run unit -t 'parser' --watch

#### `--reporter <REPORTER>`

| Reporter | Output |
| --- | --- |
| Reporter | Output |
| -------- | ---------------------------------------------------------------- |
| `pretty` | **Default.** Nested suites, inline diffs, slowest tests, summary |
| `json` | The event log, one JSON object per line |
| `junit` | JUnit XML, for CI annotations |
| `json` | The event log, one JSON object per line |
| `junit` | JUnit XML, for CI annotations |

All reporters consume the same merged stream regardless of which backend
produced it, tagging each suite's section with the environment it ran in.
Expand Down Expand Up @@ -183,7 +183,9 @@ Re-running is safe: an existing `lest.toml` prompts
untouched; with `--yes` init refuses and exits 2), and an alias already bound
to `lest` in `.luaurc` is left alone. `.luaurc` is only rewritten when it
parses as plain JSON with no comments; otherwise init prints the snippet for
you to paste. Key order is preserved.
you to paste. Key order is preserved. A project using `.config.luau` never
gains a `.luaurc` (the Luau config RFC makes the pair an error) — init prints
the alias line to paste into it instead.

See [Getting started](getting-started.md#scaffold-a-project).

Expand All @@ -200,11 +202,11 @@ being baked into whatever they expanded to that day.

## Exit codes

| Code | Meaning |
| --- | --- |
| `0` | Everything passed |
| `1` | Test failures — including timeouts, spec load errors, and a `--min` shortfall |
| `2` | Tool error — bad config, a backend that couldn't start, an undecodable event stream, a filter or coverage gate over nothing |
| Code | Meaning |
| ---- | --------------------------------------------------------------------------------------------------------------------------- |
| `0` | Everything passed |
| `1` | Test failures — including timeouts, spec load errors, and a `--min` shortfall |
| `2` | Tool error — bad config, a backend that couldn't start, an undecodable event stream, a filter or coverage gate over nothing |

These are never conflated, which is the whole point. A test that times out or a
spec that fails to load is a *test* failure (1). A backend that can't start, a
Expand All @@ -214,11 +216,11 @@ be a lie.

## Environment variables

| Variable | Effect |
| --- | --- |
| `ROBLOX_API_KEY` | Open Cloud API key for `cloud` suites |
| `LEST_API_KEY` | Alternative name for the same key |
| `CI` | When set (and not empty, `0`, or `false`), suites with `default = false` run automatically |
| `NO_COLOR` | When set, disables ANSI color everywhere, same as `--no-color` |
| Variable | Effect |
| ---------------- | ------------------------------------------------------------------------------------------ |
| `ROBLOX_API_KEY` | Open Cloud API key for `cloud` suites |
| `LEST_API_KEY` | Alternative name for the same key |
| `CI` | When set (and not empty, `0`, or `false`), suites with `default = false` run automatically |
| `NO_COLOR` | When set, disables ANSI color everywhere, same as `--no-color` |

A `.env` file at the project root is loaded automatically.
4 changes: 2 additions & 2 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ never re-run on save — not running it at all is the consistent reading.
Where this suite's specs run, overriding the top-level default. See
[Backends](backends.md).

- **Type:** `"native"` · `"lune"` · `"lute"` · `"cloud"` · `"studio"`
- **Type:** `"native"` · `"lune"` · `"lute"` · `"cloud"` · `"studio"` · `"gargantuan"`
- **Default:** `[settings] backend`

### `default`
Expand Down Expand Up @@ -167,7 +167,7 @@ file at the project root. See [Backends → cloud](backends.md#cloud).

The default backend for suites that don't declare one.

- **Type:** `"native"` · `"lune"` · `"lute"` · `"cloud"` · `"studio"`
- **Type:** `"native"` · `"lune"` · `"lute"` · `"cloud"` · `"studio"` · `"gargantuan"`
- **Default:** `"native"`

Precedence for where a suite runs: the `--backend` CLI flag, then the
Expand Down
6 changes: 5 additions & 1 deletion docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,11 @@ return nil
Three things to notice:

- **`require('@lest')`** works because `lest init` wrote that alias into
`.luaurc`. If you declined it, require the framework by path instead —
`.luaurc`. In a project configured with `.config.luau`, init prints the
alias line to paste instead (aliases resolve from either format) — and
until you paste it, the generated example requires core by relative path.
If you declined the alias,
require the framework by path instead —
`require('../.lest/core')`, adjusted for where the spec sits.
- **The destructuring line** (`local describe, it, expect = ...`) is
deliberate. Lest has no ambient globals: your specs require real `.luau`
Expand Down
126 changes: 126 additions & 0 deletions luau/runtime/encode.luau
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
--!strict
--- Pure-Luau JSON encoding for sentinel-framed protocol events, embedded into
--- bundles that run where no engine JSON service exists (gargantuan has no
--- HttpService). This is the twin of the spawned-runtime harness template's
--- inline encoder — same rules, same degradations, kept in step by hand: the
--- harness must stay a single self-contained file, so it cannot require this
--- module.

--- Replaces every byte that is not part of a valid UTF-8 sequence with U+FFFD
--- (the replacement character). Luau strings are byte arrays, but this stream
--- ends in a real JSON decoder: a raw invalid byte makes the sentinel line
--- undecodable, and one bad message becomes a tool error for the whole run.
--- The invariant, shared with the cloud sanitizer: degrade one string, never
--- the run. Valid multi-byte UTF-8 passes through untouched.
local REPLACEMENT = utf8.char(0xFFFD)

local function toValidUtf8 (s: string): string
-- Fast path: `utf8.len` validates the whole string in one pass.
if utf8.len(s) ~= nil then
return s
end
local parts = {}
local index = 1
while index <= #s do
local remaining, invalidAt = utf8.len(s, index)
if remaining ~= nil then
-- Everything from `index` to the end is valid.
table.insert(parts, string.sub(s, index))
break
end
-- Bytes before the reported position are valid; the byte there is not,
-- so it degrades alone and scanning resumes right after it.
local bad = invalidAt :: number
if bad > index then
table.insert(parts, string.sub(s, index, bad - 1))
end
table.insert(parts, REPLACEMENT)
index = bad + 1
end
return table.concat(parts)
end

local function escapeJson (s: string): string
s = toValidUtf8(s)
return (
s:gsub('[%c"\\]', function (c: string): string
if c == '"' then
return '\\"'
elseif c == '\\' then
return '\\\\'
elseif c == '\n' then
return '\\n'
elseif c == '\r' then
return '\\r'
elseif c == '\t' then
return '\\t'
end
return string.format('\\u%04x', string.byte(c))
end)
)
end

local function encodeJson (value: any): string
local t = type(value)
if t == 'nil' then
return 'null'
elseif t == 'boolean' then
return tostring(value)
elseif t == 'number' then
-- JSON has no NaN or Infinity. On the cloud path a Luau table cannot
-- hold nil, so `null` is unrepresentable there; `0` is the substitute
-- every wire path can produce, so both paths emit `0` and backends
-- agree on the shape of the same pathological event.
if value ~= value or value == math.huge or value == -math.huge then
return '0'
end
return tostring(value)
elseif t == 'string' then
return '"' .. escapeJson(value) .. '"'
elseif t == 'table' then
-- A table is an array only when its keys are exactly 1..#value: the
-- ipairs walk below would silently drop any other key, so a mixed table
-- (`{1, 2, foo = 'bar'}`) encodes as an object instead. An empty table
-- encodes as an empty array — protocol objects always carry at least a
-- `kind`/`type` field, so empty can only mean an empty array (e.g. a
-- top-level test's path).
local arrayLength = #value
local keyCount = 0
local contiguous = true
for key in pairs(value) do
keyCount += 1
if type(key) ~= 'number' or key ~= math.floor(key) or key < 1 or key > arrayLength then
contiguous = false
end
end
if keyCount == 0 or (contiguous and keyCount == arrayLength) then
local parts = {}
for _, item in ipairs(value) do
table.insert(parts, encodeJson(item))
end
return '[' .. table.concat(parts, ',') .. ']'
end
-- Object: every key stringified (JSON objects hold only string keys).
-- Two distinct keys can stringify to the same text (1 and '1'), so a
-- collision takes a ` #n` suffix rather than emitting duplicate JSON
-- keys — the same rule as the cloud sanitizer.
local parts = {}
local taken: { [string]: boolean } = {}
for key, item in pairs(value) do
local keyString = tostring(key)
if taken[keyString] then
local suffix = 2
while taken[`{keyString} #{suffix}`] do
suffix += 1
end
keyString = `{keyString} #{suffix}`
end
taken[keyString] = true
table.insert(parts, '"' .. escapeJson(keyString) .. '":' .. encodeJson(item))
end
return '{' .. table.concat(parts, ',') .. '}'
end
return '"<' .. t .. '>"'
end

return { value = encodeJson }
Loading
Loading