Runway

Static HTML previews of EDSL human-survey questions.

Runway turns a saved EDSL survey into a self-contained HTML page that looks like the live web survey a respondent would see — same markup, same stylesheet, same font. No browser, no node, no server: a file in, a file out.

Scope today: the choice family — multiple_choice, likert_five, yes_no and linear_scale — plus matrix, checkbox, checkbox_with_other, free_text and survey_message. Every other type renders a "No preview is available" notice. Types are added one at a time because each is held to byte-for-byte agreement with the live survey's own components — see SPEC.md.

Copy and paste into a coding agent

# runway — static HTML previews of EDSL human surveys

Turns an EDSL survey into a self-contained HTML page that looks like the live
web survey a respondent would see. No browser, no node, no server.

## Install

Not on PyPI (it pins edsl to a git branch, which PyPI rejects), so install from
git. Needs Python >=3.10, git, and network access.

```bash
uv tool install git+https://github.com/expectedparrot/runway.git   # CLI
uv add git+https://github.com/expectedparrot/runway.git            # as a dep
pip install git+https://github.com/expectedparrot/runway.git
```

## Input

A survey saved by edsl. `.ep`, `.json.gz` and `.json` all work and all give the
same preview. The humanize schema, if there is one, is separate — a survey
carries none.

```python
import json
from pathlib import Path

survey.save("survey")                                    # -> survey.ep
Path("schema.json").write_text(json.dumps(humanize_schema, indent=2))
```

Do not hand-write a survey file. A bare list of question dicts, or a document
that stubs `memory_plan` or `rule_collection`, is refused.

## Commands

A verb is always required.

```bash
runway check  survey.ep --schema schema.json   # what each question will render as
runway render survey.ep --schema schema.json   # write the HTML
runway types                                   # which types have a control
runway guide                                   # what it does and cannot do
runway version
```

`render` writes `./previews/<survey>.html` — one file, all questions, with a
toolbar to move between the pages. `-o DIR` to change it, `--split` for one file
per page (much larger; the stylesheet is re-inlined each time). Several surveys
may be given at once, so long as their names differ.

## Scenarios

A humanized survey can be bound to a scenario list, and a respondent is assigned
**one scenario for their whole response** — so each scenario is a different
rendering of the survey, and previewing one is previewing a page somebody is
actually served.

```bash
runway render survey.ep --scenarios scenarios.ep
runway render survey.ep --scenarios scenarios.ep --scenario-index 0,17,204
```

The bundle gains a second dropdown beside the question one; picking a scenario
keeps the question you are on. A scenario list opens from the same three formats
a survey does, through `ScenarioList.load()`.

Both spellings resolve — `{{ city }}` and `{{ scenario.city }}` — as do piped
`question_options` and matrix `question_items`.

**A question that pipes nothing renders once** and is shown for every scenario,
so the file grows only where the survey actually differs. A twenty-question
survey with three piped questions, under ten scenarios, is 17 + 30 panels rather
than 200.

The indices are the scenario list's own and stay so in the dropdown, since that
is what the live survey identifies a scenario by. A list of more than 25 is
refused rather than truncated — pick from it with `--scenario-index`.

## Question groups

A survey whose schema says `{"survey": {"presentation": "group"}}` is served a
whole question group per page, and previews that way: one panel per group, one
file per group under `--split`, progress read from where the page begins. The
groups come from the survey file and `presentation` from `--schema`; one without
the other pages per question, as the live survey does.

`.edsl-survey-item` wraps each item of a page — the `custom_css` hook for
separating them, since nothing separates them by default.

A question in no group is never served. Its page is drawn all the same, with the
question's text and a warning in place of the control, and `check` reports it as
a warning. `examples/src/group_survey.py` is the worked example.

`check` writes nothing and is the fast way to see what you'll get:

```
mixed_survey.json  -  7 items

  drawn      commute_mode          multiple_choice
  drawn      commute_time          multiple_choice
  drawn      commute_enjoyment     likert_five
  drawn      commute_satisfaction  linear_scale
  drawn      commute_switch        yes_no
  note       commute_barriers      rank    (no preview built for this type yet)
  warning    commute_breakdown     dict    (never shown to a respondent)

5 drawn, 1 note, 1 warning
```

Four outcomes, and they mean different things:

- `drawn` — the real control; the preview is accurate.
- `automatic` — `compute`, `image_generation`, or any type wrapped by
  `thinking_question()`. Nobody is ever asked it; nothing is missing.
- `note` — a type a human survey supports but runway hasn't transcribed yet.
  The survey is fine; the tool is behind.
- `warning` — nothing a respondent could be served: a type with no
  human-survey rendering anywhere, or a question no page carries because it is
  in no question group. Fix the survey.

`check` exits 1 on `warning`, and on a file it cannot read. `--json` on
`check`, `types` and `version`.

## Drawn today (9)

`multiple_choice`, `yes_no`, `likert_five`, `linear_scale`, `matrix`,
`checkbox`, `checkbox_with_other`, `free_text`, `survey_message`. Everything
else renders a complete page with a note where the control would be — so wording
and position are still checkable.

## Caveats

- Scenario values resolve with `--scenarios`; without it they render as
  written. Agent traits and prior answers render as written — except the answer
  of an `image_generation` question, which draws a placeholder picture, since
  that type answers with an image and nothing else.
- A matrix carousel (`format: {type: carousel}`) is drawn, but you cannot swipe
  it — use the arrows, and try it on a phone before sending the survey out.
- Position is inferred from authored order, so skip logic will differ.
- Controls tick but mostly don't behave. Checkbox Select-all and exclusive
  options do work; validation, limits and Next do not.
- Media in question text and in option labels is drawn — an image, video or
  PDF held in a scenario is inlined into the page, so the file's bytes are in
  the HTML.
- Math (`$$…$$`) shows as the TeX source; the live page typesets it.
- A logo (`survey.branding.logo`) draws as a placeholder. `render
  --fetch-assets` fetches it from Coop with your API key, embeds it, and keeps
  a copy in `<out>/assets/` so the next render there needs no network.

Install

Not on PyPI — it pins edsl to a git branch, which PyPI rejects — so install from git. Needs Python 3.10+, git, and network access.

uv tool install git+https://github.com/expectedparrot/runway.git   # the CLI
uv add git+https://github.com/expectedparrot/runway.git            # as a dependency
pip install git+https://github.com/expectedparrot/runway.git

Or from a checkout, which is also how to work on it:

git clone https://github.com/expectedparrot/runway.git
cd runway
uv sync                 # --extra dev adds pytest and ruff
uv run runway check examples/mixed_survey.json

Three runtime dependencies: Jinja2, markdown-it-py[linkify] and edsl.

Usage

runway check  survey.ep       # what each question will render as
runway render survey.ep       # write the HTML
runway types                  # which question types have a control
runway guide                  # what this does, and what a preview cannot show
runway version                # version, and the types it draws

The survey file is .ep, .json.gz or .json — see Input format. A verb is always required: runway survey.ep is not a shortcut for runway render survey.ep, so what a command does never depends on what its argument happens to be named. python -m runway is the same entry point if you would rather not rely on the script being on PATH.

check

Start here. It writes nothing, and tells you what each question will become:

$ runway check examples/mixed_survey.json

mixed_survey.json  -  7 items

  drawn      commute_mode          multiple_choice
  drawn      commute_enjoyment     likert_five
  drawn      commute_satisfaction  linear_scale
  note       commute_barriers      rank             (no preview built for this type yet)
  warning    commute_breakdown     dict             (never shown to a respondent)

5 drawn, 1 note, 1 warning

Four verdicts, and they are genuinely different news:

verdict	meaning
drawn	previews with its real control
automatic	answered on the server, so nobody is ever shown it and nothing is missing
note	no control transcribed for the type yet — the survey is fine, the preview is behind
warning	nothing a respondent could be served — a type with no human-survey rendering anywhere, or a question left out of every question group

Only warning exits non-zero, and it is the only one that is about the survey rather than about this package.

check is also where the thinking wrapper shows up — a multiple_choice that would otherwise look perfectly previewable, and that nobody is ever asked. It is the case worth having a command for at all:

  automatic  pet_category    multiple_choice   (thinking)

render

runway render examples/mixed_survey.json            # -> previews/mixed_survey.html
runway render survey.ep                             # -> previews/survey.html
runway render examples/mixed_survey.json --split    # -> one file per page
runway render examples/*.json                       # -> one .html per survey
runway render examples/*.json -o build/review       # -> somewhere else

Each survey is written under its own name — the file's, with the format suffix taken off — so several can share one output directory. Two whose names agree, like survey.ep and survey.json, are refused rather than one silently replacing the other.

By default the whole survey lands in one HTML file, with a toolbar across the top to jump between questions (arrows, a dropdown, arrow keys). --split writes one file per page instead, which is far larger for anything but a short survey because each file re-inlines the stylesheet. --json on check, types and version gives machine-readable output.

A survey whose schema names a logo gets its banner on every page (examples/src/logo_survey.py has one), with the logo drawn as a placeholder — the logo is an asset on Coop, and fetching it is a network call, so it is not done unless you ask:

runway render survey.ep --schema schema.json --fetch-assets

That fetches the logo with your Coop API key, which works for your own assets and for any survey's logo you can view. The image is embedded in the page, so the preview is still one file you can send anywhere, and a copy is kept in <out>/assets/: rendering into the same directory again uses it rather than the network, and deleting the folder is how to forget it. A logo that cannot be fetched is a warning and a placeholder, not a failed render.

Questions with no control here still get a full page — same shell, same progress indicator, and their question text rendered in the usual markup — with a note or a warning standing in for the input, so a mixed survey previews end to end and wording can still be checked. The exception is questions whose text a respondent never sees, where it is suppressed rather than implying the survey shows it.

Runway is importable too: load, load_schema, render_bundle, render_page and render_survey are the public surface — see SPEC.md.

Question groups

A survey can be served a question group at a time rather than a question at a time — the groups come from the survey, and humanize_schema["survey"]["presentation"] = "group" asks for them. Runway previews it the way it is served: one panel per group in the bundle, one file per group under --split (named after the group), and the progress indicator read from where each page begins rather than from each question.

Both halves are needed and they arrive in different files — the groups in the survey, presentation with --schema. Given one without the other, runway pages per question, which is what the live survey does too; check says which of the two is missing rather than leaving you to guess.

Two things worth knowing:

• A question in no group is never served. A survey paging by group serves its groups and nothing else, and EDSL refuses to create a human survey that leaves a question out. Runway still draws the page, because you have to see which question fell out, but it draws the question's text with a warning where the control would be — the treatment a question nobody is asked gets — and check reports it as a warning, so a build that checks its previews fails. check also gains a column naming the group each question is served on.
• Nothing separates the questions on a page by default. The live survey wraps each item in .edsl-survey-item; what goes between them is custom_css. examples/src/group_survey.py shows one way to write it.

Instructions are the gap: they have no preview here yet, so a group page draws its questions and leaves any instruction out. Where it sits still counts, since that is where the page begins. A survey_message — text with nothing to answer — is drawn, and is an ordinary member of a group.

Question types

These are the types a human survey can be configured for, and where each one stands here. A type still awaiting a preview gets a note in place of its control; the rest of its page is complete, so its wording and position in the survey can be checked meanwhile.

question type	preview		question type	preview
budget	— not yet		linear_scale	✅ available
checkbox	✅ available		list	— not yet
checkbox_with_other	✅ available		matrix	✅ available
compute	✅ automatic		multiple_choice	✅ available
distribution	— not yet		multiple_choice_with_other	✅ available
file_upload	— not yet		numerical	✅ available
free_text	✅ available		rank	— not yet
image_generation	✅ automatic		survey_message	✅ available
interview	— not yet		top_k	— not yet
likert_five	✅ available		yes_no	✅ available

Four results are worth expecting: checkbox draws a Select all row nothing in the question asked for unless the schema removes it, checkbox_with_other never draws one, a matrix configured as a carousel draws neither the grid nor the stacked list — the carousel replaces the pair rather than joining them — and a numerical configured as a slider gets the note rather than a control, since only its number-field form is transcribed. SPEC.md has why.

A type outside this table — dict, for instance — cannot be shown to a respondent at all, so its page carries a warning rather than a note: what needs changing is the survey.

compute, image_generation and anything wrapped by thinking_question() are marked automatic — the survey answers them itself and no respondent ever sees them. Their pages still show the question's text, since a prompt for a model is the one part of one of these that can be wrong; the notice under it says who answers. examples/background_survey.json has all three.

survey_message is the one type drawn with no control at all, and that is the whole page rather than a gap in it: a message is text a respondent reads and then continues past with Next. Nothing is missing from that page, so it carries no note.

Input format

Save the survey, then point runway at it. .ep, .json.gz and .json all work, and all give the same preview:

survey.save("survey")            # -> survey.ep
survey.save("survey.json")       # -> survey.json

A humanize schema is saved separately, so pass it separately:

Path("schema.json").write_text(json.dumps(humanize_schema, indent=2))

runway render survey.ep --schema schema.json

The schema file is the same object you would pass to Survey.humanize(), so one written for a real deployment works here unchanged:

{
  "questions": { "question_name": { "format": {"type": "dropdown"} } },
  "survey":    { "custom_css": ".edsl-question-text { font-size: 1.4rem }" }
}

Writing the schema into the survey file instead does nothing — pass it with --schema. Anything that is not a saved survey is refused with a message saying what to do about it. SPEC.md has the reasoning behind both.

Known gaps

What a preview cannot show you. SPEC.md explains why in each case.

• Fonts. Plus Jakarta Sans is linked from Google Fonts, matching the live survey — the one external request a page makes, so it renders with fallback metrics offline.
• Video and PDFs in an option label. Question text draws all three; an option label draws images only. A player or a scroller inside a <label> makes every play or scroll a click on the radio, so those degrade to "(image unavailable)" rather than becoming a trap.
• Dragging a matrix carousel. The layout is drawn and the arrows work, and answering a row advances to the next one as the live page does. What is missing is the gesture: the live survey follows a finger through a swipe and decides where to land from how fast it moved, and a preview does neither, so a swipe does nothing at all. On a touch screen that is the difference between reading the layout and using it — preview the format here, then try a real one on a phone before the survey goes out.
• Math. The live page typesets $$…$$ with KaTeX in question text, option labels and instructions — double dollars only, so a price such as "$5 or $10" stays text. A preview shows the TeX as written, delimiters included: Solve $$x^2 = 4$$ previews as exactly that. Nothing is lost or mangled, so the source can still be proofread, but how the equation looks — its size, and the line height around a displayed one — is not shown. No recorded case holds math yet, which is the first step to drawing it.
• Option randomization. Not applied; the authored order is shown.
• Agent traits and prior answers. {{ agent.x }} and {{ q_name.answer }} render as written — there is no agent and no earlier answer to resolve them from. Scenario values do resolve, given --scenarios; see above.
• An image a survey has not generated yet is still drawn. The one exception to the line above: image_generation answers with a picture and with nothing else, so {{ img.answer }} piped into a later question's text or options draws a placeholder image — the same one an offloaded scenario image draws, in the shape, the size and the place the real one will occupy — rather than the template text. Nothing else about a prior answer is knowable from its type, so nothing else resolves. Your CSS sizes and frames it exactly as it will the real picture (it is an ordinary <img>), but cannot reach inside it: the tint, icon and label are fixed, since an SVG loaded through src is an isolated document. [src^="data:image/svg+xml"] selects one if you want the box treated differently from a real picture.
• A filter on one of those stops the question piping. {{ q.answer | length }} makes the whole question render as written, scenario keys in it included. The alternative was a stand-in that answers length from its own placeholder text, which puts a plausible wrong number on the page instead of a visible unsubstituted one.
• A scenario key the list does not carry fails silently, exactly as it does on the live page: {{ typo }} renders as nothing, and {{ typo.attr }} leaves the whole question unpiped. Neither leaves a mark on the page, so check --scenarios names them.
• A file-valued scenario key is drawn, not named. The question text is split around each <see file key> marker and the file goes in between: an <img> for an image, a <video> for a video, an <object> for a PDF, and the live page's own "Unsupported file type" for anything else, audio included. Where the live page has a link to a file it has fetched, a preview has the bytes — a scenario list carries each file base64-encoded, so it is inlined as a data: URI. The page is therefore as large as the media in it: a survey of small images costs nothing, twenty scenarios of video does not. An image the list has offloaded to the platform after an upload draws the same placeholder a generated image does, at the size the real one will be: the picture exists, and it is only the preview that cannot fetch it. An image carrying no bytes and no offload receipt is broken rather than elsewhere, so it stays "(image unavailable)" — as does a marker naming a key that holds no file. Option labels split the same way, images only — see above.
• Markup in a scenario value shows as plaintext, which is right, but the live page shows slightly less of it: question text is sanitized server-side there before it is rendered, so a tag outside its allowlist (<div>, <script>) is dropped where a preview shows it. Nothing executes in either — HTML is not supported in scenarios.
• A logo is a placeholder unless fetched. See --fetch-assets above. Each page of a bundle embeds its own copy, as with scenario media, so a large logo on a long survey makes a large file.
• The page layout is transcribed by hand, for now. Everything else here is held byte for byte to markup recorded from the live page. The page's outer layout (the container, the column everything lines up on, the footer) can't be recorded at the moment, so it is copied from the live page's source instead. It matches today, but nothing would catch it drifting until recording resumes. See SPEC.md.
• Position is inferred from the authored item list, where the live page resolves it from survey flow — so a survey with skip logic will differ, the further a respondent skips the more so. The renderings themselves are exact.
• Settings that govern submission, not markup. optional, custom_validation, submitting_indicator and attention checks are accepted and ignored — none is visible on a static page. comment and progress are rendered.
• The Next button does nothing. The <form> is rendered for layout parity but has no action. Use the toolbar to move between questions.
• A clicked option is redrawn by a script, not by React. The live page redraws a control when React re-renders it; a preview cannot, so the page script marks the control chosen the same way React would and the styling follows on its own. What it does not restore is anything the reference animates or measures on re-render. A page drawing no radio or checkbox ships no script at all.
• Only two checkbox rules work. Select all and exclusive_options do; selection limits and validation do not.
• A Coop-linked .ep can change when you open it. Previewing one may fetch a newer version from Coop and rewrite the file, which is edsl's behaviour for any load().

Documentation

SPEC.md is how a preview is built and why its output can be trusted: the design constraints the markup is held to, the recorded goldens that hold it there, how a survey is loaded, the two markdown surfaces, the toolbar and progress indicator, the behaviour a preview reproduces, how to add a question type, how to regenerate the stylesheet, and the file layout.

AGENTS.md is the operating contract for changing this repository.

License

MIT — see LICENSE. Third-party notices for vendored icon geometry, reproduced attribute names and generated CSS are in LICENSES.md.