User guide

Prefly is a Mac app where virtual customers use your product and show you where they get stuck. They see, click, and type in a dedicated Chrome on your Mac, so you can test a dev server that only runs on your computer (localhost) as it is.

• A Mac
• An address to test: a dev server (like localhost:3000) or a public address
• One AI model. Customers use it to decide what to do next (see "Connect an AI model")

Prefly is not released yet. We send the app to people on the waitlist first.

1. 1Open the app and connect an AI model.
2. 2Enter the address and describe the job in one sentence. For example: "Sign up and place a first order."
3. 3Prefly suggests a test plan (cases). Check which customer does what, edit if needed, then start.
4. 4Watch customers use your product live. You can pause, or take over to help.
5. 5When it ends, read the report. Each issue comes with the screen at that moment and what the customer did.

Customers never do what a person must do, such as captchas, identity checks, or real payments. They stop there and hand it to you.

At every step, a customer asks an AI model what to do on the current screen. Connect one of these.

Coding agents such as Claude Code, Codex, and Cursor can run a test with the prefly command and get the results back in chat. It is handy right after a code change. Open the app only when you want a closer look.

1. Install the prefly command. For now it needs Rust (we are working on shipping it inside the app).

cargo install --git https://github.com/inha-developers/prefly prefly-cli

2. Connect it to your coding agents. This writes a skill (instructions the agent reads). Add --mcp to also register MCP (a tool connection for agents).

prefly install
prefly install --mcp

3. Ask your agent in plain words.

Test signing up and placing a first order with Prefly

Common commands when you run it yourself.

• Pick a model with --model sonnet or --model haiku. By default it follows the agent that called it.
• In CI, add --fail-on bug to fail when a confirmed bug is found.
• Agents without a shell (Cursor, VS Code, and others) connect with prefly mcp.
• Change the output language with --lang en. Korean, English, Japanese, and Chinese (Simplified, Traditional) are available.

Give Prefly the job in one sentence and it suggests cases (which customer does what). To keep using the same cases, save the suggestion to a file and run from it.

prefly plan localhost:3000 "Sign up and place a first order" --json > plan.json
prefly test localhost:3000 --cases plan.json

• To rerun the test you just ran after a fix, prefly test --again is simpler. It also shows what changed since last time.
• To change customers add --persona <customer id>. To repeat the same setup, add something like --repeat 3.

Saving cases as YAML files (.prefly/cases/) and running them is coming soon. For now, prefly case validate only checks the format.

A virtual customer is a face, a name, and one to three traits you can check. Traits come in three kinds only: conditions (values the engine sets on the screen and environment), behaviors (things the customer does), and give-up rules (when they stop). Age and job are only for the face and name tag and are never used to guess behavior. The same screen can trip up different customers in different places.

• Conditions: screen (phone, narrow phone, large phone, small phone, phone held sideways, tablet, tablet held sideways, half window, laptop, desktop, large monitor), zoomed-in screen, bigger default font, dark mode, more contrast, reduced motion, slow network, slow device, another language and time zone, keyboard only (no mouse), screen reader, bigger targets, color vision differences, blurry vision, low contrast
• Behaviors: pressing the same button twice, going back, typing odd values
• Give-up rules: number rules (wait time, total actions, actions on one screen) and situation rules (leaves when asked to sign up first, quits when the full total is not shown up front, quits when the delivery date is not shown, quits when there is no return information, quits when there is no easy pay)

How the list is chosen: it uses only traits you can check were applied in a test, only combinations that do not overlap with another customer, and, where possible, a public source that shows why the customer is on the list. We do not make up customers to fill a number. The list keeps growing as the engine learns to set more conditions.

Evidence: Evidence on a customer's page gives the reason they are on the list from public sources: the source name, when it was surveyed, and the country if it is not Korea. A figure is shown only when the source's terms allow it; otherwise only the source and date are shown. Values from public standards such as WCAG are marked Standard. These are facts from outside sources, not rates from Prefly runs. Click a source name to open it. Evidence is only shown on screen and never reaches the AI. Some customers have no evidence.

Customers fall into four groups by what they are good at finding (devices and screens, body and tools, hurry and patience, mistakes). When you hand over a job, Prefly picks customers that fit it. To choose yourself, pick one of five sets (Default, Checkout check, Accessibility check: seeing, Accessibility check: input, Mobile check) in the customer picker on the start screen or on the Virtual customers screen. One set is one test (up to 10 customers).

When there are many customers, the list on the Virtual customers screen shows two rows per group; use Show n more to expand. Type a name, trait text, or an evidence source name (for example WebAIM) into Find by name or trait above the list to keep only the matching customers.

Checkout customers: most customers in the Checkout check set have a situation rule. The customer decides from the screen when a situation rule applies and quits, and the screen where they quit and the reason are kept in the record. Number rules are counted and ended by the engine. They never press payment or order confirmation buttons.

Your own customers: when you want the conditions of your real customers, make customers from the same traits. Use Add customer at the end of the list on the Virtual customers screen and choose a name, an age, and one to three traits. To change a built-in customer, use Copy and change. Up to 64 customers together with the built-in ones. Notes are for people only and never reach the AI. Pick several and Save as set to choose them as a set on the start screen.

prefly customers add --name Minji --age 63 --condition pageZoom=200%
prefly customers export > .prefly/customers.json
prefly test localhost:3000 "Up to the first order" --customers-file .prefly/customers.json --persona <my customer id>

Reopen your site's screens under customers' screen conditions (phone, small phone, zoom, bigger font, dark mode, reduced motion, slow network, slow device) and measure them without AI. It compares horizontal overflow, clipped text, small targets, targets Tab can't reach, low contrast, non-stop motion, and slow first screens (over 2.5 seconds) with the screen without conditions, and shows what appears only with a condition. It sends no input and uses no AI.

When: after changing layout, CSS, fonts, or colors. It is worth doing before a customer test.

1. 1Crawl the site once with Crawl on the project map.
2. 2On the Virtual customers screen, open By screen, press Sweep with conditions, and choose conditions (up to 8; up to 4 for slow network and slow device).
3. 3When it finishes, each condition shows how many screens it flagged. Expand a screen row to see the values and screenshots. If there is an earlier sweep, "Compared with the last sweep: gone, new, unchanged" is shown too.
4. 4After a fix, sweep again and check that the line is gone.

Not measured: slow network and slow device reopen only the first 10 screens and time how long the first screen takes to draw. When a screen opens but the time could not be measured, it is listed separately, like "Slow 3G: 10 screens not measured", and an expanded row shows "Slow 3G: not measured". Cells that were not measured are never counted as 0 or as swept.

prefly sweep localhost:3000 --crawl --customer-set mobile
prefly sweep localhost:3000 --customers <customer id>,<customer id> --fail-on overflow,slow

In CI, set a threshold with --fail-on. Going over it exits with code 1; not enough values to judge exits with 3. To compare with a saved result (prefly sweep --json) instead of the engine's earlier sweep, use --compare-with.

When you test with several customers with different conditions, Prefly gathers the values the engine measured (horizontal overflow, clipped text, small targets, accessibility rules, errors, a slow first screen) that appeared on a screen only for customers with one condition, among the customers who saw that screen.

When: after a run with three or more customers with different conditions. Running the default set is enough.

• App: By condition under Automated checks in the result, the By screen view on the Virtual customers screen, and a single customer's screen
• Terminal: the Differences by condition section of the result, and runs[].conditionDiffs in JSON

- Only for pageZoom=200%: shop.test/checkout horizontal overflow 340px (1 of 1; not seen for 2 compared customers)

"It appeared only for customers with this condition" is a fact, not a verdict that the condition causes it. For a customer with two conditions you can't tell which one. To find out, use Rerun with a condition changed.

Prefly replays the presses, scrolls, and going back a customer recorded, without AI, and compares the same steps with one condition removed or changed. Typed text is not entered again.

When: to tell whether an issue or a difference comes from that condition, or to look at a passed path under other customers' conditions.

• Rerun without the condition on an issue: reruns once with each of that customer's conditions removed.
• Rerun with other conditions on a single customer's screen, and Rerun a passed path with other conditions in the By screen table: reruns once under each chosen customer's conditions. It shows how many runs it will make before you start.

prefly contrast --review <review id> --issue <issue id>
prefly contrast --run <run id> --customer <passed customer id> --with <customer id>

It sends the presses again (something like adding to the cart may happen once more), so use it only on test environments. It does not press payment, order confirmation, or login buttons, and does not rerun customers who started from a saved login.

• Run prefly doctor first. It tells you what is not ready.
• If a test does not start, check that an AI model is connected.
• When testing a dev server, make sure it is running.