Engine
AI to find it, no AI to keep it fixed
Finding bugs takes a model reading the screen. Checking them again does not. Saved actions replay without a model, and CI runs them in one line.
The Prefly team · 1 min read
There is no reason to call a model twice to confirm the same bug. Finding problems is exploration, so it needs a model. Confirming that a problem is real, or that it stays fixed, only needs the same actions in the same order. So we save what the customer did and replay it without a model. It is faster, cheaper and gives the same answer every time.
- 1Reproduction checkFor each issue, the customer’s path replays from saved actions, twice by default. You get how many times it happened again, with the screen and error log each time. With a previous passing run, it also points to the first step that diverged.
- 2Success conditionsEach test has success conditions (URL, visible text, visible element, page title) that the engine checks on the screens the customer saw. If the customer says it is done but a condition is missing, it did not pass.
- 3One line in CISaved tests rerun from one CLI line, with results as JSON and as markdown for a PR comment. A failure exits non-zero.
prefly test --project yeoul --saved all
