Plans: run many browser steps in one call

Format

A JSON object with steps. Each step has an action and that action's fields. This one runs as written:

newest.json

{ "steps": [
  { "action": "goto", "url": "https://news.ycombinator.com" },
  { "action": "click", "target": "link \"new\"" },
  { "action": "extract", "target": ".titleline", "limit": 3, "name": "newest" }
] }

okyra run newest.json
{
  "ok": true,
  "results": {
    "newest": [
      "I'm building SEPPY to destroy the trillion-dollar digital advertising market (seppy.de)",
      "Employee-like access to METR by Anthropic for 3rd party assessment (anthropic.com)",
      "Unity Mass-Suspends Accounts",
      "… 27 more, raise limit to see them"
    ]
  },
  "steps_run": 3
}

A login has the same shape. Role and name targets work before the page is seen, so the whole form is one plan:

{ "steps": [
  { "action": "goto", "url": "https://example.com/login" },
  { "action": "type", "target": "textbox \"Email\"", "text": "ana@example.com" },
  { "action": "type", "target": "textbox \"Password\"", "text": "hunter2" },
  { "action": "click", "target": "button \"Log in\"" },
  { "action": "text", "target": ".balance", "name": "balance" }
] }

Running a plan

From a file: okyra run plan.json.
From stdin: okyra run -, so a script can pipe a plan in without writing a file.
From an agent: the okyra_plan MCP tool takes the same steps array.

cat plan.json | okyra run -
okyra --page p2 run plan.json        # steps run on page p2

okyra run exits with 0 when the plan finished and 1 when it stopped on a failed step. A plan that is not valid JSON or names an unknown action fails before anything runs, with bad plan: ....

Actions and their fields

Action	Fields	Result
`goto`	`url`	`{url, title, settled}`
`click`	`target`	none
`type`	`target`, `text`	none
`select`	`target`, `value`	the option picked
`upload`	`target`, `files` (absolute paths)	`{files}`
`hover`	`target`	the text that appeared, or null
`press`	`key`	none
`scroll`	`target`, or `dy` and `dx`	`{x, y}`
`text`	`target`	visible text
`extract`	`target`, `limit` (default 200)	JSON array
`evaluate`	`expression` (also `script` or `js`)	JSON value
`wait`	`ms`, or `quiet_ms` and `timeout_ms`	`{waited_ms}`
`snapshot`	`interactive`, `max_lines`	the outline text
`screenshot`	`path`, `annotate`	`{path, refs}`
`repeat`	`steps`, `next`, `until`, `max`	`{runs, stopped}`
`each`	`target`, `steps`, `open`, `back`, `until`, `max`	one record per item

The CLI command eval is the action evaluate in a plan. repeat and each work only inside a plan. Targets are described on the targets page.

Any step may also have name (keep its result), optional (go on if it fails) and page (run it on another page label). The plan itself may set page for every step that does not name one.

Named results

Only steps with a name keep their result, under results. Unnamed results are dropped to save tokens, except the last step's, which comes back as last when it has no name.

echo '{"steps":[{"action":"goto","url":"https://example.com"},{"action":"evaluate","expression":"document.title"}]}' | okyra run -
{
  "last": "Example Domain",
  "ok": true,
  "steps_run": 2
}

Give every value you need a name, and end the plan with the read that answers the task. Then the answer arrives in the same call and the agent does not need a snapshot to check.

Optional steps and failures

A step that fails stops the plan. The result has ok: false, the failed step's index (from 0), its action and error, and an interactive snapshot of the page where it stopped, up to 80 lines. The caller can pick a fix without another call. A step with "optional": true is recorded under skipped and the plan goes on.

okyra run - <<'EOF'
{ "steps": [
  { "action": "goto", "url": "https://news.ycombinator.com/login" },
  { "action": "click", "target": "button \"Accept cookies\"", "optional": true },
  { "action": "type", "target": "#username", "text": "ana" },
  { "action": "text", "target": "#welcome", "name": "welcome" }
] }
EOF

{
  "failed": {
    "action": { "action": "type", "target": "#username", "text": "ana" },
    "error": "no element matches #username",
    "index": 2
  },
  "last": { "settled": true, "title": "", "url": "https://news.ycombinator.com/login" },
  "ok": false,
  "skipped": [
    {
      "action": { "action": "click", "target": "button \"Accept cookies\"" },
      "error": "no element matches button \"Accept cookies\"",
      "index": 1
    }
  ],
  "snapshot": "page \"\" https://news.ycombinator.com/login\nscroll 0 of 819\nform:\n  textbox \"\" @e464\n  textbox \"\" @e465 (password)\n  button \"login\" @e466\n...",
  "steps_run": 3
}

The snapshot shows the fields have no id of username, and gives refs to use instead. Over MCP the same result comes back as a tool error, so the agent sees it failed.

repeat: pagers, Load more and feeds

A repeat runs its steps, then clicks next, again and again. Named steps inside collect one result per run; when every run gives an array, as extract does, they are joined into one array. Reading three pages of a list is one step:

{ "steps": [
  { "action": "goto", "url": "https://news.ycombinator.com" },
  { "action": "repeat", "next": "link \"More\"", "max": 3, "name": "paging",
    "steps": [{ "action": "extract", "target": ".titleline", "name": "titles" }] }
] }

That returns titles with 90 entries, 30 per page, and paging as {"runs": 3, "stopped": "max"}. A repeat stops at the first of these:

stopped	When
`until <condition>`	`until` holds. It is checked before each run, so a repeat whose condition already holds runs 0 times.
`next disabled or gone`	The `next` target is disabled, missing or hidden: the last page.
`unchanged`	A run left the page text as it was, such as a feed with nothing more to load.
`max`	It ran `max` times. Default 20, at most 100.

Three common shapes: steps plus next read every page; only next presses Load more until it goes; a scroll step with no next scrolls a feed to its end.

{ "action": "repeat", "next": "button \"Load more\"", "max": 10 }
{ "action": "repeat", "steps": [{ "action": "scroll", "dy": 2000 }] }

Use repeat for a pager on one list only. To visit each item of a list, use each.

until conditions

until is an object with any of these fields. The first that holds stops the repeat or each:

Field	Holds when
`{"gone": target}`	The target is missing or hidden.
`{"disabled": target}`	The target is disabled, missing or hidden.
`{"visible": target}`	The target is on the page and visible.
`{"text": "..."}`	The page text contains it.

each: visit every item

An each visits every item its target matches and runs steps on each. It returns one record per item: its item text, its url when it is a link, and every named step's result. Opening ten search results to read a field on each is one step, not ten turns.

stories.json

{ "steps": [
  { "action": "goto", "url": "https://news.ycombinator.com" },
  { "action": "each", "target": ".subline .age a", "max": 2, "name": "stories",
    "steps": [
      { "action": "text", "target": ".fatitem .titleline", "name": "title" },
      { "action": "text", "target": ".fatitem .score", "name": "points", "optional": true }
    ] }
] }

{
  "ok": true,
  "results": {
    "stories": [
      {
        "item": "2 hours ago",
        "points": "27 points",
        "title": "Yet another way to invoke functions without parentheses (ikaes.de)",
        "url": "https://news.ycombinator.com/item?id=50077225"
      },
      {
        "item": "11 hours ago",
        "points": "318 points",
        "title": "Meta’s new commercial terms will allow the scraping of news sites: experts (niemanlab.org)",
        "url": "https://news.ycombinator.com/item?id=50111852"
      },
      "… 28 more items, raise max to visit them"
    ]
  },
  "steps_run": 2
}

Field	Meaning
`target`	A target that matches one list, table or container visits the links in it, one per row or item. Otherwise every match is an item.
`steps`	Steps to run on each item. Required; name the ones you want back.
`open`	`page` follows each link in a page of its own and closes it after, so the list stays as it was. `same` follows links in the list's own page, then returns to the list. `click` clicks each item in place, for rows that open a panel. Default: `page` when every item is a link, else `click`.
`back`	In click mode, a target clicked after each item, such as `button "Close"`. Without it, a click that changed the address is undone with the browser's Back.
`until`	Checked after each item; stops the visits when it holds.
`max`	Most items visited. Default 20, at most 100. Items left over are counted in the last record.

If a step fails on an item, the error names it, such as each item 1 ("Ana Silva"), step 2: no element matches text "Berlin" (items count from 1, steps from 0), and the records so far are still returned under the step's name.

hover: tooltips and charts

For a value shown only on hover, hover moves the real mouse over the target, waits for the page to settle and returns the text that appeared or changed. To read every point of a chart, combine each in click mode with a named hover on @item:

{ "steps": [
  { "action": "goto", "url": "https://example.com/your-chart" },
  { "action": "each", "target": "circle.dot", "open": "click", "name": "points",
    "steps": [{ "action": "hover", "target": "@item", "name": "tip" }] }
] }

Replace the URL with your chart page. On our test chart this returns [{"tip": "Lisbon: 24 °C"}, {"tip": "Porto: 19 °C"}, {"tip": "Faro: 27 °C"}]. Chart points have no text, so their records have no item field.

extract: tables and lists

extract reads every element the target matches in one step. A table or table row gives one array of cell texts per row. A ul or ol gives one string per item. Anything else gives one string per match, its lines joined with | . Past limit, the last entry says how many more there are.

{ "action": "extract", "target": "#rows tr", "limit": 2, "name": "rows" }

"rows": [["item 1", "10"], ["item 2", "11"], "… 8 more, raise limit to see them"]