Commit Graph

603 Commits

Author SHA1 Message Date
Karl Seguin
4ad8282e75 Merge pull request #2047 from lightpanda-io/fancy-wait
Add --wait-selector, --wait-script and --wait-script-file options to …
2026-03-31 20:59:48 +08:00
Karl Seguin
26653120fa Removing remaining CDP generic
Follow up to https://github.com/lightpanda-io/browser/pull/1990 which makes
both BrowserContext and Command non-generic.
2026-03-31 16:53:58 +08:00
Adrià Arrufat
008235222b SemanticTree: reorder getNodeDetails params 2026-03-31 07:29:33 +02:00
Karl Seguin
ab6c63b24b Add --wait-selector, --wait-script and --wait-script-file options to fetch
These new optional parameter run AFTER --wait-until, allowing the (imo) useful
combination of `--wait-until load --wait-script "report.complete === true"`.
However, if `--wait-until` IS NOT specified but `--wait-selector/script` IS,
then there is no default wait and it'll just check the selector/script. If
neither `--wait-selector` or `--wait-script/--wait-script-file` are specified
 then  `--wait-until` continues to default to `done`.

These waiters were added to the Runner, and the existing Action.waitForSelector
now uses the runner's version. Selector querying has been split into distinct
parse and query functions, so that we can parse once, and query on every tick.

We could potentially optimize --wait-script to compile the script once and call
it on each tick, but we'd have to detect page navigation to recompile the script
in the new context. Something I'd rather optimize separately.
2026-03-31 12:30:46 +08:00
Adrià Arrufat
16f17ead9a Merge pull request #2045 from lightpanda-io/semantic-tree-node-details
SemanticTree: Add nodeDetails tool
2026-03-31 05:28:24 +02:00
Karl Seguin
568fa25add Remove DOMContentLoaded and Loaded events from page_navigated
These were moved to their own distinct events, and should have been removed from
here.
2026-03-31 06:56:01 +08:00
Karl Seguin
752184b12b Improve/Fix CDP navigation event order
These changes all better align with chrome's event ordering/timing.

There are two big changes. The first is that our internal page_navigated event,
which is kind of our heavy hitter, is sent once the header is received as
opposed to (much later) on document load. The main goal of this internal event
is to trigger the "Page.frameNavigated" CDP event which is meant to happen
once the URL is committed, which _is_ on header response.

To accommodate this earlier trigger, new explicit events for DOMContentLoaded
and load have be added.

This drastically changes the flow of events as things go from:
Start Page Navigation
Response Received
  Start Frame Navigation
  Response Received
  End Frame Navigation
End Page Navigation
context clear + reset
DOMContentLoaded
Loaded

TO:
Start Page Navigation
Response Received
End Page Navigation
context clear + reset
Start Frame Navigation
Response Received
End Frame Navigation
DOMContentLoaded
Loaded

So not only does it remove the nesting, but it ensures that the context are
cleared and reset once the main page's navigation is locked in, and before any
frame is created.
2026-03-31 06:56:00 +08:00
Adrià Arrufat
9c8fe9b20f SemanticTree: Add nodeDetails tool
Adds a tool to retrieve detailed node metadata and updates the
semantic tree to track and display the disabled state of elements.
2026-03-30 16:38:23 +02:00
Karl Seguin
75dc4d5b0e Merge pull request #2031 from lightpanda-io/cdp-add-script-to-evaluate-on-new-document
Cdp add script to evaluate on new document
2026-03-30 11:16:39 +08:00
Karl Seguin
0d40aed1b7 zig fmt 2026-03-30 09:32:22 +08:00
Karl Seguin
78cb766298 Log for unimplemented parameter
Wrap script_on_new_document execution in try/catch for better error reporting.

Improve test for script_on_new_document
2026-03-30 09:31:13 +08:00
Nikolay Govorov
649d8d1024 Remove duplication in cookies instalation 2026-03-27 09:49:13 +00:00
Nikolay Govorov
d33edc5697 Fixup cookies management 2026-03-27 09:49:05 +00:00
Nikolay Govorov
16ca8d4b14 Fix cleanup connections in HttpClient 2026-03-27 09:49:03 +00:00
Karl Seguin
ea422075c7 Remove unused imports
And some smaller cleanups.
2026-03-27 12:45:26 +08:00
Navid EMAD
886aa3abba CDP: implement Page.addScriptToEvaluateOnNewDocument
Replace the hardcoded stub with a working implementation that stores
registered scripts and evaluates them in each new document.

Changes:
- Add ScriptOnNewDocument struct and storage list on BrowserContext
- Store scripts with unique identifiers when addScript is called
- Evaluate all registered scripts in pageNavigated, after the execution
  context is created but before frameNavigated/loadEventFired events
  are sent to the CDP client
- Add removeScriptToEvaluateOnNewDocument for cleanup
- Return unique identifiers per the CDP spec (was hardcoded to "1")

Scripts are evaluated with error suppression (warns on failure) to
avoid breaking navigation if a script has issues.

This unblocks CDP clients that rely on auto-injected scripts (polyfills,
monitoring, test helpers) persisting across navigations. Previously
clients had to manually re-inject after every Page.navigate.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 19:48:07 +01:00
Adrià Arrufat
7e778a17d6 MCP/CDP: unify node registration
This fixes a bug in MCP where interactive elements were not assigned
a backendNodeId, preventing agents from clicking or filling them. Also
extracts link collection to a shared browser module.
2026-03-26 23:51:43 +09:00
Navid EMAD
c6b0c75106 Address review: use arena.dupeZ for URL copy, add try to testing.context()
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 10:09:48 +01:00
Navid EMAD
93485c1ef3 CDP: implement Page.reload
Add `Page.reload` to the CDP Page domain dispatch. Reuses the existing
`page.navigate()` path with `NavigationKind.reload`, matching what
`Location.reload` already does for the JS `location.reload()` API.

Accepts the standard CDP params (`ignoreCache`, `scriptToEvaluateOnLoad`)
per the Chrome DevTools Protocol spec.

The current page URL is copied to the stack before `replacePage()` to
avoid a use-after-free when the old page's arena is freed.

This unblocks CDP clients (Puppeteer, capybara-lightpanda, etc.) that
call `Page.reload` and currently get `UnknownMethod`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 10:09:48 +01:00
Karl Seguin
cf641ed458 Merge pull request #1990 from lightpanda-io/remove_cdp_generic
Remove cdp generic
2026-03-26 07:49:13 +08:00
Karl Seguin
ca41bb5fa2 fix import casing 2026-03-25 17:54:24 +08:00
Karl Seguin
0dd0495ab8 Removes CDPT (generic CDP)
CDPT used to be a generic so that we could inject Browser, Session, Page and
Client. At some point, it [thankfully] became a generic only to inject Client.

This commit removes the generic and bakes the *Server.Client instance in CDP.
It uses a socketpair for testing.

BrowserContext is still generic, but that's generic for a very different reason
and, while I'd like to remove that generic too, it belongs in a different PR.
2026-03-25 17:43:30 +08:00
Adrià Arrufat
8e315e551a forms: extract form node registration logic 2026-03-25 09:30:06 +09:00
Adrià Arrufat
567cd97312 webapi.Element: centralize disabled state logic 2026-03-24 13:13:53 +09:00
Adrià Arrufat
260768463b Merge branch 'main' into osc/feat-mcp-detect-forms 2026-03-24 09:25:47 +09:00
Karl Seguin
5453630955 Merge pull request #1958 from lightpanda-io/runner
Extract Session.wait into a Runner
2026-03-24 07:28:18 +08:00
Pierre Tachoire
a94b0bec93 Merge pull request #1946 from lightpanda-io/cdp-response-body
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / wba-demo-scripts (push) Has been cancelled
e2e-test / wba-test (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig fmt (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
nightly build / build-linux-x86_64 (push) Has been cancelled
nightly build / build-linux-aarch64 (push) Has been cancelled
nightly build / build-macos-aarch64 (push) Has been cancelled
nightly build / build-macos-x86_64 (push) Has been cancelled
wpt / zig build release (push) Has been cancelled
wpt / build wpt runner (push) Has been cancelled
wpt / web platform tests json output (push) Has been cancelled
wpt / perf-fmt (push) Has been cancelled
e2e-integration-test / zig build release (push) Has been cancelled
e2e-integration-test / demo-integration-scripts (push) Has been cancelled
Encode non-utf8 Network.getResponseBody in base64
2026-03-23 16:46:12 +01:00
Pierre Tachoire
797cae2ef8 encode captured response body during CDP call 2026-03-23 14:26:27 +01:00
Adrià Arrufat
c3a2318eca fix: pass allocator as first parameter in forms.zig 2026-03-23 15:27:49 +09:00
Adrià Arrufat
4f1b499d0f zig fmt 2026-03-23 13:52:28 +09:00
Karl Seguin
c9bc370d6a Extract Session.wait into a Runner
This is done for a couple reasons. The first is just to have things a little
more self-contained for eventually supporting more advanced "wait" logic, e.g.
waiting for a selector.

The other is to provide callers with more fine-grained controlled. Specifically
the ability to manually "tick", so that they can [presumably] do something
after every tick. This is needed by the test runner to support more advanced
cases (cases that need to test beyond 'load') and it also improves (and fixes
potential use-after-free, the lp.waitForSelector)
2026-03-23 12:30:41 +08:00
Adrià Arrufat
4b29823a5b refactor: simplify form extraction and remove const casts 2026-03-23 13:24:21 +09:00
Karl Seguin
a69a22ccd7 Merge pull request #1948 from lightpanda-io/cdp-waitforselector
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / wba-demo-scripts (push) Has been cancelled
e2e-test / wba-test (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig fmt (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
CDP: add waitForSelector to lp.actions
2026-03-23 10:09:09 +08:00
Adrià Arrufat
a6d2ec7610 refactor: share form node ID serialization between MCP and CDP 2026-03-23 10:18:24 +09:00
Karl Seguin
c1fc2b1301 Merge pull request #1949 from lightpanda-io/1800-fix-startup-frame-id
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / wba-demo-scripts (push) Has been cancelled
e2e-test / wba-test (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig fmt (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
Fix Page.getFrameId on STARTUP when a browser context and a target exist
2026-03-22 07:14:33 +08:00
Matt Van Horn
78c6def2b1 mcp: add detectForms tool for structured form discovery
Add a detectForms MCP tool and lp.detectForms CDP command that return
structured form metadata from the current page. Each form includes its
action URL, HTTP method, and fields with names, types, required status,
values, select options, and backendNodeIds for use with the fill tool.

This lets AI agents discover and fill forms in a single step instead of
calling interactiveElements, filtering for form fields, and guessing
which fields belong to which form.

New files:
- src/browser/forms.zig: FormInfo/FormField structs, collectForms()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-21 08:40:50 -07:00
Pierre Tachoire
fbc71d6ff7 cdp: handle STARTUP session into Page.getFrameTree gracefully 2026-03-21 16:29:58 +01:00
Adrià Arrufat
e10ccd846d CDP: add waitForSelector to lp.actions
It refactors the implementation from MCP to be reused.
2026-03-22 00:09:02 +09:00
Pierre Tachoire
384b2f7614 cdp: call Page.getFrameTree on startup when possible 2026-03-21 16:07:48 +01:00
Pierre Tachoire
30f387d361 encode captured response depending of the content type 2026-03-21 14:11:06 +01:00
Pierre Tachoire
00d06dbe8c encode all captured responses body in base64 2026-03-21 13:29:58 +01:00
Pierre Tachoire
2107ade3a5 use a CapturedResponse struct for captured responses 2026-03-21 13:11:18 +01:00
gilangjavier
b5b012bd5d refactor(cdp): always return base64-encoded Network.getResponseBody 2026-03-21 07:06:09 +07:00
Karl Seguin
b4b7a7d58a Merge pull request #1901 from lightpanda-io/goodbye_origin
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / wba-demo-scripts (push) Has been cancelled
e2e-test / wba-test (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig fmt (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
Remove Origins
2026-03-21 07:19:47 +08:00
Karl Seguin
a5378feb1d Merge pull request #1927 from lightpanda-io/feat/fetch-wait-options
Feat/fetch wait options
2026-03-21 07:18:59 +08:00
Karl Seguin
88681b1fdb Fix Context's call_arena
The Context's call_arena should be based on the source, e.g. the IsolateWorld
or the Page, not always the page. There's no rule that says all Contexts have
to be a subset of the Page, and thus some might live longer and by doing so
outlive the page_arena.

Also, on context cleanup, isolate worlds now cleanup their identity.
2026-03-20 16:50:03 +08:00
Matt Van Horn
6925fc3f70 fix(cdp): return real frame ID in STARTUP getFrameTree when page exists
dispatchStartupCommand hard-codes "TID-STARTUP" as the frame ID in
Page.getFrameTree. When a driver connects via connectOverCDP after a
real page already exists, subsequent lifecycle events (frameNavigated)
use the actual page frame ID. The driver's frame tracking was
initialized with "TID-STARTUP", causing a mismatch that hangs
navigation.

Check for an existing browser context with a target_id in
dispatchStartupCommand. If present, return the real frame ID and URL.
Fall back to "TID-STARTUP" only when no page exists yet.

Fixes #1800

This contribution was developed with AI assistance (Claude Code + Codex).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 16:45:17 -07:00
Matt Van Horn
84557cb4e6 fix(cdp): send Target.detachedFromTarget event on detach
detachFromTarget and setAutoAttach(false) both null bc.session_id
without notifying the client. Per the CDP spec, detaching a session
must fire a Target.detachedFromTarget event so the driver stops
sending messages on the stale session ID.

Capture the session_id before nulling it and fire the event in both
code paths. Add tests covering the event emission and the no-session
edge case.

Fixes #1819

This contribution was developed with AI assistance (Claude Code + Codex).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-19 16:42:32 -07:00
Karl Seguin
a4cb5031d1 Tweak wait_until option
Small tweaks to https://github.com/lightpanda-io/browser/pull/1896

Improve the wait ergonomics with an Option with default parameter. Revert
page pointer logic to original (don't think that change was necessary).
2026-03-19 20:29:20 +08:00
Adrià Arrufat
9c2393351d SemanticTree: simplify max_depth logic 2026-03-19 20:25:20 +09:00