Compare commits

..

103 Commits

Author SHA1 Message Date
Karl Seguin
fe3faa0a5a Merge pull request #1825 from sjhddh/fix-tracking-allocator-resize
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / wba-demo-scripts (push) Has been cancelled
e2e-test / wba-test (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
fix: only increment TrackingAllocator reallocation_count on successful resizes
2026-03-14 15:48:49 +08:00
Karl Seguin
39d5a25258 Merge pull request #1820 from hobostay/fix-tracking-allocator-stats
Fix TrackingAllocator reallocation_count being incremented on failed operations
2026-03-14 15:48:13 +08:00
Karl Seguin
f4044230fd Merge pull request #1824 from sjhddh/fix-option-gettext-leak
fix: resolve memory leak in Option.getText() by using page arena
2026-03-14 15:45:53 +08:00
sjhddh
4d6d8d9a83 fix(test): properly count successful reallocations in TrackingAllocator 2026-03-14 06:57:04 +00:00
sjhddh
c4176a282f fix: resolve memory leak in Option.getText() by using page arena 2026-03-14 06:50:26 +00:00
Karl Seguin
535128da71 Merge pull request #1814 from lightpanda-io/nikneym/window-onload-alias
Make `body.onload` getter/setter alias to `window.onload`
2026-03-14 13:30:10 +08:00
hobostay
7fe26bc966 Fix TrackingAllocator reallocation_count being incremented on failed operations
The reallocation_count counter was being incremented regardless of whether
the resize/remap operations succeeded. This led to inaccurate memory
allocation statistics.

- resize: Only increment when rawResize returns true (success)
- remap: Only increment when rawRemap returns non-null (success)

This fixes the TODO comments that were present in the code.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-14 13:10:11 +08:00
Halil Durak
cc6587d6e5 make body.onload getter/setter alias to window.onload 2026-03-13 18:49:26 +03:00
Karl Seguin
be8ba53263 Merge pull request #1811 from lightpanda-io/script_handling
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / wba-demo-scripts (push) Has been cancelled
e2e-test / wba-test (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
nightly build / build-linux-x86_64 (push) Has been cancelled
nightly build / build-linux-aarch64 (push) Has been cancelled
nightly build / build-macos-aarch64 (push) Has been cancelled
nightly build / build-macos-x86_64 (push) Has been cancelled
wpt / zig build release (push) Has been cancelled
wpt / build wpt runner (push) Has been cancelled
wpt / web platform tests json output (push) Has been cancelled
wpt / perf-fmt (push) Has been cancelled
e2e-integration-test / zig build release (push) Has been cancelled
e2e-integration-test / demo-integration-scripts (push) Has been cancelled
Better script handling.
2026-03-13 21:40:19 +08:00
Pierre Tachoire
043d48d1c7 Merge pull request #1812 from lightpanda-io/longer-sleep
ci: add a longer sleep to wait for node start on wba test
2026-03-13 13:59:03 +01:00
Karl Seguin
e8fe80189b Merge pull request #1808 from lightpanda-io/cdp_startup_frames
Tweak CDP startup messages.
2026-03-13 19:24:14 +08:00
Pierre Tachoire
0e48f317cb ci: add a longer sleep to wait for node start on wba test 2026-03-13 12:22:48 +01:00
Karl Seguin
867745c71d Tweak CDP startup messages.
1 - When Target.setAutoAttach is called, send the `Target.attachedToTarget`
    event before sending the response. This matches Chrome's behavior and
    it stops playwright from thinking there's no target and making extra calls,
    e.g. to Target.attachedToTarget.

2 - Use the same dummy frameId for all startup messages. I'm not sure why we
    have STARTUP-P and STARTUP-B. Using the same frame (a) makes more sense to
    me (b) doesn't break any existing integration tests, and (c) improves this
    scenario: https://github.com/lightpanda-io/browser/issues/1800
2026-03-13 19:07:47 +08:00
Karl Seguin
a1a7919f74 Better script handling.
Dynamic scripts have script.async == true by default (we handled this correctly
in the ScriptManager, but we didn't return the right value when .async was
accessed).

Inline scripts only consider direct children, not the entire tree.

Empty inline scripts are executed at a later time if text is inserted into them
2026-03-13 19:05:23 +08:00
Pierre Tachoire
c3de47de90 Merge pull request #1810 from lightpanda-io/fix_cookie_loading
Ensure valid cookie isn't interpreted as null
2026-03-13 11:26:24 +01:00
Pierre Tachoire
dd35bdfeb4 Merge pull request #1809 from lightpanda-io/fix_flaky_test
Fix a flaky frame test
2026-03-13 10:17:01 +01:00
Karl Seguin
07c3aec34f Ensure valid cookie isn't interpreted as null
Use an explicit type when @ptrCast() is assigned to an optional to ensure the
value isn't interpreted as null.
2026-03-13 17:00:59 +08:00
Karl Seguin
bce3e8f7c6 Fix a flaky frame test
Loading `sub 1.html` has a side effect - it increments window.top..sub1_count).
So it should be used careful. It was being used in `about_blank_renavigate` as
a placeholder which _should_ not get navigated, but there's no strict guarantee
about when it gets canceled.
2026-03-13 15:39:34 +08:00
Pierre Tachoire
ba9777e754 Merge pull request #1609 from lightpanda-io/web-bot-auth
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / wba-demo-scripts (push) Has been cancelled
e2e-test / wba-test (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
Web Bot Auth
2026-03-13 08:31:25 +01:00
Pierre Tachoire
7040801dfa Merge pull request #1790 from lightpanda-io/structuredClone_serializer
Add window.structuredClone
2026-03-13 08:29:49 +01:00
Karl Seguin
4f8a6b62b8 Add window.structuredClone
Depends on https://github.com/lightpanda-io/zig-v8-fork/pull/156

Uses V8::Serializer and V8::Deserializer which handles built-in types, e.g.
regex. But it doesn't handle Zig types by default. This is something we need
to hook in, using the delegate callbacks. Which we can do after.

Meant to replace https://github.com/lightpanda-io/browser/pull/1785
2026-03-13 07:28:33 +08:00
Karl Seguin
d3dad772cf Merge pull request #1806 from lightpanda-io/update_zig_v8_action
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
nightly build / build-linux-x86_64 (push) Has been cancelled
nightly build / build-linux-aarch64 (push) Has been cancelled
nightly build / build-macos-aarch64 (push) Has been cancelled
nightly build / build-macos-x86_64 (push) Has been cancelled
wpt / zig build release (push) Has been cancelled
wpt / build wpt runner (push) Has been cancelled
wpt / web platform tests json output (push) Has been cancelled
wpt / perf-fmt (push) Has been cancelled
e2e-integration-test / zig build release (push) Has been cancelled
e2e-integration-test / demo-integration-scripts (push) Has been cancelled
update action.yml to latest zig-v8
2026-03-13 07:26:18 +08:00
Karl Seguin
944b672fea Merge pull request #1792 from lightpanda-io/Canvas_getImageData
Add dummy getImageData to canvas
2026-03-13 07:23:05 +08:00
Karl Seguin
b1c54aa92d Merge pull request #1795 from lightpanda-io/navigate_blob_url
Allow navigation from a blob URL.
2026-03-13 07:22:50 +08:00
Karl Seguin
4ca6f43aeb Merge pull request #1803 from lightpanda-io/fix-redirection-cookies
parse cookies on redirection during header callback
2026-03-13 07:17:51 +08:00
Karl Seguin
f09e66e1cc update action.yml to latest zig-v8 2026-03-13 07:15:23 +08:00
Karl Seguin
8b7a4ceaaa Merge pull request #1794 from lightpanda-io/update-docker-zig
update zig-v8 in dockerfile
2026-03-13 07:14:35 +08:00
Pierre Tachoire
51e90f5971 parse cookies on redirection during header callback
THe change to handle bot `\n` and `\r\n` for end HTTP headers skip the
cookie parsing in case of redirection.
2026-03-12 18:42:51 +01:00
Muki Kiboigo
8db64772b7 add URL getHost test 2026-03-12 09:04:13 -07:00
Muki Kiboigo
bf0be60b89 use new validator for e2e test 2026-03-12 09:04:13 -07:00
Muki Kiboigo
172481dd72 add e2e tests w/ web bot auth 2026-03-12 09:04:13 -07:00
Muki Kiboigo
c6c0492c33 fix request authentication with web bot auth 2026-03-12 09:04:13 -07:00
Muki Kiboigo
fca29a8be2 add WebBotAuth unit tests 2026-03-12 09:04:13 -07:00
Muki Kiboigo
d365240f91 fix cli argument for WebBotAuth domain 2026-03-12 09:04:12 -07:00
Muki Kiboigo
1ed61d4783 simplify parsePemPrivateKey 2026-03-12 09:04:12 -07:00
Muki Kiboigo
a1fb11ae33 make pem private key buffers smaller with comments 2026-03-12 09:04:12 -07:00
Muki Kiboigo
9971816711 use transfer arena to sign webbotauth request 2026-03-12 09:04:12 -07:00
Muki Kiboigo
c38d9a3098 auth challenge only on use_proxy 2026-03-12 09:04:12 -07:00
Muki Kiboigo
02198de455 add support for WebBotAuth in Client 2026-03-12 09:04:10 -07:00
Muki Kiboigo
6cd8202310 add WebBotAuth and support for ed25119 to crypto 2026-03-12 09:03:15 -07:00
Muki Kiboigo
4d7b7d1d42 add web bot auth args 2026-03-12 09:03:15 -07:00
Karl Seguin
e4e21f52b5 Allow navigation from a blob URL.
These are used a lot in WPT test.
2026-03-12 18:58:10 +08:00
Pierre Tachoire
84e1cd08b6 update zig-v8 in dockerfile 2026-03-12 11:54:06 +01:00
Pierre Tachoire
7796753e7a Merge pull request #1791 from lightpanda-io/wp/mrdimidium/update-v8
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
Update zig-v8
2026-03-12 11:48:27 +01:00
Karl Seguin
880205e874 Add dummy getImageData to canvas
Probably doesn't solve many (if any) WPT tests, but it moves them further along.
2026-03-12 17:53:00 +08:00
Nikolay Govorov
1b96087b08 Update zig-v8 2026-03-12 08:50:33 +00:00
Karl Seguin
aa246c9e9f Merge pull request #1788 from lightpanda-io/range_cleanup
Add cleanup to Range
2026-03-12 16:45:05 +08:00
Karl Seguin
f1d311d232 Merge pull request #1781 from lightpanda-io/wp/mrdimidium/telemetry-network
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
Use global connections poll
2026-03-12 13:46:51 +08:00
Karl Seguin
e4f7fca10d Merge pull request #1789 from lightpanda-io/fix-test-warnings
testing: add LogFilter utility for scoped log suppression
2026-03-12 13:40:13 +08:00
Adrià Arrufat
3d6d669a50 testing: add LogFilter utility for scoped log suppression 2026-03-12 13:56:53 +09:00
Nikolay Govorov
c4097e2b7e remove dead-code 2026-03-12 03:55:48 +00:00
Karl Seguin
619d27c773 Add cleanup to Range
In https://github.com/lightpanda-io/browser/pull/1774 we started to track Ranges
in the page in order to correctly make them "live". But, without correct
lifetime, they would continue to be "live" even if out of scope in JS.

This commit adds finalizers to Range via reference counting similar to Events.
It _is_ possible for a Range to outlive its page, so we can't just remove the
range from the Page's _live_range list - the page might not be valid. This
commit gives every page an unique id and the ability to try and get the page
by id from the session. By capturing the page_id at creation-time, a Range
can defensively remove itself from the page's list. If the page is already
gone, then there's no need to do anything.
2026-03-12 10:38:07 +08:00
Karl Seguin
1522c90294 Merge pull request #1787 from lightpanda-io/dummy-filelist
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
nightly build / build-linux-x86_64 (push) Has been cancelled
nightly build / build-linux-aarch64 (push) Has been cancelled
nightly build / build-macos-aarch64 (push) Has been cancelled
nightly build / build-macos-x86_64 (push) Has been cancelled
wpt / zig build release (push) Has been cancelled
wpt / build wpt runner (push) Has been cancelled
wpt / web platform tests json output (push) Has been cancelled
wpt / perf-fmt (push) Has been cancelled
e2e-integration-test / zig build release (push) Has been cancelled
e2e-integration-test / demo-integration-scripts (push) Has been cancelled
Add FileList Web API stub
2026-03-12 06:37:00 +08:00
Karl Seguin
794e15ce21 Merge pull request #1786 from lightpanda-io/fontfaceset-load
FontFaceSet is now an EventTarget
2026-03-12 06:36:22 +08:00
Karl Seguin
34771b835e Merge pull request #1783 from lightpanda-io/custom_element_dynamic_markup_handling
Throw on dynamic markup in custom element callbacks during parsing
2026-03-12 06:27:22 +08:00
Karl Seguin
8df51b232a Merge pull request #1784 from lightpanda-io/origin_arena
Use origin.arena for values that are tied to the origin
2026-03-12 06:26:20 +08:00
Karl Seguin
13b8ce18b2 Merge pull request #1780 from lightpanda-io/anchor_and_form_target
Add support for target attribute on anchors and forms
2026-03-12 06:26:08 +08:00
Pierre Tachoire
448386e52b Add FileList Web API stub
Next.js hydration references FileList as a global for feature detection.
Register a minimal stub (length=0, item()→null) so the type exists in
the global scope and the reference check doesn't throw.
2026-03-11 22:31:12 +01:00
Pierre Tachoire
bf07659dd5 FontFaceSet is now an EventTarget
Dispatch loading and loaddone events on load() call
2026-03-11 22:18:42 +01:00
Karl Seguin
16dfad0895 Use origin.arena for values that are tied to the origin
Of note, the TAO and identity map entry has to use the origin arena, not
the context arena, as those can outlive the context.
2026-03-11 21:55:58 +08:00
Adrià Arrufat
f61449c31c Merge pull request #1776 from lightpanda-io/semantic-tree
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
e2e-test / browser fetch (push) Has been cancelled
zig-test / zig test using v8 in debug mode (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
Add native Semantic Tree extraction engine for AI agents
2026-03-11 21:01:04 +09:00
Adrià Arrufat
60699229ca Merge branch 'main' into semantic-tree 2026-03-11 20:52:39 +09:00
Karl Seguin
e1dd26b307 Throw on dynamic markup in custom element callbacks during parsing
Custom element callbacks aren't allowed to call document.open/close/write while
parsing.

Fixes WPT crash:
/custom-elements/throw-on-dynamic-markup-insertion-counter-reactions.html
2026-03-11 18:41:06 +08:00
Karl Seguin
ee637c3662 Add support for target attribute on anchors and forms 2026-03-11 15:49:30 +08:00
Adrià Arrufat
65d7a39554 SemanticTree: use payload captures for CData.Text checks
Improves conciseness and idiomatic Zig style by replacing .is(CData.Text) != null and .as() with direct payload captures in if statements.
2026-03-11 16:39:59 +09:00
Adrià Arrufat
37735b1caa SemanticTree: use StaticStringMap for structural role check
Improves performance and readability of isStructuralRole. Also includes minor syntax cleanup in AXNode.
2026-03-11 16:37:24 +09:00
Adrià Arrufat
1866e7141e SemanticTree: cast with as
Co-authored-by: Karl Seguin <karlseguin@users.noreply.github.com>
2026-03-11 16:33:39 +09:00
Adrià Arrufat
feccc9f5ce AXNode: remove unused mock JSON lifecycle methods
Simplifies TextCaptureWriter by removing unused methods, ensuring future changes to writeName will fail at build time if new methods are required.
2026-03-11 16:25:34 +09:00
Adrià Arrufat
af803da5c8 cdp.lp: use enum for getSemanticTree format param
Leverages std.json.parse to automatically validate the format param into a type-safe enum.
2026-03-11 16:21:43 +09:00
Adrià Arrufat
5329d05005 interactive: optimize getTextContent single-chunk path
Avoids an unnecessary double allocation and maintains a zero-copy fast path for single-chunk text extraction.
2026-03-11 15:27:12 +09:00
Adrià Arrufat
2e6dd3edfe browser.EventManager: remove unused hasListener function 2026-03-11 15:18:14 +09:00
Nikolay Govorov
a95b4ea7b9 Use global connections poll 2026-03-11 05:44:59 +00:00
Adrià Arrufat
ca931a11be AXNode: add spacing between concatenated text nodes
When calculating accessible names for elements without explicit labels, multiple descendant text nodes were previously concatenated directly together. This adds a space between distinct text node contents to prevent words from sticking together.
2026-03-11 10:45:07 +09:00
Adrià Arrufat
6c7272061c cli: enable pruning for semantic_tree_text dump mode
Previously, semantic_tree_text hardcoded prune = false, which bypassed the structural node filters and allowed empty none nodes to pollute the root of the text dump.
2026-03-11 10:38:12 +09:00
Adrià Arrufat
4f262e5bed SemanticTree: filter computed names for generic containers
This prevents token bloat in JSON/text dumps and ensures that StaticText leaf nodes are not incorrectly pruned when structural containers (like none, table) hoist their text.
2026-03-11 10:22:40 +09:00
Adrià Arrufat
a6ccc72d15 interactive: properly concatenate text content for accessible names
This fixes a bug where only the first text node was being returned, causing fragmented text nodes (e.g. <span>Sub</span><span>mit</span>) to be missing their trailing text.
2026-03-11 09:57:08 +09:00
Adrià Arrufat
d1ee0442ea Merge branch 'main' into semantic-tree 2026-03-10 21:48:49 +09:00
Adrià Arrufat
064e7b404b SemanticTree: unify interactivity detection logic 2026-03-10 19:02:55 +09:00
Adrià Arrufat
56f47ee574 Merge branch 'main' into semantic-tree 2026-03-10 17:26:34 +09:00
Adrià Arrufat
a318c6263d SemanticTree: improve visibility, AX roles and xpath generation
- Use `checkVisibility` for more accurate element visibility detection.
- Add support for color, date, file, and month AX roles.
- Optimize XPath generation by tracking sibling indices during the walk.
- Refine interactivity detection for form elements.
2026-03-10 09:23:06 +09:00
Adrià Arrufat
83ba974f94 SemanticTree: optimize tree walking and xpath generation
- Use a reusable buffer for XPaths to reduce allocations.
- Improve `display: none` detection with proper CSS parsing.
- Pass parent name to children to avoid redundant AXNode lookups.
- Use `getElementById` for faster datalist lookups.
2026-03-09 22:53:39 +09:00
Adrià Arrufat
85ebbe8759 SemanticTree: improve accessibility tree and name calculation
- Add more structural roles (banner, navigation, main, list, etc.).
- Implement fallback for accessible names (SVG titles, image alt text).
- Skip children for leaf-like semantic nodes to reduce redundancy.
- Disable pruning in the default semantic tree view.
2026-03-09 21:04:47 +09:00
Adrià Arrufat
61cba3f6eb Merge branch 'main' into semantic-tree 2026-03-09 20:13:47 +09:00
Adrià Arrufat
3c97332fd8 feat(dump): add semantic_tree and semantic_tree_text formats
Adds support for dumping the semantic tree in JSON or text format
via the --dump option. Updates the Config enum and usage help.
2026-03-09 18:23:52 +09:00
Adrià Arrufat
c77cb317c4 Merge branch 'main' into semantic-tree 2026-03-09 18:08:10 +09:00
Adrià Arrufat
c3a53752e7 CDP: simplify AXNode name extraction logic 2026-03-09 15:34:59 +09:00
Adrià Arrufat
0a5eb93565 SemanticTree: Implement compound component metadata 2026-03-09 13:42:53 +09:00
Adrià Arrufat
b8a3135835 SemanticTree: add pruning support and move logic to walk 2026-03-09 13:02:03 +09:00
Adrià Arrufat
330dfccb89 webapi/Element: add missing block tags and reorganize checks 2026-03-09 11:23:52 +09:00
Adrià Arrufat
d80e926015 SemanticTree: unify tree traversal using visitor pattern 2026-03-09 11:09:27 +09:00
Adrià Arrufat
2a2b067633 mcp: fix wrong merge 2026-03-09 10:37:21 +09:00
Adrià Arrufat
be73c14395 SemanticTree: rename dump to dumpJson and update log tags 2026-03-09 10:29:32 +09:00
Adrià Arrufat
9cd5afe5b6 Merge branch 'main' into semantic-tree 2026-03-09 10:18:54 +09:00
Adrià Arrufat
4ba40f2295 CDP: implement intelligent pruning for textified semantic tree output 2026-03-08 22:48:22 +09:00
Adrià Arrufat
b674c2e448 CDP/MCP: add highly compressed text format for semantic tree 2026-03-08 22:42:00 +09:00
Adrià Arrufat
b8139a6e83 CDP/MCP: improve Stagehand compatibility for semantic tree 2026-03-08 15:48:44 +09:00
Adrià Arrufat
bde5fc9264 Merge branch 'main' into semantic-tree 2026-03-08 08:18:08 +09:00
Adrià Arrufat
45705a3e29 webapi: move tag category logic to Tag enum 2026-03-06 16:34:23 +09:00
Adrià Arrufat
e0f0b9f210 SemanticTree: use AXRole enum for interactive role check 2026-03-06 16:26:08 +09:00
Adrià Arrufat
f2832447d4 SemanticTree: optimize tag and role filtering
* Refactored tag ignoring logic to use the el.getTag() enum switch
  instead of string comparisons, improving performance and safety.
* Replaced string comparisons for interactive roles with
  std.StaticStringMap.
* Renamed internal dumpNode method to dump for brevity.
2026-03-06 16:12:57 +09:00
Adrià Arrufat
471ba5baf6 String: refactor isAllWhitespace into String 2026-03-06 15:52:53 +09:00
Adrià Arrufat
248851701f Refactor: move SemanticTree to core and expose via MCP tools 2026-03-06 15:44:03 +09:00
Adrià Arrufat
0f46277b1f CDP: implement LP.getSemanticTree for native semantic DOM extraction 2026-03-06 15:29:32 +09:00
74 changed files with 2450 additions and 372 deletions

View File

@@ -13,7 +13,7 @@ inputs:
zig-v8:
description: 'zig v8 version to install'
required: false
default: 'v0.3.2'
default: 'v0.3.3'
v8:
description: 'v8 version to install'
required: false

View File

@@ -117,6 +117,105 @@ jobs:
BASE_URL=https://demo-browser.lightpanda.io/ node playwright/proxy_auth.js
kill `cat LPD.pid` `cat PROXY.id`
# e2e tests w/ web-bot-auth configuration on.
wba-demo-scripts:
name: wba-demo-scripts
needs: zig-build-release
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
with:
repository: 'lightpanda-io/demo'
fetch-depth: 0
- run: npm install
- name: download artifact
uses: actions/download-artifact@v4
with:
name: lightpanda-build-release
- run: chmod a+x ./lightpanda
- run: echo "${{ secrets.WBA_PRIVATE_KEY_PEM }}" > private_key.pem
- name: run end to end tests
run: |
./lightpanda serve \
--web_bot_auth_key_file private_key.pem \
--web_bot_auth_keyid ${{ vars.WBA_KEY_ID }} \
--web_bot_auth_domain ${{ vars.WBA_DOMAIN }} \
& echo $! > LPD.pid
go run runner/main.go
kill `cat LPD.pid`
- name: build proxy
run: |
cd proxy
go build
- name: run end to end tests through proxy
run: |
./proxy/proxy & echo $! > PROXY.id
./lightpanda serve \
--web_bot_auth_key_file private_key.pem \
--web_bot_auth_keyid ${{ vars.WBA_KEY_ID }} \
--web_bot_auth_domain ${{ vars.WBA_DOMAIN }} \
--http_proxy 'http://127.0.0.1:3000' \
& echo $! > LPD.pid
go run runner/main.go
kill `cat LPD.pid` `cat PROXY.id`
- name: run request interception through proxy
run: |
export PROXY_USERNAME=username PROXY_PASSWORD=password
./proxy/proxy & echo $! > PROXY.id
./lightpanda serve & echo $! > LPD.pid
URL=https://demo-browser.lightpanda.io/campfire-commerce/ node puppeteer/proxy_auth.js
BASE_URL=https://demo-browser.lightpanda.io/ node playwright/proxy_auth.js
kill `cat LPD.pid` `cat PROXY.id`
wba-test:
name: wba-test
needs: zig-build-release
env:
LIGHTPANDA_DISABLE_TELEMETRY: true
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
with:
repository: 'lightpanda-io/demo'
fetch-depth: 0
- run: echo "${{ secrets.WBA_PRIVATE_KEY_PEM }}" > private_key.pem
- name: download artifact
uses: actions/download-artifact@v4
with:
name: lightpanda-build-release
- run: chmod a+x ./lightpanda
- name: run wba test
run: |
node webbotauth/validator.js &
VALIDATOR_PID=$!
sleep 2
./lightpanda fetch http://127.0.0.1:8989/ \
--web_bot_auth_key_file private_key.pem \
--web_bot_auth_keyid ${{ vars.WBA_KEY_ID }} \
--web_bot_auth_domain ${{ vars.WBA_DOMAIN }}
wait $VALIDATOR_PID
cdp-and-hyperfine-bench:
name: cdp-and-hyperfine-bench
needs: zig-build-release

View File

@@ -3,7 +3,7 @@ FROM debian:stable-slim
ARG MINISIG=0.12
ARG ZIG_MINISIG=RWSGOq2NVecA2UPNdBUZykf1CCb147pkmdtYxgb3Ti+JO/wCYvhbAb/U
ARG V8=14.0.365.4
ARG ZIG_V8=v0.3.2
ARG ZIG_V8=v0.3.3
ARG TARGETPLATFORM
RUN apt-get update -yq && \

View File

@@ -5,8 +5,8 @@
.minimum_zig_version = "0.15.2",
.dependencies = .{
.v8 = .{
.url = "https://github.com/lightpanda-io/zig-v8-fork/archive/refs/tags/v0.3.2.tar.gz",
.hash = "v8-0.0.0-xddH6wx-BABNgL7YIDgbnFgKZuXZ68yZNngNSrV6OjrY",
.url = "https://github.com/lightpanda-io/zig-v8-fork/archive/refs/tags/v0.3.3.tar.gz",
.hash = "v8-0.0.0-xddH6yx3BAAGD9jSoq_ttt_bk9MectTU44s_HZxxE5LD",
},
// .v8 = .{ .path = "../zig-v8-fork" },
.brotli = .{

View File

@@ -23,6 +23,8 @@ const Allocator = std.mem.Allocator;
const log = @import("log.zig");
const dump = @import("browser/dump.zig");
const WebBotAuthConfig = @import("network/WebBotAuth.zig").Config;
pub const RunMode = enum {
help,
fetch,
@@ -161,6 +163,17 @@ pub fn cdpTimeout(self: *const Config) usize {
};
}
pub fn webBotAuth(self: *const Config) ?WebBotAuthConfig {
return switch (self.mode) {
inline .serve, .fetch, .mcp => |opts| WebBotAuthConfig{
.key_file = opts.common.web_bot_auth_key_file orelse return null,
.keyid = opts.common.web_bot_auth_keyid orelse return null,
.domain = opts.common.web_bot_auth_domain orelse return null,
},
.help, .version => null,
};
}
pub fn maxConnections(self: *const Config) u16 {
return switch (self.mode) {
.serve => |opts| opts.cdp_max_connections,
@@ -200,6 +213,8 @@ pub const DumpFormat = enum {
html,
markdown,
wpt,
semantic_tree,
semantic_tree_text,
};
pub const Fetch = struct {
@@ -225,6 +240,10 @@ pub const Common = struct {
log_format: ?log.Format = null,
log_filter_scopes: ?[]log.Scope = null,
user_agent_suffix: ?[]const u8 = null,
web_bot_auth_key_file: ?[]const u8 = null,
web_bot_auth_keyid: ?[]const u8 = null,
web_bot_auth_domain: ?[]const u8 = null,
};
/// Pre-formatted HTTP headers for reuse across Http and Client.
@@ -332,6 +351,14 @@ pub fn printUsageAndExit(self: *const Config, success: bool) void {
\\--user_agent_suffix
\\ Suffix to append to the Lightpanda/X.Y User-Agent
\\
\\--web_bot_auth_key_file
\\ Path to the Ed25519 private key PEM file.
\\
\\--web_bot_auth_keyid
\\ The JWK thumbprint of your public key.
\\
\\--web_bot_auth_domain
\\ Your domain e.g. yourdomain.com
;
// MAX_HELP_LEN|
@@ -346,7 +373,7 @@ pub fn printUsageAndExit(self: *const Config, success: bool) void {
\\
\\Options:
\\--dump Dumps document to stdout.
\\ Argument must be 'html' or 'markdown'.
\\ Argument must be 'html', 'markdown', 'semantic_tree', or 'semantic_tree_text'.
\\ Defaults to no dump.
\\
\\--strip_mode Comma separated list of tag groups to remove from dump
@@ -853,5 +880,32 @@ fn parseCommonArg(
return true;
}
if (std.mem.eql(u8, "--web_bot_auth_key_file", opt)) {
const str = args.next() orelse {
log.fatal(.app, "missing argument value", .{ .arg = "--web_bot_auth_key_file" });
return error.InvalidArgument;
};
common.web_bot_auth_key_file = try allocator.dupe(u8, str);
return true;
}
if (std.mem.eql(u8, "--web_bot_auth_keyid", opt)) {
const str = args.next() orelse {
log.fatal(.app, "missing argument value", .{ .arg = "--web_bot_auth_keyid" });
return error.InvalidArgument;
};
common.web_bot_auth_keyid = try allocator.dupe(u8, str);
return true;
}
if (std.mem.eql(u8, "--web_bot_auth_domain", opt)) {
const str = args.next() orelse {
log.fatal(.app, "missing argument value", .{ .arg = "--web_bot_auth_domain" });
return error.InvalidArgument;
};
common.web_bot_auth_domain = try allocator.dupe(u8, str);
return true;
}
return false;
}

450
src/SemanticTree.zig Normal file
View File

@@ -0,0 +1,450 @@
// Copyright (C) 2023-2026 Lightpanda (Selecy SAS)
//
// Francis Bouvier <francis@lightpanda.io>
// Pierre Tachoire <pierre@lightpanda.io>
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as
// published by the Free Software Foundation, either version 3 of the
// License, or (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. See <https://www.gnu.org/licenses/>.
const std = @import("std");
const lp = @import("lightpanda");
const log = @import("log.zig");
const isAllWhitespace = @import("string.zig").isAllWhitespace;
const Page = lp.Page;
const interactive = @import("browser/interactive.zig");
const CData = @import("browser/webapi/CData.zig");
const Element = @import("browser/webapi/Element.zig");
const Node = @import("browser/webapi/Node.zig");
const AXNode = @import("cdp/AXNode.zig");
const CDPNode = @import("cdp/Node.zig");
const Self = @This();
dom_node: *Node,
registry: *CDPNode.Registry,
page: *Page,
arena: std.mem.Allocator,
prune: bool = false,
pub fn jsonStringify(self: @This(), jw: *std.json.Stringify) error{WriteFailed}!void {
var visitor = JsonVisitor{ .jw = jw, .tree = self };
var xpath_buffer: std.ArrayList(u8) = .{};
const listener_targets = interactive.buildListenerTargetMap(self.page, self.arena) catch |err| {
log.err(.app, "listener map failed", .{ .err = err });
return error.WriteFailed;
};
self.walk(self.dom_node, &xpath_buffer, null, &visitor, 1, listener_targets) catch |err| {
log.err(.app, "semantic tree json dump failed", .{ .err = err });
return error.WriteFailed;
};
}
pub fn textStringify(self: @This(), writer: *std.Io.Writer) error{WriteFailed}!void {
var visitor = TextVisitor{ .writer = writer, .tree = self, .depth = 0 };
var xpath_buffer: std.ArrayList(u8) = .empty;
const listener_targets = interactive.buildListenerTargetMap(self.page, self.arena) catch |err| {
log.err(.app, "listener map failed", .{ .err = err });
return error.WriteFailed;
};
self.walk(self.dom_node, &xpath_buffer, null, &visitor, 1, listener_targets) catch |err| {
log.err(.app, "semantic tree text dump failed", .{ .err = err });
return error.WriteFailed;
};
}
const OptionData = struct {
value: []const u8,
text: []const u8,
selected: bool,
};
const NodeData = struct {
id: u32,
axn: AXNode,
role: []const u8,
name: ?[]const u8,
value: ?[]const u8,
options: ?[]OptionData = null,
xpath: []const u8,
is_interactive: bool,
node_name: []const u8,
};
fn walk(self: @This(), node: *Node, xpath_buffer: *std.ArrayList(u8), parent_name: ?[]const u8, visitor: anytype, index: usize, listener_targets: interactive.ListenerTargetMap) !void {
// 1. Skip non-content nodes
if (node.is(Element)) |el| {
const tag = el.getTag();
if (tag.isMetadata() or tag == .svg) return;
// We handle options/optgroups natively inside their parents, skip them in the general walk
if (tag == .datalist or tag == .option or tag == .optgroup) return;
// Check visibility using the engine's checkVisibility which handles CSS display: none
if (!el.checkVisibility(self.page)) {
return;
}
if (el.is(Element.Html)) |html_el| {
if (html_el.getHidden()) return;
}
} else if (node.is(CData.Text)) |text_node| {
const text = text_node.getWholeText();
if (isAllWhitespace(text)) {
return;
}
} else if (node._type != .document and node._type != .document_fragment) {
return;
}
const cdp_node = try self.registry.register(node);
const axn = AXNode.fromNode(node);
const role = try axn.getRole();
var is_interactive = false;
var value: ?[]const u8 = null;
var options: ?[]OptionData = null;
var node_name: []const u8 = "text";
if (node.is(Element)) |el| {
node_name = el.getTagNameLower();
if (el.is(Element.Html.Input)) |input| {
value = input.getValue();
if (el.getAttributeSafe(comptime lp.String.wrap("list"))) |list_id| {
options = try extractDataListOptions(list_id, self.page, self.arena);
}
} else if (el.is(Element.Html.TextArea)) |textarea| {
value = textarea.getValue();
} else if (el.is(Element.Html.Select)) |select| {
value = select.getValue(self.page);
options = try extractSelectOptions(el.asNode(), self.page, self.arena);
}
if (el.is(Element.Html)) |html_el| {
if (interactive.classifyInteractivity(el, html_el, listener_targets) != null) {
is_interactive = true;
}
}
} else if (node._type == .document or node._type == .document_fragment) {
node_name = "root";
}
const initial_xpath_len = xpath_buffer.items.len;
try appendXPathSegment(node, xpath_buffer.writer(self.arena), index);
const xpath = xpath_buffer.items;
var name = try axn.getName(self.page, self.arena);
const has_explicit_label = if (node.is(Element)) |el|
el.getAttributeSafe(.wrap("aria-label")) != null or el.getAttributeSafe(.wrap("title")) != null
else
false;
const structural = isStructuralRole(role);
// Filter out computed concatenated names for generic containers without explicit labels.
// This prevents token bloat and ensures their StaticText children aren't incorrectly pruned.
// We ignore interactivity because a generic wrapper with an event listener still shouldn't hoist all text.
if (name != null and structural and !has_explicit_label) {
name = null;
}
var data = NodeData{
.id = cdp_node.id,
.axn = axn,
.role = role,
.name = name,
.value = value,
.options = options,
.xpath = xpath,
.is_interactive = is_interactive,
.node_name = node_name,
};
var should_visit = true;
if (self.prune) {
if (structural and !is_interactive and !has_explicit_label) {
should_visit = false;
}
if (std.mem.eql(u8, role, "StaticText") and node._parent != null) {
if (parent_name != null and name != null and std.mem.indexOf(u8, parent_name.?, name.?) != null) {
should_visit = false;
}
}
}
var did_visit = false;
var should_walk_children = true;
if (should_visit) {
should_walk_children = try visitor.visit(node, &data);
did_visit = true; // Always true if should_visit was true, because visit() executed and opened structures
} else {
// If we skip the node, we must NOT tell the visitor to close it later
did_visit = false;
}
if (should_walk_children) {
// If we are printing this node normally OR skipping it and unrolling its children,
// we walk the children iterator.
var it = node.childrenIterator();
var tag_counts = std.StringArrayHashMap(usize).init(self.arena);
while (it.next()) |child| {
var tag: []const u8 = "text()";
if (child.is(Element)) |el| {
tag = el.getTagNameLower();
}
const gop = try tag_counts.getOrPut(tag);
if (!gop.found_existing) {
gop.value_ptr.* = 0;
}
gop.value_ptr.* += 1;
try self.walk(child, xpath_buffer, name, visitor, gop.value_ptr.*, listener_targets);
}
}
if (did_visit) {
try visitor.leave();
}
xpath_buffer.shrinkRetainingCapacity(initial_xpath_len);
}
fn extractSelectOptions(node: *Node, page: *Page, arena: std.mem.Allocator) ![]OptionData {
var options = std.ArrayListUnmanaged(OptionData){};
var it = node.childrenIterator();
while (it.next()) |child| {
if (child.is(Element)) |el| {
if (el.getTag() == .option) {
if (el.is(Element.Html.Option)) |opt| {
const text = opt.getText(page);
const value = opt.getValue(page);
const selected = opt.getSelected();
try options.append(arena, .{ .text = text, .value = value, .selected = selected });
}
} else if (el.getTag() == .optgroup) {
var group_it = child.childrenIterator();
while (group_it.next()) |group_child| {
if (group_child.is(Element.Html.Option)) |opt| {
const text = opt.getText(page);
const value = opt.getValue(page);
const selected = opt.getSelected();
try options.append(arena, .{ .text = text, .value = value, .selected = selected });
}
}
}
}
}
return options.toOwnedSlice(arena);
}
fn extractDataListOptions(list_id: []const u8, page: *Page, arena: std.mem.Allocator) !?[]OptionData {
if (page.document.getElementById(list_id, page)) |referenced_el| {
if (referenced_el.getTag() == .datalist) {
return try extractSelectOptions(referenced_el.asNode(), page, arena);
}
}
return null;
}
fn appendXPathSegment(node: *Node, writer: anytype, index: usize) !void {
if (node.is(Element)) |el| {
const tag = el.getTagNameLower();
try std.fmt.format(writer, "/{s}[{d}]", .{ tag, index });
} else if (node.is(CData.Text)) |_| {
try std.fmt.format(writer, "/text()[{d}]", .{index});
}
}
const JsonVisitor = struct {
jw: *std.json.Stringify,
tree: Self,
pub fn visit(self: *JsonVisitor, node: *Node, data: *NodeData) !bool {
try self.jw.beginObject();
try self.jw.objectField("nodeId");
try self.jw.write(try std.fmt.allocPrint(self.tree.arena, "{d}", .{data.id}));
try self.jw.objectField("backendDOMNodeId");
try self.jw.write(data.id);
try self.jw.objectField("nodeName");
try self.jw.write(data.node_name);
try self.jw.objectField("xpath");
try self.jw.write(data.xpath);
if (node.is(Element)) |el| {
try self.jw.objectField("nodeType");
try self.jw.write(1);
try self.jw.objectField("isInteractive");
try self.jw.write(data.is_interactive);
try self.jw.objectField("role");
try self.jw.write(data.role);
if (data.name) |name| {
if (name.len > 0) {
try self.jw.objectField("name");
try self.jw.write(name);
}
}
if (data.value) |value| {
try self.jw.objectField("value");
try self.jw.write(value);
}
if (el._attributes) |attrs| {
try self.jw.objectField("attributes");
try self.jw.beginObject();
var iter = attrs.iterator();
while (iter.next()) |attr| {
try self.jw.objectField(attr._name.str());
try self.jw.write(attr._value.str());
}
try self.jw.endObject();
}
if (data.options) |options| {
try self.jw.objectField("options");
try self.jw.beginArray();
for (options) |opt| {
try self.jw.beginObject();
try self.jw.objectField("value");
try self.jw.write(opt.value);
try self.jw.objectField("text");
try self.jw.write(opt.text);
try self.jw.objectField("selected");
try self.jw.write(opt.selected);
try self.jw.endObject();
}
try self.jw.endArray();
}
} else if (node.is(CData.Text)) |text_node| {
try self.jw.objectField("nodeType");
try self.jw.write(3);
try self.jw.objectField("nodeValue");
try self.jw.write(text_node.getWholeText());
} else {
try self.jw.objectField("nodeType");
try self.jw.write(9);
}
try self.jw.objectField("children");
try self.jw.beginArray();
if (data.options != null) {
// Signal to not walk children, as we handled them natively
return false;
}
return true;
}
pub fn leave(self: *JsonVisitor) !void {
try self.jw.endArray();
try self.jw.endObject();
}
};
fn isStructuralRole(role: []const u8) bool {
const structural_roles = std.StaticStringMap(void).initComptime(.{
.{ "none", {} },
.{ "generic", {} },
.{ "InlineTextBox", {} },
.{ "banner", {} },
.{ "navigation", {} },
.{ "main", {} },
.{ "list", {} },
.{ "listitem", {} },
.{ "table", {} },
.{ "rowgroup", {} },
.{ "row", {} },
.{ "cell", {} },
.{ "region", {} },
});
return structural_roles.has(role);
}
const TextVisitor = struct {
writer: *std.Io.Writer,
tree: Self,
depth: usize,
pub fn visit(self: *TextVisitor, node: *Node, data: *NodeData) !bool {
// Format: " [12] link: Hacker News (value)"
for (0..(self.depth * 2)) |_| {
try self.writer.writeByte(' ');
}
try self.writer.print("[{d}] {s}: ", .{ data.id, data.role });
if (data.name) |n| {
if (n.len > 0) {
try self.writer.writeAll(n);
}
} else if (node.is(CData.Text)) |text_node| {
const trimmed = std.mem.trim(u8, text_node.getWholeText(), " \t\r\n");
if (trimmed.len > 0) {
try self.writer.writeAll(trimmed);
}
}
if (data.value) |v| {
if (v.len > 0) {
try self.writer.print(" (value: {s})", .{v});
}
}
if (data.options) |options| {
try self.writer.writeAll(" options: [");
for (options, 0..) |opt, i| {
if (i > 0) try self.writer.writeAll(", ");
try self.writer.print("'{s}'", .{opt.value});
if (opt.selected) {
try self.writer.writeAll(" (selected)");
}
}
try self.writer.writeAll("]\n");
self.depth += 1;
return false; // Native handling complete, do not walk children
}
try self.writer.writeByte('\n');
self.depth += 1;
// If this is a leaf-like semantic node and we already have a name,
// skip children to avoid redundant StaticText or noise.
const is_leaf_semantic = std.mem.eql(u8, data.role, "link") or
std.mem.eql(u8, data.role, "button") or
std.mem.eql(u8, data.role, "heading") or
std.mem.eql(u8, data.role, "code");
if (is_leaf_semantic and data.name != null and data.name.?.len > 0) {
return false;
}
return true;
}
pub fn leave(self: *TextVisitor) !void {
if (self.depth > 0) {
self.depth -= 1;
}
}
};

View File

@@ -265,13 +265,15 @@ pub fn blob(_: *const Factory, arena: Allocator, child: anytype) !*@TypeOf(child
return chain.get(1);
}
pub fn abstractRange(self: *Factory, child: anytype, page: *Page) !*@TypeOf(child) {
const allocator = self._slab.allocator();
const chain = try PrototypeChain(&.{ AbstractRange, @TypeOf(child) }).allocate(allocator);
pub fn abstractRange(_: *const Factory, arena: Allocator, child: anytype, page: *Page) !*@TypeOf(child) {
const chain = try PrototypeChain(&.{ AbstractRange, @TypeOf(child) }).allocate(arena);
const doc = page.document.asNode();
const abstract_range = chain.get(0);
abstract_range.* = AbstractRange{
._rc = 0,
._arena = arena,
._page_id = page.id,
._type = unionInit(AbstractRange.Type, chain.get(1)),
._end_offset = 0,
._start_offset = 0,

View File

@@ -30,6 +30,7 @@ const Notification = @import("../Notification.zig");
const CookieJar = @import("../browser/webapi/storage/Cookie.zig").Jar;
const Robots = @import("../network/Robots.zig");
const RobotStore = Robots.RobotStore;
const WebBotAuth = @import("../network/WebBotAuth.zig");
const Allocator = std.mem.Allocator;
const ArenaAllocator = std.heap.ArenaAllocator;
@@ -66,9 +67,18 @@ active: usize,
// 'networkAlmostIdle' Page.lifecycleEvent in CDP).
intercepted: usize,
// Our easy handles, managed by a curl multi.
// Our curl multi handle.
handles: Net.Handles,
// Connections currently in this client's curl_multi.
in_use: std.DoublyLinkedList = .{},
// Connections that failed to be removed from curl_multi during perform.
dirty: std.DoublyLinkedList = .{},
// Whether we're currently inside a curl_multi_perform call.
performing: bool = false,
// Use to generate the next request ID
next_request_id: u32 = 0,
@@ -88,8 +98,8 @@ pending_robots_queue: std.StringHashMapUnmanaged(std.ArrayList(Request)) = .empt
// request. These wil come and go with each request.
transfer_pool: std.heap.MemoryPool(Transfer),
// only needed for CDP which can change the proxy and then restore it. When
// restoring, this originally-configured value is what it goes to.
// The current proxy. CDP can change it, restoreOriginalProxy restores
// from config.
http_proxy: ?[:0]const u8 = null,
// track if the client use a proxy for connections.
@@ -97,6 +107,9 @@ http_proxy: ?[:0]const u8 = null,
// CDP.
use_proxy: bool,
// Current TLS verification state, applied per-connection in makeRequest.
tls_verify: bool = true,
cdp_client: ?CDPClient = null,
// libcurl can monitor arbitrary sockets, this lets us use libcurl to poll
@@ -126,13 +139,8 @@ pub fn init(allocator: Allocator, network: *Network) !*Client {
const client = try allocator.create(Client);
errdefer allocator.destroy(client);
var handles = try Net.Handles.init(allocator, network.ca_blob, network.config);
errdefer handles.deinit(allocator);
// Set transfer callbacks on each connection.
for (handles.connections) |*conn| {
try conn.setCallbacks(Transfer.headerCallback, Transfer.dataCallback);
}
var handles = try Net.Handles.init(network.config);
errdefer handles.deinit();
const http_proxy = network.config.httpProxy();
@@ -145,6 +153,7 @@ pub fn init(allocator: Allocator, network: *Network) !*Client {
.network = network,
.http_proxy = http_proxy,
.use_proxy = http_proxy != null,
.tls_verify = network.config.tlsVerifyHost(),
.transfer_pool = transfer_pool,
};
@@ -153,7 +162,7 @@ pub fn init(allocator: Allocator, network: *Network) !*Client {
pub fn deinit(self: *Client) void {
self.abort();
self.handles.deinit(self.allocator);
self.handles.deinit();
self.transfer_pool.deinit();
@@ -182,14 +191,14 @@ pub fn abortFrame(self: *Client, frame_id: u32) void {
// but abort can avoid the frame_id check at comptime.
fn _abort(self: *Client, comptime abort_all: bool, frame_id: u32) void {
{
var q = &self.handles.in_use;
var q = &self.in_use;
var n = q.first;
while (n) |node| {
n = node.next;
const conn: *Net.Connection = @fieldParentPtr("node", node);
var transfer = Transfer.fromConnection(conn) catch |err| {
// Let's cleanup what we can
self.handles.remove(conn);
self.removeConn(conn);
log.err(.http, "get private info", .{ .err = err, .source = "abort" });
continue;
};
@@ -226,8 +235,7 @@ fn _abort(self: *Client, comptime abort_all: bool, frame_id: u32) void {
}
if (comptime IS_DEBUG and abort_all) {
std.debug.assert(self.handles.in_use.first == null);
std.debug.assert(self.handles.available.len() == self.handles.connections.len);
std.debug.assert(self.in_use.first == null);
const running = self.handles.perform() catch |err| {
lp.assert(false, "multi perform in abort", .{ .err = err });
@@ -237,15 +245,12 @@ fn _abort(self: *Client, comptime abort_all: bool, frame_id: u32) void {
}
pub fn tick(self: *Client, timeout_ms: u32) !PerformStatus {
while (true) {
if (self.handles.hasAvailable() == false) {
while (self.queue.popFirst()) |queue_node| {
const conn = self.network.getConnection() orelse {
self.queue.prepend(queue_node);
break;
}
const queue_node = self.queue.popFirst() orelse break;
};
const transfer: *Transfer = @fieldParentPtr("_node", queue_node);
// we know this exists, because we checked hasAvailable() above
const conn = self.handles.get().?;
try self.makeRequest(conn, transfer);
}
return self.perform(@intCast(timeout_ms));
@@ -529,8 +534,8 @@ fn waitForInterceptedResponse(self: *Client, transfer: *Transfer) !bool {
fn process(self: *Client, transfer: *Transfer) !void {
// libcurl doesn't allow recursive calls, if we're in a `perform()` operation
// then we _have_ to queue this.
if (self.handles.performing == false) {
if (self.handles.get()) |conn| {
if (self.performing == false) {
if (self.network.getConnection()) |conn| {
return self.makeRequest(conn, transfer);
}
}
@@ -644,10 +649,7 @@ fn requestFailed(transfer: *Transfer, err: anyerror, comptime execute_callback:
// can be changed at any point in the easy's lifecycle.
pub fn changeProxy(self: *Client, proxy: [:0]const u8) !void {
try self.ensureNoActiveConnection();
for (self.handles.connections) |*conn| {
try conn.setProxy(proxy.ptr);
}
self.http_proxy = proxy;
self.use_proxy = true;
}
@@ -656,31 +658,21 @@ pub fn changeProxy(self: *Client, proxy: [:0]const u8) !void {
pub fn restoreOriginalProxy(self: *Client) !void {
try self.ensureNoActiveConnection();
const proxy = if (self.http_proxy) |p| p.ptr else null;
for (self.handles.connections) |*conn| {
try conn.setProxy(proxy);
}
self.use_proxy = proxy != null;
self.http_proxy = self.network.config.httpProxy();
self.use_proxy = self.http_proxy != null;
}
// Enable TLS verification on all connections.
pub fn enableTlsVerify(self: *Client) !void {
pub fn setTlsVerify(self: *Client, verify: bool) !void {
// Remove inflight connections check on enable TLS b/c chromiumoxide calls
// the command during navigate and Curl seems to accept it...
for (self.handles.connections) |*conn| {
try conn.setTlsVerify(true, self.use_proxy);
}
}
// Disable TLS verification on all connections.
pub fn disableTlsVerify(self: *Client) !void {
// Remove inflight connections check on disable TLS b/c chromiumoxide calls
// the command during navigate and Curl seems to accept it...
for (self.handles.connections) |*conn| {
try conn.setTlsVerify(false, self.use_proxy);
var it = self.in_use.first;
while (it) |node| : (it = node.next) {
const conn: *Net.Connection = @fieldParentPtr("node", node);
try conn.setTlsVerify(verify, self.use_proxy);
}
self.tls_verify = verify;
}
fn makeRequest(self: *Client, conn: *Net.Connection, transfer: *Transfer) anyerror!void {
@@ -691,9 +683,14 @@ fn makeRequest(self: *Client, conn: *Net.Connection, transfer: *Transfer) anyerr
errdefer {
transfer._conn = null;
transfer.deinit();
self.handles.isAvailable(conn);
self.releaseConn(conn);
}
// Set callbacks and per-client settings on the pooled connection.
try conn.setCallbacks(Transfer.headerCallback, Transfer.dataCallback);
try conn.setProxy(self.http_proxy);
try conn.setTlsVerify(self.tls_verify, self.use_proxy);
try conn.setURL(req.url);
try conn.setMethod(req.method);
if (req.body) |b| {
@@ -706,6 +703,12 @@ fn makeRequest(self: *Client, conn: *Net.Connection, transfer: *Transfer) anyerr
try conn.secretHeaders(&header_list, &self.network.config.http_headers); // Add headers that must be hidden from intercepts
try conn.setHeaders(&header_list);
// If we have WebBotAuth, sign our request.
if (self.network.web_bot_auth) |*wba| {
const authority = URL.getHost(req.url);
try wba.signRequest(transfer.arena.allocator(), &header_list, authority);
}
// Add cookies.
if (header_list.cookies) |cookies| {
try conn.setCookies(cookies);
@@ -728,10 +731,12 @@ fn makeRequest(self: *Client, conn: *Net.Connection, transfer: *Transfer) anyerr
// fails BEFORE `curl_multi_add_handle` succeeds, the we still need to do
// cleanup. But if things fail after `curl_multi_add_handle`, we expect
// perfom to pickup the failure and cleanup.
self.in_use.append(&conn.node);
self.handles.add(conn) catch |err| {
transfer._conn = null;
transfer.deinit();
self.handles.isAvailable(conn);
self.in_use.remove(&conn.node);
self.releaseConn(conn);
return err;
};
@@ -752,7 +757,22 @@ pub const PerformStatus = enum {
};
fn perform(self: *Client, timeout_ms: c_int) !PerformStatus {
const running = try self.handles.perform();
const running = blk: {
self.performing = true;
defer self.performing = false;
break :blk try self.handles.perform();
};
// Process dirty connections — return them to Runtime pool.
while (self.dirty.popFirst()) |node| {
const conn: *Net.Connection = @fieldParentPtr("node", node);
self.handles.remove(conn) catch |err| {
log.fatal(.http, "multi remove handle", .{ .err = err, .src = "perform" });
@panic("multi_remove_handle");
};
self.releaseConn(conn);
}
// We're potentially going to block for a while until we get data. Process
// whatever messages we have waiting ahead of time.
@@ -871,11 +891,26 @@ fn processMessages(self: *Client) !bool {
fn endTransfer(self: *Client, transfer: *Transfer) void {
const conn = transfer._conn.?;
self.handles.remove(conn);
self.removeConn(conn);
transfer._conn = null;
self.active -= 1;
}
fn removeConn(self: *Client, conn: *Net.Connection) void {
self.in_use.remove(&conn.node);
if (self.handles.remove(conn)) {
self.releaseConn(conn);
} else |_| {
// Can happen if we're in a perform() call, so we'll queue this
// for cleanup later.
self.dirty.append(&conn.node);
}
}
fn releaseConn(self: *Client, conn: *Net.Connection) void {
self.network.releaseConnection(conn);
}
fn ensureNoActiveConnection(self: *const Client) !void {
if (self.active > 0) {
return error.InflightConnection;
@@ -898,7 +933,7 @@ pub const RequestCookie = struct {
if (arr.items.len > 0) {
try arr.append(temp, 0); //null terminate
headers.cookies = @ptrCast(arr.items.ptr);
headers.cookies = @as([*c]const u8, @ptrCast(arr.items.ptr));
}
}
};
@@ -1023,7 +1058,7 @@ pub const Transfer = struct {
fn deinit(self: *Transfer) void {
self.req.headers.deinit();
if (self._conn) |conn| {
self.client.handles.remove(conn);
self.client.removeConn(conn);
}
self.arena.deinit();
self.client.transfer_pool.destroy(self);
@@ -1093,7 +1128,7 @@ pub const Transfer = struct {
requestFailed(self, err, true);
const client = self.client;
if (self._performing or client.handles.performing) {
if (self._performing or client.performing) {
// We're currently in a curl_multi_perform. We cannot call endTransfer
// as that calls curl_multi_remove_handle, and you can't do that
// from a curl callback. Instead, we flag this transfer and all of
@@ -1258,6 +1293,16 @@ pub const Transfer = struct {
if (buf_len < 3) {
// could be \r\n or \n.
// We get the last header line.
if (transfer._redirecting) {
// parse and set cookies for the redirection.
redirectionCookies(transfer, &conn) catch |err| {
if (comptime IS_DEBUG) {
log.debug(.http, "redirection cookies", .{ .err = err });
}
return 0;
};
}
return buf_len;
}
@@ -1324,38 +1369,22 @@ pub const Transfer = struct {
transfer.bytes_received += buf_len;
}
if (buf_len > 2) {
if (transfer._auth_challenge != null) {
// try to parse auth challenge.
if (std.ascii.startsWithIgnoreCase(header, "WWW-Authenticate") or
std.ascii.startsWithIgnoreCase(header, "Proxy-Authenticate"))
{
const ac = AuthChallenge.parse(
transfer._auth_challenge.?.status,
header,
) catch |err| {
// We can't parse the auth challenge
log.err(.http, "parse auth challenge", .{ .err = err, .header = header });
// Should we cancel the request? I don't think so.
return buf_len;
};
transfer._auth_challenge = ac;
}
if (transfer._auth_challenge != null) {
// try to parse auth challenge.
if (std.ascii.startsWithIgnoreCase(header, "WWW-Authenticate") or
std.ascii.startsWithIgnoreCase(header, "Proxy-Authenticate"))
{
const ac = AuthChallenge.parse(
transfer._auth_challenge.?.status,
header,
) catch |err| {
// We can't parse the auth challenge
log.err(.http, "parse auth challenge", .{ .err = err, .header = header });
// Should we cancel the request? I don't think so.
return buf_len;
};
transfer._auth_challenge = ac;
}
return buf_len;
}
// Starting here, we get the last header line.
if (transfer._redirecting) {
// parse and set cookies for the redirection.
redirectionCookies(transfer, &conn) catch |err| {
if (comptime IS_DEBUG) {
log.debug(.http, "redirection cookies", .{ .err = err });
}
return 0;
};
return buf_len;
}
return buf_len;

View File

@@ -80,6 +80,8 @@ pub const BUF_SIZE = 1024;
const Page = @This();
id: u32,
// This is the "id" of the frame. It can be re-used from page-to-page, e.g.
// when navigating.
_frame_id: u32,
@@ -254,6 +256,7 @@ pub fn init(self: *Page, frame_id: u32, session: *Session, parent: ?*Page) !void
})).asDocument();
self.* = .{
.id = session.nextPageId(),
.js = undefined,
.parent = parent,
.arena = session.page_arena,
@@ -404,6 +407,18 @@ pub fn isSameOrigin(self: *const Page, url: [:0]const u8) !bool {
return std.mem.startsWith(u8, url, current_origin);
}
/// Look up a blob URL in this page's registry, walking up the parent chain.
pub fn lookupBlobUrl(self: *Page, url: []const u8) ?*Blob {
var current: ?*Page = self;
while (current) |page| {
if (page._blob_urls.get(url)) |blob| {
return blob;
}
current = page.parent;
}
return null;
}
pub fn navigate(self: *Page, request_url: [:0]const u8, opts: NavigateOpts) !void {
lp.assert(self._load_state == .waiting, "page.renavigate", .{});
const session = self._session;
@@ -419,12 +434,17 @@ pub fn navigate(self: *Page, request_url: [:0]const u8, opts: NavigateOpts) !voi
.type = self._type,
});
// if the url is about:blank, we load an empty HTML document in the
// page and dispatch the events.
if (std.mem.eql(u8, "about:blank", request_url)) {
self.url = "about:blank";
// Handle synthetic navigations: about:blank and blob: URLs
const is_about_blank = std.mem.eql(u8, "about:blank", request_url);
const is_blob = !is_about_blank and std.mem.startsWith(u8, request_url, "blob:");
if (self.parent) |parent| {
if (is_about_blank or is_blob) {
self.url = if (is_about_blank) "about:blank" else try self.arena.dupeZ(u8, request_url);
if (is_blob) {
// strip out blob:
self.origin = try URL.getOrigin(self.arena, request_url[5.. :0]);
} else if (self.parent) |parent| {
self.origin = parent.origin;
} else {
self.origin = null;
@@ -435,10 +455,22 @@ pub fn navigate(self: *Page, request_url: [:0]const u8, opts: NavigateOpts) !voi
// It's important to force a reset during the following navigation.
self._parse_state = .complete;
self.document.injectBlank(self) catch |err| {
log.err(.browser, "inject blank", .{ .err = err });
return error.InjectBlankFailed;
};
// Content injection
if (is_blob) {
const blob = self.lookupBlobUrl(request_url) orelse {
log.warn(.js, "invalid blob", .{ .url = request_url });
return error.BlobNotFound;
};
const parse_arena = try self.getArena(.{ .debug = "Page.parseBlob" });
defer self.releaseArena(parse_arena);
var parser = Parser.init(parse_arena, self.document.asNode(), self);
parser.parse(blob._slice);
} else {
self.document.injectBlank(self) catch |err| {
log.err(.browser, "inject blank", .{ .err = err });
return error.InjectBlankFailed;
};
}
self.documentIsComplete();
session.notification.dispatch(.page_navigate, &.{
@@ -452,7 +484,7 @@ pub fn navigate(self: *Page, request_url: [:0]const u8, opts: NavigateOpts) !voi
// Record telemetry for navigation
session.browser.app.telemetry.record(.{
.navigate = .{
.tls = false, // about:blank is not TLS
.tls = false, // about:blank and blob: are not TLS
.proxy = session.browser.app.config.httpProxy() != null,
},
});
@@ -562,12 +594,9 @@ fn scheduleNavigationWithArena(originator: *Page, arena: Allocator, request_url:
};
const target = switch (nt) {
.form, .anchor => |p| p,
.script => |p| p orelse originator,
.iframe => |iframe| iframe._window.?._page, // only an frame with existing content (i.e. a window) can be navigated
.anchor, .form => |node| blk: {
const doc = node.ownerDocument(originator) orelse break :blk originator;
break :blk doc._page orelse originator;
},
};
const session = target._session;
@@ -763,6 +792,10 @@ fn _documentIsComplete(self: *Page) !void {
try self._event_manager.dispatchDirect(window_target, pageshow_event, self.window._on_pageshow, .{ .context = "page show" });
}
if (comptime IS_DEBUG) {
log.debug(.page, "load", .{ .url = self.url, .type = self._type });
}
self.notifyParentLoadComplete();
}
@@ -977,6 +1010,14 @@ pub fn scriptAddedCallback(self: *Page, comptime from_parser: bool, script: *Ele
return;
}
if (comptime from_parser) {
// parser-inserted scripts have force-async set to false, but only if
// they have src or non-empty content
if (script._src.len > 0 or script.asNode().firstChild() != null) {
script._force_async = false;
}
}
self._script_manager.addFromElement(from_parser, script, "parsing") catch |err| {
log.err(.page, "page.scriptAddedCallback", .{
.err = err,
@@ -1459,6 +1500,8 @@ pub fn adoptNodeTree(self: *Page, node: *Node, new_owner: *Document) !void {
}
pub fn createElementNS(self: *Page, namespace: Element.Namespace, name: []const u8, attribute_iterator: anytype) !*Node {
const from_parser = @TypeOf(attribute_iterator) == Parser.AttributeIterator;
switch (namespace) {
.html => {
switch (name.len) {
@@ -2129,6 +2172,15 @@ pub fn createElementNS(self: *Page, namespace: Element.Namespace, name: []const
self.js.localScope(&ls);
defer ls.deinit();
if (from_parser) {
// There are some things custom elements aren't allowed to do
// when we're parsing.
self.document._throw_on_dynamic_markup_insertion_counter += 1;
}
defer if (from_parser) {
self.document._throw_on_dynamic_markup_insertion_counter -= 1;
};
var caught: JS.TryCatch.Caught = undefined;
_ = ls.toLocal(def.constructor).newInstance(&caught) catch |err| {
log.warn(.js, "custom element constructor", .{ .name = name, .err = err, .caught = caught, .type = self._type, .url = self.url });
@@ -2599,6 +2651,8 @@ pub fn _insertNodeRelative(self: *Page, comptime from_parser: bool, parent: *Nod
}
}
const parent_is_connected = parent.isConnected();
// Tri-state behavior for mutations:
// 1. from_parser=true, parse_mode=document -> no mutations (initial document parse)
// 2. from_parser=true, parse_mode=fragment -> mutations (innerHTML additions)
@@ -2614,6 +2668,15 @@ pub fn _insertNodeRelative(self: *Page, comptime from_parser: bool, parent: *Nod
// When the parser adds the node, nodeIsReady is only called when the
// nodeComplete() callback is executed.
try self.nodeIsReady(false, child);
// Check if text was added to a script that hasn't started yet.
if (child._type == .cdata and parent_is_connected) {
if (parent.is(Element.Html.Script)) |script| {
if (!script._executed) {
try self.nodeIsReady(false, parent);
}
}
}
}
// Notify mutation observers about childList change
@@ -2652,7 +2715,6 @@ pub fn _insertNodeRelative(self: *Page, comptime from_parser: bool, parent: *Nod
}
const parent_in_shadow = parent.is(ShadowRoot) != null or parent.isInShadowTree();
const parent_is_connected = parent.isConnected();
if (!parent_in_shadow and !parent_is_connected) {
return;
@@ -3106,9 +3168,9 @@ const NavigationType = enum {
};
const Navigation = union(NavigationType) {
form: *Node,
form: *Page,
script: ?*Page,
anchor: *Node,
anchor: *Page,
iframe: *IFrame,
};
@@ -3120,6 +3182,69 @@ pub const QueuedNavigation = struct {
navigation_type: NavigationType,
};
/// Resolves a target attribute value (e.g., "_self", "_parent", "_top", or frame name)
/// to the appropriate Page to navigate.
/// Returns null if the target is "_blank" (which would open a new window/tab).
/// Note: Callers should handle empty target separately (for owner document resolution).
pub fn resolveTargetPage(self: *Page, target_name: []const u8) ?*Page {
if (std.ascii.eqlIgnoreCase(target_name, "_self")) {
return self;
}
if (std.ascii.eqlIgnoreCase(target_name, "_blank")) {
return null;
}
if (std.ascii.eqlIgnoreCase(target_name, "_parent")) {
return self.parent orelse self;
}
if (std.ascii.eqlIgnoreCase(target_name, "_top")) {
var page = self;
while (page.parent) |p| {
page = p;
}
return page;
}
// Named frame lookup: search current page's descendants first, then from root
// This follows the HTML spec's "implementation-defined" search order.
if (findFrameByName(self, target_name)) |frame_page| {
return frame_page;
}
// If not found in descendants, search from root (catches siblings and ancestors' descendants)
var root = self;
while (root.parent) |p| {
root = p;
}
if (root != self) {
if (findFrameByName(root, target_name)) |frame_page| {
return frame_page;
}
}
// If no frame found with that name, navigate in current page
// (this matches browser behavior - unknown targets act like _self)
return self;
}
fn findFrameByName(page: *Page, name: []const u8) ?*Page {
for (page.frames.items) |frame| {
if (frame.iframe) |iframe| {
const frame_name = iframe.asElement().getAttributeSafe(comptime .wrap("name")) orelse "";
if (std.mem.eql(u8, frame_name, name)) {
return frame;
}
}
// Recursively search child frames
if (findFrameByName(frame, name)) |found| {
return found;
}
}
return null;
}
pub fn triggerMouseClick(self: *Page, x: f64, y: f64) !void {
const target = (try self.window._document.elementFromPoint(x, y, self)) orelse return;
if (comptime IS_DEBUG) {
@@ -3158,29 +3283,27 @@ pub fn handleClick(self: *Page, target: *Node) !void {
return;
}
// Check target attribute - don't navigate if opening in new window/tab
const target_val = anchor.getTarget();
if (target_val.len > 0 and !std.mem.eql(u8, target_val, "_self")) {
log.warn(.not_implemented, "a.target", .{ .type = self._type, .url = self.url });
return;
}
if (try element.hasAttribute(comptime .wrap("download"), self)) {
log.warn(.browser, "a.download", .{ .type = self._type, .url = self.url });
return;
}
// TODO: We need to support targets properly, but this is the most
// common case: a click on an anchor navigates the page/frame that
// anchor is in.
const target_page = blk: {
const target_name = anchor.getTarget();
if (target_name.len == 0) {
break :blk target.ownerPage(self);
}
break :blk self.resolveTargetPage(target_name) orelse {
log.warn(.not_implemented, "target", .{ .type = self._type, .url = self.url, .target = target_name });
return;
};
};
// ownerDocument only returns null when `target` is a document, which
// it is NOT in this case. Even for a detched node, it'll return self.document
try element.focus(self);
try self.scheduleNavigation(href, .{
.reason = .script,
.kind = .{ .push = null },
}, .{ .anchor = target });
}, .{ .anchor = target_page });
},
.input => |input| {
try element.focus(self);
@@ -3273,6 +3396,25 @@ pub fn submitForm(self: *Page, submitter_: ?*Element, form_: ?*Element.Html.Form
const form_element = form.asElement();
const target_name_: ?[]const u8 = blk: {
if (submitter_) |submitter| {
if (submitter.getAttributeSafe(comptime .wrap("formtarget"))) |ft| {
break :blk ft;
}
}
break :blk form_element.getAttributeSafe(comptime .wrap("target"));
};
const target_page = blk: {
const target_name = target_name_ orelse {
break :blk form_element.asNode().ownerPage(self);
};
break :blk self.resolveTargetPage(target_name) orelse {
log.warn(.not_implemented, "target", .{ .type = self._type, .url = self.url, .target = target_name });
return;
};
};
if (submit_opts.fire_event) {
const submit_event = try Event.initTrusted(comptime .wrap("submit"), .{ .bubbles = true, .cancelable = true }, self);
@@ -3315,7 +3457,8 @@ pub fn submitForm(self: *Page, submitter_: ?*Element, form_: ?*Element.Html.Form
} else {
action = try URL.concatQueryString(arena, action, buf.written());
}
return self.scheduleNavigationWithArena(arena, action, opts, .{ .form = form_element.asNode() });
return self.scheduleNavigationWithArena(arena, action, opts, .{ .form = target_page });
}
// insertText is a shortcut to insert text into the active element.
@@ -3364,6 +3507,9 @@ fn asUint(comptime string: anytype) std.meta.Int(
const testing = @import("../testing.zig");
test "WebApi: Page" {
const filter: testing.LogFilter = .init(.http);
defer filter.deinit();
try testing.htmlRunner("page", .{});
}

View File

@@ -159,7 +159,6 @@ pub fn addFromElement(self: *ScriptManager, comptime from_parser: bool, script_e
// <script> has already been processed.
return;
}
script_element._executed = true;
const element = script_element.asElement();
if (element.getAttributeSafe(comptime .wrap("nomodule")) != null) {
@@ -204,10 +203,22 @@ pub fn addFromElement(self: *ScriptManager, comptime from_parser: bool, script_e
source = .{ .remote = .{} };
}
} else {
const inline_source = try element.asNode().getTextContentAlloc(page.arena);
var buf = std.Io.Writer.Allocating.init(page.arena);
try element.asNode().getChildTextContent(&buf.writer);
try buf.writer.writeByte(0);
const data = buf.written();
const inline_source: [:0]const u8 = data[0 .. data.len - 1 :0];
if (inline_source.len == 0) {
// we haven't set script_element._executed = true yet, which is good.
// If content is appended to the script, we will execute it then.
return;
}
source = .{ .@"inline" = inline_source };
}
// Only set _executed (already-started) when we actually have content to execute
script_element._executed = true;
const script = try self.script_pool.create();
errdefer self.script_pool.destroy(script);

View File

@@ -84,6 +84,7 @@ queued_navigation: std.ArrayList(*Page),
// about:blank navigations (which may add to queued_navigation).
queued_queued_navigation: std.ArrayList(*Page),
page_id_gen: u32,
frame_id_gen: u32,
pub fn init(self: *Session, browser: *Browser, notification: *Notification) !void {
@@ -103,6 +104,7 @@ pub fn init(self: *Session, browser: *Browser, notification: *Notification) !voi
.page_arena = page_arena,
.factory = Factory.init(page_arena),
.history = .{},
.page_id_gen = 0,
.frame_id_gen = 0,
// The prototype (EventTarget) for Navigation is created when a Page is created.
.navigation = .{ ._proto = undefined },
@@ -297,9 +299,24 @@ pub const WaitResult = enum {
cdp_socket,
};
pub fn findPage(self: *Session, frame_id: u32) ?*Page {
pub fn findPageByFrameId(self: *Session, frame_id: u32) ?*Page {
const page = self.currentPage() orelse return null;
return if (page._frame_id == frame_id) page else null;
return findPageBy(page, "_frame_id", frame_id);
}
pub fn findPageById(self: *Session, id: u32) ?*Page {
const page = self.currentPage() orelse return null;
return findPageBy(page, "id", id);
}
fn findPageBy(page: *Page, comptime field: []const u8, id: u32) ?*Page {
if (@field(page, field) == id) return page;
for (page.frames.items) |f| {
if (findPageBy(f, field, id)) |found| {
return found;
}
}
return null;
}
pub fn wait(self: *Session, wait_ms: u32) WaitResult {
@@ -636,3 +653,9 @@ pub fn nextFrameId(self: *Session) u32 {
self.frame_id_gen = id;
return id;
}
pub fn nextPageId(self: *Session) u32 {
const id = self.page_id_gen +% 1;
self.page_id_gen = id;
return id;
}

View File

@@ -277,6 +277,11 @@ pub fn isCompleteHTTPUrl(url: []const u8) bool {
return false;
}
// blob: and data: URLs are complete but don't follow scheme:// pattern
if (std.mem.startsWith(u8, url, "blob:") or std.mem.startsWith(u8, url, "data:")) {
return true;
}
// Check if there's a scheme (protocol) ending with ://
const colon_pos = std.mem.indexOfScalar(u8, url, ':') orelse return false;
@@ -1400,3 +1405,12 @@ test "URL: unescape" {
try testing.expectEqual("hello%2", result);
}
}
test "URL: getHost" {
try testing.expectEqualSlices(u8, "example.com:8080", getHost("https://example.com:8080/path"));
try testing.expectEqualSlices(u8, "example.com", getHost("https://example.com/path"));
try testing.expectEqualSlices(u8, "example.com:443", getHost("https://example.com:443/"));
try testing.expectEqualSlices(u8, "example.com", getHost("https://user:pass@example.com/page"));
try testing.expectEqualSlices(u8, "example.com:8080", getHost("https://user:pass@example.com:8080/page"));
try testing.expectEqualSlices(u8, "", getHost("not-a-url"));
}

View File

@@ -157,7 +157,7 @@ pub fn collectInteractiveElements(
.node = node,
.tag_name = el.getTagNameLower(),
.role = getRole(el),
.name = getAccessibleName(el),
.name = try getAccessibleName(el, arena),
.interactivity_type = itype,
.listener_types = listener_types,
.disabled = isDisabled(el),
@@ -178,12 +178,12 @@ pub fn collectInteractiveElements(
return results.items;
}
const ListenerTargetMap = std.AutoHashMapUnmanaged(usize, std.ArrayList([]const u8));
pub const ListenerTargetMap = std.AutoHashMapUnmanaged(usize, std.ArrayList([]const u8));
/// Pre-build a map from event_target pointer → list of event type names.
/// This lets both classifyInteractivity (O(1) "has any?") and
/// getListenerTypes (O(1) "which ones?") avoid re-iterating per element.
fn buildListenerTargetMap(page: *Page, arena: Allocator) !ListenerTargetMap {
pub fn buildListenerTargetMap(page: *Page, arena: Allocator) !ListenerTargetMap {
var map = ListenerTargetMap{};
// addEventListener registrations
@@ -209,7 +209,7 @@ fn buildListenerTargetMap(page: *Page, arena: Allocator) !ListenerTargetMap {
return map;
}
fn classifyInteractivity(
pub fn classifyInteractivity(
el: *Element,
html_el: *Element.Html,
listener_targets: ListenerTargetMap,
@@ -296,7 +296,7 @@ fn getRole(el: *Element) ?[]const u8 {
};
}
fn getAccessibleName(el: *Element) ?[]const u8 {
fn getAccessibleName(el: *Element, arena: Allocator) !?[]const u8 {
// aria-label
if (el.getAttributeSafe(comptime .wrap("aria-label"))) |v| {
if (v.len > 0) return v;
@@ -325,11 +325,15 @@ fn getAccessibleName(el: *Element) ?[]const u8 {
}
// Text content (first non-empty text node, trimmed)
return getTextContent(el.asNode());
return try getTextContent(el.asNode(), arena);
}
fn getTextContent(node: *Node) ?[]const u8 {
var tw = TreeWalker.FullExcludeSelf.init(node, .{});
fn getTextContent(node: *Node, arena: Allocator) !?[]const u8 {
var tw: TreeWalker.FullExcludeSelf = .init(node, .{});
var arr: std.ArrayList(u8) = .empty;
var single_chunk: ?[]const u8 = null;
while (tw.next()) |child| {
// Skip text inside script/style elements.
if (child.is(Element)) |el| {
@@ -344,13 +348,29 @@ fn getTextContent(node: *Node) ?[]const u8 {
if (child.is(Node.CData)) |cdata| {
if (cdata.is(Node.CData.Text)) |text| {
const content = std.mem.trim(u8, text.getWholeText(), &std.ascii.whitespace);
if (content.len > 0) return content;
if (content.len > 0) {
if (single_chunk == null and arr.items.len == 0) {
single_chunk = content;
} else {
if (single_chunk) |sc| {
try arr.appendSlice(arena, sc);
try arr.append(arena, ' ');
single_chunk = null;
}
try arr.appendSlice(arena, content);
try arr.append(arena, ' ');
}
}
}
}
}
return null;
}
if (single_chunk) |sc| return sc;
if (arr.items.len == 0) return null;
// strip out trailing space
return arr.items[0 .. arr.items.len - 1];
}
fn isDisabled(el: *Element) bool {
if (el.getAttributeSafe(comptime .wrap("disabled")) != null) return true;
return isDisabledByFieldset(el);

View File

@@ -171,6 +171,7 @@ pub fn setOrigin(self: *Context, key: ?[]const u8) !void {
errdefer self.session.releaseOrigin(origin);
try self.origin.transferTo(origin);
lp.assert(self.origin.rc == 1, "Ref opaque origin", .{ .rc = self.origin.rc });
self.origin.deinit(env.app);
self.origin = origin;

View File

@@ -326,7 +326,7 @@ pub fn createContext(self: *Env, page: *Page) !*Context {
.script_manager = &page._script_manager,
.scheduler = .init(context_arena),
};
try context.origin.identity_map.putNoClobber(context_arena, @intFromPtr(page.window), global_global);
try context.origin.identity_map.putNoClobber(origin.arena, @intFromPtr(page.window), global_global);
// Store a pointer to our context inside the v8 context so that, given
// a v8 context, we can get our context out

View File

@@ -202,20 +202,20 @@ pub fn compileAndRun(self: *const Local, src: []const u8, name: ?[]const u8) !js
// we can just grab it from the identity_map)
pub fn mapZigInstanceToJs(self: *const Local, js_obj_handle: ?*const v8.Object, value: anytype) !js.Object {
const ctx = self.ctx;
const arena = ctx.arena;
const origin_arena = ctx.origin.arena;
const T = @TypeOf(value);
switch (@typeInfo(T)) {
.@"struct" => {
// Struct, has to be placed on the heap
const heap = try arena.create(T);
const heap = try origin_arena.create(T);
heap.* = value;
return self.mapZigInstanceToJs(js_obj_handle, heap);
},
.pointer => |ptr| {
const resolved = resolveValue(value);
const gop = try ctx.origin.identity_map.getOrPut(arena, @intFromPtr(resolved.ptr));
const gop = try ctx.origin.addIdentity(@intFromPtr(resolved.ptr));
if (gop.found_existing) {
// we've seen this instance before, return the same object
return (js.Object.Global{ .handle = gop.value_ptr.* }).local(self);
@@ -244,7 +244,7 @@ pub fn mapZigInstanceToJs(self: *const Local, js_obj_handle: ?*const v8.Object,
// The TAO contains the pointer to our Zig instance as
// well as any meta data we'll need to use it later.
// See the TaggedOpaque struct for more details.
const tao = try arena.create(TaggedOpaque);
const tao = try origin_arena.create(TaggedOpaque);
tao.* = .{
.value = resolved.ptr,
.prototype_chain = resolved.prototype_chain.ptr,

View File

@@ -129,6 +129,19 @@ pub fn trackGlobal(self: *Origin, global: v8.Global) !void {
return self.globals.append(self.arena, global);
}
pub const IdentityResult = struct {
value_ptr: *v8.Global,
found_existing: bool,
};
pub fn addIdentity(self: *Origin, ptr: usize) !IdentityResult {
const gop = try self.identity_map.getOrPut(self.arena, ptr);
return .{
.value_ptr = gop.value_ptr,
.found_existing = gop.found_existing,
};
}
pub fn trackTemp(self: *Origin, global: v8.Global) !void {
return self.temps.put(self.arena, global.data_ptr, global);
}

View File

@@ -56,7 +56,7 @@ fn _toSlice(self: String, comptime null_terminate: bool, allocator: Allocator) !
pub fn toSSO(self: String, comptime global: bool) !(if (global) SSO.Global else SSO) {
if (comptime global) {
return .{ .str = try self.toSSOWithAlloc(self.local.ctx.arena) };
return .{ .str = try self.toSSOWithAlloc(self.local.ctx.origin.arena) };
}
return self.toSSOWithAlloc(self.local.call_arena);
}

View File

@@ -245,6 +245,46 @@ pub fn toJson(self: Value, allocator: Allocator) ![]u8 {
return js.String.toSliceWithAlloc(.{ .local = local, .handle = str_handle }, allocator);
}
// Currently does not support host objects (Blob, File, etc.) or transferables
// which require delegate callbacks to be implemented.
pub fn structuredClone(self: Value) !Value {
const local = self.local;
const v8_context = local.handle;
const v8_isolate = local.isolate.handle;
const size, const data = blk: {
const serializer = v8.v8__ValueSerializer__New(v8_isolate, null) orelse return error.JsException;
defer v8.v8__ValueSerializer__DELETE(serializer);
var write_result: v8.MaybeBool = undefined;
v8.v8__ValueSerializer__WriteHeader(serializer);
v8.v8__ValueSerializer__WriteValue(serializer, v8_context, self.handle, &write_result);
if (!write_result.has_value or !write_result.value) {
return error.JsException;
}
var size: usize = undefined;
const data = v8.v8__ValueSerializer__Release(serializer, &size) orelse return error.JsException;
break :blk .{ size, data };
};
defer v8.v8__ValueSerializer__FreeBuffer(data);
const cloned_handle = blk: {
const deserializer = v8.v8__ValueDeserializer__New(v8_isolate, data, size, null) orelse return error.JsException;
defer v8.v8__ValueDeserializer__DELETE(deserializer);
var read_header_result: v8.MaybeBool = undefined;
v8.v8__ValueDeserializer__ReadHeader(deserializer, v8_context, &read_header_result);
if (!read_header_result.has_value or !read_header_result.value) {
return error.JsException;
}
break :blk v8.v8__ValueDeserializer__ReadValue(deserializer, v8_context) orelse return error.JsException;
};
return .{ .local = local, .handle = cloned_handle };
}
pub fn persist(self: Value) !Global {
return self._persist(true);
}

View File

@@ -885,6 +885,7 @@ pub const JsApis = flattenTypes(&.{
@import("../webapi/IdleDeadline.zig"),
@import("../webapi/Blob.zig"),
@import("../webapi/File.zig"),
@import("../webapi/FileList.zig"),
@import("../webapi/FileReader.zig"),
@import("../webapi/Screen.zig"),
@import("../webapi/VisualViewport.zig"),

View File

@@ -24,6 +24,7 @@ const TreeWalker = @import("webapi/TreeWalker.zig");
const CData = @import("webapi/CData.zig");
const Element = @import("webapi/Element.zig");
const Node = @import("webapi/Node.zig");
const isAllWhitespace = @import("../string.zig").isAllWhitespace;
pub const Opts = struct {
// Options for future customization (e.g., dialect)
@@ -46,13 +47,6 @@ const State = struct {
last_char_was_newline: bool = true,
};
fn isBlock(tag: Element.Tag) bool {
return switch (tag) {
.p, .div, .section, .article, .main, .header, .footer, .nav, .aside, .h1, .h2, .h3, .h4, .h5, .h6, .ul, .ol, .blockquote, .pre, .table, .hr => true,
else => false,
};
}
fn shouldAddSpacing(tag: Element.Tag) bool {
return switch (tag) {
.p, .h1, .h2, .h3, .h4, .h5, .h6, .blockquote, .pre, .table => true,
@@ -99,26 +93,18 @@ fn isSignificantText(node: *Node) bool {
}
fn isVisibleElement(el: *Element) bool {
return switch (el.getTag()) {
.script, .style, .noscript, .template, .head, .meta, .link, .title, .svg => false,
else => true,
};
const tag = el.getTag();
return !tag.isMetadata() and tag != .svg;
}
fn getAnchorLabel(el: *Element) ?[]const u8 {
return el.getAttributeSafe(comptime .wrap("aria-label")) orelse el.getAttributeSafe(comptime .wrap("title"));
}
fn isAllWhitespace(text: []const u8) bool {
return for (text) |c| {
if (!std.ascii.isWhitespace(c)) break false;
} else true;
}
fn hasBlockDescendant(root: *Node) bool {
var tw = TreeWalker.FullExcludeSelf.Elements.init(root, .{});
while (tw.next()) |el| {
if (isBlock(el.getTag())) return true;
if (el.getTag().isBlock()) return true;
}
return false;
}
@@ -192,7 +178,7 @@ fn renderElement(el: *Element, state: *State, writer: *std.Io.Writer, page: *Pag
// --- Opening Tag Logic ---
// Ensure block elements start on a new line (double newline for paragraphs etc)
if (isBlock(tag) and !state.in_table) {
if (tag.isBlock() and !state.in_table) {
try ensureNewline(state, writer);
if (shouldAddSpacing(tag)) {
try writer.writeByte('\n');
@@ -431,7 +417,7 @@ fn renderElement(el: *Element, state: *State, writer: *std.Io.Writer, page: *Pag
}
// Post-block newlines
if (isBlock(tag) and !state.in_table) {
if (tag.isBlock() and !state.in_table) {
try ensureNewline(state, writer);
}
}

View File

@@ -23,6 +23,9 @@ const h5e = @import("html5ever.zig");
const Page = @import("../Page.zig");
const Node = @import("../webapi/Node.zig");
const Element = @import("../webapi/Element.zig");
pub const AttributeIterator = h5e.AttributeIterator;
const Allocator = std.mem.Allocator;
const IS_DEBUG = @import("builtin").mode == .Debug;

View File

@@ -89,6 +89,41 @@
}
</script>
<script id="CanvasRenderingContext2D#getImageData">
{
const element = document.createElement("canvas");
element.width = 100;
element.height = 50;
const ctx = element.getContext("2d");
const imageData = ctx.getImageData(0, 0, 10, 20);
testing.expectEqual(true, imageData instanceof ImageData);
testing.expectEqual(imageData.width, 10);
testing.expectEqual(imageData.height, 20);
testing.expectEqual(imageData.data.length, 10 * 20 * 4);
testing.expectEqual(true, imageData.data instanceof Uint8ClampedArray);
// Undrawn canvas should return transparent black pixels.
testing.expectEqual(imageData.data[0], 0);
testing.expectEqual(imageData.data[1], 0);
testing.expectEqual(imageData.data[2], 0);
testing.expectEqual(imageData.data[3], 0);
}
</script>
<script id="CanvasRenderingContext2D#getImageData invalid">
{
const element = document.createElement("canvas");
const ctx = element.getContext("2d");
// Zero or negative width/height should throw IndexSizeError.
testing.expectError('Index or size', () => ctx.getImageData(0, 0, 0, 10));
testing.expectError('Index or size', () => ctx.getImageData(0, 0, 10, 0));
testing.expectError('Index or size', () => ctx.getImageData(0, 0, -5, 10));
testing.expectError('Index or size', () => ctx.getImageData(0, 0, 10, -5));
}
</script>
<script id="getter">
{

View File

@@ -62,3 +62,26 @@
testing.expectEqual(offscreen.height, 96);
}
</script>
<script id=OffscreenCanvasRenderingContext2D#getImageData>
{
const canvas = new OffscreenCanvas(100, 50);
const ctx = canvas.getContext("2d");
const imageData = ctx.getImageData(0, 0, 10, 20);
testing.expectEqual(true, imageData instanceof ImageData);
testing.expectEqual(imageData.width, 10);
testing.expectEqual(imageData.height, 20);
testing.expectEqual(imageData.data.length, 10 * 20 * 4);
// Undrawn canvas should return transparent black pixels.
testing.expectEqual(imageData.data[0], 0);
testing.expectEqual(imageData.data[1], 0);
testing.expectEqual(imageData.data[2], 0);
testing.expectEqual(imageData.data[3], 0);
// Zero or negative dimensions should throw.
testing.expectError('Index or size', () => ctx.getImageData(0, 0, 0, 10));
testing.expectError('Index or size', () => ctx.getImageData(0, 0, 10, -5));
}
</script>

View File

@@ -56,3 +56,25 @@
testing.expectEqual('FontFaceSet', document.fonts.constructor.name);
}
</script>
<script id="document_fonts_addEventListener">
{
let loading = false;
document.fonts.addEventListener('loading', function() {
loading = true;
});
let loadingdone = false;
document.fonts.addEventListener('loadingdone', function() {
loadingdone = true;
});
document.fonts.load("italic bold 16px Roboto");
testing.eventually(() => {
testing.expectEqual(true, loading);
testing.expectEqual(true, loadingdone);
});
testing.expectEqual(true, true);
}
</script>

View File

@@ -0,0 +1,66 @@
<!DOCTYPE html>
<head>
<script src="../testing.js"></script>
<script>
// Test that document.open/write/close throw InvalidStateError during custom element
// reactions when the element is parsed from HTML
window.constructorOpenException = null;
window.constructorWriteException = null;
window.constructorCloseException = null;
window.constructorCalled = false;
class ThrowTestElement extends HTMLElement {
constructor() {
super();
window.constructorCalled = true;
// Try document.open on the same document during constructor - should throw
try {
document.open();
} catch (e) {
window.constructorOpenException = e;
}
// Try document.write on the same document during constructor - should throw
try {
document.write('<b>test</b>');
} catch (e) {
window.constructorWriteException = e;
}
// Try document.close on the same document during constructor - should throw
try {
document.close();
} catch (e) {
window.constructorCloseException = e;
}
}
}
customElements.define('throw-test-element', ThrowTestElement);
</script>
</head>
<body>
<!-- This element will be parsed from HTML, triggering the constructor -->
<throw-test-element id="test-element"></throw-test-element>
<script id="verify_throws">
{
// Verify the constructor was called
testing.expectEqual(true, window.constructorCalled);
// Verify document.open threw InvalidStateError
testing.expectEqual(true, window.constructorOpenException !== null);
testing.expectEqual('InvalidStateError', window.constructorOpenException.name);
// Verify document.write threw InvalidStateError
testing.expectEqual(true, window.constructorWriteException !== null);
testing.expectEqual('InvalidStateError', window.constructorWriteException.name);
// Verify document.close threw InvalidStateError
testing.expectEqual(true, window.constructorCloseException !== null);
testing.expectEqual('InvalidStateError', window.constructorCloseException.name);
}
</script>
</body>

View File

@@ -0,0 +1,61 @@
<!DOCTYPE html>
<script src="../../../testing.js"></script>
<script id=force_async>
{
// Dynamically created scripts have async=true by default
let s = document.createElement('script');
testing.expectEqual(true, s.async);
// Setting async=false clears the force async flag and removes attribute
s.async = false;
testing.expectEqual(false, s.async);
testing.expectEqual(false, s.hasAttribute('async'));
// Setting async=true adds the attribute
s.async = true;
testing.expectEqual(true, s.async);
testing.expectEqual(true, s.hasAttribute('async'));
}
</script>
<script></script>
<script id=empty>
{
// Empty parser-inserted script should have async=true (force async retained)
let scripts = document.getElementsByTagName('script');
let emptyScript = scripts[scripts.length - 2];
testing.expectEqual(true, emptyScript.async);
}
</script>
<script id=text_content>
{
let s = document.createElement('script');
s.appendChild(document.createComment('COMMENT'));
s.appendChild(document.createTextNode(' TEXT '));
s.appendChild(document.createProcessingInstruction('P', 'I'));
let a = s.appendChild(document.createElement('a'));
a.appendChild(document.createTextNode('ELEMENT'));
// script.text should return only direct Text node children
testing.expectEqual(' TEXT ', s.text);
// script.textContent should return all descendant text
testing.expectEqual(' TEXT ELEMENT', s.textContent);
}
</script>
<script id=lazy_inline>
{
// Empty script in DOM, then append text - should execute
window.lazyScriptRan = false;
let s = document.createElement('script');
document.head.appendChild(s);
// Script is in DOM but empty, so not yet executed
testing.expectEqual(false, window.lazyScriptRan);
// Append text node with code
s.appendChild(document.createTextNode('window.lazyScriptRan = true;'));
// Now it should have executed
testing.expectEqual(true, window.lazyScriptRan);
}
</script>

View File

@@ -108,7 +108,7 @@
{
let f5 = document.createElement('iframe');
f5.id = 'f5';
f5.src = "support/sub 1.html";
f5.src = "support/page.html";
document.documentElement.appendChild(f5);
f5.src = "about:blank";

View File

@@ -0,0 +1,2 @@
<!DOCTYPE html>
a-page

View File

@@ -0,0 +1,42 @@
<!DOCTYPE html>
<script src="../testing.js"></script>
<iframe name=f1 id=frame1></iframe>
<a id=l1 target=f1 href=support/page.html></a>
<script id=anchor>
$('#l1').click();
testing.eventually(() => {
testing.expectEqual('<html><head></head><body>a-page\n</body></html>', $('#frame1').contentDocument.documentElement.outerHTML);
});
</script>
<script id=form>
{
let frame2 = document.createElement('iframe');
frame2.name = 'frame2';
document.documentElement.appendChild(frame2);
let form = document.createElement('form');
form.target = 'frame2';
form.action = 'support/page.html';
form.submit();
testing.eventually(() => {
testing.expectEqual('<html><head></head><body>a-page\n</body></html>', frame2.contentDocument.documentElement.outerHTML);
});
}
</script>
<iframe name=frame3 id=f3></iframe>
<form target="_top" action="support/page.html">
<input type=submit id=submit1 formtarget="frame3">
</form>
<script id=formtarget>
{
$('#submit1').click();
testing.eventually(() => {
testing.expectEqual('<html><head></head><body>a-page\n</body></html>', $('#f3').contentDocument.documentElement.outerHTML);
});
}
</script>

View File

@@ -0,0 +1,41 @@
<!DOCTYPE html>
<body></body>
<script src="../testing.js"></script>
<script id="basic_blob_navigation">
{
const html = '<html><head></head><body><div id="test">Hello Blob</div></body></html>';
const blob = new Blob([html], { type: 'text/html' });
const blob_url = URL.createObjectURL(blob);
const iframe = document.createElement('iframe');
document.body.appendChild(iframe);
iframe.src = blob_url;
testing.eventually(() => {
testing.expectEqual('Hello Blob', iframe.contentDocument.getElementById('test').textContent);
});
}
</script>
<script id="multiple_blobs">
{
const blob1 = new Blob(['<html><body>First</body></html>'], { type: 'text/html' });
const blob2 = new Blob(['<html><body>Second</body></html>'], { type: 'text/html' });
const url1 = URL.createObjectURL(blob1);
const url2 = URL.createObjectURL(blob2);
const iframe1 = document.createElement('iframe');
document.body.appendChild(iframe1);
iframe1.src = url1;
const iframe2 = document.createElement('iframe');
document.body.appendChild(iframe2);
iframe2.src = url2;
testing.eventually(() => {
testing.expectEqual('First', iframe1.contentDocument.body.textContent);
testing.expectEqual('Second', iframe2.contentDocument.body.textContent);
});
}
</script>

View File

@@ -115,30 +115,6 @@
}
</script>
<script id=structuredClone>
// Basic types
testing.expectEqual(42, structuredClone(42));
testing.expectEqual('hello', structuredClone('hello'));
testing.expectEqual(true, structuredClone(true));
testing.expectEqual(null, structuredClone(null));
// Object deep clone
const obj = { a: 1, b: { c: 2 } };
const cloned = structuredClone(obj);
testing.expectEqual(1, cloned.a);
testing.expectEqual(2, cloned.b.c);
cloned.b.c = 99;
testing.expectEqual(2, obj.b.c); // original unchanged
// Array deep clone
const arr = [1, [2, 3]];
const clonedArr = structuredClone(arr);
testing.expectEqual(1, clonedArr[0]);
testing.expectEqual(2, clonedArr[1][0]);
clonedArr[1][0] = 99;
testing.expectEqual(2, arr[1][0]); // original unchanged
</script>
<script id=screen>
testing.expectEqual(1920, screen.width);
testing.expectEqual(1080, screen.height);
@@ -149,6 +125,143 @@
testing.expectEqual(screen, window.screen);
</script>
<script id=structuredClone>
// Basic types
testing.expectEqual(42, structuredClone(42));
testing.expectEqual('hello', structuredClone('hello'));
testing.expectEqual(true, structuredClone(true));
testing.expectEqual(null, structuredClone(null));
testing.expectEqual(undefined, structuredClone(undefined));
// Objects and arrays (these work with JSON too, but verify they're cloned)
const obj = { a: 1, b: { c: 2 } };
const clonedObj = structuredClone(obj);
testing.expectEqual(1, clonedObj.a);
testing.expectEqual(2, clonedObj.b.c);
clonedObj.b.c = 999;
testing.expectEqual(2, obj.b.c); // original unchanged
const arr = [1, [2, 3]];
const clonedArr = structuredClone(arr);
testing.expectEqual(1, clonedArr[0]);
testing.expectEqual(2, clonedArr[1][0]);
clonedArr[1][0] = 999;
testing.expectEqual(2, arr[1][0]); // original unchanged
// Date - JSON would stringify to ISO string
const date = new Date('2024-01-15T12:30:00Z');
const clonedDate = structuredClone(date);
testing.expectEqual(true, clonedDate instanceof Date);
testing.expectEqual(date.getTime(), clonedDate.getTime());
testing.expectEqual(date.toISOString(), clonedDate.toISOString());
// RegExp - JSON would stringify to {}
const regex = /test\d+/gi;
const clonedRegex = structuredClone(regex);
testing.expectEqual(true, clonedRegex instanceof RegExp);
testing.expectEqual(regex.source, clonedRegex.source);
testing.expectEqual(regex.flags, clonedRegex.flags);
testing.expectEqual(true, clonedRegex.test('test123'));
// Map - JSON can't handle
const map = new Map([['a', 1], ['b', 2]]);
const clonedMap = structuredClone(map);
testing.expectEqual(true, clonedMap instanceof Map);
testing.expectEqual(2, clonedMap.size);
testing.expectEqual(1, clonedMap.get('a'));
testing.expectEqual(2, clonedMap.get('b'));
// Set - JSON can't handle
const set = new Set([1, 2, 3]);
const clonedSet = structuredClone(set);
testing.expectEqual(true, clonedSet instanceof Set);
testing.expectEqual(3, clonedSet.size);
testing.expectEqual(true, clonedSet.has(1));
testing.expectEqual(true, clonedSet.has(2));
testing.expectEqual(true, clonedSet.has(3));
// ArrayBuffer
const buffer = new ArrayBuffer(8);
const view = new Uint8Array(buffer);
view[0] = 42;
view[7] = 99;
const clonedBuffer = structuredClone(buffer);
testing.expectEqual(true, clonedBuffer instanceof ArrayBuffer);
testing.expectEqual(8, clonedBuffer.byteLength);
const clonedView = new Uint8Array(clonedBuffer);
testing.expectEqual(42, clonedView[0]);
testing.expectEqual(99, clonedView[7]);
// TypedArray
const typedArr = new Uint32Array([100, 200, 300]);
const clonedTypedArr = structuredClone(typedArr);
testing.expectEqual(true, clonedTypedArr instanceof Uint32Array);
testing.expectEqual(3, clonedTypedArr.length);
testing.expectEqual(100, clonedTypedArr[0]);
testing.expectEqual(200, clonedTypedArr[1]);
testing.expectEqual(300, clonedTypedArr[2]);
// Special number values - JSON can't preserve these
testing.expectEqual(true, Number.isNaN(structuredClone(NaN)));
testing.expectEqual(Infinity, structuredClone(Infinity));
testing.expectEqual(-Infinity, structuredClone(-Infinity));
// Object with undefined value - JSON would omit it
const objWithUndef = { a: 1, b: undefined, c: 3 };
const clonedObjWithUndef = structuredClone(objWithUndef);
testing.expectEqual(1, clonedObjWithUndef.a);
testing.expectEqual(undefined, clonedObjWithUndef.b);
testing.expectEqual(true, 'b' in clonedObjWithUndef);
testing.expectEqual(3, clonedObjWithUndef.c);
// Error objects
const error = new Error('test error');
const clonedError = structuredClone(error);
testing.expectEqual(true, clonedError instanceof Error);
testing.expectEqual('test error', clonedError.message);
// TypeError
const typeError = new TypeError('type error');
const clonedTypeError = structuredClone(typeError);
testing.expectEqual(true, clonedTypeError instanceof TypeError);
testing.expectEqual('type error', clonedTypeError.message);
// BigInt
const bigInt = BigInt('9007199254740993');
const clonedBigInt = structuredClone(bigInt);
testing.expectEqual(bigInt, clonedBigInt);
// Circular references ARE supported by structuredClone (unlike JSON)
const circular = { a: 1 };
circular.self = circular;
const clonedCircular = structuredClone(circular);
testing.expectEqual(1, clonedCircular.a);
testing.expectEqual(clonedCircular, clonedCircular.self); // circular ref preserved
// Functions cannot be cloned - should throw
{
let threw = false;
try {
structuredClone(() => {});
} catch (err) {
threw = true;
// Just verify an error was thrown - V8's message format may vary
}
testing.expectEqual(true, threw);
}
// Symbols cannot be cloned - should throw
{
let threw = false;
try {
structuredClone(Symbol('test'));
} catch (err) {
threw = true;
}
testing.expectEqual(true, threw);
}
</script>
<script id=unhandled_rejection>
{
let unhandledCalled = 0;

View File

@@ -19,15 +19,22 @@
const std = @import("std");
const js = @import("../js/js.zig");
const Session = @import("../Session.zig");
const Node = @import("Node.zig");
const Range = @import("Range.zig");
const Allocator = std.mem.Allocator;
const IS_DEBUG = @import("builtin").mode == .Debug;
const AbstractRange = @This();
pub const _prototype_root = true;
_rc: u8,
_type: Type,
_page_id: u32,
_arena: Allocator,
_end_offset: u32,
_start_offset: u32,
_end_container: *Node,
@@ -36,6 +43,27 @@ _start_container: *Node,
// Intrusive linked list node for tracking live ranges on the Page.
_range_link: std.DoublyLinkedList.Node = .{},
pub fn acquireRef(self: *AbstractRange) void {
self._rc += 1;
}
pub fn deinit(self: *AbstractRange, shutdown: bool, session: *Session) void {
_ = shutdown;
const rc = self._rc;
if (comptime IS_DEBUG) {
std.debug.assert(rc != 0);
}
if (rc == 1) {
if (session.findPageById(self._page_id)) |page| {
page._live_ranges.remove(&self._range_link);
}
session.releaseArena(self._arena);
return;
}
self._rc = rc - 1;
}
pub const Type = union(enum) {
range: *Range,
// TODO: static_range: *StaticRange,
@@ -310,6 +338,8 @@ pub const JsApi = struct {
pub const name = "AbstractRange";
pub const prototype_chain = bridge.prototypeChain();
pub var class_id: bridge.ClassId = undefined;
pub const weak = true;
pub const finalizer = bridge.finalizer(AbstractRange.deinit);
};
pub const startContainer = bridge.accessor(AbstractRange.getStartContainer, null, .{});

View File

@@ -151,8 +151,13 @@ pub fn asNode(self: *CData) *Node {
pub fn is(self: *CData, comptime T: type) ?*T {
inline for (@typeInfo(Type).@"union".fields) |f| {
if (f.type == T and @field(Type, f.name) == self._type) {
return &@field(self._type, f.name);
if (@field(Type, f.name) == self._type) {
if (f.type == T) {
return &@field(self._type, f.name);
}
if (f.type == *T) {
return @field(self._type, f.name);
}
}
}
return null;

View File

@@ -63,6 +63,11 @@ _script_created_parser: ?Parser.Streaming = null,
_adopted_style_sheets: ?js.Object.Global = null,
_selection: Selection = .init,
// https://html.spec.whatwg.org/multipage/dynamic-markup-insertion.html#throw-on-dynamic-markup-insertion-counter
// Incremented during custom element reactions when parsing. When > 0,
// document.open/close/write/writeln must throw InvalidStateError.
_throw_on_dynamic_markup_insertion_counter: u32 = 0,
_on_selectionchange: ?js.Function.Global = null,
pub fn getOnSelectionChange(self: *Document) ?js.Function.Global {
@@ -641,6 +646,10 @@ pub fn write(self: *Document, text: []const []const u8, page: *Page) !void {
return error.InvalidStateError;
}
if (self._throw_on_dynamic_markup_insertion_counter > 0) {
return error.InvalidStateError;
}
const html = blk: {
var joined: std.ArrayList(u8) = .empty;
for (text) |str| {
@@ -723,6 +732,10 @@ pub fn open(self: *Document, page: *Page) !*Document {
return error.InvalidStateError;
}
if (self._throw_on_dynamic_markup_insertion_counter > 0) {
return error.InvalidStateError;
}
if (page._load_state == .parsing) {
return self;
}
@@ -761,6 +774,10 @@ pub fn close(self: *Document, page: *Page) !void {
return error.InvalidStateError;
}
if (self._throw_on_dynamic_markup_insertion_counter > 0) {
return error.InvalidStateError;
}
if (self._script_created_parser == null) {
return;
}

View File

@@ -1580,6 +1580,36 @@ pub const Tag = enum {
else => tag,
};
}
pub fn isBlock(self: Tag) bool {
// zig fmt: off
return switch (self) {
// Semantic Layout
.article, .aside, .footer, .header, .main, .nav, .section,
// Grouping / Containers
.address, .div, .fieldset, .figure, .p,
// Headings
.h1, .h2, .h3, .h4, .h5, .h6,
// Lists
.dl, .ol, .ul,
// Preformatted / Quotes
.blockquote, .pre,
// Tables
.table,
// Other
.hr,
=> true,
else => false,
};
// zig fmt: on
}
pub fn isMetadata(self: Tag) bool {
return switch (self) {
.base, .head, .link, .meta, .noscript, .script, .style, .template, .title => true,
else => false,
};
}
};
pub const JsApi = struct {

View File

@@ -44,6 +44,7 @@ pub const Type = union(enum) {
screen_orientation: *@import("Screen.zig").Orientation,
visual_viewport: *@import("VisualViewport.zig"),
file_reader: *@import("FileReader.zig"),
font_face_set: *@import("css/FontFaceSet.zig"),
};
pub fn init(page: *Page) !*EventTarget {
@@ -139,6 +140,7 @@ pub fn format(self: *EventTarget, writer: *std.Io.Writer) !void {
.screen_orientation => writer.writeAll("<ScreenOrientation>"),
.visual_viewport => writer.writeAll("<VisualViewport>"),
.file_reader => writer.writeAll("<FileReader>"),
.font_face_set => writer.writeAll("<FontFaceSet>"),
};
}
@@ -157,6 +159,7 @@ pub fn toString(self: *EventTarget) []const u8 {
.screen_orientation => return "[object ScreenOrientation]",
.visual_viewport => return "[object VisualViewport]",
.file_reader => return "[object FileReader]",
.font_face_set => return "[object FontFaceSet]",
};
}

View File

@@ -0,0 +1,28 @@
const js = @import("../js/js.zig");
const FileList = @This();
/// Padding to avoid zero-size struct, which causes identity_map pointer collisions.
_pad: bool = false,
pub fn getLength(_: *const FileList) u32 {
return 0;
}
pub fn item(_: *const FileList, _: u32) ?*@import("File.zig") {
return null;
}
pub const JsApi = struct {
pub const bridge = js.Bridge(FileList);
pub const Meta = struct {
pub const name = "FileList";
pub const prototype_chain = bridge.prototypeChain();
pub var class_id: bridge.ClassId = undefined;
pub const empty_with_no_proto = true;
};
pub const length = bridge.accessor(FileList.getLength, null, .{});
pub const item = bridge.function(FileList.item, .{});
};

View File

@@ -52,7 +52,7 @@ pub const ConstructorSettings = struct {
/// ```
///
/// We currently support only the first 2.
pub fn constructor(
pub fn init(
width: u32,
height: u32,
maybe_settings: ?ConstructorSettings,
@@ -106,7 +106,7 @@ pub const JsApi = struct {
pub var class_id: bridge.ClassId = undefined;
};
pub const constructor = bridge.constructor(ImageData.constructor, .{ .dom_exception = true });
pub const constructor = bridge.constructor(ImageData.init, .{ .dom_exception = true });
pub const colorSpace = bridge.property("srgb", .{ .template = false, .readonly = true });
pub const pixelFormat = bridge.property("rgba-unorm8", .{ .template = false, .readonly = true });

View File

@@ -285,6 +285,19 @@ pub fn getTextContentAlloc(self: *Node, allocator: Allocator) error{WriteFailed}
return data[0 .. data.len - 1 :0];
}
/// Returns the "child text content" which is the concatenation of the data
/// of all the Text node children of the node, in tree order.
/// This differs from textContent which includes all descendant text.
/// See: https://dom.spec.whatwg.org/#concept-child-text-content
pub fn getChildTextContent(self: *Node, writer: *std.Io.Writer) error{WriteFailed}!void {
var it = self.childrenIterator();
while (it.next()) |child| {
if (child.is(CData.Text)) |text| {
try writer.writeAll(text._proto._data.str());
}
}
}
pub fn setTextContent(self: *Node, data: []const u8, page: *Page) !void {
switch (self._type) {
.element => |el| {
@@ -493,6 +506,11 @@ pub fn ownerDocument(self: *const Node, page: *const Page) ?*Document {
return page.document;
}
pub fn ownerPage(self: *const Node, default: *Page) *Page {
const doc = self.ownerDocument(default) orelse return default;
return doc._page orelse default;
}
pub fn isSameDocumentAs(self: *const Node, other: *const Node, page: *const Page) bool {
// Get the root document for each node
const self_doc = if (self._type == .document) self._type.document else self.ownerDocument(page);

View File

@@ -21,22 +21,31 @@ const String = @import("../../string.zig").String;
const js = @import("../js/js.zig");
const Page = @import("../Page.zig");
const Session = @import("../Session.zig");
const Node = @import("Node.zig");
const DocumentFragment = @import("DocumentFragment.zig");
const AbstractRange = @import("AbstractRange.zig");
const DOMRect = @import("DOMRect.zig");
const Allocator = std.mem.Allocator;
const Range = @This();
_proto: *AbstractRange,
pub fn asAbstractRange(self: *Range) *AbstractRange {
return self._proto;
pub fn init(page: *Page) !*Range {
const arena = try page.getArena(.{ .debug = "Range" });
errdefer page.releaseArena(arena);
return page._factory.abstractRange(arena, Range{ ._proto = undefined }, page);
}
pub fn init(page: *Page) !*Range {
return page._factory.abstractRange(Range{ ._proto = undefined }, page);
pub fn deinit(self: *Range, shutdown: bool, session: *Session) void {
self._proto.deinit(shutdown, session);
}
pub fn asAbstractRange(self: *Range) *AbstractRange {
return self._proto;
}
pub fn setStart(self: *Range, node: *Node, offset: u32) !void {
@@ -309,7 +318,10 @@ pub fn intersectsNode(self: *const Range, node: *Node) bool {
}
pub fn cloneRange(self: *const Range, page: *Page) !*Range {
const clone = try page._factory.abstractRange(Range{ ._proto = undefined }, page);
const arena = try page.getArena(.{ .debug = "Range.clone" });
errdefer page.releaseArena(arena);
const clone = try page._factory.abstractRange(arena, Range{ ._proto = undefined }, page);
clone._proto._end_offset = self._proto._end_offset;
clone._proto._start_offset = self._proto._start_offset;
clone._proto._end_container = self._proto._end_container;
@@ -687,6 +699,8 @@ pub const JsApi = struct {
pub const name = "Range";
pub const prototype_chain = bridge.prototypeChain();
pub var class_id: bridge.ClassId = undefined;
pub const weak = true;
pub const finalizer = bridge.finalizer(Range.deinit);
};
// Constants for compareBoundaryPoints

View File

@@ -21,6 +21,8 @@ const log = @import("../../log.zig");
const js = @import("../js/js.zig");
const Page = @import("../Page.zig");
const Session = @import("../Session.zig");
const Range = @import("Range.zig");
const AbstractRange = @import("AbstractRange.zig");
const Node = @import("Node.zig");
@@ -37,13 +39,22 @@ _direction: SelectionDirection = .none,
pub const init: Selection = .{};
pub fn deinit(self: *Selection, shutdown: bool, session: *Session) void {
if (self._range) |r| {
r.deinit(shutdown, session);
self._range = null;
}
}
fn dispatchSelectionChangeEvent(page: *Page) !void {
const event = try Event.init("selectionchange", .{}, page);
try page._event_manager.dispatch(page.document.asEventTarget(), event);
}
fn isInTree(self: *const Selection) bool {
if (self._range == null) return false;
if (self._range == null) {
return false;
}
const anchor_node = self.getAnchorNode() orelse return false;
const focus_node = self.getFocusNode() orelse return false;
return anchor_node.isConnected() and focus_node.isConnected();
@@ -104,21 +115,33 @@ pub fn getIsCollapsed(self: *const Selection) bool {
}
pub fn getRangeCount(self: *const Selection) u32 {
if (self._range == null) return 0;
if (!self.isInTree()) return 0;
if (self._range == null) {
return 0;
}
if (!self.isInTree()) {
return 0;
}
return 1;
}
pub fn getType(self: *const Selection) []const u8 {
if (self._range == null) return "None";
if (!self.isInTree()) return "None";
if (self.getIsCollapsed()) return "Caret";
if (self._range == null) {
return "None";
}
if (!self.isInTree()) {
return "None";
}
if (self.getIsCollapsed()) {
return "Caret";
}
return "Range";
}
pub fn addRange(self: *Selection, range: *Range, page: *Page) !void {
if (self._range != null) return;
if (self._range != null) {
return;
}
// Only add the range if its root node is in the document associated with this selection
const start_node = range.asAbstractRange().getStartContainer();
@@ -126,22 +149,25 @@ pub fn addRange(self: *Selection, range: *Range, page: *Page) !void {
return;
}
self._range = range;
self.setRange(range, page);
try dispatchSelectionChangeEvent(page);
}
pub fn removeRange(self: *Selection, range: *Range, page: *Page) !void {
if (self._range == range) {
self._range = null;
try dispatchSelectionChangeEvent(page);
return;
} else {
const existing_range = self._range orelse return error.NotFound;
if (existing_range != range) {
return error.NotFound;
}
self.setRange(null, page);
try dispatchSelectionChangeEvent(page);
}
pub fn removeAllRanges(self: *Selection, page: *Page) !void {
self._range = null;
if (self._range == null) {
return;
}
self.setRange(null, page);
self._direction = .none;
try dispatchSelectionChangeEvent(page);
}
@@ -157,7 +183,7 @@ pub fn collapseToEnd(self: *Selection, page: *Page) !void {
try new_range.setStart(last_node, last_offset);
try new_range.setEnd(last_node, last_offset);
self._range = new_range;
self.setRange(new_range, page);
self._direction = .none;
try dispatchSelectionChangeEvent(page);
}
@@ -173,7 +199,7 @@ pub fn collapseToStart(self: *Selection, page: *Page) !void {
try new_range.setStart(first_node, first_offset);
try new_range.setEnd(first_node, first_offset);
self._range = new_range;
self.setRange(new_range, page);
self._direction = .none;
try dispatchSelectionChangeEvent(page);
}
@@ -255,7 +281,7 @@ pub fn extend(self: *Selection, node: *Node, _offset: ?u32, page: *Page) !void {
},
}
self._range = new_range;
self.setRange(new_range, page);
try dispatchSelectionChangeEvent(page);
}
@@ -560,7 +586,8 @@ fn applyModify(self: *Selection, alter: ModifyAlter, new_node: *Node, new_offset
const new_range = try Range.init(page);
try new_range.setStart(new_node, new_offset);
try new_range.setEnd(new_node, new_offset);
self._range = new_range;
self.setRange(new_range, page);
self._direction = .none;
try dispatchSelectionChangeEvent(page);
},
@@ -582,7 +609,7 @@ pub fn selectAllChildren(self: *Selection, parent: *Node, page: *Page) !void {
const child_count = parent.getChildrenCount();
try range.setEnd(parent, @intCast(child_count));
self._range = range;
self.setRange(range, page);
self._direction = .forward;
try dispatchSelectionChangeEvent(page);
}
@@ -630,7 +657,7 @@ pub fn setBaseAndExtent(
},
}
self._range = range;
self.setRange(range, page);
try dispatchSelectionChangeEvent(page);
}
@@ -656,7 +683,7 @@ pub fn collapse(self: *Selection, _node: ?*Node, _offset: ?u32, page: *Page) !vo
try range.setStart(node, offset);
try range.setEnd(node, offset);
self._range = range;
self.setRange(range, page);
self._direction = .none;
try dispatchSelectionChangeEvent(page);
}
@@ -666,6 +693,16 @@ pub fn toString(self: *const Selection, page: *Page) ![]const u8 {
return try range.toString(page);
}
fn setRange(self: *Selection, new_range: ?*Range, page: *Page) void {
if (self._range) |existing| {
existing.deinit(false, page._session);
}
if (new_range) |nr| {
nr.asAbstractRange().acquireRef();
}
self._range = new_range;
}
pub const JsApi = struct {
pub const bridge = js.Bridge(Selection);
@@ -673,6 +710,7 @@ pub const JsApi = struct {
pub const name = "Selection";
pub const prototype_chain = bridge.prototypeChain();
pub var class_id: bridge.ClassId = undefined;
pub const finalizer = bridge.finalizer(Selection.deinit);
};
pub const anchorNode = bridge.accessor(Selection.getAnchorNode, null, .{});

View File

@@ -249,6 +249,8 @@ pub fn createObjectURL(blob: *Blob, page: *Page) ![]const u8 {
.{ page.origin orelse "null", uuid_buf },
);
try page._blob_urls.put(page.arena, blob_url, blob);
// prevent GC from cleaning up the blob while it's in the registry
page.js.strongRef(blob);
return blob_url;
}
@@ -258,8 +260,10 @@ pub fn revokeObjectURL(url: []const u8, page: *Page) void {
return;
}
// Remove from registry (no-op if not found)
_ = page._blob_urls.remove(url);
// Remove from registry and release strong ref (no-op if not found)
if (page._blob_urls.fetchRemove(url)) |entry| {
page.js.weakRef(entry.value);
}
}
pub const JsApi = struct {

View File

@@ -413,15 +413,7 @@ pub fn atob(_: *const Window, input: []const u8, page: *Page) ![]const u8 {
}
pub fn structuredClone(_: *const Window, value: js.Value) !js.Value {
// Simplified structured clone using JSON round-trip.
// Handles JSON-serializable types (objects, arrays, strings, numbers, booleans, null).
const local = value.local;
const str_handle = js.v8.v8__JSON__Stringify(local.handle, value.handle, null) orelse return error.DataCloneError;
const cloned_handle = js.v8.v8__JSON__Parse(local.handle, str_handle) orelse return error.DataCloneError;
return js.Value{
.local = local,
.handle = cloned_handle,
};
return value.structuredClone();
}
pub fn getFrame(self: *Window, idx: usize) !?*Window {

View File

@@ -64,15 +64,30 @@ pub fn createImageData(
switch (width_or_image_data) {
.width => |width| {
const height = maybe_height orelse return error.TypeError;
return ImageData.constructor(width, height, maybe_settings, page);
return ImageData.init(width, height, maybe_settings, page);
},
.image_data => |image_data| {
return ImageData.constructor(image_data._width, image_data._height, null, page);
return ImageData.init(image_data._width, image_data._height, null, page);
},
}
}
pub fn putImageData(_: *const CanvasRenderingContext2D, _: *ImageData, _: f64, _: f64, _: ?f64, _: ?f64, _: ?f64, _: ?f64) void {}
pub fn getImageData(
_: *const CanvasRenderingContext2D,
_: i32, // sx
_: i32, // sy
sw: i32,
sh: i32,
page: *Page,
) !*ImageData {
if (sw <= 0 or sh <= 0) {
return error.IndexSizeError;
}
return ImageData.init(@intCast(sw), @intCast(sh), null, page);
}
pub fn save(_: *CanvasRenderingContext2D) void {}
pub fn restore(_: *CanvasRenderingContext2D) void {}
pub fn scale(_: *CanvasRenderingContext2D, _: f64, _: f64) void {}
@@ -125,6 +140,7 @@ pub const JsApi = struct {
pub const createImageData = bridge.function(CanvasRenderingContext2D.createImageData, .{ .dom_exception = true });
pub const putImageData = bridge.function(CanvasRenderingContext2D.putImageData, .{ .noop = true });
pub const getImageData = bridge.function(CanvasRenderingContext2D.getImageData, .{ .dom_exception = true });
pub const save = bridge.function(CanvasRenderingContext2D.save, .{ .noop = true });
pub const restore = bridge.function(CanvasRenderingContext2D.restore, .{ .noop = true });
pub const scale = bridge.function(CanvasRenderingContext2D.scale, .{ .noop = true });

View File

@@ -63,15 +63,30 @@ pub fn createImageData(
switch (width_or_image_data) {
.width => |width| {
const height = maybe_height orelse return error.TypeError;
return ImageData.constructor(width, height, maybe_settings, page);
return ImageData.init(width, height, maybe_settings, page);
},
.image_data => |image_data| {
return ImageData.constructor(image_data._width, image_data._height, null, page);
return ImageData.init(image_data._width, image_data._height, null, page);
},
}
}
pub fn putImageData(_: *const OffscreenCanvasRenderingContext2D, _: *ImageData, _: f64, _: f64, _: ?f64, _: ?f64, _: ?f64, _: ?f64) void {}
pub fn getImageData(
_: *const OffscreenCanvasRenderingContext2D,
_: i32, // sx
_: i32, // sy
sw: i32,
sh: i32,
page: *Page,
) !*ImageData {
if (sw <= 0 or sh <= 0) {
return error.IndexSizeError;
}
return ImageData.init(@intCast(sw), @intCast(sh), null, page);
}
pub fn save(_: *OffscreenCanvasRenderingContext2D) void {}
pub fn restore(_: *OffscreenCanvasRenderingContext2D) void {}
pub fn scale(_: *OffscreenCanvasRenderingContext2D, _: f64, _: f64) void {}
@@ -124,6 +139,7 @@ pub const JsApi = struct {
pub const createImageData = bridge.function(OffscreenCanvasRenderingContext2D.createImageData, .{ .dom_exception = true });
pub const putImageData = bridge.function(OffscreenCanvasRenderingContext2D.putImageData, .{ .noop = true });
pub const getImageData = bridge.function(OffscreenCanvasRenderingContext2D.getImageData, .{ .dom_exception = true });
pub const save = bridge.function(OffscreenCanvasRenderingContext2D.save, .{ .noop = true });
pub const restore = bridge.function(OffscreenCanvasRenderingContext2D.restore, .{ .noop = true });
pub const scale = bridge.function(OffscreenCanvasRenderingContext2D.scale, .{ .noop = true });

View File

@@ -21,28 +21,34 @@ const js = @import("../../js/js.zig");
const Page = @import("../../Page.zig");
const Session = @import("../../Session.zig");
const FontFace = @import("FontFace.zig");
const EventTarget = @import("../EventTarget.zig");
const Event = @import("../Event.zig");
const Allocator = std.mem.Allocator;
const FontFaceSet = @This();
_proto: *EventTarget,
_arena: Allocator,
pub fn init(page: *Page) !*FontFaceSet {
const arena = try page.getArena(.{ .debug = "FontFaceSet" });
errdefer page.releaseArena(arena);
const self = try arena.create(FontFaceSet);
self.* = .{
return page._factory.eventTargetWithAllocator(arena, FontFaceSet{
._proto = undefined,
._arena = arena,
};
return self;
});
}
pub fn deinit(self: *FontFaceSet, _: bool, session: *Session) void {
session.releaseArena(self._arena);
}
pub fn asEventTarget(self: *FontFaceSet) *EventTarget {
return self._proto;
}
// FontFaceSet.ready - returns an already-resolved Promise.
// In a headless browser there is no font loading, so fonts are always ready.
pub fn getReady(_: *FontFaceSet, page: *Page) !js.Promise {
@@ -56,8 +62,24 @@ pub fn check(_: *const FontFaceSet, font: []const u8) bool {
}
// load(font, text?) - resolves immediately with an empty array.
pub fn load(_: *FontFaceSet, font: []const u8, page: *Page) !js.Promise {
pub fn load(self: *FontFaceSet, font: []const u8, page: *Page) !js.Promise {
// TODO parse font to check if the font has been added before dispatching
// events.
_ = font;
// Dispatch loading event
const target = self.asEventTarget();
if (page._event_manager.hasDirectListeners(target, "loading", null)) {
const event = try Event.initTrusted(comptime .wrap("loading"), .{}, page);
try page._event_manager.dispatchDirect(target, event, null, .{ .context = "load font face set" });
}
// Dispatch loadingdone event
if (page._event_manager.hasDirectListeners(target, "loadingdone", null)) {
const event = try Event.initTrusted(comptime .wrap("loadingdone"), .{}, page);
try page._event_manager.dispatchDirect(target, event, null, .{ .context = "load font face set" });
}
return page.js.local.?.resolvePromise({});
}

View File

@@ -36,6 +36,16 @@ pub fn asNode(self: *Body) *Node {
return self.asElement().asNode();
}
/// Special-case: `body.onload` is actually an alias for `window.onload`.
pub fn setOnLoad(_: *Body, callback: ?js.Function.Global, page: *Page) !void {
page.window._on_load = callback;
}
/// Special-case: `body.onload` is actually an alias for `window.onload`.
pub fn getOnLoad(_: *Body, page: *Page) ?js.Function.Global {
return page.window._on_load;
}
pub const JsApi = struct {
pub const bridge = js.Bridge(Body);
@@ -44,6 +54,8 @@ pub const JsApi = struct {
pub const prototype_chain = bridge.prototypeChain();
pub var class_id: bridge.ClassId = undefined;
};
pub const onload = bridge.accessor(getOnLoad, setOnLoad, .{ .null_as_undefined = false });
};
pub const Build = struct {

View File

@@ -100,6 +100,14 @@ pub fn setAction(self: *Form, value: []const u8, page: *Page) !void {
try self.asElement().setAttributeSafe(comptime .wrap("action"), .wrap(value), page);
}
pub fn getTarget(self: *Form) []const u8 {
return self.asElement().getAttributeSafe(comptime .wrap("target")) orelse "";
}
pub fn setTarget(self: *Form, value: []const u8, page: *Page) !void {
try self.asElement().setAttributeSafe(comptime .wrap("target"), .wrap(value), page);
}
pub fn getLength(self: *Form, page: *Page) !u32 {
const elements = try self.getElements(page);
return elements.length(page);
@@ -120,6 +128,7 @@ pub const JsApi = struct {
pub const name = bridge.accessor(Form.getName, Form.setName, .{});
pub const method = bridge.accessor(Form.getMethod, Form.setMethod, .{});
pub const action = bridge.accessor(Form.getAction, Form.setAction, .{});
pub const target = bridge.accessor(Form.getTarget, Form.setTarget, .{});
pub const elements = bridge.accessor(Form.getElements, null, .{});
pub const length = bridge.accessor(Form.getLength, null, .{});
pub const submit = bridge.function(Form.submit, .{});

View File

@@ -65,6 +65,14 @@ pub fn setSrc(self: *IFrame, src: []const u8, page: *Page) !void {
}
}
pub fn getName(self: *IFrame) []const u8 {
return self.asElement().getAttributeSafe(comptime .wrap("name")) orelse "";
}
pub fn setName(self: *IFrame, value: []const u8, page: *Page) !void {
try self.asElement().setAttributeSafe(comptime .wrap("name"), .wrap(value), page);
}
pub const JsApi = struct {
pub const bridge = js.Bridge(IFrame);
@@ -75,6 +83,7 @@ pub const JsApi = struct {
};
pub const src = bridge.accessor(IFrame.getSrc, IFrame.setSrc, .{});
pub const name = bridge.accessor(IFrame.getName, IFrame.setName, .{});
pub const contentWindow = bridge.accessor(IFrame.getContentWindow, null, .{});
pub const contentDocument = bridge.accessor(IFrame.getContentDocument, null, .{});
};

View File

@@ -61,10 +61,9 @@ pub fn setValue(self: *Option, value: []const u8, page: *Page) !void {
self._value = owned;
}
pub fn getText(self: *const Option) []const u8 {
pub fn getText(self: *const Option, page: *Page) []const u8 {
const node: *Node = @constCast(self.asConstElement().asConstNode());
const allocator = std.heap.page_allocator; // TODO: use proper allocator
return node.getTextContentAlloc(allocator) catch "";
return node.getTextContentAlloc(page.call_arena) catch "";
}
pub fn setText(self: *Option, value: []const u8, page: *Page) !void {

View File

@@ -31,6 +31,8 @@ const Script = @This();
_proto: *HtmlElement,
_src: []const u8 = "",
_executed: bool = false,
// dynamic scripts are forced to be async by default
_force_async: bool = true,
pub fn asElement(self: *Script) *Element {
return self._proto._proto;
@@ -83,10 +85,11 @@ pub fn setCharset(self: *Script, value: []const u8, page: *Page) !void {
}
pub fn getAsync(self: *const Script) bool {
return self.asConstElement().getAttributeSafe(comptime .wrap("async")) != null;
return self._force_async or self.asConstElement().getAttributeSafe(comptime .wrap("async")) != null;
}
pub fn setAsync(self: *Script, value: bool, page: *Page) !void {
self._force_async = false;
if (value) {
try self.asElement().setAttributeSafe(comptime .wrap("async"), .wrap(""), page);
} else {
@@ -136,7 +139,12 @@ pub const JsApi = struct {
try self.asNode().getTextContent(&buf.writer);
return buf.written();
}
pub const text = bridge.accessor(_innerText, Script.setInnerText, .{});
pub const text = bridge.accessor(_text, Script.setInnerText, .{});
fn _text(self: *Script, page: *const Page) ![]const u8 {
var buf = std.Io.Writer.Allocating.init(page.call_arena);
try self.asNode().getChildTextContent(&buf.writer);
return buf.written();
}
};
pub const Build = struct {

View File

@@ -557,13 +557,13 @@ pub const Writer = struct {
pub const AXRole = enum(u8) {
// zig fmt: off
none, article, banner, blockquote, button, caption, cell, checkbox, code,
columnheader, combobox, complementary, contentinfo, definition, deletion,
dialog, document, emphasis, figure, form, group, heading, image, insertion,
link, list, listbox, listitem, main, marquee, meter, navigation, option,
none, article, banner, blockquote, button, caption, cell, checkbox, code, color,
columnheader, combobox, complementary, contentinfo, date, definition, deletion,
dialog, document, emphasis, figure, file, form, group, heading, image, insertion,
link, list, listbox, listitem, main, marquee, menuitem, meter, month, navigation, option,
paragraph, presentation, progressbar, radio, region, row, rowgroup,
rowheader, searchbox, separator, slider, spinbutton, status, strong,
subscript, superscript, table, term, textbox, time, RootWebArea, LineBreak,
subscript, superscript, @"switch", table, term, textbox, time, RootWebArea, LineBreak,
StaticText,
// zig fmt: on
@@ -620,9 +620,13 @@ pub const AXRole = enum(u8) {
.number => .spinbutton,
.search => .searchbox,
.checkbox => .checkbox,
.color => .color,
.date => .date,
.file => .file,
.month => .month,
.@"datetime-local", .week, .time => .combobox,
// zig fmt: off
.password, .@"datetime-local", .hidden, .month, .color,
.week, .time, .file, .date => .none,
.password, .hidden => .none,
// zig fmt: on
};
},
@@ -738,6 +742,44 @@ const AXSource = enum(u8) {
value, // input value
};
pub fn getName(self: AXNode, page: *Page, allocator: std.mem.Allocator) !?[]const u8 {
var aw: std.Io.Writer.Allocating = .init(allocator);
defer aw.deinit();
// writeName expects a std.json.Stringify instance.
const TextCaptureWriter = struct {
aw: *std.Io.Writer.Allocating,
writer: *std.Io.Writer,
pub fn write(w: @This(), val: anytype) !void {
const T = @TypeOf(val);
if (T == []const u8 or T == [:0]const u8 or T == *const [val.len]u8) {
try w.aw.writer.writeAll(val);
} else if (comptime std.meta.hasMethod(T, "format")) {
try std.fmt.format(w.aw.writer, "{s}", .{val});
} else {
// Ignore unexpected types (e.g. booleans) to avoid garbage output
}
}
// Mock JSON Stringifier lifecycle methods
pub fn beginWriteRaw(_: @This()) !void {}
pub fn endWriteRaw(_: @This()) void {}
};
const w: TextCaptureWriter = .{ .aw = &aw, .writer = &aw.writer };
const source = try self.writeName(w, page);
if (source != null) {
// Remove literal quotes inserted by writeString.
var raw_text = std.mem.trim(u8, aw.written(), "\"");
raw_text = std.mem.trim(u8, raw_text, &std.ascii.whitespace);
return try allocator.dupe(u8, raw_text);
}
return null;
}
fn writeName(axnode: AXNode, w: anytype, page: *Page) !?AXSource {
const node = axnode.dom;
@@ -823,15 +865,17 @@ fn writeName(axnode: AXNode, w: anytype, page: *Page) !?AXSource {
.object, .progress, .meter, .main, .nav, .aside, .header,
.footer, .form, .section, .article, .ul, .ol, .dl, .menu,
.thead, .tbody, .tfoot, .tr, .td, .div, .span, .p, .details, .li,
.style, .script,
.style, .script, .html, .body,
// zig fmt: on
=> {},
else => {
// write text content if exists.
var buf = std.Io.Writer.Allocating.init(page.call_arena);
try el.getInnerText(&buf.writer);
try writeString(buf.written(), w);
return .contents;
var buf: std.Io.Writer.Allocating = .init(page.call_arena);
try writeAccessibleNameFallback(node, &buf.writer, page);
if (buf.written().len > 0) {
try writeString(buf.written(), w);
return .contents;
}
},
}
@@ -855,6 +899,48 @@ fn writeName(axnode: AXNode, w: anytype, page: *Page) !?AXSource {
};
}
fn writeAccessibleNameFallback(node: *DOMNode, writer: *std.Io.Writer, page: *Page) !void {
var it = node.childrenIterator();
while (it.next()) |child| {
switch (child._type) {
.cdata => |cd| switch (cd._type) {
.text => |*text| {
const content = std.mem.trim(u8, text.getWholeText(), &std.ascii.whitespace);
if (content.len > 0) {
try writer.writeAll(content);
try writer.writeByte(' ');
}
},
else => {},
},
.element => |el| {
if (el.getTag() == .img) {
if (el.getAttributeSafe(.wrap("alt"))) |alt| {
try writer.writeAll(alt);
try writer.writeByte(' ');
}
} else if (el.getTag() == .svg) {
// Try to find a <title> inside SVG
var sit = child.childrenIterator();
while (sit.next()) |s_child| {
if (s_child.is(DOMNode.Element)) |s_el| {
if (std.mem.eql(u8, s_el.getTagNameLower(), "title")) {
try writeAccessibleNameFallback(s_child, writer, page);
try writer.writeByte(' ');
}
}
}
} else {
if (!el.getTag().isMetadata()) {
try writeAccessibleNameFallback(child, writer, page);
}
}
},
else => {},
}
}
}
fn isHidden(elt: *DOMNode.Element) bool {
if (elt.getAttributeSafe(comptime .wrap("aria-hidden"))) |value| {
if (std.mem.eql(u8, value, "true")) {
@@ -987,7 +1073,7 @@ fn isIgnore(self: AXNode, page: *Page) bool {
return false;
}
fn getRole(self: AXNode) ![]const u8 {
pub fn getRole(self: AXNode) ![]const u8 {
if (self.role_attr) |role_value| {
// TODO the role can have multiple comma separated values.
return role_value;

View File

@@ -196,7 +196,7 @@ pub fn CDPT(comptime TypeProvider: type) type {
return command.sendResult(.{
.frameTree = .{
.frame = .{
.id = "TID-STARTUP-B",
.id = "TID-STARTUP",
.loaderId = "LOADERID24DD2FD56CF1EF33C965C79C",
.securityOrigin = URL_BASE,
.url = "about:blank",

View File

@@ -53,8 +53,8 @@ fn getFullAXTree(cmd: anytype) !void {
const frame_id = params.frameId orelse {
break :blk session.currentPage() orelse return error.PageNotLoaded;
};
const page_id = try id.toPageId(.frame_id, frame_id);
break :blk session.findPage(page_id) orelse {
const page_frame_id = try id.toPageId(.frame_id, frame_id);
break :blk session.findPageByFrameId(page_frame_id) orelse {
return cmd.sendError(-32000, "Frame with the given id does not belong to the target.", .{});
};
};

View File

@@ -502,9 +502,9 @@ fn getFrameOwner(cmd: anytype) !void {
})) orelse return error.InvalidParams;
const bc = cmd.browser_context orelse return error.BrowserContextNotLoaded;
const page_id = try id.toPageId(.frame_id, params.frameId);
const page_frame_id = try id.toPageId(.frame_id, params.frameId);
const page = bc.session.findPage(page_id) orelse {
const page = bc.session.findPageByFrameId(page_frame_id) orelse {
return cmd.sendError(-32000, "Frame with the given id does not belong to the target.", .{});
};

View File

@@ -18,25 +18,67 @@
const std = @import("std");
const lp = @import("lightpanda");
const log = @import("../../log.zig");
const markdown = lp.markdown;
const SemanticTree = lp.SemanticTree;
const interactive = lp.interactive;
const structured_data = lp.structured_data;
const Node = @import("../Node.zig");
const DOMNode = @import("../../browser/webapi/Node.zig");
pub fn processMessage(cmd: anytype) !void {
const action = std.meta.stringToEnum(enum {
getMarkdown,
getSemanticTree,
getInteractiveElements,
getStructuredData,
}, cmd.input.action) orelse return error.UnknownMethod;
switch (action) {
.getMarkdown => return getMarkdown(cmd),
.getSemanticTree => return getSemanticTree(cmd),
.getInteractiveElements => return getInteractiveElements(cmd),
.getStructuredData => return getStructuredData(cmd),
}
}
fn getSemanticTree(cmd: anytype) !void {
const Params = struct {
format: ?enum { text } = null,
prune: ?bool = null,
};
const params = (try cmd.params(Params)) orelse Params{};
const bc = cmd.browser_context orelse return error.NoBrowserContext;
const page = bc.session.currentPage() orelse return error.PageNotLoaded;
const dom_node = page.document.asNode();
var st = SemanticTree{
.dom_node = dom_node,
.registry = &bc.node_registry,
.page = page,
.arena = cmd.arena,
.prune = params.prune orelse false,
};
if (params.format) |format| {
if (format == .text) {
st.prune = params.prune orelse true;
var aw: std.Io.Writer.Allocating = .init(cmd.arena);
defer aw.deinit();
try st.textStringify(&aw.writer);
return cmd.sendResult(.{
.semanticTree = aw.written(),
}, .{});
}
}
return cmd.sendResult(.{
.semanticTree = st,
}, .{});
}
fn getMarkdown(cmd: anytype) !void {
const Params = struct {
nodeId: ?Node.Id = null,
@@ -51,7 +93,7 @@ fn getMarkdown(cmd: anytype) !void {
else
page.document.asNode();
var aw = std.Io.Writer.Allocating.init(cmd.arena);
var aw: std.Io.Writer.Allocating = .init(cmd.arena);
defer aw.deinit();
try markdown.dump(dom_node, .{}, &aw.writer, page);

View File

@@ -238,7 +238,7 @@ pub fn httpRequestStart(bc: anytype, msg: *const Notification.RequestStart) !voi
const transfer = msg.transfer;
const req = &transfer.req;
const frame_id = req.frame_id;
const page = bc.session.findPage(frame_id) orelse return;
const page = bc.session.findPageByFrameId(frame_id) orelse return;
// Modify request with extra CDP headers
for (bc.extra_headers.items) |extra| {

View File

@@ -662,6 +662,10 @@ test "cdp.page: getFrameTree" {
}
test "cdp.page: captureScreenshot" {
const LogFilter = @import("../../testing.zig").LogFilter;
const filter: LogFilter = .init(.not_implemented);
defer filter.deinit();
var ctx = testing.context();
defer ctx.deinit();
{

View File

@@ -35,12 +35,7 @@ fn setIgnoreCertificateErrors(cmd: anytype) !void {
ignore: bool,
})) orelse return error.InvalidParams;
if (params.ignore) {
try cmd.cdp.browser.http_client.disableTlsVerify();
} else {
try cmd.cdp.browser.http_client.enableTlsVerify();
}
try cmd.cdp.browser.http_client.setTlsVerify(!params.ignore);
return cmd.sendResult(null, .{});
}

View File

@@ -340,7 +340,7 @@ fn getTargetInfo(cmd: anytype) !void {
return cmd.sendResult(.{
.targetInfo = TargetInfo{
.targetId = "TID-STARTUP-B",
.targetId = "TID-STARTUP",
.type = "browser",
.title = "",
.url = "about:blank",
@@ -424,14 +424,13 @@ fn setAutoAttach(cmd: anytype) !void {
// set a flag to send Target.attachedToTarget events
cmd.cdp.target_auto_attach = params.autoAttach;
try cmd.sendResult(null, .{});
if (cmd.cdp.target_auto_attach == false) {
// detach from all currently attached targets.
if (cmd.browser_context) |bc| {
bc.session_id = null;
// TODO should we send a Target.detachedFromTarget event?
}
try cmd.sendResult(null, .{});
return;
}
@@ -444,7 +443,7 @@ fn setAutoAttach(cmd: anytype) !void {
try doAttachtoTarget(cmd, &bc.target_id.?);
}
}
// should we send something here?
try cmd.sendResult(null, .{});
return;
}
@@ -460,12 +459,14 @@ fn setAutoAttach(cmd: anytype) !void {
.sessionId = "STARTUP",
.targetInfo = TargetInfo{
.type = "page",
.targetId = "TID-STARTUP-P",
.targetId = "TID-STARTUP",
.title = "",
.url = "about:blank",
.browserContextId = "BID-STARTUP",
},
}, .{});
try cmd.sendResult(null, .{});
}
fn doAttachtoTarget(cmd: anytype, target_id: []const u8) !void {

View File

@@ -228,6 +228,8 @@ pub extern fn X25519_keypair(out_public_value: *[32]u8, out_private_key: *[32]u8
pub const NID_X25519 = @as(c_int, 948);
pub const EVP_PKEY_X25519 = NID_X25519;
pub const NID_ED25519 = 949;
pub const EVP_PKEY_ED25519 = NID_ED25519;
pub extern fn EVP_PKEY_new_raw_private_key(@"type": c_int, unused: ?*ENGINE, in: [*c]const u8, len: usize) [*c]EVP_PKEY;
pub extern fn EVP_PKEY_new_raw_public_key(@"type": c_int, unused: ?*ENGINE, in: [*c]const u8, len: usize) [*c]EVP_PKEY;
@@ -236,3 +238,11 @@ pub extern fn EVP_PKEY_CTX_free(ctx: ?*EVP_PKEY_CTX) void;
pub extern fn EVP_PKEY_derive_init(ctx: ?*EVP_PKEY_CTX) c_int;
pub extern fn EVP_PKEY_derive(ctx: ?*EVP_PKEY_CTX, key: [*c]u8, out_key_len: [*c]usize) c_int;
pub extern fn EVP_PKEY_derive_set_peer(ctx: ?*EVP_PKEY_CTX, peer: [*c]EVP_PKEY) c_int;
pub extern fn EVP_PKEY_free(pkey: ?*EVP_PKEY) void;
pub extern fn EVP_DigestSignInit(ctx: ?*EVP_MD_CTX, pctx: ?*?*EVP_PKEY_CTX, typ: ?*const EVP_MD, e: ?*ENGINE, pkey: ?*EVP_PKEY) c_int;
pub extern fn EVP_DigestSign(ctx: ?*EVP_MD_CTX, sig: [*c]u8, sig_len: *usize, data: [*c]const u8, data_len: usize) c_int;
pub extern fn EVP_MD_CTX_new() ?*EVP_MD_CTX;
pub extern fn EVP_MD_CTX_free(ctx: ?*EVP_MD_CTX) void;
pub const struct_evp_md_ctx_st = opaque {};
pub const EVP_MD_CTX = struct_evp_md_ctx_st;

View File

@@ -22,6 +22,7 @@ pub const Network = @import("network/Runtime.zig");
pub const Server = @import("Server.zig");
pub const Config = @import("Config.zig");
pub const URL = @import("browser/URL.zig");
pub const String = @import("string.zig").String;
pub const Page = @import("browser/Page.zig");
pub const Browser = @import("browser/Browser.zig");
pub const Session = @import("browser/Session.zig");
@@ -31,6 +32,8 @@ pub const log = @import("log.zig");
pub const js = @import("browser/js/js.zig");
pub const dump = @import("browser/dump.zig");
pub const markdown = @import("browser/markdown.zig");
pub const SemanticTree = @import("SemanticTree.zig");
pub const CDPNode = @import("cdp/Node.zig");
pub const interactive = @import("browser/interactive.zig");
pub const structured_data = @import("browser/structured_data.zig");
pub const mcp = @import("mcp.zig");
@@ -110,6 +113,24 @@ pub fn fetch(app: *App, url: [:0]const u8, opts: FetchOpts) !void {
switch (mode) {
.html => try dump.root(page.window._document, opts.dump, writer, page),
.markdown => try markdown.dump(page.window._document.asNode(), .{}, writer, page),
.semantic_tree, .semantic_tree_text => {
var registry = CDPNode.Registry.init(app.allocator);
defer registry.deinit();
const st: SemanticTree = .{
.dom_node = page.window._document.asNode(),
.registry = &registry,
.page = page,
.arena = page.call_arena,
.prune = (mode == .semantic_tree_text),
};
if (mode == .semantic_tree) {
try std.json.Stringify.value(st, .{}, writer);
} else {
try st.textStringify(writer);
}
},
.wpt => try dumpWPT(page, writer),
}
}

View File

@@ -7,6 +7,7 @@ const HttpClient = @import("../browser/HttpClient.zig");
const testing = @import("../testing.zig");
const protocol = @import("protocol.zig");
const router = @import("router.zig");
const CDPNode = @import("../cdp/Node.zig");
const Self = @This();
@@ -17,6 +18,7 @@ http_client: *HttpClient,
notification: *lp.Notification,
browser: lp.Browser,
session: *lp.Session,
node_registry: CDPNode.Registry,
writer: *std.io.Writer,
mutex: std.Thread.Mutex = .{},
@@ -44,12 +46,15 @@ pub fn init(allocator: std.mem.Allocator, app: *App, writer: *std.io.Writer) !*S
.http_client = http_client,
.notification = notification,
.session = undefined,
.node_registry = CDPNode.Registry.init(allocator),
};
self.session = try self.browser.newSession(self.notification);
return self;
}
pub fn deinit(self: *Self) void {
self.node_registry.deinit();
self.aw.deinit();
self.browser.deinit();
self.notification.deinit();
@@ -82,7 +87,7 @@ pub fn sendResult(self: *Self, id: std.json.Value, result: anytype) !void {
}
pub fn sendError(self: *Self, id: std.json.Value, code: protocol.ErrorCode, message: []const u8) !void {
try self.sendResponse(protocol.Response{
try self.sendResponse(.{
.id = id,
.@"error" = protocol.Error{
.code = @intFromEnum(code),

View File

@@ -114,6 +114,7 @@ pub const Tool = struct {
};
pub fn minify(comptime json: []const u8) []const u8 {
@setEvalBranchQuota(100000);
return comptime blk: {
var res: []const u8 = "";
var in_string = false;

View File

@@ -133,9 +133,8 @@ test "MCP.router - handleMessage - synchronous unit tests" {
// 4. Parse error
{
const old_filter = log.opts.filter_scopes;
log.opts.filter_scopes = &.{.mcp};
defer log.opts.filter_scopes = old_filter;
const filter: testing.LogFilter = .init(.mcp);
defer filter.deinit();
try handleMessage(server, aa, "invalid json");
try testing.expectJson("{\"id\": null, \"error\": {\"code\": -32700}}", out_alloc.writer.buffered());

View File

@@ -8,6 +8,7 @@ const Element = @import("../browser/webapi/Element.zig");
const Selector = @import("../browser/webapi/selector/Selector.zig");
const protocol = @import("protocol.zig");
const Server = @import("Server.zig");
const CDPNode = @import("../cdp/Node.zig");
pub const tool_list = [_]protocol.Tool{
.{
@@ -61,6 +62,18 @@ pub const tool_list = [_]protocol.Tool{
\\}
),
},
.{
.name = "semantic_tree",
.description = "Get the page content as a simplified semantic DOM tree for AI reasoning. If a url is provided, it navigates to that url first.",
.inputSchema = protocol.minify(
\\{
\\ "type": "object",
\\ "properties": {
\\ "url": { "type": "string", "description": "Optional URL to navigate to before fetching the semantic tree." }
\\ }
\\}
),
},
.{
.name = "interactiveElements",
.description = "Extract interactive elements from the opened page. If a url is provided, it navigates to that url first.",
@@ -103,13 +116,16 @@ const EvaluateParams = struct {
const ToolStreamingText = struct {
page: *lp.Page,
action: enum { markdown, links },
action: enum { markdown, links, semantic_tree },
registry: ?*CDPNode.Registry = null,
arena: ?std.mem.Allocator = null,
pub fn jsonStringify(self: @This(), jw: *std.json.Stringify) !void {
try jw.beginWriteRaw();
try jw.writer.writeByte('"');
var escaped = protocol.JsonEscapingWriter.init(jw.writer);
var escaped: protocol.JsonEscapingWriter = .init(jw.writer);
const w = &escaped.writer;
switch (self.action) {
.markdown => lp.markdown.dump(self.page.document.asNode(), .{}, w, self.page) catch |err| {
log.err(.mcp, "markdown dump failed", .{ .err = err });
@@ -137,7 +153,21 @@ const ToolStreamingText = struct {
log.err(.mcp, "query links failed", .{ .err = err });
}
},
.semantic_tree => {
const st = lp.SemanticTree{
.dom_node = self.page.document.asNode(),
.registry = self.registry.?,
.page = self.page,
.arena = self.arena.?,
.prune = true,
};
st.textStringify(w) catch |err| {
log.err(.mcp, "semantic tree dump failed", .{ .err = err });
};
},
}
try jw.writer.writeByte('"');
jw.endWriteRaw();
}
@@ -151,6 +181,7 @@ const ToolAction = enum {
interactiveElements,
structuredData,
evaluate,
semantic_tree,
};
const tool_map = std.StaticStringMap(ToolAction).initComptime(.{
@@ -161,6 +192,7 @@ const tool_map = std.StaticStringMap(ToolAction).initComptime(.{
.{ "interactiveElements", .interactiveElements },
.{ "structuredData", .structuredData },
.{ "evaluate", .evaluate },
.{ "semantic_tree", .semantic_tree },
});
pub fn handleCall(server: *Server, arena: std.mem.Allocator, req: protocol.Request) !void {
@@ -188,6 +220,7 @@ pub fn handleCall(server: *Server, arena: std.mem.Allocator, req: protocol.Reque
.interactiveElements => try handleInteractiveElements(server, arena, req.id.?, call_params.arguments),
.structuredData => try handleStructuredData(server, arena, req.id.?, call_params.arguments),
.evaluate => try handleEvaluate(server, arena, req.id.?, call_params.arguments),
.semantic_tree => try handleSemanticTree(server, arena, req.id.?, call_params.arguments),
}
}
@@ -241,6 +274,27 @@ fn handleLinks(server: *Server, arena: std.mem.Allocator, id: std.json.Value, ar
try server.sendResult(id, protocol.CallToolResult(ToolStreamingText){ .content = &content });
}
fn handleSemanticTree(server: *Server, arena: std.mem.Allocator, id: std.json.Value, arguments: ?std.json.Value) !void {
const TreeParams = struct {
url: ?[:0]const u8 = null,
};
if (arguments) |args_raw| {
if (std.json.parseFromValueLeaky(TreeParams, arena, args_raw, .{ .ignore_unknown_fields = true })) |args| {
if (args.url) |u| {
try performGoto(server, u, id);
}
} else |_| {}
}
const page = server.session.currentPage() orelse {
return server.sendError(id, .PageNotLoaded, "Page not loaded");
};
const content = [_]protocol.TextContent(ToolStreamingText){.{
.text = .{ .page = page, .action = .semantic_tree, .registry = &server.node_registry, .arena = arena },
}};
try server.sendResult(id, protocol.CallToolResult(ToolStreamingText){ .content = &content });
}
fn handleInteractiveElements(server: *Server, arena: std.mem.Allocator, id: std.json.Value, arguments: ?std.json.Value) !void {
const Params = struct {
url: ?[:0]const u8 = null,

View File

@@ -28,6 +28,7 @@ const libcurl = @import("../sys/libcurl.zig");
const net_http = @import("http.zig");
const RobotStore = @import("Robots.zig").RobotStore;
const WebBotAuth = @import("WebBotAuth.zig");
const Runtime = @This();
@@ -42,6 +43,11 @@ allocator: Allocator,
config: *const Config,
ca_blob: ?net_http.Blob,
robot_store: RobotStore,
web_bot_auth: ?WebBotAuth,
connections: []net_http.Connection,
available: std.DoublyLinkedList = .{},
conn_mutex: std.Thread.Mutex = .{},
pollfds: []posix.pollfd,
listener: ?Listener = null,
@@ -191,11 +197,29 @@ pub fn init(allocator: Allocator, config: *const Config) !Runtime {
ca_blob = try loadCerts(allocator);
}
const count: usize = config.httpMaxConcurrent();
const connections = try allocator.alloc(net_http.Connection, count);
errdefer allocator.free(connections);
var available: std.DoublyLinkedList = .{};
for (0..count) |i| {
connections[i] = try net_http.Connection.init(ca_blob, config);
available.append(&connections[i].node);
}
const web_bot_auth = if (config.webBotAuth()) |wba_cfg|
try WebBotAuth.fromConfig(allocator, &wba_cfg)
else
null;
return .{
.allocator = allocator,
.config = config,
.ca_blob = ca_blob,
.robot_store = RobotStore.init(allocator),
.connections = connections,
.available = available,
.web_bot_auth = web_bot_auth,
.pollfds = pollfds,
.wakeup_pipe = pipe,
};
@@ -216,7 +240,15 @@ pub fn deinit(self: *Runtime) void {
self.allocator.free(data[0..ca_blob.len]);
}
for (self.connections) |*conn| {
conn.deinit();
}
self.allocator.free(self.connections);
self.robot_store.deinit();
if (self.web_bot_auth) |wba| {
wba.deinit(self.allocator);
}
globalDeinit();
}
@@ -310,6 +342,25 @@ pub fn stop(self: *Runtime) void {
_ = posix.write(self.wakeup_pipe[1], &.{1}) catch {};
}
pub fn getConnection(self: *Runtime) ?*net_http.Connection {
self.conn_mutex.lock();
defer self.conn_mutex.unlock();
const node = self.available.popFirst() orelse return null;
return @fieldParentPtr("node", node);
}
pub fn releaseConnection(self: *Runtime, conn: *net_http.Connection) void {
conn.reset() catch |err| {
lp.assert(false, "couldn't reset curl easy", .{ .err = err });
};
self.conn_mutex.lock();
defer self.conn_mutex.unlock();
self.available.append(&conn.node);
}
pub fn newConnection(self: *Runtime) !net_http.Connection {
return net_http.Connection.init(self.ca_blob, self.config);
}

284
src/network/WebBotAuth.zig Normal file
View File

@@ -0,0 +1,284 @@
// Copyright (C) 2023-2026 Lightpanda (Selecy SAS)
//
// Francis Bouvier <francis@lightpanda.io>
// Pierre Tachoire <pierre@lightpanda.io>
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as
// published by the Free Software Foundation, either version 3 of the
// License, or (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
const std = @import("std");
const crypto = @import("../crypto.zig");
const Http = @import("../network/http.zig");
const WebBotAuth = @This();
pkey: *crypto.EVP_PKEY,
keyid: []const u8,
directory_url: [:0]const u8,
pub const Config = struct {
key_file: []const u8,
keyid: []const u8,
domain: []const u8,
};
fn parsePemPrivateKey(pem: []const u8) !*crypto.EVP_PKEY {
const begin = "-----BEGIN PRIVATE KEY-----";
const end = "-----END PRIVATE KEY-----";
const start_idx = std.mem.indexOf(u8, pem, begin) orelse return error.InvalidPem;
const end_idx = std.mem.indexOf(u8, pem, end) orelse return error.InvalidPem;
const b64 = std.mem.trim(u8, pem[start_idx + begin.len .. end_idx], &std.ascii.whitespace);
// decode base64 into 48-byte DER buffer
var der: [48]u8 = undefined;
try std.base64.standard.Decoder.decode(der[0..48], b64);
// Ed25519 PKCS#8 structure always places the 32-byte raw private key at offset 16.
const key_bytes = der[16..48];
const pkey = crypto.EVP_PKEY_new_raw_private_key(crypto.EVP_PKEY_ED25519, null, key_bytes.ptr, 32);
return pkey orelse error.InvalidKey;
}
fn signEd25519(pkey: *crypto.EVP_PKEY, message: []const u8, out: *[64]u8) !void {
const ctx = crypto.EVP_MD_CTX_new() orelse return error.OutOfMemory;
defer crypto.EVP_MD_CTX_free(ctx);
if (crypto.EVP_DigestSignInit(ctx, null, null, null, pkey) != 1)
return error.SignInit;
var sig_len: usize = 64;
if (crypto.EVP_DigestSign(ctx, out.ptr, &sig_len, message.ptr, message.len) != 1)
return error.SignFailed;
}
pub fn fromConfig(allocator: std.mem.Allocator, config: *const Config) !WebBotAuth {
const pem = try std.fs.cwd().readFileAlloc(allocator, config.key_file, 1024 * 4);
defer allocator.free(pem);
const pkey = try parsePemPrivateKey(pem);
errdefer crypto.EVP_PKEY_free(pkey);
const directory_url = try std.fmt.allocPrintSentinel(
allocator,
"https://{s}/.well-known/http-message-signatures-directory",
.{config.domain},
0,
);
errdefer allocator.free(directory_url);
return .{
.pkey = pkey,
// Owned by the Config so it's okay.
.keyid = config.keyid,
.directory_url = directory_url,
};
}
pub fn signRequest(
self: *const WebBotAuth,
allocator: std.mem.Allocator,
headers: *Http.Headers,
authority: []const u8,
) !void {
const now = std.time.timestamp();
const expires = now + 60;
// build the signature-input value (without the sig1= label)
const sig_input_value = try std.fmt.allocPrint(
allocator,
"(\"@authority\" \"signature-agent\");created={d};expires={d};keyid=\"{s}\";alg=\"ed25519\";tag=\"web-bot-auth\"",
.{ now, expires, self.keyid },
);
defer allocator.free(sig_input_value);
// build the canonical string to sign
const canonical = try std.fmt.allocPrint(
allocator,
"\"@authority\": {s}\n\"signature-agent\": \"{s}\"\n\"@signature-params\": {s}",
.{ authority, self.directory_url, sig_input_value },
);
defer allocator.free(canonical);
// sign it
var sig: [64]u8 = undefined;
try signEd25519(self.pkey, canonical, &sig);
// base64 encode
const encoded_len = std.base64.standard.Encoder.calcSize(sig.len);
const encoded = try allocator.alloc(u8, encoded_len);
defer allocator.free(encoded);
_ = std.base64.standard.Encoder.encode(encoded, &sig);
// build the 3 headers and add them
const sig_agent = try std.fmt.allocPrintSentinel(
allocator,
"Signature-Agent: \"{s}\"",
.{self.directory_url},
0,
);
defer allocator.free(sig_agent);
const sig_input = try std.fmt.allocPrintSentinel(
allocator,
"Signature-Input: sig1={s}",
.{sig_input_value},
0,
);
defer allocator.free(sig_input);
const signature = try std.fmt.allocPrintSentinel(
allocator,
"Signature: sig1=:{s}:",
.{encoded},
0,
);
defer allocator.free(signature);
try headers.add(sig_agent);
try headers.add(sig_input);
try headers.add(signature);
}
pub fn deinit(self: WebBotAuth, allocator: std.mem.Allocator) void {
crypto.EVP_PKEY_free(self.pkey);
allocator.free(self.directory_url);
}
test "parsePemPrivateKey: valid Ed25519 PKCS#8 PEM" {
const pem =
\\-----BEGIN PRIVATE KEY-----
\\MC4CAQAwBQYDK2VwBCIEIBuCRBIEFNtXcMBsyOOkFBFTJcEWTkbgSwKExhOjKFHT
\\-----END PRIVATE KEY-----
\\
;
const pkey = try parsePemPrivateKey(pem);
defer crypto.EVP_PKEY_free(pkey);
}
test "parsePemPrivateKey: missing BEGIN marker returns error" {
const bad_pem = "-----END PRIVATE KEY-----\n";
try std.testing.expectError(error.InvalidPem, parsePemPrivateKey(bad_pem));
}
test "parsePemPrivateKey: missing END marker returns error" {
const bad_pem = "-----BEGIN PRIVATE KEY-----\nMC4CAQA=\n";
try std.testing.expectError(error.InvalidPem, parsePemPrivateKey(bad_pem));
}
test "signEd25519: signature length is always 64 bytes" {
const pem =
\\-----BEGIN PRIVATE KEY-----
\\MC4CAQAwBQYDK2VwBCIEIBuCRBIEFNtXcMBsyOOkFBFTJcEWTkbgSwKExhOjKFHT
\\-----END PRIVATE KEY-----
\\
;
const pkey = try parsePemPrivateKey(pem);
defer crypto.EVP_PKEY_free(pkey);
var sig: [64]u8 = @splat(0);
try signEd25519(pkey, "hello world", &sig);
var all_zero = true;
for (sig) |b| if (b != 0) {
all_zero = false;
break;
};
try std.testing.expect(!all_zero);
}
test "signEd25519: same key + message produces same signature (deterministic)" {
const pem =
\\-----BEGIN PRIVATE KEY-----
\\MC4CAQAwBQYDK2VwBCIEIBuCRBIEFNtXcMBsyOOkFBFTJcEWTkbgSwKExhOjKFHT
\\-----END PRIVATE KEY-----
\\
;
const pkey = try parsePemPrivateKey(pem);
defer crypto.EVP_PKEY_free(pkey);
var sig1: [64]u8 = undefined;
var sig2: [64]u8 = undefined;
try signEd25519(pkey, "deterministic test", &sig1);
try signEd25519(pkey, "deterministic test", &sig2);
try std.testing.expectEqualSlices(u8, &sig1, &sig2);
}
test "signEd25519: same key + diff message produces different signature (deterministic)" {
const pem =
\\-----BEGIN PRIVATE KEY-----
\\MC4CAQAwBQYDK2VwBCIEIBuCRBIEFNtXcMBsyOOkFBFTJcEWTkbgSwKExhOjKFHT
\\-----END PRIVATE KEY-----
\\
;
const pkey = try parsePemPrivateKey(pem);
defer crypto.EVP_PKEY_free(pkey);
var sig1: [64]u8 = undefined;
var sig2: [64]u8 = undefined;
try signEd25519(pkey, "msg 1", &sig1);
try signEd25519(pkey, "msg 2", &sig2);
try std.testing.expect(!std.mem.eql(u8, &sig1, &sig2));
}
test "signRequest: adds headers with correct names" {
const allocator = std.testing.allocator;
const pem =
\\-----BEGIN PRIVATE KEY-----
\\MC4CAQAwBQYDK2VwBCIEIBuCRBIEFNtXcMBsyOOkFBFTJcEWTkbgSwKExhOjKFHT
\\-----END PRIVATE KEY-----
\\
;
const pkey = try parsePemPrivateKey(pem);
const directory_url = try allocator.dupeZ(
u8,
"https://example.com/.well-known/http-message-signatures-directory",
);
var auth = WebBotAuth{
.pkey = pkey,
.keyid = "test-key-id",
.directory_url = directory_url,
};
defer auth.deinit(allocator);
var headers = try Http.Headers.init("User-Agent: Test-Agent");
defer headers.deinit();
try auth.signRequest(allocator, &headers, "example.com");
var it = headers.iterator();
var found_sig_agent = false;
var found_sig_input = false;
var found_signature = false;
var count: usize = 0;
while (it.next()) |h| {
count += 1;
if (std.ascii.eqlIgnoreCase(h.name, "Signature-Agent")) found_sig_agent = true;
if (std.ascii.eqlIgnoreCase(h.name, "Signature-Input")) found_sig_input = true;
if (std.ascii.eqlIgnoreCase(h.name, "Signature")) found_signature = true;
}
try std.testing.expect(count >= 3);
try std.testing.expect(found_sig_agent);
try std.testing.expect(found_sig_input);
try std.testing.expect(found_signature);
}

View File

@@ -237,7 +237,7 @@ pub const ResponseHead = struct {
pub const Connection = struct {
easy: *libcurl.Curl,
node: Handles.HandleList.Node = .{},
node: std.DoublyLinkedList.Node = .{},
pub fn init(
ca_blob_: ?libcurl.CurlBlob,
@@ -385,8 +385,16 @@ pub const Connection = struct {
try libcurl.curl_easy_setopt(self.easy, .write_function, data_cb);
}
pub fn setProxy(self: *const Connection, proxy: ?[*:0]const u8) !void {
try libcurl.curl_easy_setopt(self.easy, .proxy, proxy);
pub fn reset(self: *const Connection) !void {
try libcurl.curl_easy_setopt(self.easy, .header_data, null);
try libcurl.curl_easy_setopt(self.easy, .header_function, null);
try libcurl.curl_easy_setopt(self.easy, .write_data, null);
try libcurl.curl_easy_setopt(self.easy, .write_function, null);
try libcurl.curl_easy_setopt(self.easy, .proxy, null);
}
pub fn setProxy(self: *const Connection, proxy: ?[:0]const u8) !void {
try libcurl.curl_easy_setopt(self.easy, .proxy, if (proxy) |p| p.ptr else null);
}
pub fn setTlsVerify(self: *const Connection, verify: bool, use_proxy: bool) !void {
@@ -467,111 +475,32 @@ pub const Connection = struct {
};
pub const Handles = struct {
connections: []Connection,
dirty: HandleList,
in_use: HandleList,
available: HandleList,
multi: *libcurl.CurlM,
performing: bool = false,
pub const HandleList = std.DoublyLinkedList;
pub fn init(
allocator: Allocator,
ca_blob: ?libcurl.CurlBlob,
config: *const Config,
) !Handles {
const count: usize = config.httpMaxConcurrent();
if (count == 0) return error.InvalidMaxConcurrent;
pub fn init(config: *const Config) !Handles {
const multi = libcurl.curl_multi_init() orelse return error.FailedToInitializeMulti;
errdefer libcurl.curl_multi_cleanup(multi) catch {};
try libcurl.curl_multi_setopt(multi, .max_host_connections, config.httpMaxHostOpen());
const connections = try allocator.alloc(Connection, count);
errdefer allocator.free(connections);
var available: HandleList = .{};
for (0..count) |i| {
connections[i] = try Connection.init(ca_blob, config);
available.append(&connections[i].node);
}
return .{
.dirty = .{},
.in_use = .{},
.connections = connections,
.available = available,
.multi = multi,
};
return .{ .multi = multi };
}
pub fn deinit(self: *Handles, allocator: Allocator) void {
for (self.connections) |*conn| {
conn.deinit();
}
allocator.free(self.connections);
pub fn deinit(self: *Handles) void {
libcurl.curl_multi_cleanup(self.multi) catch {};
}
pub fn hasAvailable(self: *const Handles) bool {
return self.available.first != null;
}
pub fn get(self: *Handles) ?*Connection {
if (self.available.popFirst()) |node| {
self.in_use.append(node);
return @as(*Connection, @fieldParentPtr("node", node));
}
return null;
}
pub fn add(self: *Handles, conn: *const Connection) !void {
try libcurl.curl_multi_add_handle(self.multi, conn.easy);
}
pub fn remove(self: *Handles, conn: *Connection) void {
if (libcurl.curl_multi_remove_handle(self.multi, conn.easy)) {
self.isAvailable(conn);
} else |err| {
// can happen if we're in a perform() call, so we'll queue this
// for cleanup later.
const node = &conn.node;
self.in_use.remove(node);
self.dirty.append(node);
log.warn(.http, "multi remove handle", .{ .err = err });
}
}
pub fn isAvailable(self: *Handles, conn: *Connection) void {
const node = &conn.node;
self.in_use.remove(node);
self.available.append(node);
pub fn remove(self: *Handles, conn: *const Connection) !void {
try libcurl.curl_multi_remove_handle(self.multi, conn.easy);
}
pub fn perform(self: *Handles) !c_int {
self.performing = true;
defer self.performing = false;
const multi = self.multi;
var running: c_int = undefined;
try libcurl.curl_multi_perform(self.multi, &running);
{
const list = &self.dirty;
while (list.first) |node| {
list.remove(node);
const conn: *Connection = @fieldParentPtr("node", node);
if (libcurl.curl_multi_remove_handle(multi, conn.easy)) {
self.available.append(node);
} else |err| {
log.fatal(.http, "multi remove handle", .{ .err = err, .src = "perform" });
@panic("multi_remove_handle");
}
}
}
return running;
}

View File

@@ -305,6 +305,12 @@ pub const String = packed struct {
}
};
pub fn isAllWhitespace(text: []const u8) bool {
return for (text) |c| {
if (!std.ascii.isWhitespace(c)) break false;
} else true;
}
// Discriminatory type that signals the bridge to use arena instead of call_arena
// Use this for strings that need to persist beyond the current call
// The caller can unwrap and store just the underlying .str field

View File

@@ -590,7 +590,10 @@ pub fn curl_easy_setopt(easy: *Curl, comptime option: CurlOption, value: anytype
.header_data,
.write_data,
=> blk: {
const ptr: *anyopaque = @ptrCast(value);
const ptr: ?*anyopaque = switch (@typeInfo(@TypeOf(value))) {
.null => null,
else => @ptrCast(value),
};
break :blk c.curl_easy_setopt(easy, opt, ptr);
},

View File

@@ -501,7 +501,7 @@ pub const TrackingAllocator = struct {
defer self.mutex.unlock();
const result = self.parent_allocator.rawResize(old_mem, alignment, new_len, ra);
self.reallocation_count += 1; // TODO: only if result is not null?
if (result) self.reallocation_count += 1;
return result;
}
@@ -531,7 +531,7 @@ pub const TrackingAllocator = struct {
defer self.mutex.unlock();
const result = self.parent_allocator.rawRemap(memory, alignment, new_len, ret_addr);
self.reallocation_count += 1; // TODO: only if result is not null?
if (result != null) self.reallocation_count += 1;
return result;
}
};

View File

@@ -610,3 +610,23 @@ fn testHTTPHandler(req: *std.http.Server.Request) !void {
unreachable;
}
/// LogFilter provides a scoped way to suppress specific log categories during tests.
/// This is useful for tests that trigger expected errors or warnings.
pub const LogFilter = struct {
old_filter: []const log.Scope,
/// Sets the log filter to only include the specified scope.
/// Returns a LogFilter that should be deinitialized to restore previous filters.
pub fn init(comptime scope: log.Scope) LogFilter {
const old_filter = log.opts.filter_scopes;
const new_filter = comptime &[_]log.Scope{scope};
log.opts.filter_scopes = new_filter;
return .{ .old_filter = old_filter };
}
/// Restores the log filters to their previous state.
pub fn deinit(self: LogFilter) void {
log.opts.filter_scopes = self.old_filter;
}
};