Commit Graph

1942 Commits

Author SHA1 Message Date
Karl Seguin
31f3c2771a fix build error...sorry 2025-08-14 18:07:14 +08:00
Karl Seguin
f9352e26cb same memory leak, different place 2025-08-14 18:00:56 +08:00
Karl Seguin
4fa542bc38 fix memory leak 2025-08-14 17:51:40 +08:00
Pierre Tachoire
a707e10af7 Merge pull request #922 from lightpanda-io/nonblocking_libcurl
Some checks failed
e2e-test / zig build release (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
zig-test / zig build dev (push) Has been cancelled
zig-test / browser fetch (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
Nonblocking libcurl
2025-08-14 11:43:19 +02:00
Karl Seguin
96b10f4b85 Optimize Network.responseReceived
Add a header iterator to the transfer. This removes the need for NetworkState,
duping header name/values, and the http_header_received event.
2025-08-14 15:50:56 +08:00
Karl Seguin
5100e06f38 fix header done callback 2025-08-14 14:51:02 +08:00
Pierre Tachoire
8d2d4ffdd2 fix integer overflow for sleeping delay 2025-08-13 19:44:06 +02:00
sjorsdonkers
7d05712f40 setExtraHTTPHeaders 2025-08-13 14:54:59 +02:00
sjorsdonkers
c0106a238b http_headers_done_receiving 2025-08-13 14:29:23 +02:00
Karl Seguin
f6c68e4580 fix release build (constness via telemetry, not seen in debug) 2025-08-13 20:16:14 +08:00
Karl Seguin
3c8065fdee fix fmt 2025-08-13 20:12:39 +08:00
Karl Seguin
9bd8b2fc43 fix wpt runner 2025-08-13 19:39:49 +08:00
Karl Seguin
5a3d5f5512 improve elapsed display for larger numbers 2025-08-13 18:17:59 +08:00
Karl Seguin
ca9e850ac7 Create Client.Transfer earlier.
On client.request(req) we now immediately wrap the request into a Transfer. This
results in less copying of the Request object. It also makes the transfer.uri
available, so CDP no longer needs to std.Uri(request.url) anymore.

The main advantage is that it's easier to manage resources. There was a use-
after free before due to the sensitive nature of the tranfer's lifetime. There
were also corner cases where some resources might not be freed. This is
hopefully fixed with the lifetime of Transfer being extended.
2025-08-13 18:05:00 +08:00
sjorsdonkers
a49154acf4 http_request_fail 2025-08-12 15:20:48 +02:00
sjorsdonkers
77eee7f087 Cookies 2025-08-12 14:40:23 +02:00
sjorsdonkers
03694b54f0 3# This is a combination of 3 commits.
intercept continue and abort

feedback

First version of headers, no cookies yet
2025-08-12 13:49:20 +02:00
Pierre Tachoire
971524fa3b finalize document loading with non-HTML pages
Avoid infinite the loop of loading non-HTML documents with CDP.
2025-08-12 12:55:44 +02:00
Karl Seguin
3ef4ba6b8b Fix Node.isConnected
The previous implementation just checked if a node had a parent. But it should
check the node has a document ancestor:

```
const d1 = document.createElement('div');
document.createElement('div').appendChild(d1);
d1.isConnected
```

should be `false`.

In addition to this fix, also added support for DocumentFragments which are
part of the ShadowRoot. This, like events, is one of those apis that DOES break
the ShadowRoot isolation.
2025-08-12 17:23:38 +08:00
Karl Seguin
ea0bbaf332 Revert "Treat pending requests as active"
This reverts commit 19c908035b.
2025-08-12 11:27:28 +08:00
Karl Seguin
19c908035b Treat pending requests as active
This ensures that page.wait won't unblock too early. As-is, this isn't an issue
since active can only be 0 if there are no active OR pending requests. However,
with request interception (https://github.com/lightpanda-io/browser/pull/930)
it's possible to have no active requests and no pending requests - from the
http client's point of view - but still have pending-on-intercept requests.

An alternative to this would be to undo these changes, and instead change
Page.wait to be intercept-aware. That is, Page.wait would continue to block on
http activity and scheduled tasks, as well as intercepted requests. However,
since the Page doesn't know anything about CDP right now, and it does know
about the http client, maybe doing this in the client is fine.
2025-08-12 11:13:19 +08:00
Karl Seguin
079ce5e9de whitelist application/ld+json 2025-08-11 21:38:36 +08:00
Karl Seguin
ff742c0169 don't allow concurrent blocking calls 2025-08-11 21:38:36 +08:00
Karl Seguin
332e264437 remove unimportant todos 2025-08-11 21:38:34 +08:00
Karl Seguin
3554634c1c cleanup optional request headers 2025-08-11 21:37:03 +08:00
Karl Seguin
c96fb3c2f2 support CDP proxy override 2025-08-11 21:37:03 +08:00
Karl Seguin
1e612e4166 Add command line options to control HTTP client
http_timeout_ms
http_connect_timeout_ms
http_max_host_open
http_max_concurrent
2025-08-11 21:37:03 +08:00
Karl Seguin
06984ace21 fix overflow and debug units 2025-08-11 21:37:03 +08:00
Karl Seguin
cabd4fa718 re-enable datauris 2025-08-11 21:37:03 +08:00
Karl Seguin
ddb549cb45 cookie support 2025-08-11 21:37:02 +08:00
Karl Seguin
c7484c69c0 Increase max concurrent request to 10
Improve wait analysis dump.

De-prioritize secondary schedules.

Don't log warning for application/json scripts

Change pretty log timer to display time from start.
2025-08-11 21:37:02 +08:00
Karl Seguin
9876d79680 Add Accept-Encoding
This is necessary because of CloudFront which will send gzip content even if
we don't ask for it.

Properly handle scripts that are both async and defer.

Add a helper to print state of page wait. This can be helpful in identifying
what's causing the page to hang on page.wait.
2025-08-11 21:37:02 +08:00
Karl Seguin
32566ccc80 Set window location on load
Set SUPPRESS_CONNECT_HEADERS option.
2025-08-11 21:37:02 +08:00
Karl Seguin
7f9e309ae8 Shutdown clean async scripts
Set parent current script
2025-08-11 21:37:02 +08:00
Karl Seguin
7831aabe5a connect proxy 2025-08-11 21:37:02 +08:00
Karl Seguin
74b40b97ec fix ScriptManager wrong order execution 2025-08-11 21:37:02 +08:00
Karl Seguin
f45726d61f ScriptManager & HttpClient support for JS modules
Improve cleanup/shutdown (*cough* memory leaks *cough*)
2025-08-11 21:37:01 +08:00
Karl Seguin
3c0d027306 dynamic script support 2025-08-11 21:37:01 +08:00
Karl Seguin
dc83765808 fix build 2025-08-11 21:37:01 +08:00
Karl Seguin
4244b572d1 Improve page.wait
Allow page.wait to transition page mode.

Optimize initial page load. No point running scheduler until the initial
page is loaded.

Support ISO-8859-1 charset
2025-08-11 21:37:01 +08:00
Karl Seguin
77475ca5e4 Re-enable --insecure_disable_tls_host_verification
Better error logs on http callback error

Fix wait timing
2025-08-11 21:37:01 +08:00
Karl Seguin
3555680335 Working navigation events (clicks, form submission) 2025-08-11 21:37:01 +08:00
Karl Seguin
f65a39a3e3 Re-enable telemetry
Start work on supporting navigation events (clicks, form submission).
2025-08-11 21:37:00 +08:00
Karl Seguin
94e8964f69 add custom scheduler 2025-08-11 21:37:00 +08:00
Karl Seguin
254d22e2cc don't poll libcurl if we have no running transfers 2025-08-11 21:37:00 +08:00
Karl Seguin
54ab1326e5 Switch XHR to new http client
get puppeteer/cdp.js working again

make test are all passing
2025-08-11 21:37:00 +08:00
Karl Seguin
b0fe5d60ab Initial work on integrating libcurl and making all http nonblocking 2025-08-11 21:36:56 +08:00
Karl Seguin
4b1eb2794f Add ShadowRoot get/set innerHTML
Adds event.composedPath()

This depends on https://github.com/lightpanda-io/libdom/pull/34
2025-08-11 16:32:08 +08:00
Karl Seguin
f5da89b50b lit compatibility
Aims to improve compatibility for the lit framework (e.g. what Reddit is using).

1 - Adds support for adoptedStyleSheets to the Document and ShadowRoot
2 - Adds mock support for replace and replaceSync to the CSSStyleSheet
3 - Optionally include shadowroot in dump
4 - Special-case setting innerHTML on a TemplateElement
2025-08-09 07:43:27 +08:00
Karl Seguin
bede244598 Merge pull request #934 from lightpanda-io/with-base
Some checks failed
e2e-test / zig build release (push) Has been cancelled
zig-test / zig build dev (push) Has been cancelled
zig-test / zig test (push) Has been cancelled
e2e-test / demo-scripts (push) Has been cancelled
e2e-test / cdp-and-hyperfine-bench (push) Has been cancelled
e2e-test / perf-fmt (push) Has been cancelled
zig-test / browser fetch (push) Has been cancelled
zig-test / perf-fmt (push) Has been cancelled
nightly build / build-linux-x86_64 (push) Has been cancelled
nightly build / build-linux-aarch64 (push) Has been cancelled
nightly build / build-macos-aarch64 (push) Has been cancelled
nightly build / build-macos-x86_64 (push) Has been cancelled
wpt / web platform tests json output (push) Has been cancelled
wpt / perf-fmt (push) Has been cancelled
add a --with_base option to fetch
2025-08-09 07:37:32 +08:00