2532 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Karl Seguin	211012d367	move intercept_state and extra_headers from CDP instance to BrowserContext	2025-08-18 13:23:17 +08:00
Karl Seguin	c1319d1f27	add proper resourceType	2025-08-18 12:42:18 +08:00
Karl Seguin	01223601f2	Reduce allocations made during request interception ... Stream (to json) the Transfer as a request and response object in the various network interception-related events (e.g. Network.responseReceived). Add a page.request_intercepted boolean flag for CDP to signal the page that requests have been intercepted, allowing Page.wait to prioritize intercept handling (or, at least, not block it).	2025-08-15 14:01:57 +08:00
Karl Seguin	d9ed4cfca8	Merge pull request #940 from lightpanda-io/redirect-cookies ... enable curl cookie engine	2025-08-15 08:50:25 +08:00
Pierre Tachoire	7d0e4b6270	use CURLOPT_COOKIE to set cookies	2025-08-14 15:33:02 +02:00
Pierre Tachoire	b2f645a5ce	enable curl cookie engine ... Enabling Curl cookie engine brings advantage: * handle cookies during a redirection: when a srv redirects including cookies, curl sends back the cookies correctly during the next request	2025-08-14 15:32:56 +02:00
Karl Seguin	6a29d6711c	Merge pull request #945 from lightpanda-io/remove_unecessary_content_type_parse ... Remove unecessary content type parse	2025-08-14 19:07:41 +08:00
Karl Seguin	5b2806a784	expose response header amount	2025-08-14 18:57:57 +08:00
Karl Seguin	a2f15ce0b2	Remove unecessary content type parse ... getResponseHeader takes header index	2025-08-14 18:26:01 +08:00
Karl Seguin	68400f3bcf	Merge pull request #944 from lightpanda-io/fix_memory_leak ... fix memory leak	2025-08-14 18:20:53 +08:00
Karl Seguin	31f3c2771a	fix build error...sorry	2025-08-14 18:07:14 +08:00
Karl Seguin	f9352e26cb	same memory leak, different place	2025-08-14 18:00:56 +08:00
Karl Seguin	4fa542bc38	fix memory leak	2025-08-14 17:51:40 +08:00
Pierre Tachoire	a707e10af7	Merge pull request #922 from lightpanda-io/nonblocking_libcurl ... Nonblocking libcurl	2025-08-14 11:43:19 +02:00
Pierre Tachoire	1e095fede5	zig fmt build.zig	2025-08-14 11:29:56 +02:00
Karl Seguin	96b10f4b85	Optimize Network.responseReceived ... Add a header iterator to the transfer. This removes the need for NetworkState, duping header name/values, and the http_header_received event.	2025-08-14 15:50:56 +08:00
Karl Seguin	5100e06f38	fix header done callback	2025-08-14 14:51:02 +08:00
Karl Seguin	35e2fa5058	Merge pull request #943 from lightpanda-io/integer-overflow ... fix integer overflow for sleeping delay	2025-08-14 05:43:49 +08:00
Pierre Tachoire	8d2d4ffdd2	fix integer overflow for sleeping delay	2025-08-13 19:44:06 +02:00
sjorsdonkers	7d05712f40	setExtraHTTPHeaders	2025-08-13 14:54:59 +02:00
sjorsdonkers	c0106a238b	http_headers_done_receiving	2025-08-13 14:29:23 +02:00
Karl Seguin	f6c68e4580	fix release build (constness via telemetry, not seen in debug)	2025-08-13 20:16:14 +08:00
Karl Seguin	3c8065fdee	fix fmt	2025-08-13 20:12:39 +08:00
Karl Seguin	9bd8b2fc43	fix wpt runner	2025-08-13 19:39:49 +08:00
Karl Seguin	5a3d5f5512	improve elapsed display for larger numbers	2025-08-13 18:17:59 +08:00
Karl Seguin	ca9e850ac7	Create Client.Transfer earlier. ... On client.request(req) we now immediately wrap the request into a Transfer. This results in less copying of the Request object. It also makes the transfer.uri available, so CDP no longer needs to std.Uri(request.url) anymore. The main advantage is that it's easier to manage resources. There was a use- after free before due to the sensitive nature of the tranfer's lifetime. There were also corner cases where some resources might not be freed. This is hopefully fixed with the lifetime of Transfer being extended.	2025-08-13 18:05:00 +08:00
Karl Seguin	2dc09c799f	Merge pull request #930 from lightpanda-io/request_interception ... request interception	2025-08-13 14:44:26 +08:00
sjorsdonkers	a49154acf4	http_request_fail	2025-08-12 15:20:48 +02:00
sjorsdonkers	77eee7f087	Cookies	2025-08-12 14:40:23 +02:00
sjorsdonkers	03694b54f0	3# This is a combination of 3 commits. ... intercept continue and abort feedback First version of headers, no cookies yet	2025-08-12 13:49:20 +02:00
Karl Seguin	bed320204d	Merge pull request #939 from lightpanda-io/raw-done ... finalize document loading with non-HTML pages	2025-08-12 19:09:31 +08:00
Pierre Tachoire	971524fa3b	finalize document loading with non-HTML pages ... Avoid infinite the loop of loading non-HTML documents with CDP.	2025-08-12 12:55:44 +02:00
Karl Seguin	4758456069	Merge pull request #938 from lightpanda-io/node_isConnected ... Fix Node.isConnected	2025-08-12 18:17:28 +08:00
Karl Seguin	3ef4ba6b8b	Fix Node.isConnected ... The previous implementation just checked if a node had a parent. But it should check the node has a document ancestor: ``` const d1 = document.createElement('div'); document.createElement('div').appendChild(d1); d1.isConnected ``` should be `false`. In addition to this fix, also added support for DocumentFragments which are part of the ShadowRoot. This, like events, is one of those apis that DOES break the ShadowRoot isolation.	2025-08-12 17:23:38 +08:00
Karl Seguin	a504f051e7	Merge pull request #937 from lightpanda-io/event_composedPath ... Add ShadowRoot get/set innerHTML	2025-08-12 17:22:25 +08:00
Karl Seguin	ea0bbaf332	Revert "Treat pending requests as active" ... This reverts commit 19c908035b.	2025-08-12 11:27:28 +08:00
Karl Seguin	19c908035b	Treat pending requests as active ... This ensures that page.wait won't unblock too early. As-is, this isn't an issue since active can only be 0 if there are no active OR pending requests. However, with request interception (https://github.com/lightpanda-io/browser/pull/930) it's possible to have no active requests and no pending requests - from the http client's point of view - but still have pending-on-intercept requests. An alternative to this would be to undo these changes, and instead change Page.wait to be intercept-aware. That is, Page.wait would continue to block on http activity and scheduled tasks, as well as intercepted requests. However, since the Page doesn't know anything about CDP right now, and it does know about the http client, maybe doing this in the client is fine.	2025-08-12 11:13:19 +08:00
Muki Kiboigo	05192b6850	update flake	2025-08-11 12:09:22 -07:00
Karl Seguin	079ce5e9de	whitelist application/ld+json	2025-08-11 21:38:36 +08:00
Karl Seguin	ff742c0169	don't allow concurrent blocking calls	2025-08-11 21:38:36 +08:00
Karl Seguin	332e264437	remove unimportant todos	2025-08-11 21:38:34 +08:00
Karl Seguin	3554634c1c	cleanup optional request headers	2025-08-11 21:37:03 +08:00
Karl Seguin	c96fb3c2f2	support CDP proxy override	2025-08-11 21:37:03 +08:00
Karl Seguin	1e612e4166	Add command line options to control HTTP client ... http_timeout_ms http_connect_timeout_ms http_max_host_open http_max_concurrent	2025-08-11 21:37:03 +08:00
Karl Seguin	06984ace21	fix overflow and debug units	2025-08-11 21:37:03 +08:00
Karl Seguin	cabd4fa718	re-enable datauris	2025-08-11 21:37:03 +08:00
Karl Seguin	ddb549cb45	cookie support	2025-08-11 21:37:02 +08:00
Karl Seguin	c7484c69c0	Increase max concurrent request to 10 ... Improve wait analysis dump. De-prioritize secondary schedules. Don't log warning for application/json scripts Change pretty log timer to display time from start.	2025-08-11 21:37:02 +08:00
Karl Seguin	9876d79680	Add Accept-Encoding ... This is necessary because of CloudFront which will send gzip content even if we don't ask for it. Properly handle scripts that are both async and defer. Add a helper to print state of page wait. This can be helpful in identifying what's causing the page to hang on page.wait.	2025-08-11 21:37:02 +08:00
Karl Seguin	32566ccc80	Set window location on load ... Set SUPPRESS_CONNECT_HEADERS option.	2025-08-11 21:37:02 +08:00