Stream (to json) the Transfer as a request and response object in the various
network interception-related events (e.g. Network.responseReceived).
Add a page.request_intercepted boolean flag for CDP to signal the page that
requests have been intercepted, allowing Page.wait to prioritize intercept
handling (or, at least, not block it).
Enabling Curl cookie engine brings advantage:
* handle cookies during a redirection: when a srv redirects including
cookies, curl sends back the cookies correctly during the next request
On client.request(req) we now immediately wrap the request into a Transfer. This
results in less copying of the Request object. It also makes the transfer.uri
available, so CDP no longer needs to std.Uri(request.url) anymore.
The main advantage is that it's easier to manage resources. There was a use-
after free before due to the sensitive nature of the tranfer's lifetime. There
were also corner cases where some resources might not be freed. This is
hopefully fixed with the lifetime of Transfer being extended.
This ensures that page.wait won't unblock too early. As-is, this isn't an issue
since active can only be 0 if there are no active OR pending requests. However,
with request interception (https://github.com/lightpanda-io/browser/pull/930)
it's possible to have no active requests and no pending requests - from the
http client's point of view - but still have pending-on-intercept requests.
An alternative to this would be to undo these changes, and instead change
Page.wait to be intercept-aware. That is, Page.wait would continue to block on
http activity and scheduled tasks, as well as intercepted requests. However,
since the Page doesn't know anything about CDP right now, and it does know
about the http client, maybe doing this in the client is fine.
This is necessary because of CloudFront which will send gzip content even if
we don't ask for it.
Properly handle scripts that are both async and defer.
Add a helper to print state of page wait. This can be helpful in identifying
what's causing the page to hang on page.wait.
Allow page.wait to transition page mode.
Optimize initial page load. No point running scheduler until the initial
page is loaded.
Support ISO-8859-1 charset
If it wasn't for the fact that the HTTP client is likely going to see a major
refactor, it would definitely be time to create a specific state instance for
synchronous requests.
