When set (defaults to not set/false), --dump will include iframe contents.
I was hoping I could add a mode to strip_mode to this, but since dump is used
extensively (e.g. innerHTML), this is something that has to be off by default
(for correctness).
Adds a new `mcp` run mode to start an MCP server over stdio.
Implements tools for navigation and JS evaluation, along with
resources for HTML and Markdown page content.
At a high level, this does for Events what was recently done for XHR, Fetch and
Observers. Events are self-contained in their own arena from the ArenaPool and
are registered with v8 to be finalized.
But events are more complicated than those other types. For one, events have
a prototype chain. (XHR also does, but it's always the top-level object that's
created, whereas it's valid to create a base Event or something that inherits
from Event). But the _real_ complication is that Events, unlike previous types,
can be created from Zig or from V8.
This is something that Fetch had to deal with too, because the Response is only
given to V8 on success. So in Fetch, there's a period of time where Zig is
solely responsible for the Response, until it's passed to v8. But with events
it's a lot more subtle.
There are 3 possibilities:
1 - An Event is created from v8. This is the simplest, and it simply becomes a
a weak reference for us. When v8 is done with it, the finalizer is called.
2 - An Event is created in Zig (e.g. window.load) and dispatched to v8. Again
we can rely on the v8 finalizer.
3 - An event is created in Zig, but not dispatched to v8 (e.g. there are no
listeners), Zig has to release the event.
(It's worth pointing out that one thing that still keeps this relatively
straightforward is that we never hold on to Events past some pretty clear point)
Now, it would seem that #3 is the only issue we have to deal with, and maybe
we can do something like:
```
if (event_manager.hasListener("load", capture)) {
try event_manager.dispatch(event);
} else {
event.deinit();
}
```
In fact, in many cases, we could use this to optimize not even creating the
event:
```
if (event_manager.hasListener("load, capture)) {
const event = try createEvent("load", capture);
try event_manager.dispatch(event);
}
```
And that's an optimization worth considering, but it isn't good enough to
properly manage memory. Do you see the issue? There could be a listener (so we
think v8 owns it), but we might never give the value to v8. Any failure between
hasListener and actually handing the value to v8 would result in a leak.
To solve this, the bridge will now set a _v8_handover flag (if present) once it
has created the finalizer_callback entry. So dispatching code now becomes:
```
const event = try createEvent("load", capture);
defer if (!event._v8_handover) event.deinit(false);
try event_manager.dispatch(event);
```
The v8 finalizer callback was also improved. Previously, we just embedded the
pointer to the zig object. In the v8 callback, we could cast that back to T
and call deinit. But, because of possible timing issues between when (if) v8
calls the finalizer, and our own cleanup, the code would check in the context to
see if the ptr was still valid. Wait, what? We're using the ptr to get the
context to see if the ptr is valid?
We now store a pointer to the FinalizerCallback which contains the context.
So instead of something stupid like:
```
// note, if the identity_map doesn't contain the value, then value is likely
// invalid, and value.page will segfault
value.page.js.identity_map.contains(@intFromPtr(value))
```
We do:
```
if (fc.ctx.finalizer_callbacks.contains(@intFromPtr(fc.value)) {
// fc.value is safe to use
}
```
Currently the sighandler is setup regardless of the running mode, but it only
does something in "serve" mode. In fetch mode, since there are no registered
listeners, it intercepts the signal and does nothing. On MacOS at least, this
isn't a great experience as it can leave the process running in the background.
This adds a crash handler which reports a crash (if telemetry is enabled). On a
crash, this looks for `curl` (using the PATH env), and forks the process to then
call execve. This relies on a new endpoint to be setup to accept the "report".
Also, we include very little data..I figured just knowing about crashes would
be a good place to start.
A panic handler is provided, which override's Zig default handler and hooks
into the crash handler.
An `assert` function is added and hooks into the crash handler. This is
currently only used in one place (Session.zig) to demonstrate its use. In
addition to reporting a failed assert, the assert aborts execution in
ReleaseFast (as opposed to an undefined behavior with std.debug.assert).
I want to hook this into the v8 global error handler, but only after direct_v8
is merged.
Much of this is inspired by bun's code. They have their own assert (1) and
a [more sophisticated] crashHandler (2).
:
(1) beccd01647/src/bun.zig (L2987)
(2) beccd01647/src/crash_handler.zig (L198)
There are two layers here. The first is that, on startup, a v8 SnapshotCreator
is created, and a snapshot-specific isolate/context is setup with our browser
environment. This contains most of what was in Env.init and a good chunk of
what was in ExecutionWorld.createContext. From this, we create a v8.StartupData
which is used for the creation of all subsequent contexts. The snapshot sits
at the application level, above the Env - it's re-used for all envs/isolates, so
this gives a nice performance boost for both 1 connection opening multiple pages
or multiple connections opening 1 page.
The second layer is that the Snapshot data can be embedded into the binary, so
that it doesn't have to be created on startup, but rather created at build-time.
This improves the startup time (though, I'm not really sure how to measure that
accurately...).
The first layer is the big win (and just works as-is without any build / usage
changes).
with snapshot
total runs 1000
total duration (ms) 7527
avg run duration (ms) 7
min run duration (ms) 5
max run duration (ms) 41
without snapshot
total runs 1000
total duration (ms) 9350
avg run duration (ms) 9
min run duration (ms) 8
max run duration (ms) 42
To embed a snapshot into the binary, we first need to create the snapshot file:
zig build -Doptimize=ReleaseFast snapshot_creator -- src/snapshot.bin
And then build using the new snapshot_path argument:
zig build -Dsnapshot_path=../../snapshot.bin -Doptimize=ReleaseFast
The paths are weird, I know...since it's embedded, it needs to be inside the
project path, hence we put it in src/snapshot.bin. And since it's embedded
relative to the embedder (src/browser/js/Snapshot.zig) the path has to be
relative to that, hence ../../snapshot.bin. I'm open to suggestions on
improving this.
First, this exposes the v8 Profiler. Right now it's just a commented-out block
in `fetch` and meant for internal debugging.
Depends on: https://github.com/lightpanda-io/zig-v8-fork/pull/105
Use postAttach on Window to attach "static" properties. This comes from
profiling (lightpanda.io) and seeing window.get_self called tens of thousands
of times.
--noscript is deprecated (warning) and automatically maps to --strip_mode js
--strip_mode takes a comma separated list of values. From the help:
- "js" script and link[as=script, rel=preload]
- "ui" includes img, picture, video, css and svg
- "css" includes style and link[rel=stylesheet]
- "full" includes js, ui and css
Maybe this is overkill, but i sometimes find myself looking --dump outputs over
and over again, and removing noise (like HUGE svgs) seems like a small
improvement.
There is some risk to this change. The first is that I made a mistake. The
other is that one of the APIs that doesn't currently return an error changes
in the future.
Further reducing bouncing between page and server for loop polling. If there is
a page, the page polls. If there isn't a page, the server polls. Simpler.
Fix events.get_timeStamp (was events.get_timestamp, wrong casing).
Rename `newRunner` to `htmlRunner`.
move tests to src/tests (from src/browser/tests). src/runtime and possibly other
parts might want to have html tests too.
Follows up on https://github.com/lightpanda-io/browser/pull/994 and replaces
the jsRunner with a new page.navigation-based test runner.
Currently only implemented for the Window tests, looking for feedback and
converting every existing test will take time - so for a while, newRunner (to be
renamed) will sit side-by-side with jsRunner.
In addition to the benefits outlined in 994, largely around code simplicity and
putting more of the actual code under tests, I think our WebAPI tests
particularly benefit from:
1 - No need to recompile when modifying the html tests
2 - Much better assertions, e.g. you can assert that something is actually an
array, not just a string representation of an array
3 - Ability to test some edge cases (e.g. dynamic script loading)
I've put some effort into testing.js to make sure that, if the encapsulating
zig test passes, it's because it actually passed, not because it didn't run.
For the time being, console tests are removed. I think it's more useful to have
access to the console within tests, than it is to test the console (which is
just a wrapper around log, which is both tested and heavily used).
Depends on https://github.com/lightpanda-io/browser/pull/993
There's currently 3 ways to execute a page:
1 - page.navigate (as used in both the 'fetch' and 'serve' commands)
2 - jsRunner as used in unit tests
3 - main_wpt as used in the WPT runner
Both jsRunner and main_wpt replicate the page.navigate code, but in their own
hack-ish way. main_wpt re-implements the DOM walking in order to extract and
execute <script> tags, as well as the needed page lifecycle events.
This PR replaces the existing main_wpt loader with a call to page.navigate. To
support this, a test HTTP server was added. (The test HTTP server is extracted
from the existing unit test test server, and re-used between the two).
There are benefits to this approach:
1 - The code is simpler
2 - More of the actual code and flow is tested
3 - There's 1 way to do things (page.navigate)
4 - Having an HTTP server might unlock some WPT tests
Technically, we're replacing file IO with network IO i.e. http requests). This
has potential downsides:
1 - The tests might be more brittle
2 - The tests might be slower
I think we need to run it for a while to see if we get flaky behavior.
The goal for following PRs is to bring this unification to the jsRunner.
Removes optional platform, which only existed for tests.
There is now a global `@import("testing.zig").test_app` available. This is setup
when the test runner starts, and cleaned up at the end of tests. Individual
tests don't have to worry about creating app, which I assume was the reason I
Platform optional, since that woul dhave been something else that needed to be
setup.
Previously, the IO loop was doing three things:
1 - Managing timeouts (either from scripts or for our own needs)
2 - Handling browser IO events (page/script/xhr)
3 - Handling CDP events (accept, read, write, timeout)
With the libcurl merge, 1 was moved to an in-process scheduler and 2 was moved
to libcurl's own event loop. That means the entire loop code, including
the dependency on tigerbeetle-io existed for handling a single TCP client.
Not only is that a lot of code, there was also friction between the two loops
(the libcurl one and our IO loop), which would result in latency - while one
loop is waiting for the events, any events on the other loop go un-processed.
This PR removes our IO loop. To accomplish this:
1 - The main accept loop is blocking. This is simpler and works perfectly well,
given we only allow 1 active connection.
2 - The client socket is passed to libcurl - yes, libcurl's loop can take
arbitrary FDs and poll them along with its own.
In addition to having one less dependency, the CDP code is quite a bit simpler,
especially around shutdowns and writes. This also removes _some_ of the latency
caused by the friction between page process and CDP processing. Specifically,
when CDP now blocks for input, http page events (script loading, xhr, ...) will
still be processed.
There's still friction. For one, the reverse isn't true: when the page is
waiting for events, CDP events aren't going to be processed. But the page.wait
already have some sensitivity to this (e.g. the page.request_intercepted flag).
Also, when CDP waits, while we will process network events, page timeouts are
still not processed. Because of both these remaining issues, we still need to
jump between the two loops - but being able to block on CDP (even for a short
time) WITHOUT stopping the page's network I/O, should reduce some latency.
