public/browser
1
0
Fork 0
mirror of https://github.com/lightpanda-io/browser.git synced 2025-10-29 15:13:28 +00:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

Update README

Browse Source 
Signed-off-by: Francis Bouvier <francis@lightpanda.io>
This commit is contained in:
Francis Bouvier
2024-05-15 18:08:21 +02:00
parent 7ec5a8d15b
commit 0636240a58
3 changed files with 110 additions and 57 deletions
Download Patch File Download Diff File Expand all files Collapse all files

19
Makefile
View File

@@ -23,7 +23,7 @@ help:
# $(ZIG) commands # $(ZIG) commands
# ------------ # ------------
.PHONY: build build-release run run-release shell test bench download-zig wpt .PHONY: build build-dev run run-release shell test bench download-zig wpt
zig_version = $(shell grep 'recommended_zig_version = "' "vendor/zig-js-runtime/build.zig" | cut -d'"' -f2) zig_version = $(shell grep 'recommended_zig_version = "' "vendor/zig-js-runtime/build.zig" | cut -d'"' -f2)
kernel = $(shell uname -ms) kernel = $(shell uname -ms)
@@ -54,23 +54,24 @@ endif
@curl -o "$(dest)" -L "$(url)" || (printf "\e[33mBuild ERROR\e[0m\n"; exit 1;) @curl -o "$(dest)" -L "$(url)" || (printf "\e[33mBuild ERROR\e[0m\n"; exit 1;)
@printf "\e[33mDownloaded $(dest)\e[0m\n" @printf "\e[33mDownloaded $(dest)\e[0m\n"
## Build in debug mode ## Build in release-safe mode
build: build:
@printf "\e[36mBuilding (debug)...\e[0m\n"
@$(ZIG) build -Dengine=v8 || (printf "\e[33mBuild ERROR\e[0m\n"; exit 1;)
@printf "\e[33mBuild OK\e[0m\n"
build-release:
@printf "\e[36mBuilding (release safe)...\e[0m\n" @printf "\e[36mBuilding (release safe)...\e[0m\n"
@$(ZIG) build -Doptimize=ReleaseSafe -Dengine=v8 || (printf "\e[33mBuild ERROR\e[0m\n"; exit 1;) @$(ZIG) build -Doptimize=ReleaseSafe -Dengine=v8 || (printf "\e[33mBuild ERROR\e[0m\n"; exit 1;)
@printf "\e[33mBuild OK\e[0m\n" @printf "\e[33mBuild OK\e[0m\n"
## Run the server ## Build in debug mode
build-dev:
@printf "\e[36mBuilding (debug)...\e[0m\n"
@$(ZIG) build -Dengine=v8 || (printf "\e[33mBuild ERROR\e[0m\n"; exit 1;)
@printf "\e[33mBuild OK\e[0m\n"
## Run the server in debug mode
run: build run: build
@printf "\e[36mRunning...\e[0m\n" @printf "\e[36mRunning...\e[0m\n"
@./zig-out/bin/browsercore || (printf "\e[33mRun ERROR\e[0m\n"; exit 1;) @./zig-out/bin/browsercore || (printf "\e[33mRun ERROR\e[0m\n"; exit 1;)
## Run a JS shell in release-safe mode ## Run a JS shell in debug mode
shell: shell:
@printf "\e[36mBuilding shell...\e[0m\n" @printf "\e[36mBuilding shell...\e[0m\n"
@$(ZIG) build shell -Dengine=v8 || (printf "\e[33mBuild ERROR\e[0m\n"; exit 1;) @$(ZIG) build shell -Dengine=v8 || (printf "\e[33mBuild ERROR\e[0m\n"; exit 1;)

146
README.md
View File

@@ -1,20 +1,69 @@
# Browsercore <p align="center">
<a href="https://lightpanda.io"><img src="https://lpd-cdn.s3.eu-west-3.amazonaws.com/assets/images/logo/lpd-logo.png" alt="Logo" height=170></a>
</p>
## Build <h1 align="center">Lightpanda</h1>
<div align="center">
<br />
</div>
Lightpanda is the open-source browser made for headless usage:
- Javascript execution
- Support of the Web APIs (partial, WIP)
- Compatible with Playwright, Puppeteer through CDP (WIP)
Fast scraping and web automation with minimal memory footprint:
- Ultra-low memory footprint (12x less than Chrome)
- Blazingly fast & instant startup (64x faster than Chrome)
See [benchmark details](https://github.com/lightpanda-io/demo).
## Why?
### Javascript execution is mandatory for the modern web
Back in the good old times, grabbing a webpage was as easy as making an HTTP request, cURL-like. It’s not possible anymore, because Javascript is everywhere, like it or not:
- Ajax, Single Page App, Infinite loading, “click to display”, instant search, etc.
- JS web frameworks: React, Vue, Angular & others
### Chrome is not the right tool
So if we need Javascript, why not use a real web browser. Let’s take a huge desktop application, hack it, and run it on the server, right? Hundreds of instance of Chrome if you use it at scale. Are you sure it’s such a good idea?
- Heavy on RAM and CPU, expensive to run
- Hard to package, deploy and maintain at scale
- Bloated, lots of features are not useful in headless usage
### Lightpanda is built for performance
If we want both Javascript and performance, for a real headless browser, we need to start from scratch. Not yet another iteration of Chromium, really from a blank page. Crazy right? But that’s we did:
- Not based on Chromium, Blink or WebKit
- Low-level system programming language (Zig) with optimisations in mind
- Opinionated, no rendering
## Build from sources
We do not provide yet binary versions of Lightpanda, you have to compile it from source.
### Prerequisites ### Prerequisites
Browsercore is written with [Zig](https://ziglang.org/) `0.12`. You have to Lightpanda is written with [Zig](https://ziglang.org/) `0.12`. You have to
install it with the right version in order to build the project. install it with the right version in order to build the project.
Browsercore also depends on Lightpanda also depends on
[js-runtimelib](https://github.com/francisbouvier/zig-js-runtime/), [zig-js-runtime](https://github.com/lightpanda-io/zig-js-runtime/) (with v8),
[Netsurf libs](https://www.netsurf-browser.org/) and [Netsurf libs](https://www.netsurf-browser.org/) and
[Mimalloc](https://microsoft.github.io/mimalloc) libs. [Mimalloc](https://microsoft.github.io/mimalloc).
To be able to build the v8 engine for js-runtimelib, you have to install some libs: To be able to build the v8 engine for zig-js-runtime, you have to install some libs:
For Debian/Ubuntu based Linux: For Debian/Ubuntu based Linux:
``` ```
sudo apt install xz-utils \ sudo apt install xz-utils \
python3 ca-certificates git \ python3 ca-certificates git \
@@ -23,100 +72,103 @@ sudo apt install xz-utils \
cmake clang cmake clang
``` ```
For MacOS, you only need Python 3 and cmake. For MacOS, you only need cmake:
```
brew install cmake
```
### Install and build dependencies ### Install and build dependencies
The project uses git submodule for dependencies. #### All in one build
The `make install-submodule` will init and update the submodules in the `vendor/`
directory. You can run `make install` to install deps all in one (or `make install-dev` if you need the development versions).
Be aware that the build task is very long and cpu consuming, as you will build from sources all dependancies, including the v8 Javascript engine.
#### Step by step build dependancy
The project uses git submodules for dependencies.
To init or update the submodules in the `vendor/` directory:
``` ```
make install-submodule make install-submodule
``` ```
### Build Netsurf **Netsurf libs**
Netsurf libs are used for HTML parsing and DOM tree generation.
The command `make install-netsurf` will build Netsurf libs used by browsercore.
``` ```
make install-netsurf make install-netsurf
``` ```
For dev env, use `make install-netsurf-dev`. For dev env, use `make install-netsurf-dev`.
### Build Mimalloc **Mimalloc**
Mimalloc is used as a C memory allocator.
The command `make install-mimalloc` will build Mimalloc lib used by browsercore.
``` ```
make install-mimalloc make install-mimalloc
``` ```
For dev env, use `make install-mimalloc-dev`. For dev env, use `make install-mimalloc-dev`.
Note, when Mimalloc is built in dev mode, you can dump memory stats with the Note: when Mimalloc is built in dev mode, you can dump memory stats with the
env var `MIMALLOC_SHOW_STATS=1`. See env var `MIMALLOC_SHOW_STATS=1`. See
https://microsoft.github.io/mimalloc/environment.html [https://microsoft.github.io/mimalloc/environment.html](https://microsoft.github.io/mimalloc/environment.html).
### Build zig-js-runtime **zig-js-runtime**
The command `make install-zig-js-runtime-dev` uses zig-js-runtime's `zig-v8` dependency to build v8 engine lib. Our own Zig/Javascript runtime, which includes the v8 Javascript engine.
Be aware the build task is very long and cpu consuming.
Build v8 engine for debug/dev version, it creates This build task is very long and cpu consuming, as you will build v8 from sources.
`vendor/zig-js-runtime/vendor/v8/$ARCH/debug/libc_v8.a` file.
```
make install-zig-js-runtime-dev
```
You should also build a release vesion of v8 with:
``` ```
make install-zig-js-runtime make install-zig-js-runtime
``` ```
### All in one build For dev env, use `make iinstall-zig-js-runtime-dev`.
You can run `make intall` and `make install-dev` to install deps all in one.
## Test ## Test
### Unit Tests ### Unit Tests
You can test browsercore by running `make test`. You can test Lightpanda by running `make test`.
### Web Platform Tests ### Web Platform Tests
Browsercore is tested against the standardized [Web Platform Lightpanda is tested against the standardized [Web Platform
Tests](https://web-platform-tests.org/). Tests](https://web-platform-tests.org/).
The relevant tests cases for Browsercore are commit with the project. The relevant tests cases are committed in a [dedicated repository](https://github.com/lightpanda-io/wpt) which is fetched by the `make install-submodule` command.
All the tests cases executed are located in `tests/wpt` dir and come from an
external repository: https://github.com/lightpanda-io/wpt All the tests cases executed are located in the `tests/wpt` sub-directory.
For reference, you can easily execute a WPT test case with your browser via For reference, you can easily execute a WPT test case with your browser via
[wpt.live](https://wpt.live). [wpt.live](https://wpt.live).
*Run WPT test suite* #### Run WPT test suite
To run all the tests:
You can run all the test.
The runner execute all the tests ending with `.html`.
``` ```
make wpt make wpt
``` ```
Or one specific test by using a suffix. Or one specific test:
``` ```
make wpt Node-childNodes.html make wpt Node-childNodes.html
``` ```
*Add a new WPT test case* #### Add a new WPT test case
We add new tests cases files with implemented changes in Browsercore. We add new relevant tests cases files when we implemented changes in Lightpanda.
Copy the test case you want to add from the [WPT To add a new test, copy the file you want from the [WPT
repo](https://github.com/web-platform-tests/wpt) into `tests/wpt` dir, commit repo](https://github.com/web-platform-tests/wpt) into the `tests/wpt` directory.
the files in the https://github.com/lightpanda-io/wpt repository and update the
git submodule in browsercore.
:warning: Please keep the original directory tree structure into `tests/wpt`. :warning: Please keep the original directory tree structure of `tests/wpt`.

2
vendor/zig-js-runtime vendored

Submodule vendor/zig-js-runtime updated: e8b74e1983...fff1a6778d