public/browser
1
0
Fork 0
mirror of https://github.com/lightpanda-io/browser.git synced 2026-03-22 04:34:44 +00:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity

add --obey_robots into README examples

Browse Source
This commit is contained in:
Pierre Tachoire
2026-02-10 14:24:38 +01:00
parent e43fc98c0d
commit c68692d78e
1 changed files with 3 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@@ -78,7 +78,7 @@ docker run -d --name lightpanda -p 9222:9222 lightpanda/browser:nightly
### Dump a URL ### Dump a URL
```console ```console
./lightpanda fetch --dump https://lightpanda.io ./lightpanda fetch --obey_robots --dump https://lightpanda.io
``` ```
```console ```console
info(browser): GET https://lightpanda.io/ http.Status.ok info(browser): GET https://lightpanda.io/ http.Status.ok
@@ -90,7 +90,7 @@ info(browser): eval remote https://api.website.lightpanda.io/js/script.js: TypeE
### Start a CDP server ### Start a CDP server
```console ```console
./lightpanda serve --host 127.0.0.1 --port 9222 ./lightpanda serve --obey_robots --host 127.0.0.1 --port 9222
``` ```
```console ```console
info(websocket): starting blocking worker to listen on 127.0.0.1:9222 info(websocket): starting blocking worker to listen on 127.0.0.1:9222
@@ -156,6 +156,7 @@ Here are the key features we have implemented:
- [x] Custom HTTP headers - [x] Custom HTTP headers
- [x] Proxy support - [x] Proxy support
- [x] Network interception - [x] Network interception
- [x] Respect `robots.txt` with option `--obey_robots`
NOTE: There are hundreds of Web APIs. Developing a browser (even just for headless mode) is a huge task. Coverage will increase over time. NOTE: There are hundreds of Web APIs. Developing a browser (even just for headless mode) is a huge task. Coverage will increase over time.