add some anti crawler detection

add useragent override flag
copyright update
2026-02-15 05:45:17 +00:00 · 2024-01-02 21:55:39 -08:00 · 2024-01-02 21:54:59 -08:00 · 2024-01-02 02:09:38 -08:00 · 2024-01-02 02:08:14 -08:00 · 2024-01-02 02:00:30 -08:00
6 changed files with 298 additions and 192 deletions
--- a/1
+++ b/1
@@ -10,7 +10,6 @@ cross:
 	GOOS=darwin GOARCH=amd64 go build -a -o wrp-amd64-macos wrp.go
 	GOOS=darwin GOARCH=arm64 go build -a -o wrp-arm64-macos wrp.go
 	GOOS=windows GOARCH=amd64 go build -a -o wrp-amd64-windows.exe wrp.go
-	GOOS=windows GOARCH=386 go build -a -o wrp-386-windows.exe wrp.go
 	GOOS=linux GOARCH=arm go build -a -o wrp-arm-linux wrp.go
 	GOOS=linux GOARCH=arm64 go build -a -o wrp-arm64-linux wrp.go

--- a/README.md
+++ b/README.md
@@ -4,24 +4,57 @@ A browser-in-browser "proxy" server that allows to use historical / vintage web

 ![Internet Explorer 1.5 doing Gmail](wrp.png)

-## Usage
+## Usage Instructions

-* [Download a WRP binary](https://github.com/tenox7/wrp/releases/) and run it on a machine that will become your WRP gateway/server. 
-This machine should be pretty modern, high spec and Google Chrome / Chromium Browser is required to be preinstalled.
-* Point your legacy browser to `http://address:port` of WRP server. Do not set or use it as a "proxy server".
-* Type a search string or a http/https URL and click **Go**.
+* [Download a WRP binary](https://github.com/tenox7/wrp/releases/) and run it on a machine that will become your WRP gateway/server. This should be modern hardware, OS and Google Chrome / Chromium Browser is required to be preinstalled. Do not try to run WRP on an old machine like Windows XP or 98.
+* Make sure you have disabled firewall or open port WRP is listening on (by default 8080).
+* Point your legacy browser to `http://address:port` of the WRP server. Do not set or use it as a "proxy server".
+* Type a search string or a full http/https URL and click **Go**.
 * Adjust your screen **W**idth/**H**eight/**S**cale/**C**olors to fit in your old browser.
 * Scroll web page by clicking on the in-image scroll bar.
+* WRP also allows **a single tall image without the vertical scrollbar** and use client scrolling. To enable this, simply height **H** to `0` . However this should not be used with old and low spec clients. Such tall images will be very large, take a lot of memory and long time to process, especially for GIFs.
 * Do not use client browser history-back, instead use **Bk** button in the app.
-* To send keystrokes, fill **K** input box and press **Go**. There also are buttons for backspace, enter and arrow keys.
-* You can set height **H** to `0` to render pages in to **a single tall image without the vertical scrollbar** and use client scrolling. However this should not be used with old and low spec clients. Such tall images will be very large and take long time to process, especially for GIFs.
-* Prefer PNG over GIF if your browser supports it. PNG is much faster, whereas GIF requires a lot of additional processing on both client and server.
-* You can re-capture page screenshot without reloading by using **St** (Stop).
+* You can re-capture page screenshot without reloading by using **St** (Stop). This is useful if page didn't render fully before screenshot is taken.
 * You can also reload and re-capture current page with **Re** (Reload).
+* To send keystrokes, fill **K** input box and press **Go**. There also are buttons for backspace, enter and arrow keys.
+* Prefer PNG over GIF if your browser supports it. PNG is much faster, whereas GIF requires a lot of additional processing on both client and server to encode/decode. Jpeg encoding is also quite fast.
+* GIF images are by default encoded with 216 colors, "web safe" palette. This uses an ultra fast but not very accurate color mapping algorithm. If you want better color representation switch to 256 color mode.

-## Customization
+## UI explanation

-Since 4.5.1 WRP supports customizing it's own UI using HTML Template file. Download [wrp.html](wrp.html) place in the same directory with wrp binary customize it to your liking.
+The first unnamed input box is either search (google) or URL starting with http/https
+
+**Go** instructs browser to navigate to the url or perform search
+
+**Bk** is History Back
+
+**St** is Stop, also re-capture screenshot without refreshing page, for example if page
+render takes a long time or it changes periodically
+
+**Re** is Reload
+
+**W** is width in pixels, adjust it to get rid of horizontal scroll bar
+
+**H** is height in pixels, adjust it to get rid of vertical scroll bar.
+It can also be set to 0 to produce one very tall image and use
+client scroll. This 0 size is experimental, buggy and should be
+used with PNG and lots of memory on a client side.
+
+**Z** is zoom or scale
+
+**C** is colors, for GIF images only (unused in PNG, JPG)
+
+**K** is keystroke input, you can type some letters in it and when you click Go it will be typed in the remote browser.
+
+**Bs** is backspace
+
+**Rt** is return / enter
+
+**< ^ v >** are arrow keys, typically for navigating a map, buggy.
+
+### UI Customization 
+
+WRP supports customizing it's own UI using HTML Template file. Download [wrp.html](wrp.html) place in the same directory with wrp binary customize it to your liking.

 ## Docker

@@ -49,49 +82,20 @@ Or from the [Azure Console](https://portal.azure.com/#create/Microsoft.Container

 Fortunately ACI allows port 80 without encryption.

-
 ## Flags

+```text
+-l   listen address:port (default :8080)
+-t   image type gif, png or jpg (default gif)
+-g   image geometry, WxHxC, height can be 0 for unlimited (default 1152x600x216)
+     C (number of colors) is only used for GIF
+-q   Jpeg image quality, default 80%
+-h   headless mode, hide browser window on the server (default true)
+-d   chromedp debug logging (default false)
+-n   do not free maps and images after use (default false)
+-ui  html template file (default "wrp.html")
+-s   delay/sleep after page is rendered before screenshot is taken (default 2s)
 ```
-l  listen address:port (default :8080)
-t  image type gif or png (default gif) 
-g  image geometry, WxHxC, height can be 0 for unlimited (default 1152x600x256)
-h  headless mode, hide browser window on the server (default true)
-d  chromedp debug logging (default false)
-n  do not free maps and gif images after use (default false)
-```
-
-## UI explanation
-
-The first unnamed input box is either search (google) or URL starting with http/https
-
-**Go** instructs browser to navigate to the url or perform search
-
-**Bk** is History Back
-
-**St** is Stop, also re-capture screenshot without refreshing page, for example if page
-render takes a long time or it changes periodically
-
-**Re** is Reload
-
-**W** is width in pixels, adjust it to get rid of horizontal scroll bar
-
-**H** is height in pixels, adjust it to get rid of vertical scroll bar.
-It can also be set to 0 to produce one very tall image and use
-client scroll. This 0 size is experimental, buggy and should be
-used with PNG and lots of memory on a client side.
-
-**Z** is zoom or scale
-
-**C** is colors, for GIF images only (unused in PNG)
-
-**K** is keystroke input, you can type some letters in it and when you click Go it will be typed in the remote browser.
-
-**Bs** is backspace
-
-**Rt** is return / enter
-
-**< ^ v >** are arrow keys, typically for navigating a map, buggy.

 ## Minimal Requirements

@@ -104,7 +108,7 @@ used with PNG and lots of memory on a client side.

 This program does not have a GUI and is run from the command line. You may need to enable executable bit on Unix systems, for example:

-```bash
+```shell
 $ cd ~/Downloads
 $ chmod +x wrp-amd64-macos
 $ ./wrp-amd64-macos -t png
@@ -117,8 +121,9 @@ $ ./wrp-amd64-macos -t png
 * In 2016 thanks to EFF/Certbot the whole internet migrated to HTTPS/SSL/TLS and WRP largely stopped working. Python code became unmaintainable and there was no easy way to make it work on Windows, even under WSL.
 * Version 3.0 (2019) has been rewritten in [Go](https://golang.org/) using [Chromedp](https://github.com/chromedp) as browser-in-browser instead of http-proxy. The initial version was [less than 100 lines of code](https://gist.github.com/tenox7/b0f03c039b0a8b67f6c1bf47e2dd0df0).
 * Version 4.0 has been completely refactored to use mouse clicks via imagemap instead parsing a href nodes. 
-* Version 4.1 added sending keystrokes in to input boxes. You can now login to Gmail. Also now runs as a Docker container and on Cloud Run/Azure Containers. 
+* Version 4.1 added sending keystrokes in to input boxes. You can now login to Gmail. Also now runs as a Docker container and on Cloud Run/Azure Containers.
 * Version 4.5 introduces rendering whole pages in to a single tall image with client scrolling.
+* Version 4.6 adds blazing fast gif encoding by [Hill Ma](https://github.com/mahiuchun).

 ## Credits

@@ -126,10 +131,11 @@ $ ./wrp-amd64-macos -t png
 * Uses [go-quantize](https://github.com/ericpauley/go-quantize), thanks to [ericpauley](https://github.com/ericpauley) for developing the missing go quantizer
 * Thanks to Jason Stevens of [Fun With Virtualization](https://virtuallyfun.com/) for graciously hosting my rumblings
 * Thanks to [claunia](https://github.com/claunia/) for help with the Python/Webkit version in the past
+* Thanks to [Hill Ma](https://github.com/mahiuchun) for ultra fast gif encoding algorithm
 * Historical Python/Webkit versions and prior art can be seen in [wrp-old](https://github.com/tenox7/wrp-old) repo

 ## Legal Stuff

 License: Apache 2.0  
 Copyright (c) 2013-2018 Antoni Sawicki  
-Copyright (c) 2019-2020 Google LLC
+Copyright (c) 2019-2022 Google LLC
--- a/go.mod
+++ b/go.mod
@@ -1,11 +1,20 @@
 module github.com/tenox7/wrp

-go 1.16
+go 1.21.5

 require (
 	github.com/MaxHalford/halfgone v0.0.0-20171017091812-482157b86ccb
-	github.com/chromedp/cdproto v0.0.0-20221029224954-108014bf7279
-	github.com/chromedp/chromedp v0.8.6
-	github.com/ericpauley/go-quantize v0.0.0-20200331213906-ae555eb2afa4
-	golang.org/x/sys v0.1.0 // indirect
+	github.com/chromedp/cdproto v0.0.0-20231205062650-00455a960d61
+	github.com/chromedp/chromedp v0.9.3
+	github.com/soniakeys/quant v1.0.0
+)
+
+require (
+	github.com/chromedp/sysutil v1.0.0 // indirect
+	github.com/gobwas/httphead v0.1.0 // indirect
+	github.com/gobwas/pool v0.2.1 // indirect
+	github.com/gobwas/ws v1.3.1 // indirect
+	github.com/josharian/intern v1.0.0 // indirect
+	github.com/mailru/easyjson v0.7.7 // indirect
+	golang.org/x/sys v0.15.0 // indirect
 )
--- a/go.sum
+++ b/go.sum
@@ -1,20 +1,19 @@
 github.com/MaxHalford/halfgone v0.0.0-20171017091812-482157b86ccb h1:YQ+d0g0P0F/06oDoeEgDHeZCIrnKgLxXcqYOpe8sTuU=
 github.com/MaxHalford/halfgone v0.0.0-20171017091812-482157b86ccb/go.mod h1:J86XzS1wgzJPjpQmpriJ+SetP17JSQUd9l+HWQK86jA=
-github.com/chromedp/cdproto v0.0.0-20220924210414-0e3390be1777/go.mod h1:5Y4sD/eXpwrChIuxhSr/G20n9CdbCmoerOHnuAf0Zr0=
-github.com/chromedp/cdproto v0.0.0-20221029224954-108014bf7279 h1:7+D/pA8BoNzTpcM0Yw8issS95U/ipn0im5vzhfPzDZc=
-github.com/chromedp/cdproto v0.0.0-20221029224954-108014bf7279/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs=
-github.com/chromedp/chromedp v0.8.6 h1:KobeeqR2dpfKSG1prS3Y6+FbffMmGC6xmAobRXA9QEQ=
-github.com/chromedp/chromedp v0.8.6/go.mod h1:nBYHoD6YSNzrr82cIeuOzhw1Jo/s2o0QQ+ifTeoCZ+c=
+github.com/chromedp/cdproto v0.0.0-20231011050154-1d073bb38998/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs=
+github.com/chromedp/cdproto v0.0.0-20231205062650-00455a960d61 h1:XD280QPATe9jaz20dylKe3vBsNcH1w3mkssGY0lidn8=
+github.com/chromedp/cdproto v0.0.0-20231205062650-00455a960d61/go.mod h1:GKljq0VrfU4D5yc+2qA6OVr8pmO/MBbPEWqWQ/oqGEs=
+github.com/chromedp/chromedp v0.9.3 h1:Wq58e0dZOdHsxaj9Owmfcf+ibtpYN1N0FWVbaxa/esg=
+github.com/chromedp/chromedp v0.9.3/go.mod h1:NipeUkUcuzIdFbBP8eNNvl9upcceOfWzoJn6cRe4ksA=
 github.com/chromedp/sysutil v1.0.0 h1:+ZxhTpfpZlmchB58ih/LBHX52ky7w2VhQVKQMucy3Ic=
 github.com/chromedp/sysutil v1.0.0/go.mod h1:kgWmDdq8fTzXYcKIBqIYvRRTnYb9aNS9moAV0xufSww=
-github.com/ericpauley/go-quantize v0.0.0-20200331213906-ae555eb2afa4 h1:BBade+JlV/f7JstZ4pitd4tHhpN+w+6I+LyOS7B4fyU=
-github.com/ericpauley/go-quantize v0.0.0-20200331213906-ae555eb2afa4/go.mod h1:H7chHJglrhPPzetLdzBleF8d22WYOv7UM/lEKYiwlKM=
 github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU=
 github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
 github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og=
 github.com/gobwas/pool v0.2.1/go.mod h1:q8bcK0KcYlCgd9e7WYLm9LpyS+YeLd8JVDW6WezmKEw=
-github.com/gobwas/ws v1.1.0 h1:7RFti/xnNkMJnrK7D1yQ/iCIB5OrrY/54/H930kIbHA=
-github.com/gobwas/ws v1.1.0/go.mod h1:nzvNcVha5eUziGrbxFCo6qFIojQHjJV5cLYIbezhfL0=
+github.com/gobwas/ws v1.3.0/go.mod h1:hRKAFb8wOxFROYNsT1bqfWnhX+b5MFeJM9r2ZSwg/KY=
+github.com/gobwas/ws v1.3.1 h1:Qi34dfLMWJbiKaNbDVzM9x27nZBjmkaW6i4+Ku+pGVU=
+github.com/gobwas/ws v1.3.1/go.mod h1:hRKAFb8wOxFROYNsT1bqfWnhX+b5MFeJM9r2ZSwg/KY=
 github.com/josharian/intern v1.0.0 h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
 github.com/josharian/intern v1.0.0/go.mod h1:5DoeVV0s6jJacbCEi61lwdGj/aVlrQvzHFFd8Hwg//Y=
 github.com/ledongthuc/pdf v0.0.0-20220302134840-0c2507a12d80 h1:6Yzfa6GP0rIo/kULo2bwGEkFvCePZ3qHDDTC3/J9Swo=
@@ -23,7 +22,8 @@ github.com/mailru/easyjson v0.7.7 h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0
 github.com/mailru/easyjson v0.7.7/go.mod h1:xzfreul335JAWq5oZzymOObrkdz5UnU4kGfJJLY9Nlc=
 github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde h1:x0TT0RDC7UhAVbbWWBzr41ElhJx5tXPWkIHA2HWPRuw=
 github.com/orisano/pixelmatch v0.0.0-20220722002657-fb0b55479cde/go.mod h1:nZgzbfBr3hhjoZnS66nKrHmduYNpc34ny7RK4z5/HM0=
-golang.org/x/sys v0.0.0-20201207223542-d4d67f95c62d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
-golang.org/x/sys v0.0.0-20220928140112-f11e5e49a4ec/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.1.0 h1:kunALQeHf1/185U1i0GOB/fy1IPRDDpuoOOqRReG57U=
-golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+github.com/soniakeys/quant v1.0.0 h1:N1um9ktjbkZVcywBVAAYpZYSHxEfJGzshHCxx/DaI0Y=
+github.com/soniakeys/quant v1.0.0/go.mod h1:HI1k023QuVbD4H8i9YdfZP2munIHU4QpjsImz6Y6zds=
+golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.15.0 h1:h48lPFYpsTvQJZF4EKyI4aLHaev3CxivZmv7yZig9pc=
+golang.org/x/sys v0.15.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
--- a/wrp.go
+++ b/wrp.go
@@ -2,7 +2,7 @@
 // WRP - Web Rendering Proxy
 //
 // Copyright (c) 2013-2018 Antoni Sawicki
-// Copyright (c) 2019-2021 Google LLC
+// Copyright (c) 2019-2024 Google LLC
 //

 package main
@@ -15,12 +15,16 @@ import (
 	"fmt"
 	"html/template"
 	"image"
+	"image/color/palette"
 	"image/gif"
+	"image/jpeg"
 	"image/png"
+	"io"
 	"io/ioutil"
 	"log"
 	"math"
 	"math/rand"
+	"net"
 	"net/http"
 	"net/url"
 	"os"
@@ -35,23 +39,31 @@ import (
 	"github.com/chromedp/cdproto/emulation"
 	"github.com/chromedp/cdproto/page"
 	"github.com/chromedp/chromedp"
-	"github.com/ericpauley/go-quantize/quantize"
+	"github.com/soniakeys/quant/median"
 )

+const version = "4.6.0"
+
 var (
-	version  = "4.5.3"
-	srv      http.Server
-	ctx      context.Context
-	cancel   context.CancelFunc
-	img      = make(map[string]bytes.Buffer)
-	ismap    = make(map[string]wrpReq)
-	noDel    bool
-	defType  string
-	defGeom  geom
-	htmlTmpl *template.Template
+	addr        = flag.String("l", ":8080", "Listen address:port, default :8080")
+	headless    = flag.Bool("h", true, "Headless mode / hide browser window (default true)")
+	noDel       = flag.Bool("n", false, "Do not free maps and images after use")
+	defType     = flag.String("t", "gif", "Image type: png|gif|jpg")
+	jpgQual     = flag.Int("q", 80, "Jpeg image quality, default 80%")
+	fgeom       = flag.String("g", "1152x600x216", "Geometry: width x height x colors, height can be 0 for unlimited")
+	htmFnam     = flag.String("ui", "wrp.html", "HTML template file for the UI")
+	delay       = flag.Duration("s", 2*time.Second, "Delay/sleep after page is rendered and before screenshot is taken")
+	userAgent   = flag.String("ua", "", "override chrome user agent")
+	srv         http.Server
+	actx, ctx   context.Context
+	acncl, cncl context.CancelFunc
+	img         = make(map[string]bytes.Buffer)
+	ismap       = make(map[string]wrpReq)
+	defGeom     geom
+	htmlTmpl    *template.Template
 )

-// go:embed *.html
+//go:embed *.html
 var fs embed.FS

 type geom struct {
@@ -129,8 +141,12 @@ func (rq *wrpReq) parseForm() {
 	rq.keys = rq.r.FormValue("k")
 	rq.buttons = rq.r.FormValue("Fn")
 	rq.imgType = rq.r.FormValue("t")
-	if rq.imgType != "gif" && rq.imgType != "png" {
-		rq.imgType = defType
+	switch rq.imgType {
+	case "png":
+	case "gif":
+	case "jpg":
+	default:
+		rq.imgType = *defType
 	}
 	log.Printf("%s WrpReq from UI Form: %+v\n", rq.r.RemoteAddr, rq)
 }
@@ -204,19 +220,26 @@ func (rq *wrpReq) action() chromedp.Action {
 	return chromedp.Navigate(rq.url)
 }

-// Process Keyboard and Mouse events or Navigate to the desired URL.
+// Navigate to the desired URL.
 func (rq *wrpReq) navigate() {
-	err := chromedp.Run(ctx, rq.action())
-	if err != nil {
-		if err.Error() == "context canceled" {
-			log.Printf("%s Contex cancelled, try again", rq.r.RemoteAddr)
-			fmt.Fprintf(rq.w, "<BR>%s<BR> -- restarting, try again", err)
-			ctx, cancel = chromedp.NewContext(context.Background())
-			return
-		}
-		log.Printf("%s %s", rq.r.RemoteAddr, err)
-		fmt.Fprintf(rq.w, "<BR>%s<BR>", err)
+	ctxErr(chromedp.Run(ctx, rq.action()), rq.w)
+}
+
+// Handle context errors
+func ctxErr(err error, w io.Writer) {
+	// TODO: callers should have retry logic, perhaps create another function
+	// that takes ...chromedp.Action and retries with give up
+	if err == nil {
+		return
 	}
+	log.Printf("Context error: %s", err)
+	fmt.Fprintf(w, "Context error: %s<BR>\n", err)
+	if err.Error() != "context canceled" {
+		return
+	}
+	ctx, cncl = chromedp.NewContext(actx)
+	log.Printf("Created new context, try again")
+	fmt.Fprintln(w, "Created new context, try again")
 }

 // https://github.com/chromedp/chromedp/issues/979
@@ -235,13 +258,51 @@ func chromedpCaptureScreenshot(res *[]byte, h int64) chromedp.Action {
 	})
 }

+func gifPalette(i image.Image, n int64) image.Image {
+	switch n {
+	case 2:
+		i = halfgone.FloydSteinbergDitherer{}.Apply(halfgone.ImageToGray(i))
+	case 216:
+		var FastGifLut = [256]int{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5}
+		r := i.Bounds()
+		// NOTE: the color index computation below works only for palette.WebSafe!
+		p := image.NewPaletted(r, palette.WebSafe)
+		if i64, ok := i.(image.RGBA64Image); ok {
+			for y := r.Min.Y; y < r.Max.Y; y++ {
+				for x := r.Min.X; x < r.Max.X; x++ {
+					c := i64.RGBA64At(x, y)
+					r6 := FastGifLut[c.R>>8]
+					g6 := FastGifLut[c.G>>8]
+					b6 := FastGifLut[c.B>>8]
+					p.SetColorIndex(x, y, uint8(36*r6+6*g6+b6))
+				}
+			}
+		} else {
+			for y := r.Min.Y; y < r.Max.Y; y++ {
+				for x := r.Min.X; x < r.Max.X; x++ {
+					c := i.At(x, y)
+					r, g, b, _ := c.RGBA()
+					r6 := FastGifLut[r&0xff]
+					g6 := FastGifLut[g&0xff]
+					b6 := FastGifLut[b&0xff]
+					p.SetColorIndex(x, y, uint8(36*r6+6*g6+b6))
+				}
+			}
+		}
+		i = p
+	default:
+		q := median.Quantizer(n)
+		i = q.Paletted(i)
+	}
+	return i
+}
+
 // Capture currently rendered web page to an image and fake ISMAP
 func (rq *wrpReq) capture() {
-	var err error
 	var styles []*css.ComputedStyleProperty
 	var r, g, b int
 	var h int64
-	var pngcap []byte
+	var pngCap []byte
 	chromedp.Run(ctx,
 		emulation.SetDeviceMetricsOverride(int64(float64(rq.width)/rq.zoom), 10, rq.zoom, false),
 		chromedp.Location(&rq.url),
@@ -266,69 +327,74 @@ func (rq *wrpReq) capture() {
 	}
 	chromedp.Run(
 		ctx, emulation.SetDeviceMetricsOverride(int64(float64(rq.width)/rq.zoom), height, rq.zoom, false),
-		chromedp.Sleep(time.Second*2), // TODO(tenox): totally lame, find a better way to determine if page is rendered
+		chromedp.Sleep(*delay), // TODO(tenox): find a better way to determine if page is rendered
 	)
 	// Capture screenshot...
-	err = chromedp.Run(ctx, chromedpCaptureScreenshot(&pngcap, rq.height))
-	if err != nil {
-		if err.Error() == "context canceled" {
-			log.Printf("%s Contex cancelled, try again", rq.r.RemoteAddr)
-			fmt.Fprintf(rq.w, "<BR>%s<BR> -- restarting, try again", err)
-			ctx, cancel = chromedp.NewContext(context.Background())
-			return
-		}
-		log.Printf("%s Failed to capture screenshot: %s\n", rq.r.RemoteAddr, err)
-		fmt.Fprintf(rq.w, "<BR>Unable to capture screenshot:<BR>%s<BR>\n", err)
-		return
-	}
+	ctxErr(chromedp.Run(ctx, chromedpCaptureScreenshot(&pngCap, rq.height)), rq.w)
 	seq := rand.Intn(9999)
-	imgpath := fmt.Sprintf("/img/%04d.%s", seq, rq.imgType)
-	mappath := fmt.Sprintf("/map/%04d.map", seq)
-	ismap[mappath] = *rq
-	var ssize string
-	var iw, ih int
+	imgPath := fmt.Sprintf("/img/%04d.%s", seq, rq.imgType)
+	mapPath := fmt.Sprintf("/map/%04d.map", seq)
+	ismap[mapPath] = *rq
+	var sSize string
+	var iW, iH int
 	switch rq.imgType {
+	case "png":
+		pngBuf := bytes.NewBuffer(pngCap)
+		img[imgPath] = *pngBuf
+		cfg, _, _ := image.DecodeConfig(pngBuf)
+		sSize = fmt.Sprintf("%.0f KB", float32(len(pngBuf.Bytes()))/1024.0)
+		iW = cfg.Width
+		iH = cfg.Height
+		log.Printf("%s Got PNG image: %s, Size: %s, Res: %dx%d\n", rq.r.RemoteAddr, imgPath, sSize, iW, iH)
 	case "gif":
-		i, err := png.Decode(bytes.NewReader(pngcap))
+		i, err := png.Decode(bytes.NewReader(pngCap))
 		if err != nil {
-			log.Printf("%s Failed to decode screenshot: %s\n", rq.r.RemoteAddr, err)
-			fmt.Fprintf(rq.w, "<BR>Unable to decode page screenshot:<BR>%s<BR>\n", err)
+			log.Printf("%s Failed to decode PNG screenshot: %s\n", rq.r.RemoteAddr, err)
+			fmt.Fprintf(rq.w, "<BR>Unable to decode page PNG screenshot:<BR>%s<BR>\n", err)
 			return
 		}
-		if rq.colors == 2 {
-			gray := halfgone.ImageToGray(i)
-			i = halfgone.FloydSteinbergDitherer{}.Apply(gray)
-		}
-		var gifbuf bytes.Buffer
 		st := time.Now()
-		err = gif.Encode(&gifbuf, i, &gif.Options{NumColors: int(rq.colors), Quantizer: quantize.MedianCutQuantizer{}})
+		var gifBuf bytes.Buffer
+		err = gif.Encode(&gifBuf, gifPalette(i, rq.colors), &gif.Options{})
 		if err != nil {
 			log.Printf("%s Failed to encode GIF: %s\n", rq.r.RemoteAddr, err)
 			fmt.Fprintf(rq.w, "<BR>Unable to encode GIF:<BR>%s<BR>\n", err)
 			return
 		}
-		img[imgpath] = gifbuf
-		ssize = fmt.Sprintf("%.0f KB", float32(len(gifbuf.Bytes()))/1024.0)
-		iw = i.Bounds().Max.X
-		ih = i.Bounds().Max.Y
-		log.Printf("%s Encoded GIF image: %s, Size: %s, Colors: %d, %dx%d, Time: %vms\n", rq.r.RemoteAddr, imgpath, ssize, rq.colors, iw, ih, time.Since(st).Milliseconds())
-	case "png":
-		pngbuf := bytes.NewBuffer(pngcap)
-		img[imgpath] = *pngbuf
-		cfg, _, _ := image.DecodeConfig(pngbuf)
-		ssize = fmt.Sprintf("%.0f KB", float32(len(pngbuf.Bytes()))/1024.0)
-		iw = cfg.Width
-		ih = cfg.Height
-		log.Printf("%s Got PNG image: %s, Size: %s, %dx%d\n", rq.r.RemoteAddr, imgpath, ssize, iw, ih)
+		img[imgPath] = gifBuf
+		sSize = fmt.Sprintf("%.0f KB", float32(len(gifBuf.Bytes()))/1024.0)
+		iW = i.Bounds().Max.X
+		iH = i.Bounds().Max.Y
+		log.Printf("%s Encoded GIF image: %s, Size: %s, Colors: %d, Res: %dx%d, Time: %vms\n", rq.r.RemoteAddr, imgPath, sSize, rq.colors, iW, iH, time.Since(st).Milliseconds())
+	case "jpg":
+		i, err := png.Decode(bytes.NewReader(pngCap))
+		if err != nil {
+			log.Printf("%s Failed to decode PNG screenshot: %s\n", rq.r.RemoteAddr, err)
+			fmt.Fprintf(rq.w, "<BR>Unable to decode page PNG screenshot:<BR>%s<BR>\n", err)
+			return
+		}
+		st := time.Now()
+		var jpgBuf bytes.Buffer
+		err = jpeg.Encode(&jpgBuf, i, &jpeg.Options{Quality: *jpgQual})
+		if err != nil {
+			log.Printf("%s Failed to encode JPG: %s\n", rq.r.RemoteAddr, err)
+			fmt.Fprintf(rq.w, "<BR>Unable to encode JPG:<BR>%s<BR>\n", err)
+			return
+		}
+		img[imgPath] = jpgBuf
+		sSize = fmt.Sprintf("%.0f KB", float32(len(jpgBuf.Bytes()))/1024.0)
+		iW = i.Bounds().Max.X
+		iH = i.Bounds().Max.Y
+		log.Printf("%s Encoded JPG image: %s, Size: %s, Quality: %d, Res: %dx%d, Time: %vms\n", rq.r.RemoteAddr, imgPath, sSize, *jpgQual, iW, iH, time.Since(st).Milliseconds())
 	}
 	rq.printHTML(printParams{
 		bgColor:    fmt.Sprintf("#%02X%02X%02X", r, g, b),
 		pageHeight: fmt.Sprintf("%d PX", h),
-		imgSize:    ssize,
-		imgURL:     imgpath,
-		mapURL:     mappath,
-		imgWidth:   iw,
-		imgHeight:  ih,
+		imgSize:    sSize,
+		imgURL:     imgPath,
+		mapURL:     mapPath,
+		imgWidth:   iW,
+		imgHeight:  iH,
 	})
 	log.Printf("%s Done with capture for %s\n", rq.r.RemoteAddr, rq.url)
 }
@@ -345,7 +411,7 @@ func pageServer(w http.ResponseWriter, r *http.Request) {
 		rq.printHTML(printParams{bgColor: "#FFFFFF"})
 		return
 	}
-	rq.navigate()
+	rq.navigate() // TODO: if error from navigate do not capture
 	rq.capture()
 }

@@ -360,7 +426,7 @@ func mapServer(w http.ResponseWriter, r *http.Request) {
 		log.Printf("Unable to find map %s\n", r.URL.Path)
 		return
 	}
-	if !noDel {
+	if !*noDel {
 		defer delete(ismap, r.URL.Path)
 	}
 	n, err := fmt.Sscanf(r.URL.RawQuery, "%d,%d", &rq.mouseX, &rq.mouseY)
@@ -374,20 +440,20 @@ func mapServer(w http.ResponseWriter, r *http.Request) {
 		rq.printHTML(printParams{bgColor: "#FFFFFF"})
 		return
 	}
-	rq.navigate()
+	rq.navigate() // TODO: if error from navigate do not capture
 	rq.capture()
 }

 // Process HTTP requests for images '/img/' url
 func imgServer(w http.ResponseWriter, r *http.Request) {
 	log.Printf("%s IMG Request for %s\n", r.RemoteAddr, r.URL.Path)
-	imgbuf, ok := img[r.URL.Path]
-	if !ok || imgbuf.Bytes() == nil {
+	imgBuf, ok := img[r.URL.Path]
+	if !ok || imgBuf.Bytes() == nil {
 		fmt.Fprintf(w, "Unable to find image %s\n", r.URL.Path)
 		log.Printf("%s Unable to find image %s\n", r.RemoteAddr, r.URL.Path)
 		return
 	}
-	if !noDel {
+	if !*noDel {
 		defer delete(img, r.URL.Path)
 	}
 	switch {
@@ -395,12 +461,14 @@ func imgServer(w http.ResponseWriter, r *http.Request) {
 		w.Header().Set("Content-Type", "image/gif")
 	case strings.HasPrefix(r.URL.Path, ".png"):
 		w.Header().Set("Content-Type", "image/png")
+	case strings.HasPrefix(r.URL.Path, ".jpg"):
+		w.Header().Set("Content-Type", "image/jpeg")
 	}
-	w.Header().Set("Content-Length", strconv.Itoa(len(imgbuf.Bytes())))
+	w.Header().Set("Content-Length", strconv.Itoa(len(imgBuf.Bytes())))
 	w.Header().Set("Cache-Control", "max-age=0")
 	w.Header().Set("Expires", "-1")
 	w.Header().Set("Pragma", "no-cache")
-	w.Write(imgbuf.Bytes())
+	w.Write(imgBuf.Bytes())
 	w.(http.Flusher).Flush()
 }

@@ -414,7 +482,8 @@ func haltServer(w http.ResponseWriter, r *http.Request) {
 	fmt.Fprintf(w, "Shutting down WRP...\n")
 	w.(http.Flusher).Flush()
 	time.Sleep(time.Second * 2)
-	cancel()
+	cncl()
+	acncl()
 	srv.Shutdown(context.Background())
 	os.Exit(1)
 }
@@ -426,62 +495,84 @@ func tmpl(t string) string {
 	if err != nil {
 		goto builtin
 	}
+	defer fh.Close()
+
 	tmpl, err = ioutil.ReadAll(fh)
 	if err != nil {
 		goto builtin
 	}
-	log.Printf("Got UI template from %v file\n", t)
+	log.Printf("Got HTML UI template from %v file, size %v \n", t, len(tmpl))
 	return string(tmpl)

 builtin:
-	fhs, err := fs.Open("/wrp.html")
+	fhs, err := fs.Open("wrp.html")
 	if err != nil {
 		log.Fatal(err)
 	}
+	defer fhs.Close()

 	tmpl, err = ioutil.ReadAll(fhs)
 	if err != nil {
 		log.Fatal(err)
 	}
-	log.Printf("Got UI template from built-in\n")
+	log.Printf("Got HTML UI template from embed\n")
 	return string(tmpl)
 }

-// Main...
-func main() {
-	var addr, fgeom, tHTML string
-	var headless bool
-	var debug bool
-	var err error
-	flag.StringVar(&addr, "l", ":8080", "Listen address:port, default :8080")
-	flag.BoolVar(&headless, "h", true, "Headless mode - hide browser window")
-	flag.BoolVar(&debug, "d", false, "Debug ChromeDP")
-	flag.BoolVar(&noDel, "n", false, "Do not free maps and images after use")
-	flag.StringVar(&defType, "t", "gif", "Image type: gif|png")
-	flag.StringVar(&fgeom, "g", "1152x600x256", "Geometry: width x height x colors, height can be 0 for unlimited")
-	flag.StringVar(&tHTML, "ui", "wrp.html", "HTML template file for the UI")
-	flag.Parse()
-	if len(os.Getenv("PORT")) > 0 {
-		addr = ":" + os.Getenv(("PORT"))
+// Print my own IP addresses
+func printIPs(b string) {
+	ap := strings.Split(b, ":")
+	if len(ap) < 1 {
+		log.Fatal("Wrong format of ipaddress:port")
 	}
-	n, err := fmt.Sscanf(fgeom, "%dx%dx%d", &defGeom.w, &defGeom.h, &defGeom.c)
+	log.Printf("Listen address: %v", b)
+	if ap[0] != "" && ap[0] != "0.0.0.0" {
+		return
+	}
+	a, err := net.InterfaceAddrs()
+	if err != nil {
+		log.Print("Unable to get interfaces: ", err)
+		return
+	}
+	var m string
+	for _, i := range a {
+		n, ok := i.(*net.IPNet)
+		if !ok || n.IP.IsLoopback() || strings.Contains(n.IP.String(), ":") {
+			continue
+		}
+		m = m + n.IP.String() + " "
+	}
+	log.Print("My IP addresses: ", m)
+}
+
+// Main
+func main() {
+	var err error
+	flag.Parse()
+	log.Printf("Web Rendering Proxy Version %s\n", version)
+	log.Printf("Args: %q", os.Args)
+	if len(os.Getenv("PORT")) > 0 {
+		*addr = ":" + os.Getenv(("PORT"))
+	}
+	printIPs(*addr)
+	n, err := fmt.Sscanf(*fgeom, "%dx%dx%d", &defGeom.w, &defGeom.h, &defGeom.c)
 	if err != nil || n != 3 {
 		log.Fatalf("Unable to parse -g geometry flag / %s", err)
 	}

 	opts := append(chromedp.DefaultExecAllocatorOptions[:],
-		chromedp.Flag("headless", headless),
+		chromedp.Flag("headless", *headless),
 		chromedp.Flag("hide-scrollbars", false),
+		chromedp.Flag("enable-automation", false),
+		chromedp.Flag("disable-blink-features", "AutomationControlled"),
 	)
-	actx, acancel := chromedp.NewExecAllocator(context.Background(), opts...)
-	defer acancel()
-	switch debug {
-	case true:
-		ctx, cancel = chromedp.NewContext(actx, chromedp.WithDebugf(log.Printf))
-	default:
-		ctx, cancel = chromedp.NewContext(actx)
+	if *userAgent != "" {
+		opts = append(opts, chromedp.UserAgent(*userAgent))
 	}
-	defer cancel()
+	actx, acncl = chromedp.NewExecAllocator(context.Background(), opts...)
+	defer acncl()
+	ctx, cncl = chromedp.NewContext(actx)
+	defer cncl()

 	rand.Seed(time.Now().UnixNano())

@@ -490,7 +581,8 @@ func main() {
 	go func() {
 		<-c
 		log.Printf("Interrupt - shutting down.")
-		cancel()
+		cncl()
+		acncl()
 		srv.Shutdown(context.Background())
 		os.Exit(1)
 	}()
@@ -501,17 +593,15 @@ func main() {
 	http.HandleFunc("/shutdown/", haltServer)
 	http.HandleFunc("/favicon.ico", http.NotFound)

-	log.Printf("Web Rendering Proxy Version %s\n", version)
-	log.Printf("Args: %q", os.Args)
-	log.Printf("Default Img Type: %v, Geometry: %+v", defType, defGeom)
+	log.Printf("Default Img Type: %v, Geometry: %+v", *defType, defGeom)

-	htmlTmpl, err = template.New("wrp.html").Parse(tmpl(tHTML))
+	htmlTmpl, err = template.New("wrp.html").Parse(tmpl(*htmFnam))
 	if err != nil {
 		log.Fatal(err)
 	}

-	log.Printf("Starting WRP http server on %s\n", addr)
-	srv.Addr = addr
+	log.Print("Starting WRP http server")
+	srv.Addr = *addr
 	err = srv.ListenAndServe()
 	if err != nil {
 		log.Fatal(err)
--- a/wrp.html
+++ b/wrp.html
@@ -21,11 +21,13 @@
                <OPTION VALUE="1.3" {{ if eq .Zoom 1.3}}SELECTED{{end}}>1.3 x</OPTION>
            </SELECT>
            T <SELECT NAME="t">
-                <OPTION VALUE="gif" {{ if eq .ImgType "gif"}}SELECTED{{end}}>GIF</OPTION>
                <OPTION VALUE="png" {{ if eq .ImgType "png"}}SELECTED{{end}}>PNG</OPTION>
+                <OPTION VALUE="gif" {{ if eq .ImgType "gif"}}SELECTED{{end}}>GIF</OPTION>
+                <OPTION VALUE="jpg" {{ if eq .ImgType "jpg"}}SELECTED{{end}}>JPG</OPTION>
            </SELECT>
            C <SELECT NAME="c">
                <OPTION VALUE="256" {{ if eq .NColors 256}}SELECTED{{end}}>256</OPTION>
+                <OPTION VALUE="216" {{ if eq .NColors 216}}SELECTED{{end}}>216</OPTION>
                <OPTION VALUE="128" {{ if eq .NColors 128}}SELECTED{{end}}>128</OPTION>
                <OPTION VALUE="64" {{ if eq .NColors 64}}SELECTED{{end}}>64</OPTION>
                <OPTION VALUE="16" {{ if eq .NColors 16}}SELECTED{{end}}>16</OPTION>
Author	SHA1	Message	Date
Antoni Sawicki	7d9cec6297	add some anti crawler detection	2024-01-02 21:55:39 -08:00
Antoni Sawicki	43df501d97	add useragent override flag	2024-01-02 21:54:59 -08:00
Antoni Sawicki	85d64a3577	copyright update	2024-01-02 02:09:38 -08:00
Antoni Sawicki	993e5723ed	more mod updates	2024-01-02 02:08:14 -08:00
Antoni Sawicki	9056c798d6	go mod update	2024-01-02 02:00:30 -08:00
Antoni Sawicki	8fd65b3e47	readme update	2022-12-10 03:59:30 -08:00
Antoni Sawicki	8716a983ce	readme update	2022-12-10 03:42:13 -08:00
Antoni Sawicki	4757cfd32b	readme update	2022-12-10 03:38:56 -08:00
Antoni Sawicki	79e6f3a17e	readme update	2022-12-10 03:38:22 -08:00
Antoni Sawicki	d8f5c6fb28	print my own IP addresses	2022-12-08 01:49:46 -08:00
Antoni Sawicki	c816ef712a	readme update	2022-12-08 01:03:04 -08:00
Antoni Sawicki	d28924583c	move png to first case	2022-12-08 00:56:24 -08:00
Antoni Sawicki	1c17b39ea5	add jpeg encoding	2022-12-08 00:55:56 -08:00
Antoni Sawicki	ecb2cc0c06	add todos for error handling	2022-12-08 00:28:28 -08:00
Antoni Sawicki	b571df7a37	single function to handle context errors	2022-12-08 00:18:13 -08:00
Antoni Sawicki	78c568ac09	remove debug flag, never used	2022-12-07 23:54:18 -08:00
Antoni Sawicki	04a755749e	close template files	2022-12-07 23:50:58 -08:00
Antoni Sawicki	e983f244f8	move flags top top var()	2022-12-07 23:44:09 -08:00
Antoni Sawicki	9b76c045d6	fix html template embed	2022-11-30 02:00:53 -08:00
Antoni Sawicki	c8e391274a	update dependencies	2022-11-30 00:02:59 -08:00
Antoni Sawicki	7b1274b9d4	ver bump	2022-11-30 00:01:01 -08:00
Antoni Sawicki	0128b3ff8e	add delay/sleep flag	2022-11-29 23:59:43 -08:00
Antoni Sawicki	6de3fad580	Merge pull request #101 from tenox7/fastgif Fastgif	2022-11-29 23:45:53 -08:00
Antoni Sawicki	d7dcb58adc	readme update	2022-11-29 23:45:01 -08:00
Antoni Sawicki	e5f83225f7	update readme	2022-11-29 07:02:06 -08:00
Antoni Sawicki	36803c4312	update readme	2022-11-29 06:57:31 -08:00
Antoni Sawicki	40e081be77	refactor fastgif code for readability	2022-11-29 05:50:17 -08:00
Antoni Sawicki	1c18fb9b81	Merge pull request #99 from mahiuchun/master Introduce "fastgif" image type	2022-11-29 04:58:01 -08:00
Antoni Sawicki	363fbcd225	remove 32bit windows build	2022-11-24 02:11:05 -08:00
Hill Ma	bf7e7bfb2c	improve fastgif	2022-11-10 20:39:41 -08:00
Hill Ma	fec812bc32	Add "fastgif" image type that provides much faster (but lower quality) GIF quantization. The default image type is changed to PNG as it is the most "trouble free" choice.	2022-11-09 21:32:46 -08:00
Hill Ma	a238a0ea6f	gif: faster median cut quantization	2022-11-09 20:48:18 -08:00
Antoni Sawicki	444b7b31d7	update copyrights	2022-11-06 01:41:40 -08:00