Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,6 @@
.idea/
.vscode/
tmp/

# Local secrets
.env
36 changes: 35 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,23 @@ serpapi search engine=google q=coffee --all-pages --max-pages 3 --jq '.organic_r

`async=true` returns at once with `search_metadata.id`; collect the results with `serpapi archive`.

#### Image Search (Google Lens)

- `--image <file>` — Upload an image with the [Image API](https://serpapi.com/image-api) and search with its `image_id` in one step. Use `-` to read the image from stdin. Only `engine=google_lens` takes an `image_id`, so other engines are refused before the upload; to pass an upload to another engine, use [`serpapi image`](#image) and `image_id=<id>`.

```bash
# Reverse image search with Google Lens
serpapi search engine=google_lens --image ./photo.jpg

# Pipe an image in from another command
curl -s https://example.com/photo.jpg | serpapi search engine=google_lens --image -

# Keep only the visual matches
serpapi search engine=google_lens --image ./photo.jpg --jq ".visual_matches[:5]"
```

Supported formats are JPG/JPEG, PNG and WebP up to 500 KB. See the [Google Lens image upload docs](https://serpapi.com/google-lens-upload-an-image).

### archive

Fetches an earlier search by its `search_metadata.id`, for free. A search made with `async=true` has `search_metadata.status` `Queued` or `Processing` until it is `Success`, so poll. `output=` takes the same values as in search.
Expand All @@ -93,6 +110,23 @@ Shows the plan, usage and searches left, for free. The API key in the output is
serpapi account --jq .total_searches_left
```

### image

Upload an image with the [Image API](https://serpapi.com/image-api) and print its `image_id` without searching. Useful when you want to reuse the same upload across several searches; image IDs expire after 10 minutes. For a one-off search, prefer `serpapi search --image <file>`.

```bash
# Upload an image file
serpapi image ./photo.jpg

# Read the image from stdin
curl -s https://example.com/photo.jpg | serpapi image -

# Reuse the image_id across searches
IMAGE_ID=$(serpapi image ./photo.jpg --jq .image_id)
serpapi search engine=google_lens image_id=$IMAGE_ID
serpapi search engine=google_lens image_id=$IMAGE_ID type=products
```

### login

Checks an API key with a free account request, then saves it. The key comes from `--api-key`, a hidden prompt on a terminal, or else the first line of stdin, which `login` waits for.
Expand Down Expand Up @@ -184,7 +218,7 @@ A config file that can't be read or isn't valid TOML is reported by path and lin
| 130 | `interrupted` | Ctrl-C or SIGTERM stopped it before it finished, after printing what had arrived, such as the `--all-pages` pages fetched so far. |

- **Cost**: each successful search counts against your monthly searches, including each `--all-pages` page; cached, errored and failed ones don't, though errored ones count toward the hourly limit ([pricing FAQ](https://serpapi.com/pricing)). An identical search, `--fields` included, repeated within an hour comes from the cache. `no_cache=true` forces a new, counted search; cached results are at most an hour old, so add it only when that is too old. `zero_trace=true` (Enterprise) stores nothing, so it skips the cache too and leaves no search for `archive` to read. `account`, `locations` and `archive` are free.
- **No prompts without a terminal.** Only `serpapi login` reads stdin, and it waits for the first line, so pipe the key in (`printf %s "$KEY" | serpapi login`) or, simpler, set `SERPAPI_KEY`.
- **No prompts without a terminal.** Only `serpapi login` and an image given as `-` (`serpapi image -`, `search --image -`) read stdin. `login` waits for the first line, so pipe the key in (`printf %s "$KEY" | serpapi login`) or, simpler, set `SERPAPI_KEY`; an image is read until stdin closes.

## Links

Expand Down
134 changes: 134 additions & 0 deletions pkg/api/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,14 @@ import (
"errors"
"fmt"
"io"
"mime/multipart"
"net"
"net/http"
"net/http/httptrace"
"net/textproto"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"time"
Expand All @@ -32,6 +36,10 @@ const (
// maxResponseBytes caps the response body; a variable so tests can lower it.
var maxResponseBytes = 100 << 20

// quoteEscaper mirrors the escaping mime/multipart applies to form-data
// header values.
var quoteEscaper = strings.NewReplacer("\\", "\\\\", `"`, "\\\"")

// Client is an HTTP client for the SerpApi service.
type Client struct {
apiKey string
Expand Down Expand Up @@ -112,6 +120,52 @@ func (c *Client) getTraced(ctx context.Context, endpoint string, params map[stri
if err != nil {
return nil, &clierrors.NetworkError{Message: "Invalid request: " + endpoint, Cause: err}
}
return c.do(req, endpoint, tr)
}

// doPostMultipart sends a multipart/form-data POST request. params are sent as
// form fields alongside a single file part named fieldName.
func (c *Client) doPostMultipart(ctx context.Context, endpoint string, params map[string]string, fieldName, fileName string, content []byte, tr *tracer) ([]byte, error) {
var body bytes.Buffer
w := multipart.NewWriter(&body)
for k, v := range params {
if err := w.WriteField(k, v); err != nil {
return nil, &clierrors.NetworkError{Message: "Failed to encode request: " + err.Error(), Cause: err}
}
}

// CreateFormFile hardcodes application/octet-stream, so build the part
// header by hand to send the sniffed image content type.
h := make(textproto.MIMEHeader)
h.Set("Content-Disposition", fmt.Sprintf(`form-data; name="%s"; filename="%s"`,
quoteEscaper.Replace(fieldName), quoteEscaper.Replace(fileName)))
h.Set("Content-Type", http.DetectContentType(content))
part, err := w.CreatePart(h)
if err != nil {
return nil, &clierrors.NetworkError{Message: "Failed to encode request: " + err.Error(), Cause: err}
}
if _, err := part.Write(content); err != nil {
return nil, &clierrors.NetworkError{Message: "Failed to encode request: " + err.Error(), Cause: err}
}
if err := w.Close(); err != nil {
return nil, &clierrors.NetworkError{Message: "Failed to encode request: " + err.Error(), Cause: err}
}

if tr != nil {
ctx = httptrace.WithClientTrace(ctx, tr.clientTrace())
}

req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+endpoint, &body)
if err != nil {
return nil, &clierrors.NetworkError{Message: "Invalid request: " + endpoint, Cause: err}
}
req.Header.Set("Content-Type", w.FormDataContentType())
return c.do(req, endpoint, tr)
}

// do sends req and returns the body of a 200 response verbatim, logging each
// step to tr, which may be nil.
func (c *Client) do(req *http.Request, endpoint string, tr *tracer) ([]byte, error) {
req.Header.Set("User-Agent", userAgent())
tr.logRequest(req, c.http.Timeout)

Expand Down Expand Up @@ -156,6 +210,14 @@ func (c *Client) networkError(endpoint string, err error) error {
if c.debug == nil {
debugHint = "; --debug shows which step stalled"
}
// An upload has no engine, search or archive entry to advise on.
if endpoint == "/image" {
return &clierrors.NetworkError{
Message: fmt.Sprintf("Request to %s timed out after %s: retry with a higher --timeout, such as --timeout %d, or 0 to wait indefinitely%s",
endpoint, seconds(c.http.Timeout), suggestedTimeout(c.http.Timeout), debugHint),
Cause: cause,
}
}
return &clierrors.NetworkError{
Message: fmt.Sprintf("Request to %s timed out after %s. Slow engines, such as google_ai_mode, and no_cache=true can take longer: "+
"retry with a higher --timeout, such as --timeout %d, or 0 to wait indefinitely%s. "+
Expand Down Expand Up @@ -293,6 +355,78 @@ func (c *Client) Locations(ctx context.Context, params map[string]string) (json.
return body, nil
}

// UploadImage uploads an image file using the Image API. The returned payload
// contains an image_id usable with engines that accept uploaded images, such
// as google_lens. Image IDs expire after 10 minutes.
func (c *Client) UploadImage(ctx context.Context, path string) (json.RawMessage, error) {
f, err := os.Open(path)
if err != nil {
return nil, &clierrors.UsageError{Message: "Cannot read image file: " + err.Error()}
}
defer f.Close()
content, err := ReadImage(f)
if err != nil {
return nil, &clierrors.UsageError{Message: "Cannot read image file: " + err.Error()}
}
return c.UploadImageBytes(ctx, content, filepath.Base(path))
}

// MaxImageSize is the Image API's 500 KB limit, read as 500 KiB so the CLI
// never refuses an image the API would take.
const MaxImageSize = 500 << 10

// ReadImage reads an image to upload, stopping one byte past MaxImageSize:
// UploadImageBytes refuses a larger one, which needn't be read whole.
func ReadImage(r io.Reader) ([]byte, error) {
return io.ReadAll(io.LimitReader(r, MaxImageSize+1))
}

// UploadImageBytes uploads an in-memory image using the Image API. fileName
// is sent as the multipart filename; when empty, one is derived from the
// detected content type.
func (c *Client) UploadImageBytes(ctx context.Context, content []byte, fileName string) (json.RawMessage, error) {
switch {
case len(content) == 0:
return nil, &clierrors.UsageError{Message: "The image is empty"}
case len(content) > MaxImageSize:
return nil, &clierrors.UsageError{Message: "The image is over 500 KB, the Image API's limit; resize or compress it"}
}
if fileName == "" {
fileName = defaultImageFileName(content)
}

params := map[string]string{}
if c.apiKey != "" {
params["api_key"] = c.apiKey
}

body, err := c.doPostMultipart(ctx, "/image", params, "image", fileName, content, c.newTracer())
if err != nil {
return nil, err
}
if !json.Valid(body) {
return nil, &clierrors.APIError{Message: "Unexpected non-JSON response from server"}
}
if msg := errorMessage(body); msg != "" {
return nil, &clierrors.APIError{Message: msg, Status: http.StatusOK}
}
return json.RawMessage(body), nil
}

// defaultImageFileName picks a filename with an extension matching the
// sniffed image type, for uploads that have no natural filename (e.g. stdin).
func defaultImageFileName(content []byte) string {
switch http.DetectContentType(content) {
case "image/jpeg":
return "image.jpg"
case "image/png":
return "image.png"
case "image/webp":
return "image.webp"
}
return "image"
}

// Archive retrieves a previous search by ID.
func (c *Client) Archive(ctx context.Context, id string) (json.RawMessage, error) {
return c.getObject(ctx, archivePath(id), map[string]string{"api_key": c.apiKey})
Expand Down
Loading
Loading