Skip to main content

API Reference

Base URL

https://api.scrapingpros.com/v1

Authentication

All endpoints require the authorization header:

Authorization: Bearer <API-KEY>

Available Endpoints

Synchronous Operations

MethodEndpointDescription
POST/v1/sync/scrapeScrape a URL and get the result immediately
POST/v1/sync/downloadDownload a file from a URL and get its content in base64
GET/v1/sync/metricsGet global API metrics
GET/v1/sync/client-metricsGet per-client usage metrics
GET/v1/sync/billingGet monthly billing summary per client

Proxy by Country

MethodEndpointDescription
GET/v1/proxy/countriesList available countries for geographic proxies
POST/v1/proxy/request-countryRequest access to proxies from a country (requires admin approval)
GET/v1/proxy/statusView approved and pending countries for your account

Monitoring

MethodEndpointDescription
GET/v1/healthHealth check of all components (no authentication required)

Asynchronous Operations

MethodEndpointDescription
POST/v1/async/collectionsCreate a request collection
GET/v1/async/collectionsList all collections
GET/v1/async/collections/{id}Get a specific collection
PUT/v1/async/collections/{id}Update a collection
POST/v1/async/collections/{id}/runExecute a collection
GET/v1/async/collections/{id}/runs/{run_id}Query the status of an execution

Response Codes

CodeDescription
200Successful request
400Invalid request (incorrect parameters)
401Unauthorized (invalid or missing API key)
404Resource not found
500Internal server error
503Service unavailable (retry in a few seconds)

Error Handling

Most errors return a JSON with the detail field:

{
"detail": "Error description"
}

Errors by endpoint

400 -- Invalid Request

Occurs when the submitted parameters are not valid.

{
"detail": "There is a problem with the collection. Please check the requests. <reason>"
}

401 -- Unauthorized

Missing or invalid API key.

{
"detail": "Unauthorized"
}

404 -- Resource Not Found

SituationMessage
Non-existent collection"Collection not found."
Collection without requests when attempting to execute"Collection not exists or Collection does not requests associated."
Non-existent run"Run not found."
Run does not belong to the collection"Run not found for the given collection."
{
"detail": "Collection not found."
}

500 -- Internal Error

The scraping job failed or an unexpected error occurred. In the /v1/sync/scrape endpoint the error comes inside the response body (not as HTTP 500):

{
"html": null,
"statusCode": 500,
"message": "The job wasn't succesfully completed due to an unknown error.",
"executionTime": null,
"screenshot": null,
"extracted_data": null
}

In the /v1/sync/download endpoint the error does return HTTP 500:

{
"content": null,
"contentType": null,
"statusCode": 500,
"message": "error description",
"executionTime": null
}

503 -- Service Unavailable

Redis is not available. Retry the request in a few seconds.

{
"detail": "Service unavailable, please try again later."
}