Custom Headers

If you would like to add custom/additional headers (user agents, cookies, etc.), simply add them as parameters to the request, with the “wsa-” prefix.

Only use this feature to get customized results, do not use this feature to avoid blocks. Shifter's Scraping API has a powerful internal engine that takes care of everything for you.

On this page, you will find an example request used to scrape the URL, which will mirror the headers sent.

Custom Headers example


⇡ Input

curl --request GET --url "" --header "Wsa-My-header: test" --header "Wsa-User-Agent: potato"

⇣ Output

    "headers": {
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
        "Accept-Encoding": "gzip, deflate, br",
        "Host": "",
        "My-Header": "test",
        "Upgrade-Insecure-Requests": "1",
        "User-Agent": "potato",
        "X-Amzn-Trace-Id": "Root=1-6267c064-44fa9993017cfcc32e41dfc1"

Last updated