POST Requests
(BETA) Now it is possible to scrape forms or API endpoints directly. You can do this by sending a POST request to Shifter's Web Scraping API with your api_key and URL parameter.
In the right-side box, you will find an example request used to scrape the URL https://httpbin.org/post.
POST Requests examples
POST https://scrape.shifter.io/v1?api_key=api_key&url=https://httpbin.org/post
⇡ Input
curl --request POST --url "https://scrape.shifter.io/v1?api_key=api_key&url=https%3A%2F%2Fhttpbin.org%2Fpost" --data "{
"foo": "bar"
}"
⇣ Output
{
"args": {},
"data": "{\"foo\":\"bar\"}",
"files": {},
"form": {},
"headers": {
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
"Accept-Encoding": "gzip, deflate, br",
"Content-Length": "13",
"Content-Type": "application/json",
"Host": "httpbin.org",
"Upgrade-Insecure-Requests": "1",
"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36",
"X-Amzn-Trace-Id": "Root=1-6267eb77-3bbe26043611b895363f145a"
},
"json": {
"foo": "bar"
},
"origin": "185.162.168.36",
"url": "https://httpbin.org/post"
}
Last updated
Was this helpful?