使用Python爬虫库requests发送请求、传递URL参数、定制headers

使用Python爬虫库requests发送请求、传递URL参数、定制headers

首先我们先引入requests模块

import requests

一、发送请求

r = requests.get("https://api.github.com/events") # GET请求
r = requests.post("http://httpbin.org/post", data = {"key":"value"}) # POST请求
r = requests.put("http://httpbin.org/put", data = {"key":"value"}) # PUT请求
r = requests.delete("http://httpbin.org/delete") # DELETE请求
r = requests.head("http://httpbin.org/get") # HEAD请求
r = requests.options("http://httpbin.org/get") # OPTIONS请求
type(r)
requests.models.Response

二、传递URL参数

URL传递参数的形式为:httpbin.org/get?key=val。但是手动的构造很麻烦,这是可以使用params参数来方便的构造带参数URL。

payload = {"key1": "value1", "key2": "value2"}
r = requests.get("http://httpbin.org/get", params=payload)
print(r.url)
http://httpbin.org/get?key1=value1&key2=value2

同一个key可以有多个value

payload = {"key1": "value1", "key2": ["value2", "value3"]}
r = requests.get("http://httpbin.org/get", params=payload)
print(r.url)
http://httpbin.org/get?key1=value1&key2=value2&key2=value3

三、定制headers

只需要将一个dict传递给headers参数便可以定制headers

url = "https://api.github.com/some/endpoint"
headers = {"user-agent": "my-app/0.0.1"}
r = requests.get(url, headers=headers)

更多关于Python爬虫库requests的使用方法请点击下面的相关链接