使用urllib发送post请求-模拟网站买票查票的post

python 3.5.1

我装的最新的python3.6.1

python

urllib

from urllib.request import urlopen

beautifulSoup4

from bs4 import BeautifulSoup

安装BeautifulSoup4

linux:

sudo apt-get install python-bs4

mac:

sudo easy_install pip

pip install beautifulsoup4

windows:

pip install beautifulsoup4

pip3 install beautifulsoup4

3

3.1 urllib基本用法

urllib是python3.x中提供的一系列操作url的库，可以轻松的模拟用户使用浏览

器访问网页

模拟真实浏览器：

携带User-Agent头

req= request.Request(url)

req.add_header(key,value)

resp = reuqest.urlopen(req)

print(resp.read().decode(“utf-8”))

使用Post:

导入urllib库下面的parse

from urllib import parse

使用urlencode生成post数据

postData=parse.urlencode([

(key1,val1),

(key2,val2),

(keyn,valn)

])

使用postData发送post请求

request.urlopen(req,data=postData.encode(‘utf-8’))

得到请求状态

resp.status

得到服务器的类型

resp.reason

www.thsrc.com.tw/tw/TimeTable/SearchResult

模拟post查票

posttest.py

from urllib import  request
from urllib import parse
postData = parse.urlencode([
("StartStation", "977abb69-413a-4ccf-a109-0272c24fd490"),
("EndStation", "9c5ac6ca-ec89-48f8-aab0-41b738cb1814"),
("SearchDate", "2017/04/08"),
("SearchTime", "08:00"),
("SearchWay", "DepartureInMandarin")
])

req = request.Request("http://www.thsrc.com.tw/tw/TimeTable/SearchResult")
req.add_header("Origin","http://www.thsrc.com.tw")
req.add_header("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36")
rep = request.urlopen(req,data=postData.encode('utf-8'))
print(rep.read().decode("utf-8"))

urltest.py

from urllib import request
req = request.Request("https://www.baidu.com/")
req.add_header("User-Agent","Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36")
rep = request.urlopen(req)
print(rep.read().decode("utf-8"))