目錄項(xiàng)目地址：環(huán)境配置及使用結(jié)果展示完整代碼項(xiàng)目地址：

https://github.com/chen0495/pythonCrawlerForJSU

環(huán)境 python 3.5即以上 request、BeautifulSoup、numpy、pandas. 安裝BeautifulSoup使用命令pip install BeautifulSoup4配置及使用

登陸學(xué)校成績單查詢網(wǎng)站,修改cookie.

按F12后按Ctrl+R刷新一下,獲取cookie的方法見下圖:

修改爬蟲url為自己的成績單網(wǎng)址.

運(yùn)行src/main.py文件即可在/result下得到csv文件.

結(jié)果展示

完整代碼

# -*- coding: utf-8 -*-# @Time : 5/29/2021 2:13 PM# @Author : Chen0495# @Email : 1346565673@qq.com|chenweiin612@gmail.com# @File : main.py# @Software: PyCharmimport requests as rqfrom bs4 import BeautifulSoup as BSimport numpy as npimport pandas as pdrq.adapters.DEFAULT_RETRIES = 5s = rq.session()s.keep_alive = False # 關(guān)閉多余連接header = { # 請(qǐng)更改cookie ’user-agent’ : ’Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4501.0 Safari/537.36 Edg/92.0.891.1’, ’cookie’ : ’wengine_vpn_ticketwebvpn_jsu_edu_cn=xxxxxxxxxx; show_vpn=1; refresh=1’}# 請(qǐng)更改urlr = rq.get(’https://webvpn.jsu.edu.cn/https/xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/jsxsd/kscj/cjcx_list’, headers = header, verify=False)soup = BS(r.text,’html.parser’)head = []for th in soup.find_all('th'): head.append(th.text)while ’’ in head: head.remove(’’)head.remove(’序號(hào)’)context = np.array(head)x = []flag = 0for td in soup.find_all('td'): if flag!=0 and flag%11!=1:x.append(td.text) if flag%11==0 and flag!=0:context = np.row_stack((context,np.array(x)))x.clear() flag+=1context = np.delete(context,0,axis=0)data = pd.DataFrame(context,columns=head)print(data)# 生成文件,親更改文件名data.to_csv(’../result/result.csv’,encoding=’utf-8-sig’)

以上就是python 爬取吉首大學(xué)成績單的詳細(xì)內(nèi)容，更多關(guān)于python 爬取成績單的資料請(qǐng)關(guān)注好吧啦網(wǎng)其它相關(guān)文章！