Python爬取全球疫情数据,制作数据可视化图

Python爬取全球疫情数据,制作数据可视化图

前言

开发环境

  • python 3.8: 解释器
  • pycharm: 代码编辑器
  • requests 发送请求
  • pyecharts 绘制图表
  • pandas 读取数据

爬虫案例思路流程:

一. 数据来源分析:

    1. 确定需求, 采集那个网站上面什么数据
    1. 抓包分析, 通过开发者工具进行抓包分析
      <浏览器自带工具>开发者工具
      F12 刷新网页 清空数据, 点击选择
      找相关数据包 --> 请求url 请求方式 得到数据是什么样

二. 代码实现步骤过程:

    1. 发送请求, 对于刚刚分析得到url地址发送请求
    1. 获取数据, 获取服务器返回响应数据 --> 开发者工具 response
    1. 解析数据, 提取我们想要数据内容
    1. 保存数据

对于本篇文章有疑问的同学可以加【资料白嫖、解答交流群:753182387】

获取疫情数据代码

导入所需模块

import requests     # 第三方 安装
import csv

获取数据

url = "https://api.inews.qq.com/newsqa/v1/automation/modules/list?modules=FAutoCountryConfirmAdd,WomWorld,WomAboard"
response = requests.post(url)
json_data = response.json()

解析数据

WomAboard = json_data["data"]["WomAboard"]
for wom in WomAboard:
    name = wom["name"]          # 国家名称
    confirm = wom["confirm"]        # 确诊人数
    confirmAdd = wom["confirmAdd"]        # 新增人数
    dead = wom["dead"]        # 死亡人数
    heal = wom["heal"]        # 治愈人数
    nowConfirm = wom["nowConfirm"]        # 现有确诊
    print(name, confirm, confirmAdd, dead, heal, nowConfirm)

保存数据

f = open("疫情数据1.csv", mode="a", encoding="utf-8", newline="")
# 初始化 使用 csv去写入数据
csv_writer = csv.writer(f)
# 写入表头
csv_writer.writerow(["name", "confirm", "confirmAdd", "dead", "heal", "nowConfirm"])


因为国内和国外的数据是分开的,所以还要获取国内的数据

import requests
import csv

f = open("疫情数据.csv", mode="a", encoding="utf-8", newline="")
csv_writer = csv.writer(f)
url = "https://api.inews.qq.com/newsqa/v1/query/inner/publish/modules/list?modules=localCityNCOVDataList,diseaseh5Shelf"
json_data = requests.post(url).json()
chinaTotal = json_data["data"]["diseaseh5Shelf"]["chinaTotal"]
confirm = chinaTotal["confirm"]
confirmAdd = chinaTotal["confirmAdd"]
dead = chinaTotal["dead"]
heal = chinaTotal["heal"]
nowConfirm = chinaTotal["nowConfirm"]
name = "国内"
print(name, confirm, confirmAdd, dead, heal, nowConfirm)
csv_writer.writerow([name, confirm, confirmAdd, dead, heal, nowConfirm])

可视化疫情数据

导入模块

import pandas as pd     # 做表格操作的模块
from pyecharts.charts import Map  # 绘图的模块
from pyecharts import options as opts

读取数据

name_map = {
    "Singapore Rep.": "新加坡",
    "Dominican Rep.": "多米尼加",
    "Palestine": "巴勒斯坦",
    "Bahamas": "巴哈马",
    "Timor-Leste": "东帝汶",
    "Afghanistan": "阿富汗",
    "Guinea-Bissau": "几内亚比绍",
    "Côte d"Ivoire": "科特迪瓦",
    "Siachen Glacier": "锡亚琴冰川",
    "Br. Indian Ocean Ter.": "英属印度洋领土",
    "Angola": "安哥拉",
    "Albania": "阿尔巴尼亚",
    "United Arab Emirates": "阿联酋",
    "Argentina": "阿根廷",
    "Armenia": "亚美尼亚",
    "French Southern and Antarctic Lands": "法属南半球和南极领地",
    "Australia": "澳大利亚",
    "Austria": "奥地利",
    "Azerbaijan": "阿塞拜疆",
    "Burundi": "布隆迪",
    "Belgium": "比利时",
    "Benin": "贝宁",
    "Burkina Faso": "布基纳法索",
    "Bangladesh": "孟加拉国",
    "Bulgaria": "保加利亚",
    "The Bahamas": "巴哈马",
    "Bosnia and Herz.": "波斯尼亚和黑塞哥维那",
    "Belarus": "白俄罗斯",
    "Belize": "伯利兹",
    "Bermuda": "百慕大",
    "Bolivia": "玻利维亚",
    "Brazil": "巴西",
    "Brunei": "文莱",
    "Bhutan": "不丹",
    "Botswana": "博茨瓦纳",
    "Central African Rep.": "中非共和国",
    "Canada": "加拿大",
    "Switzerland": "瑞士",
    "Chile": "智利",
    "China": "中国",
    "Ivory Coast": "象牙海岸",
    "Cameroon": "喀麦隆",
    "Dem. Rep. Congo": "刚果(金)",
    "Congo": "刚果(布)",
    "Colombia": "哥伦比亚",
    "Costa Rica": "哥斯达黎加",
    "Cuba": "古巴",
    "N. Cyprus": "北塞浦路斯",
    "Cyprus": "塞浦路斯",
    "Czech Rep.": "捷克",
    "Germany": "德国",
    "Djibouti": "吉布提",
    "Denmark": "丹麦",
    "Algeria": "阿尔及利亚",
    "Ecuador": "厄瓜多尔",
    "Egypt": "埃及",
    "Eritrea": "厄立特里亚",
    "Spain": "西班牙",
    "Estonia": "爱沙尼亚",
    "Ethiopia": "埃塞俄比亚",
    "Finland": "芬兰",
    "Fiji": "斐",
    "Falkland Islands": "福克兰群岛",
    "France": "法国",
    "Gabon": "加蓬",
    "United Kingdom": "英国",
    "Georgia": "格鲁吉亚",
    "Ghana": "加纳",
    "Guinea": "几内亚",
    "Gambia": "冈比亚",
    "Guinea Bissau": "几内亚比绍",
    "Eq. Guinea": "赤道几内亚",
    "Greece": "希腊",
    "Greenland": "格陵兰",
    "Guatemala": "危地马拉",
    "French Guiana": "法属圭亚那",
    "Guyana": "圭亚那",
    "Honduras": "洪都拉斯",
    "Croatia": "克罗地亚",
    "Haiti": "海地",
    "Hungary": "匈牙利",
    "Indonesia": "印度尼西亚",
    "India": "印度",
    "Ireland": "爱尔兰",
    "Iran": "伊朗",
    "Iraq": "伊拉克",
    "Iceland": "冰岛",
    "Israel": "以色列",
    "Italy": "意大利",
    "Jamaica": "牙买加",
    "Jordan": "约旦",
    "Japan": "日本",
    "Kazakhstan": "哈萨克斯坦",
    "Kenya": "肯尼亚",
    "Kyrgyzstan": "吉尔吉斯斯坦",
    "Cambodia": "柬埔寨",
    "Korea": "韩国",
    "Kosovo": "科索沃",
    "Kuwait": "科威特",
    "Lao PDR": "老挝",
    "Lebanon": "黎巴嫩",
    "Liberia": "利比里亚",
    "Libya": "利比亚",
    "Sri Lanka": "斯里兰卡",
    "Lesotho": "莱索托",
    "Lithuania": "立陶宛",
    "Luxembourg": "卢森堡",
    "Latvia": "拉脱维亚",
    "Morocco": "摩洛哥",
    "Moldova": "摩尔多瓦",
    "Madagascar": "马达加斯加",
    "Mexico": "墨西哥",
    "Macedonia": "马其顿",
    "Mali": "马里",
    "Myanmar": "缅甸",
    "Montenegro": "黑山",
    "Mongolia": "蒙古",
    "Mozambique": "莫桑比克",
    "Mauritania": "毛里塔尼亚",
    "Malawi": "马拉维",
    "Malaysia": "马来西亚",
    "Namibia": "纳米比亚",
    "New Caledonia": "新喀里多尼亚",
    "Niger": "尼日尔",
    "Nigeria": "尼日利亚",
    "Nicaragua": "尼加拉瓜",
    "Netherlands": "荷兰",
    "Norway": "挪威",
    "Nepal": "尼泊尔",
    "New Zealand": "新西兰",
    "Oman": "阿曼",
    "Pakistan": "巴基斯坦",
    "Panama": "巴拿马",
    "Peru": "秘鲁",
    "Philippines": "菲律宾",
    "Papua New Guinea": "巴布亚新几内亚",
    "Poland": "波兰",
    "Puerto Rico": "波多黎各",
    "Dem. Rep. Korea": "朝鲜",
    "Portugal": "葡萄牙",
    "Paraguay": "巴拉圭",
    "Qatar": "卡塔尔",
    "Romania": "罗马尼亚",
    "Russia": "俄罗斯",
    "Rwanda": "卢旺达",
    "W. Sahara": "西撒哈拉",
    "Saudi Arabia": "沙特阿拉伯",
    "Sudan": "苏丹",
    "S. Sudan": "南苏丹",
    "Senegal": "塞内加尔",
    "Solomon Is.": "所罗门群岛",
    "Sierra Leone": "塞拉利昂",
    "El Salvador": "萨尔瓦多",
    "Somaliland": "索马里兰",
    "Somalia": "索马里",
    "Serbia": "塞尔维亚",
    "Suriname": "苏里南",
    "Slovakia": "斯洛伐克",
    "Slovenia": "斯洛文尼亚",
    "Sweden": "瑞典",
    "Swaziland": "斯威士兰",
    "Syria": "叙利亚",
    "Chad": "乍得",
    "Togo": "多哥",
    "Thailand": "泰国",
    "Tajikistan": "塔吉克斯坦",
    "Turkmenistan": "土库曼斯坦",
    "East Timor": "东帝汶",
    "Trinidad and Tobago": "特里尼达和多巴哥",
    "Tunisia": "突尼斯",
    "Turkey": "土耳其",
    "Tanzania": "坦桑尼亚",
    "Uganda": "乌干达",
    "Ukraine": "乌克兰",
    "Uruguay": "乌拉圭",
    "United States": "美国",
    "Uzbekistan": "乌兹别克斯坦",
    "Venezuela": "委内瑞拉",
    "Vietnam": "越南",
    "Vanuatu": "瓦努阿图",
    "West Bank": "西岸",
    "Yemen": "也门",
    "South Africa": "南非",
    "Zambia": "赞比亚",
    "Zimbabwe": "津巴布韦",
    "Comoros": "科摩罗"
}

df = pd.read_csv("疫情数据.csv")
name_list = df["name"].values.tolist()
confirm_list = df["confirm"].values.tolist()
nowConfirm_list = df["nowConfirm"].values.tolist()
# pyechars 添加地理位置信息 只能用英文

全球数据可视化

world_map = (
    Map()
    .add("累计确诊", [list(i) for i in zip(name_list, confirm_list)], "world", is_map_symbol_show=False, name_map=name_map)
    .add("现有确诊", [list(i) for i in zip(name_list, nowConfirm_list)], "world", is_map_symbol_show=False, name_map=name_map)
    .set_series_opts(label_opts=opts.LabelOpts(is_show=False))
    .set_global_opts(
        visualmap_opts=opts.VisualMapOpts(max_=1000000, is_piecewise=True, pieces=pieces)
    )
)
world_map.render("1.html")



原文地址:https://www.cnblogs.com/qshhl/archive/2022/09/24/16725733.html