tcpdump+wireshark
- wireshark Wireshark · Download
- windows tcpdump https://www.winpcap.org/windump/
- 开启tcpdump 网络监听 sudo tcpdump -Al host www.baidu.com -v -w /tmp/baidu1.txt
- -v显示日志,
- -w 表示将监听的内容保存到本地
- host 要监听的主机
- port 要监听的端口
- Al A:以ascii的方式显示数据包,抓取web数据时很有用.l:使用标准格式进行输出
- curl http://www.baidu.com 使用curl命令发送请求
curl
- 通过参数指定wd的编码
curl -G url https://www.baidu.com --data-urlencod "wd:霍格沃兹测试学院" -H 'Connection: keep-alive' -H 'Cache-Control: max-age=0' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: zh-CN,zh;q=0.9,en;q=0.8' -H 'Cookie: BAIDUID=5403DAC6AEC8B48DCC6A3D45552B9CD6:FG=1; BIDUPSID=5403DAC6AEC8B48DCC6A3D45552B9CD6; PSTM=1568602233; BDUSS=BrMX5HN2l5NmtFLWdhU3ZHYThHakY3cm94UUZlR3l-TTU1My1leWpmNHJkS2hkRVFBQUFBJCQAAAAAAAAAAAEAAAD6VgShbmljZcLt6LQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACvngF0r54BdRj; _homeland_session=lIG%2F9zPjSHHWta%2BKXwOOry%2FOxa3NGGyPzoE0Eek8rAel8wmq4FAv0Sa1PQhR1I5MNTpopx3zCJeLykMFt6xGjAB66RZ5hhJOKMXEdIpbORyYYIENsEyciBeNCYkz7IKVAKWdhvF4NzH1BFxhHc35ONOm59%2F8%2FRJ0nfj4tXD97PoiS02Zpe4WDtZTZrg%2Fmm5BN%2BOnn0qHFA%2BmkndNQOLhIyLT3Tt8nVki0cg74IV5Mzo%2FSE28u2HS9IdkoiagIllYZehGJwiMDN7UH%2Fwc1jFc%2BJMp94Alu%2Fnxrg%3D%3D--8vazfTOh2Djn3G5e--f7J5EgYyZpIiw4UZYmtpOw%3D%3D; BD_UPN=123253; delPer=0; BD_CK_SAM=1; PSINO=2; H_PS_PSSID=1421_31170_21118_31187_30905_31051_30823_31085_26350; H_PS_645EC=ddce%2FJvbox9oZltdQxsz83h1tOXe35N1k0%2BsPVHJ9oX56aUNa87qp6eTeMSvp0CPlD5f; BDORZ=B490B5EBF6F3CD402E515D22BCDA1598; BDRCVFR[feWj1Vr5u3D]=I67x6TjHwwYf0; BDSVRTM=168' --compressed
- 从google开发者工具里边copy查来的原始curl命令
curl 'https://www.baidu.com/s?wd=%E9%9C%8D%E6%A0%BC%E6%B2%83%E5%85%B9%E6%B5%8B%E8%AF%95%E5%AD%A6%E9%99%A2&rsv_spt=1&rsv_iqid=0xa6cf9ee20002237f&issp=1&f=3&rsv_bp=1&rsv_idx=2&ie=utf-8&tn=baiduhome_pg&rsv_enter=1&rsv_dl=ts_0&rsv_sug3=26&rsv_sug1=26&rsv_sug7=101&rsv_sug2=0&prefixsug=huogewozi%2520ceshix&rsp=0&inputT=7278&rsv_sug4=7622' -H 'Connection: keep-alive' -H 'Cache-Control: max-age=0' -H 'Upgrade-Insecure-Requests: 1' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: zh-CN,zh;q=0.9,en;q=0.8' -H 'Cookie: BAIDUID=5403DAC6AEC8B48DCC6A3D45552B9CD6:FG=1; BIDUPSID=5403DAC6AEC8B48DCC6A3D45552B9CD6; PSTM=1568602233; BDUSS=BrMX5HN2l5NmtFLWdhU3ZHYThHakY3cm94UUZlR3l-TTU1My1leWpmNHJkS2hkRVFBQUFBJCQAAAAAAAAAAAEAAAD6VgShbmljZcLt6LQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACvngF0r54BdRj; _homeland_session=lIG%2F9zPjSHHWta%2BKXwOOry%2FOxa3NGGyPzoE0Eek8rAel8wmq4FAv0Sa1PQhR1I5MNTpopx3zCJeLykMFt6xGjAB66RZ5hhJOKMXEdIpbORyYYIENsEyciBeNCYkz7IKVAKWdhvF4NzH1BFxhHc35ONOm59%2F8%2FRJ0nfj4tXD97PoiS02Zpe4WDtZTZrg%2Fmm5BN%2BOnn0qHFA%2BmkndNQOLhIyLT3Tt8nVki0cg74IV5Mzo%2FSE28u2HS9IdkoiagIllYZehGJwiMDN7UH%2Fwc1jFc%2BJMp94Alu%2Fnxrg%3D%3D--8vazfTOh2Djn3G5e--f7J5EgYyZpIiw4UZYmtpOw%3D%3D; BD_UPN=123253; delPer=0; BD_CK_SAM=1; PSINO=2; BDORZ=B490B5EBF6F3CD402E515D22BCDA1598; BD_HOME=1; BDRCVFR[feWj1Vr5u3D]=I67x6TjHwwYf0; H_PS_PSSID=1421_31170_21118_31187_30905_31051_30823_31085_26350; H_PS_645EC=e792dE2XBpnjK%2F9OjU2IC%2FE9Cfs3bHxD9fh62992HhAfEOxr2qwG9tnyEIswv8OA2da%2F; BDSVRTM=171' --compressed
- curl使用(思寒) curl命令参数
- curl用法指南 curl 的用法指南 - 阮一峰的网络日志
- curl重用命令
- s 静默输出
- G 将参数携带在url里边
- –data-urlencod 将参数进行编码
- o 将输出内容保存到本地
- v 打印详细发送的日志
- d 发送post请求时,携带的数据体
nc telnet 模拟发送请求
nc www.baidu.com 80 -v
GET / HTTP/1.0
Host: www.baidu.com
telnet www.baidu.com 80
GET / HTTP/1.0
Host: www.baidu.com
curl作业
1.用curl发送一个header中携带user-agent:xxx的头信息
curl -I -H 'User-Agent:hogwarts-ashin' 'http://www.baidu.com' -v
- 向雪球发起一个股票搜索
curl -G 'https://xueqiu.com/stock/search.json' --data-urlencode "code=阿里巴巴" --data-urlencode "size=3" --data-urlencode "page=1" -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: zh-CN,zh;q=0.9,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36' -H 'elastic-apm-traceparent: 00-d874fd37925fbeab5e4f84e2f32a682e-444a22d0ee0952ed-01' -H 'Accept: application/json, text/plain, */*' -H 'Referer: https://xueqiu.com/k?q=%E9%98%BF%E9%87%8C%E5%B7%B4%E5%B7%B4' -H 'Cookie: device_id=24700f9f1986800ab4fcc880530dd0ed; _ga=GA1.2.889686388.1571316755; s=ci11rec8p6; acw_tc=2760824815851901643843314e96343459c7d769337f5e8d44cfcfc1bfb32e; aliyungf_tc=AQAAACTAEkRpQwEA0bjHb59FkyABvGUT; xq_a_token=2ee68b782d6ac072e2a24d81406dd950aacaebe3; xqat=2ee68b782d6ac072e2a24d81406dd950aacaebe3; xq_r_token=f9a2c4e43ce1340d624c8b28e3634941c48f1052; xq_id_token=eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJ1aWQiOi0xLCJpc3MiOiJ1YyIsImV4cCI6MTU4NzUyMjY2MSwiY3RtIjoxNTg1NDYxOTE5MDEwLCJjaWQiOiJkOWQwbjRBWnVwIn0.Gt45VHJaeY5YdyAZDXdB4LLE88Dco1NkBKlpcSBcmqTgiddsBF6TkeqikpBylRIX-HnzRJbtRpXKipoKW3INUBhDGQoq20B5GB_W9AVUyRZtIxHUl_BTfYyBGx5kB_odiODC7c9kk6W1Ctgx5pHj-CB8unMghl_pypvHxH7i8ONqAaSAGDDqq9q0T04UlGTocjd-UTIV9sHzLR76xd2vbUpyIQ1tK9igPyyV3anP7r7qbPqMrvKf772kucHU4mA9_U770xpvHSebFyHckjuv-vY_v1nAq6RUClZCODUdcKyGq089I-CTqPwlos5ImWHMFT8PRUvRwHxGiEPNNw5lqA; u=611585461963094; Hm_lvt_1db88642e346389874251b5a1eded6e3=1585190122,1585461920; Hm_lpvt_1db88642e346389874251b5a1eded6e3=1585461944' -H 'Connection: keep-alive' --compressed -v |jq
下午课程准备
- 提前安装好charles
- pycharm准备好相关代码 一个测试http与post的flask demo
get 和 post请求分析
-
curl -s ‘http://127.0.0.1:5000/request?a=1&b=2’ -v &>/tmp/get
-
curl -s ‘http://127.0.0.1:5000/request?a=1&b=2’ -d “a=3&b=4” -v &>/tmp/post
-
下周四作业:打开一个网站发生了什么事情
####charles mock&fake
- 1.修改雪球某一支股票为自定义名字,并修改价格
- 2 .修改雪球列表页股票的列表内容,修改为上千只股票
raw=$(cat /tmp/stock.json)
raw=$(echo $raw |jq '.data.items+=.data.items'| jq '.data.items_size+=.data.items_size' )