推荐学习书目
› Learn Python the Hard Way
Python Sites
› PyPI - Python Package Index
› http://diveintopython.org/toc/index.html
› Pocoo
值得关注的项目
› PyPy
› Celery
› Jinja2
› Read the Docs
› gevent
› pyenv
› virtualenv
› Stackless Python
› Beautiful Soup
› 结巴中文分词
› Green Unicorn
› Sentry
› Shovel
› Pyflakes
› pytest
Python 编程
› pep8 Checker
Styles
› PEP 8
› Google Python Style Guide
› Code Style from The Hitchhiker's Guide
Ewig
V2EX  ›  Python

拉勾网最近换反爬了?

  •  
  •   Ewig · Jan 23, 2019 · 6902 views
    This topic created in 2841 days ago, the information mentioned may be changed or developed.
    之前写的拉钩网,我今天试了一下出现下面这个

    {"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"117.93.157.209","state":2402}

    {"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"60.30.54.82","state":2402}

    {"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"117.93.157.209","state":2402}
    网址: https://www.lagou.com/gongsi/0-1-0-0

    后来我把 cookie 换了 还是这样 这是为啥呢,ip 是动态代理? 求人指教
    from ip_pool import get_ip
    import requests
    headers={"Cookie":'_ga=GA1.2.174518896.1523111183; user_trace_token=20180407222623-a5c90692-3a6f-11e8-b740-5254005c3644; LGUID=20180407222623-a5c90b3f-3a6f-11e8-b740-5254005c3644; sensorsdata2015jssdkcross=%7B%22distinct_id%22%3A%22167a6ed15993d2-015970814fc80b-35667607-2073600-167a6ed159a938%22%2C%22%24device_id%22%3A%22167a6ed15993d2-015970814fc80b-35667607-2073600-167a6ed159a938%22%7D; index_location_city=%E5%8C%97%E4%BA%AC; JSESSIONID=ABAAABAAAGFABEF2514709505FB85F0FC824310BC7C43F2; _gid=GA1.2.1492847185.1548121054; Hm_lvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1546789367; TG-TRACK-CODE=index_search; SEARCH_ID=8cc1b952a94a496892284ac7a525daea; _gat=1; LGSID=20190123153732-bedacc90-1ee1-11e9-9486-525400f775ce; PRE_UTM=; PRE_HOST=; PRE_SITE=; PRE_LAND=https%3A%2F%2Fwww.lagou.com%2Fgongsi%2F0-1-0-0; LG_LOGIN_USER_ID=d809bbbe54ac48bf0a9ce5888befc8dbdd72485efb1d041a; _putrc=528CDA7A1053B994; login=true; unick=%E5%B2%B3%E5%BA%B7; showExpriedIndex=1; showExpriedCompanyHome=1; showExpriedMyPublish=1; hasDeliver=138; gate_login_token=b729a3ea436639fccaac9cdae984ae92c4562ed3d14bb148; LGRID=20190123153826-ded128bf-1ee1-11e9-b748-5254005c3644; Hm_lpvt_4233e74dff0ae5bd0a3d81c6ccf756e6=1548229075',"User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36","Referer": "https://www.lagou.com/gongsi/0-1-0-0",}

    form_data={'first': 'false', 'pn': '3', 'sortField': '0', 'havemark': '0'}

    res = requests.get('https://www.lagou.com/gongsi/0-1-0-0.json', headers=headers,data=form_data,proxies=get_ip())
    print(res.text)
    26 replies  •  2019-04-29 17:23:28 +08:00
    baicheng10
    baicheng10
    Jan 23, 2019❤️4
    1
    不是想抖机灵,但我确实第一眼看成了“您操的太频繁,请稍后再访问”= =|||
    loveCoding
    loveCoding
    Jan 23, 2019
    2
    拉勾用的人还多吗?
    mytsing520
    mytsing520
    PRO
    Jan 23, 2019
    3
    禁了 token 吧
    supervipcard
    supervipcard
    Jan 23, 2019
    4
    zhangslob669
    zhangslob669
    Jan 23, 2019
    5
    是的,现在需要先访问两个请求获取 cookies
    nichijou
    nichijou
    Jan 23, 2019
    6
    我就没你这么机智懂得过来问一下😅 http://nichijou.city/post/431M
    Ewig
    Ewig
    OP
    Jan 23, 2019
    7
    @zhangslob669 哪两个
    Ewig
    Ewig
    OP
    Jan 23, 2019
    8
    @supervipcard 我还要爬那个详情页公司,下一次 cookie 好像是变化的?你说获取后就可以了?
    Ewig
    Ewig
    OP
    Jan 23, 2019
    9
    @zhangslob669 你的模拟登陆 challence 参数如何破解的
    Ewig
    Ewig
    OP
    Jan 24, 2019
    10
    @zhangslob669 你是模拟登陆的吗?
    supervipcard
    supervipcard
    Jan 24, 2019
    11
    @Ewig 你自己用 fiddler 抓包找一下有 Set_Cookie 的,还要请求一个
    Ewig
    Ewig
    OP
    Jan 24, 2019
    12
    根据你们的意思 得先模拟登陆 才可以啊,但是模拟登陆杨浦个 chaallenge 参数
    daimazha
    daimazha
    Jan 24, 2019
    13
    之前都是随便请求接口就行的
    Ewig
    Ewig
    OP
    Jan 24, 2019
    14
    @daimazha 现在修改了啊,你随便请求也就少部分啊,多了不行了啊
    c4f36e5766583218
    c4f36e5766583218
    Jan 24, 2019
    15
    ip_pool 用的是啥,交流一下
    luofan004
    luofan004
    Jan 24, 2019
    16
    你在这儿问,拉勾的人看不到么,滑稽.jpg
    fakefish
    fakefish
    Jan 24, 2019
    17
    楼主这个有解决方法了吗?我也遇到同样问题, 发现 copy cookie 过来并不行啊
    Ewig
    Ewig
    OP
    Jan 24, 2019
    18
    @c4f36e5766583218 一个动态代理啊
    Ewig
    Ewig
    OP
    Jan 24, 2019
    19
    @fakefish 暂时没有解决,我还想模拟登陆呢
    Ewig
    Ewig
    OP
    Jan 25, 2019
    20
    @c4f36e5766583218 你现在有时间交流吗
    Ewig
    Ewig
    OP
    Jan 25, 2019
    21
    @fakefish 你解决了吗
    fakefish
    fakefish
    Jan 25, 2019
    22
    @Ewig 并没有, 但是只要 cookie,header 一样, 就没问题,但是问题是他的 cookie 是动态设置的, 包括 search_id 用的也和第一次 response 里 set-cookie 的值不一样
    c4f36e5766583218
    c4f36e5766583218
    Jan 26, 2019
    23
    @Ewig 把 ip_pool 资料 /官网贴一下
    Ewig
    Ewig
    OP
    Jan 26, 2019
    24
    @c4f36e5766583218 就是一个动态代理啊,我花钱买的啊,这个就不停的换 ip 的一个 module 啊,和这个没关系吧
    Ewig
    Ewig
    OP
    Jan 28, 2019
    25
    没有高手解决吗
    my1103
    my1103
    Apr 29, 2019
    26
    因为你访问的根不是真正的 url,真正的用 AJAX 提交的
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Privacy   ·   Solana   ·   3957 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 83ms · UTC 01:02 · PVG 09:02 · LAX 17:02 · JFK 20:02
    ♥ Do have faith in what you're doing.