推荐学习书目
› Learn Python the Hard Way
Python Sites
› PyPI - Python Package Index
› http://diveintopython.org/toc/index.html
› Pocoo
值得关注的项目
› PyPy
› Celery
› Jinja2
› Read the Docs
› gevent
› pyenv
› virtualenv
› Stackless Python
› Beautiful Soup
› 结巴中文分词
› Green Unicorn
› Sentry
› Shovel
› Pyflakes
› pytest
Python 编程
› pep8 Checker
Styles
› PEP 8
› Google Python Style Guide
› Code Style from The Hitchhiker's Guide
daiqiangbudainiu
V2EX  ›  Python

发现一个 scrapy 的奇怪问题

  •  
  •   daiqiangbudainiu · Aug 5, 2021 · 1675 views
    This topic created in 1918 days ago, the information mentioned may be changed or developed.
    在用 scrapy 抓一些网页内容的时候,发现被 403 了,我以为是我爬的频率太高了

    但是奇怪的是,同样的请求我放到 postman 里边,就能正常访问

    而此时 scrapy 还是会报 403
    NLL
    NLL
    Aug 6, 2021 via iPhone
    1
    抓包对比看看两个发出的请求有什么区别
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Privacy   ·   Solana   ·   1383 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 26ms · UTC 22:34 · PVG 06:34 · LAX 14:34 · JFK 17:34
    ♥ Do have faith in what you're doing.