Home Sign Up Sign In
wico77
V2EX  ›  问与答

scrapy 爬取一个站下的多个分类，每个分类在 pipeline 里保存成不同的文件。怎么实现批量爬取和管理

  •   
  •   wico77 · Apr 20, 2015 · 2700 views
    This topic created in 4225 days ago, the information mentioned may be changed or developed.
  • Scrapy
  • pipline
  • 分类
    1 replies  •  2015-04-20 16:58:47 +08:00
    zts1993
    zts1993
    Apr 20, 2015
    1
    在Item里面弄一个字段区分，在每个pipeline里面判断是否符合条件？

    也不是很熟，下面的来回答吧
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Privacy   ·   Solana   ·   1097 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 25ms · UTC 23:18 · PVG 07:18 · LAX 15:18 · JFK 18:18
    ♥ Do have faith in what you're doing.