在 Python Scrapy 中建立動態管道

在 settings.py 中啟用管道 ****

ITEM_PIPELINES = {
    'project_folder.pipelines.MyPipeline': 100 
}

然後在 items.py 中編寫此程式碼 ****

# -*- coding: utf-8 -*-
from scrapy import Item, Field
from collections import OrderedDict

class DynamicItem(Item):
    def __setitem__(self, key, value):
        self._values[key] = value
        self.fields[key] = {}

然後在你的`project_folder / spiders / spider_file.py 中

from project_folder.items import DynamicItem
       def parse(self, response):
               # create an ordered dictionary
               data = OrderedDict()
               data['first'] = ...
               data['second'] = ...
               data['third'] = ...
               .
               .
               .
               # create dictionary as long as you need
               
               # now unpack dictionary
               yield DynamicItem( **data )

               # above line is same as this line
               yield DynamicItem( first = data['first'], second = data['second'], third = data['third'])

這段程式碼有什麼好處？

無需逐個建立 items.py 中的每個專案。