I need a tool that I can run on a monthly basis that can scrape data from the site below.
I need the following vehicle and product data. This includes
Vehicle Options (Each vehicle could have multiple options or have no options)
Item part number
MAP Price (This field is hidden in the html) See screen shot in attached zip.
Primary Image url
The same part will fit multiple vehicles, so it will be a large collection of data.
Each page can have items with multiple locations (Example: Front, Rear, Front & Rear) and multiple colors (Example: Black, Gray, Smoke). I will need all combinations.
Not every vehicle is available for every part.
I need the process to be done in Python using the Scrapy framework.. I will need to run this script/program off of different PCs. I will also need to run this script/program multiple times during the year.