We'd like to develop a crawler/scraper script with filtering option. We need to download info on a regular basis from specific websites, some of which need the application of a filter (keywords, phrases, time frame, etc.) on reaching the site. Need this scraper to run on a periodic basis, period to be specified by the user.
I am looking for a mix of experience and value