We are hiring a Python developer to create a standalone crawler program for a specific website using Scrapy. Target website content: aggregated announcements for events or ticket gateway. Applicants will be asked to complete version 1.0 of this program, and deliver all source code to the client. Estimated budget is $200 with some room for negotiation.
The program requirements:
- Built using Scrapy, callable from Scrapy CLI.
- When executed, program must scrape ALL desired data from target website. Crawler may need to follow links within the target website.
- Crawled data items should be cleaned using a simple format (specified by client).
- Program MUST include a toggle-able test mode, which should check whether markup changes in the target website break the crawler. You will answer this in an Upwork screening question for applying to this job.
- [Extended] Exclusive rights to numerous followup project opportunities will be considered if the project is completed satisfactorily.
The applicant requirements:
- Strong communication skills, proficient with English, and comfortable speaking on [Offsite communication is prohibited in the site].
- Required programming languages: Python (HTML, CSS is an advantage).
- Strong preference given to those demonstrate previous experience with Scrapy.
- More consideration will be given to applicants who either 1) provide links to their own open source projects, 2) provide links to contributions to Python community via tech blogs, Q&A forums, etc 3) provide references for past clients.