Looking for a freelancer to develop a web scraper tool and integrate it with our current OctoberCMS installation.
For this project, prospective freelancers must understand MongoDB and have the ability to write a web scraper that can pull up to 2000 points of data from a HTML page into multiple seperate documents in the MongoDB database, which the developer will create according to our spec.
It is expected that administrators will be able to enter a URL to be scraped in the backend, at which point the profile linked to the URL will be either updated, or created if none exists, in the databases, and an entry will be added to the administration module. If updated, it is expected that specific data points will be added to track over-time progress of specific statistics. Full details on all data that we wish to scrape and include will be provided.
The URL scraped is from a single domain, and the overall design and HTML will remain similar, however specific details such as the order or availability of any number of the 2000 points of data may change on any particular page load.
This project is for the scraper, admin module, and database only, and no front-end or graphic design is required. Please place the words "Ironic Tortilla" in your response or it will be removed for "not matching brief". Responses without direct reference to MongoDB and freelancer's ability with MongoDB will be removed for "not matching brief".
This is a single part of a 9-part project, and prospective freelancers may have future opportunities available based on their abilities and success. Freelancers are expected to potentially work with other freelancers or employees assigned to the project to produce expected integration between projects. The entire project is expected to complete within June, and may have up to 5 different freelancer's working on it at any given time.
All work is done via GitHub, and freelancers are expected to follow proper workflow (staging to live) and properly comment their changes as necessary.