Build a data scraper https://github.com/felipecsl/wombat and add the data to a Postgresql DB
We will start with one website as the source for scraping and should be able to add more later.
There needs to be a front end UI through which we can input the source website URL. Once we input the URL and hit submit, the scraper will scrape the required data and store into the postgreSQL DB. This is the first part of the project
Fetch the data from the DB and display it in our front end UI for editing and adding info to 5 new fields.
This data will be updated in the DB and exported into Spree Ecommerce database via API. Export needs to happen in the format supported by spree.
This data will also be exported into Magento Ecommerce database via API or other means.
Receive data from Spree (via API) and magento and store it into the same DB
Build a data matching algorithm that takes the data coming from a table and matches with data from different table and display it on the front end http://yuesaa.axshare.com/#g=1&p=matching&c=1
for human curation. If you can use Google search API to match similar images that's a plus)
Send data to spree via API once the human confirms the matching of each data.