I need script what will copy Angel.co website data to my local DB. I'm going to launch it every week or two and update local copy with new data.
The difficult part of this project, that alhough Angel.co has it own API, it's not allow to get more then 250-300 records per 1 search. Than meens that this project is not just scrapping, but create prorgamm logic of harvesting data using links of different type of profliles. Some kind of mini network. Get some data, to get more data, to get more data etc.
Actually I dont need ALL data, but about 60-70%. More detailed description will be provided when we'll discussing project