Senior Python Developer | Advanced Google Scaper

Closed - This job posting has been filled and work has been completed.
Web, Mobile & Software Dev Other - Software Development Posted 2 years ago

Fixed Price

Delivery by February 14, 2013

$250.00

Budget

Details

Need an experienced python developer who can create a highly scalable Google scraper.

The software will need to scan in excess of 300,000 keywords a day.

I envision that there will be a main "hub" process that will gather keywords from a MySQL database (based off my model) and distribute tasks to external python processes in order to mitigate CPU usage and bandwidth.

Hub:
- Distribute keywords to be scraped to external python processes

External Python Process:
- Multi threaded to scan multiple keywords at once
- Gather top 100 results from Google
- Be able to use multiple TLD's (.com, .co.uk, etc)

The results will then need to be inserted into a database.

I have in excess of 15,000 IPs to do this so IP blocking isn't an issue.


About the Client

(4.91) 38 reviews

Australia
Rosanna 09:49 AM

92 Jobs Posted
58% Hire Rate, 2 Open Jobs

$9,369 Total Spent
58 Hires, 4 Active

$9.98/hr Avg Hourly Rate Paid
328 Hours

Member Since Sep 29, 2007