Crawl, scrape & API lookup - Python

Web, Mobile & Software Dev Web Development Posted 2 years ago

Fixed Price

Delivery by April 20, 2014




This project is a trial test to ensure that we can build out scalability and ensure we have got the best approach.

Needed is someone to create a multi-threaded Python script that will have the following functionality:

1. Create new Project ID and Que task
2. Scrape search suggest wildcards (python)
3. Pull 10 results for each letter of the alphabet
4. Run across 10 search operators (plus their alphabet variables)
5. Collate list and remove duplicates (while in MySQL database)
6. Batch keywords by Regex Variables (variables have already been created)
7. Run through search volumes (probably require API)
8. Filter by search volume and categorize by Batches (in point 6)

The page will simply be an input field / past projects (hidden behind a username and password), when selecting past projects, results will be displayed in JQuery data table

We have access to proxies (if they are required) and a test server for you to use.

Obviously I will provide more details upon interview.

  • Other Skills:

Activity on this Job

Last Viewed by Client: 2 years ago

Hired: 1

About the Client

(4.94) 168 reviews

Warragul 03:33 PM

289 Jobs Posted
76% Hire Rate, 1 Open Job

Over $40,000 Total Spent
237 Hires, 14 Active

$2.34/hr Avg Hourly Rate Paid
9,956 Hours

Member Since Feb 8, 2009