You've landed at the right place. oDesk is now Upwork. Learn about the new platform.

Web Crawler Jobs

35 were found based on your criteria {{ paging.total | number:0 }} were found based on your criteria

show all
  • Hourly ({{ jobTypeController.getFacetCount("hourly") | number:0}})
  • Fixed Price ({{ jobTypeController.getFacetCount("fixed") | number:0}})
show all
only
only
only
show all
only
only
only
only
only
show all
only
only
only
Looking for the Team App?
Download the New Upwork Team App
Fixed-Price - Intermediate ($$) - Est. Budget: $100 - Posted
Hello, I'm looking for someone to download all prices for every part, for all vehicle year/make/models. From an automotive parts site such as http://www.rockauto.com/en/catalog/ I would like it to be summarized in an excel spreadsheet, or itemized database that can be downloaded. Please let me know A) How you plan to do this (crawler, manually, etc) B) How long it will take you C) Price you will charge (can be more or less than $100)
Skills: Web Crawler Web scraping
Fixed-Price - Intermediate ($$) - Est. Budget: $250 - Posted
I need help in scrapping university websites sites for email address. This is a low weight work. The code can be written in any language, preferably in Ruby. I will need a csv file in the format, i specify. I will also specify the sites and techniques to scrape as i have done this before. There are close to 30 sites, you will be needing to scrape from. I will give you the university name and you will need to get names from university facebook group or use common names that i can provide. Find the student directory page for that university and use that for searching and fetching the email and other meta information from the results
  • Number of freelancers needed: 2
Skills: Web Crawler Data scraping HTML JavaScript
Fixed-Price - Expert ($$$) - Est. Budget: $50 - Posted
I'm going to Angola, a Portuguese speaking country, and need someone to conduct some research (within financial services/ banking and telecommunications) on this country, some important individuals and other aspects of these industries. A lot of this information will be in Portuguese and I need someone with excellent research skills - web research/ deep mining - and native (or near native) Portuguese: written, reading and speaking. Ideally the individual will have lived or live in Angola - though this is not completely necessary. I'll provide you with the list of names (approx. 10) , companies/ societies (approx. 10) and specific details on the aforementioned industries. I'd need this work completed by: 16th February. It should be presented in a table either using MS Word or Excel. All to be written in English. Please ask any needed questions. Price given is negotiable depending on skill, expertise, and speed of delivery.
Skills: Web Crawler Data scraping Internet research Market research
Fixed-Price - Entry Level ($) - Est. Budget: $200 - Posted
We need to develop a PHP script that crawls craiglist website. Must work as a bot. The script must do: 1) The bot must crawl all cities in craiglist on the selected craiglist section (vacation rentals) and retrieve all the data of each apartment (including email, photos, text, features and owner data). 2) The bot will insert the crawled data into our database (mysql). - First creating the user with the data in craiglist and then the apartment data. 3) Send email to inform about the listing to the owner. Our site is developed in CakePHP and the coder can help with the step 2).
  • Number of freelancers needed: 2
Skills: Web Crawler CakePHP Web Crawling MySQL Programming
Fixed-Price - Intermediate ($$) - Est. Budget: $50 - Posted
Looking for someone experience with Amazon Product advertising api before, to develop very simple tool where 1. user enter keywords and some filter 2. tool uses keyword API to display results 3. and then use ASIN lookup to get prices from different marketplace and display results.. 4. what is usually limitation on number of calls per hour or second for this API if you know ? I will provide access to linux server.. I already have tool that uses scraping but scraping has lot of issues, so decide to switch to API http://ec2-52-53-228-149.us-west-1.compute.amazonaws.com/search Tool doesn't have to be copy of this As tool should display data in html table with excel export option.. Only apply if you are available whole day today and finish tool today ... Ideally its just JSON / XML phrasing snd saving data in db... 1. please send me link/explaination of previous amazon ads API work 2. what language will you code (prefer php , python or node.js java ) 3. confirm you can work whole day oon this and finish it, Buget : $50
Skills: Web Crawler API Development JavaScript json
Fixed-Price - Intermediate ($$) - Est. Budget: $500 - Posted
Crawl website for content extraction and provide extracted content in various format such as Microsoft Excel (.xls), XML, Microsoft Access (.mdb), SQL, etc. - Will be collecting data such as Real Estate Properties - Must have experience with Scrapy and Scrapinghub Platform - Setup scraped files and images to FTP server - Salesforce API Integration and Resolving Formatting Issues
Skills: Web Crawler Data mining Data scraping Python
Fixed-Price - Intermediate ($$) - Est. Budget: $500 - Posted
Looking for someone expert in scraping data from variation website.. and save data in mysql / csv . Script has to be python or php . If python, it should work with on linux server with lamp php website. If you are really good , I dont mind you offering full time job as I need 100s of scrapping stuff needed over next 3 months .. this job is for 6 websites .. but I might need some other small scrappers before I give big project please answer these 1. write 'ddonk' before application 2. let me know if you prefer php or python 3. mention what website you have scrapped ? google, linkedin, amazon, yellow pages ? 4. show me list to any web application that does scraping if you have build any 5. do you have full time job and part time freelancer ? or you are full time freelancer ?
  • Number of freelancers needed: 3
Skills: Web Crawler Data mining Data scraping Django
Fixed-Price - Expert ($$$) - Est. Budget: $50 - Posted
Hi Our Insurance brokerage needs someone to scrap the Contractors State License Board of California's website (www.cslb.ca.gov) to obtain the Workers Compensation data listed below and format into an excel spreadsheet. We are only looking for those businesses that have "active" licenses. We have an excel list already we can provide as an example. It just needs to be updated with current dates as it has been awhile since last updating. I plan to ask for an updated list every 2-3 months and would like to find someone I can consistently go to to update the list. Please let us know if you need any further details before applying and bidding. Contractors State License Number, Business Name, Classification Type (Plumbing, Landscaping, etc.), First & Last Name of Owner (contact), Policy Number, Current Carrier, Renewal Date, Address, City, Zip Code, State, Phone Number (s), and emails if available
Skills: Web Crawler Data mining
Looking for the Team App?
Download the New Upwork Team App
Fixed Price Budget - ${{ job.amount.amount | number:0 }} to ${{ job.maxAmount.amount | number:0 }} Fixed-Price - Est. Budget: ${{ job.amount.amount | number:0 }} Open to Suggestion Hourly - Est. Time: {{ [job.duration, job.engagement].join(', ') }} - Posted
Skills: {{ skill.prettyName }}
Looking for the Team App?
Download the New Upwork Team App