You've landed at the right place. oDesk is now Upwork. Learn about the new platform.

Web Crawler Jobs

44 were found based on your criteria

show all
show all
only
only
only
show all
only
only
only
only
only
show all
only
only
only

Hourly - Est. Time: 3 to 6 months, Less than 10 hrs/week - Posted
Hello, I'm seeking an experienced data mining professional, who can compile specific data lists that will be converted to leads for a call center business. I would be able to provide the insight and direction in which you would need to focus your attention. I have specific criteria in which I would need to be able to collect on. For candidates that apply I will give the specific details of exactly what I'm looking for.

Fixed-Price - Est. Budget: $ 300 Posted
Hi, Pretty simple scraping project but the output needs to be an application that I can install and run myself (or that runs in a hosted environment). The application itself will: a) authenticate with LinkedIn as a user inputs for this step: username, password, verification code (may be required if running as a web app) b) visit a specific user's profile input for the step: target profile URL c) scrape all the 1st degree connections for that user (this requires paginating through the contacts...) outputs for this step (output as csv for all contacts for the target): Name LinkedIn profile URL Company Current Role Company URL (if available) Work history Requirements: - this can run as a web app OR as a desktop app (I'm open to either) - if creating a desktop app, it needs to run on OSX Yosemite or higher - if creating a web app, you'll need to set up the server environment and provide a secured web interface for inputting the parameters

Hourly - Est. Time: 3 to 6 months, Less than 10 hrs/week - Posted
Need A data extraction, web scraping expert needed for ongoing data mining activities. Need to be able to use import.io and or scrapy.org for extracting data. I will give you instructions on the data I need and I need you to build a crawler/extractor that will find the information and present it in csv format I also need you to find information on latest blogs, social media mentions, etc of my topics and people and compile it into data I can use. Respond with what you can do Software you work with work you have completed time frame to start getting data

Fixed-Price - Est. Budget: $ 300 Posted
### Expert Level ### Please reply with real world examples ?? If you have experience with a bloom filter please let me know. Hi I have a version of the application already built but isn't performing as I had hoped. I need a developer to help me improve the application in terms of performance and accuracy. Below are what I want to happen so be clear in your replies that you can perform this type of work. - Web scraping sort of works, but really needs to become multi-threaded and a lot more robust (currently breaks lots), the basic stack is below but I'm not adverse to other technologies being used. - The crawler aims to collect 100's millions of rows of data from numerous content networks, so the crawler needs to be able to manage that number of rows and the complications that bring to the table. - Post content to social networks, current app doesn't post in the correct format (weird categories) so that needs to be fixed. Basic headlines - Create multi-threaded...

Hourly - Est. Time: 1 to 3 months, 10-30 hrs/week - Posted
The task is to implement a focused Web crawler component comprising an API and a Web-based GUI. The focused crawler's objective is to gather user-specified pieces of data from Web pages using two strategies: (1) by carrying out a set of Web search engine queries for user-provided query terms, by downloading Search Engine Result Pages, and optionally following the hyperlinks contained therein; and (2) using a set of user-provided seed URLs, by following the hyperlinks on the Web pages identified by these URLs, using a breadth-first search and a user-specified depth. Rather than implement from scratch, the project should be pursued as wrapping existing open-source components such as crawler libraries and Web search engine APIs with additional layers of functionality, and in some cases, providing a convenience layer that hides complexity. Skills/technologies: Java, Heritrix, WARC, search engine technology, crawling, Yahoo BOSS

Fixed-Price - Est. Budget: $ 150 Posted
Hi Upwork community, We need a web scraping expert in order to get us some information about football clubs, teams and members on a Swiss website. The website that we need to scrape is www.football.ch and is available in three languages: German, French and Italian. If you know one of these, that might make it easier. We need the freelancer to crawl the site to create 4 data tables out of the information that is on the website, namely: 1) Clubs 2) Club admins 3) Teams 4) Team trainers We have put together in the enclosed excel the information that we would need to scrape for each of these tables to give you a better idea of what we are trying to get out of this project. We also added additional help and screenshots as to where the information is on the platform. Finally, we need the freelancer to use a widespread tool for doing so and to handover the code for scraping at the end of the project as we will need a yearly refresh of the information. Thanks and looking forward...

Fixed-Price - Est. Budget: $ 150 Posted
I need scrape and parser to xml akomantoso version, of http://www.concejodebucaramanga.gov.co/descargas.php?seccion=NQ==&categoria=MQ== and http://www.concejodebucaramanga.gov.co/descargas.php?seccion=NQ==&categoria=OA== I have 2 scripts templates for that only need create regualar expressions for that council chamber. Examples: https://comision6senado.files.wordpress.com/2013/03/acta-10-12-septiembre-18-de-2012.pdf converted to http://senado.felipeurrego.com/comisi%C3%B3n-sexta-senado/2012/septiembre/acta-no-10-18-09-2012.an I need test experience and skills with http://www.concejodecali.gov.co/documentos.php?id=502

Fixed-Price - Est. Budget: $ 50 Posted
I need the data from the following website: www.ncass.org.uk I need someone to get me the following information off every company on the site: Company Name Location Contact Name Telephone Mobile Email I will need to be given the data in a spreadsheet using either Google Docs, Numbers or another format that is compatible with my Apple Mac. Payment will be made upon proof that the works have been completed in the format requested. I will be choosing the cheapest option for this project. This is for a large website that will require many more directories to be scraped. Whoever wins this project and does a good job will be used to complete the other projects.