You've landed at the right place. oDesk is now Upwork. Learn about the new platform.

Web Crawler Jobs

55 were found based on your criteria {{ paging.total | number:0 }} were found based on your criteria

show all
  • Hourly ({{ jobTypeController.getFacetCount("hourly") | number:0}})
  • Fixed Price ({{ jobTypeController.getFacetCount("fixed") | number:0}})
show all
only
only
only
show all
only
only
only
only
only
show all
only
only
only
Hourly - Expert ($$$) - Est. Time: 1 to 3 months, 30+ hrs/week - Posted
Hello, we are looking for a developer that is an ABSOLUTE expert at creating software that is capable of crawling websites and scraping data rapidly and accurately. We currently have a software, however we are looking to drastically improve the software by making it -Cheaper to run -Yield far better results A simple explanation of how the software works is (there is much more to it, but this is the basics) -It pulls or is given a list of URLs -It then crawls the links on these pages, follows them and then continues the process for as many pages we deem. So for example if we set it to 300 it will crawl the links on a page, follow them and then crawl the links on the next page and repeat 300 pages deep -When a followed link shoots back an error or 404 the tool checks to see if the domain can be registered via namecheap and apis. AKA a person can purchase the domain because it is expired. -It then gathers all the available domains and presents it to the user while also pulling SEO metrics on the domains We need a Ruby developer who can 1. Significantly lower the cost of the crawl 2. Add different crawl features to the software (we will give you these when you come aboard) 3. Improve the crawls accuracy PLEASE ONLY APPLY IF 1. You are an expert on scraping solutions and have built advanced crawlers before 2. Read the above again Thank you!
Skills: Web Crawler Data scraping Ruby Ruby on Rails
Hourly - Entry Level ($) - Est. Time: Less than 1 month, 10-30 hrs/week - Posted
I need a list of the best bloggers with the highest number of readers, viewers in Slovenia, Croatia, Romania, Estonia, and Poland who write about, cover, or deal with: 1) nonfiction books/book reviews 2) religion/spirituality 3) self-help (personal betterment) 4) mental health/depression/grief/death/anxiety, etc. 5) world cultures 6) real-world problems or anything else similar to these topics I would need the name of the contact person, the URL for the media outlet, their email and/or Twitter, and the number of readers/followers. Applicants should speak one or more of the native languages of the countries listed (Estonian, Croatian, Romanian, Polish, Slovenian, Slovakian) +
Skills: Web Crawler Web Crawling Croatian Data Entry
Hourly - Entry Level ($) - Est. Time: Less than 1 week, 10-30 hrs/week - Posted
Hi, I need a trustworthy Virtual Assistant to help me with compiling website information from a list of companies that I will provide to you. Tasks: Complete my google doc by doing the following: - Google search for each company by name - Visit website - If no website, find on Manta.com or some other directory - Copy and paste into fields on my spreadsheet including: Hours, address, phone, contact info, website URL, products offered, etc Successful candidates: - Speak English well - Operate timely - Communicate any problems ahead of time - Have a 4.5+ star rating on upwork If successful I will have several other projects coming. Thanks CP
  • Number of freelancers needed: 3
Skills: Web Crawler Data Entry Internet research
Looking for the Team App?
Download the New Upwork Team App
Fixed Price Budget - ${{ job.amount.amount | number:0 }} to ${{ job.maxAmount.amount | number:0 }} Fixed-Price - Est. Budget: ${{ job.amount.amount | number:0 }} Open to Suggestion Hourly - Est. Time: {{ [job.duration, job.engagement].join(', ') }} - Posted
Skills: {{ skill.prettyName }}
Looking for the Team App?
Download the New Upwork Team App