You've landed at the right place. oDesk is now Upwork. Learn about the new platform.

Web Crawler Jobs

55 were found based on your criteria {{ paging.total | number:0 }} were found based on your criteria

show all
  • Hourly ({{ jobTypeController.getFacetCount("hourly") | number:0}})
  • Fixed Price ({{ jobTypeController.getFacetCount("fixed") | number:0}})
show all
only
only
only
show all
only
only
only
only
only
show all
only
only
only
Fixed-Price - Intermediate ($$) - Est. Budget: $250 - Posted
I need a freelancer to develop a web crawler that scrapes data off the VRBO for specific geographic markets and stores listing data and calendar availability in an Excel format. Scraper must be able to meter itself to avoid getting refused by the website for too many requests to the server. I don't want my IP to get blocked! A. Crawler to crawl VRBO.com to identify all geographic destinations on the site B. Software presents geography tree and allows user to highlight which markets should be crawled C. User selects one of two crawl functions and a third publication function: 1. Crawl: Inventory Pull 2. Crawl: Availability Check 3. Publish: Inventory statistics by market 1. Inventory Pull function -- Crawler goes to VRBO and scrapes all data about all properties for the user selected geographies --Examples of data for each listing includes: ----We need to track multiple levels (parents and children) of geographies (e.g., Hawaii -> Maui -> South -> Kihei) ----Description: Name of Property (e.g., Grand Champion), Listing Identification Number, Listing Name ----Property Type: Condo, House, 1 bedroom, 1,442 Sq. ft., etc.; ----Unit Number (for Condos; e.g., Grand Champion Unit #75) ----Number of bedrooms, number of bathrooms, how many people it sleeps ----Size of home ----Min Stay Requirements ----Low season cost per night, high season cost per night, low season cost per week, high season cost pet week, holiday cost per night, holiday cost per week, etc. ----Dates that define the low season, high season and holiday seasons ----URL Link to the details page of the property ----Number of Reviews ----Review Rating ----Information on amenities (check boxes) ----Information on Activities (check boxes) ----Contact information of the owner ----Tax ID ----Calendar last updated by Owner Date -- Crawler also downloads the current availability calendar for the property and calculated a vacancy rate by month for upcoming 12 months ----calendar data needs to be able to be archived so that future downloads do not overwrite the historical vacancy information ----we want the ability to track how quickly specific properties book over time --Data is saved in a excel format 2. Availability Check Function --Crawler goes to VRBO to check vacancy for each of the user selected markets -- Crawler will check one week increments for 52 weeks in the future for the selected markets -- Crawler will record how many properties of each bedroom count (studio, 1 bed, 2 bed, 3 bed, 4 bed, 5 bed, 6+ beds) are currently available in the markets for each of the 52 week --Spider will compare availability for each week to total property count to calculate current vacancy rate by market by week 3. Publish Inventory Statistics by market -- Need to discuss best approach with our selected developer -- Our goal is to have an easy way to read summaries of the data and drill into the details when desired
Skills: Web Crawler Data scraping Web scraping
Fixed-Price - Expert ($$$) - Est. Budget: $50 - Posted
We search for a data research expert who is able to extract for us the company details of the given sites. Your job is to deliver java code which extracts following details from the weblinks given here: A set URLs for different areas: http://www.immobilienscout24.de/anbieter/suchen/Baden-Wuerttemberg/Boeblingen-Kreis/Herrenberg?geocodeid=1276001006010&focustype=1,2,3,6,7&includeOperationAreas=true http://www.immobilienscout24.de/anbieter/suchen/Baden-Wuerttemberg/Ostalbkreis/Schwaebisch-Gmuend?geocodeid=1276001028034&focustype=1,2,3,6,7&includeOperationAreas=true A set URLs for different areas: http://www.meinestadt.de/herrenberg/berufe-branchen/handwerk http://www.meinestadt.de/herrenberg/berufe-branchen/pflegeberufe Your delivery has to be in following format: https://docs.google.com/spreadsheets/d/17Ug0grVXxb7dZGv7cQgGjd2tzZHeDCN7PTNDIMbVkuw/edit?usp=sharing Please ignore the test/checking tasks of the home pages. We need the full address, phone, email, internet address of the entries. Please ignore our placed budget and place your bid in USD along with the brief description which technology you prefer on solution. We have a lot of more work to do, as you may already see.
Skills: Web Crawler Data scraping Web scraping
Fixed-Price - Intermediate ($$) - Est. Budget: $150 - Posted
I need one java script to crawl 5 sites (only a small portion of the site and not the whole site) and create a excel sheet (tab delimited) of the results. This sites have products and I need to run this script daily so I need it to be fast. Please let me know if you have any questions. This site are public site and no password is needed to access them.
Skills: Web Crawler JavaScript
Fixed-Price - Intermediate ($$) - Est. Budget: $5,000 - Posted
The job requires creativity and analytical thinking to solve problems and move the growth metrics for the project. Skills Required 1. Web Crawling 2. Scripting: Javascript/Ruby/Python 3. File I/O: Reading and writing to CSV files. Mentorship will be provided if needed. Fixed pay for the task however I am flexible and can pay hourly once we establish a rapport.
  • Number of freelancers needed: 10
Skills: Web Crawler Web scraping
Looking for the Team App?
Download the New Upwork Team App
Fixed Price Budget - ${{ job.amount.amount | number:0 }} to ${{ job.maxAmount.amount | number:0 }} Fixed-Price - Est. Budget: ${{ job.amount.amount | number:0 }} Open to Suggestion Hourly - Est. Time: {{ [job.duration, job.engagement].join(', ') }} - Posted
Skills: {{ skill.prettyName }}
Looking for the Team App?
Download the New Upwork Team App