Web Crawler Jobs

72 were found based on your criteria {{ paging.total|number:0 }} were found based on your criteria

show all
  • Hourly ({{ jobTypeController.getFacetCount("0")|number:0}})
  • Fixed Price ({{ jobTypeController.getFacetCount("1")|number:0}})
Fixed-Price - Intermediate ($$) - Est. Budget: $20 - Posted
We are looking for someone to propose the best solution for crawling particular google searches for all the websites that are listed which are non mobile responsive. i.e - We would give the machine/script a key word such as "Mens fashion stores in melbourne" and it will run and check the css framework to identify whether or not it is mobile responsive. For the applicant that thinks they can do this I would like them to provide a short brief of how they think It could work. I am a developer, but not experience in this field.
Skills: Web Crawler CSS Scripts & Utilities
Fixed-Price - Expert ($$$) - Est. Budget: $750 - Posted
We are looking for an experienced web developer who can create a bot to search all media outlets. My company current does this manually and there has to be an easier way to generate leads. My company searches for any type of insurance claim. (fire, flood, hurricane, etc) We handle the claim for the owner of the policy. We will need to be able to target specific locations or areas and search for key words to pull the stories. Right now it will pertain to only 3 states. Will provide more details once engaged.
Skills: Web Crawler Web Crawling Makerbot
Fixed-Price - Expert ($$$) - Est. Budget: $1,500 - Posted
Dear Developer, I'm looking for a highly skilled and energetic developer for a search engine / web crawler development. In essence, the project I intend to do is about developing an intelligent algorithm + an implementation of such an algorithm that - take given inputs (A --- e.g. "Airbus") - searches the web for more information on the given input (A linked to B via relation x --- e.g. Airbus (A) is competitor (x) of Boeing (B)) - sends back a probability of the reference found (A x B --> % --- e.g. Airbus is competitor of Boing --> 99%) - hands this relations & probabilities back into a database (data base design is not necessarily part of this project, but can be added) The relation can be designed from pure time and material work to a sustainable partnership / hiring in a US based company. If you are interested, please reply with a short mail - describing your background & why you are the right one for that job - give references to similar projects / research work
Skills: Web Crawler Data Analytics Data mining Data Science
Fixed-Price - Intermediate ($$) - Est. Budget: $30 - Posted
We are looking to extract names and other information from a specific directory that we have registered too. We need to pull out all info from this directory, so that it's easier for us to work with the data that we subscribed too. It does have CAPTCHA blocker. We are looking for someone to extract the data we need and provide it in an excel spreadsheet. Please bid and show us things you have done in the past, so we know you understand the work to be done. IT IS VERY IMPORTANT THAT THE WORK THAT IS BEING DONE, DOES NOT KICK US OFF THE SYSTEM. YOU MUST DO IT IN A WAY THAT WILL NOT CAUSE ANY RED FLAGS OR ISSUES FOR OUR ACCOUNT. WE HAVE AN ACCOUNT WITH THIS DIRECTORY AND DO NOT WANT THEM TO FLAG OUR ACCOUNT. PLEASE BID.
Skills: Web Crawler Web Crawling Web scraping
Fixed-Price - Intermediate ($$) - Est. Budget: $400 - Posted
This project involves scraping the text of 726,875 movie reviews from a variety of websites. We will provide a list of the movie reviews we are looking to get. This list contains URLs and an identifier. You will find/create the tools to automatize visiting the website URLs and scrape the relevant text on those website. The scraped review text from the website will need to be stored as a flat text file with the identifier being the file name. We have previously completed a test run with a sub-sample (~10,000 yielding 7549 files, we will provide this to the person we hire) and found the following problems that we would like you to address: 1. Paywalls: we suggest you determine which websites this concerns and what their access fees are. We will review this information and inform you on further steps. 2. Broken links or no link (up to 25% based on test sample): the above-mentioned list contains title/excerpt/source/author information. Please suggest how to search for the desired information, for instance, using google search to find an alternative source for the review we are looking for and scrape it. 3. Links not actually leading to the desired information (main page or being redirected as content is no longer available/in a different place): we are open for suggestions similar to point 2. 4. Odd encoding of the website making it hard to get the correct information: results in very small files being stored, suggestions on how to deal with this are welcome. Problems named under 1, 3, and 4 likely result in small files. In the test sample, ~1100 files had file sizes under 1 kb. When you run into other problems we expect you to inform us and work with us to create solutions where those are feasible. The person taking this job has experience with text scraping and especially doing so with a diversity of web-based source material, has a proactive attitude and is a creative problem-solver. If this is you, we look forward to your application.
Skills: Web Crawler Web Crawling Data scraping Web scraping
Fixed-Price - Intermediate ($$) - Est. Budget: $500 - Posted
This project involves locating interviews with particular film industry professionals (directors, producers and actors/actresses) from a defined list of websites/magazines/newspapers, scraping the text of each interview and storing it in a separate text file (using the following naming convention: [personID][interview number (001-xxx)].txt). You are asked to collect interviews available from a shortlist of sources though at least 10 interviews per person. The project will consist of the following steps for each list of persons: 1. Determine method of access to the intended data sources (websites/magazines/newspapers), which we will provide a list of 10 websites for. 2. Query the 10 websites for interviews with the persons on the list, examine if interview contains evidence of the interviewee being quoted (quotation marks in combination with prose, name in combination with verb indicative of speech) and scrape the interview if it meets the aforementioned criterion. 3. Supplement where necessary with top hits in a Google search (name person + interview), determine which of these are from sources not included in the list used for step 1, and execute Step 2 on the additional sources found until a sufficient number of interviews per person is reached. 4. Extract the parts from the interview text that are quotes/speech/something the person you queried for said, so taking out all other parts, and storing in a 'cleaned' text file. Lists include: 1. Directors: 380 (overlaps with producers, total for both is 722) 2. Producers: 605 (overlaps with directors, total for both is 722) 3. Actors/actresses: 713 (will overlap to some degree with list 1 and 2, we have not examined this yet) The person taking this job has experience with web crawlers and text scraping, can work with a wide range of source material for text scraping, has a proactive attitude and is a creative problem-solver. If this is you, we look forward to your application.
Skills: Web Crawler Web Crawling Data scraping Web scraping
Fixed-Price - Expert ($$$) - Est. Budget: $5 - Posted
Hi, I need to scrap a particular word from a leading newspaper of India. I need this work to be done within a week. thanks, Saakshi
Skills: Web Crawler Python
Hourly - Entry Level ($) - Est. Time: More than 6 months, 30+ hrs/week - Posted
Looking for a web data scraper who will do the extracting information of millions of products from multiple websites. We will provide the Software Application for web scraping data. Can easily retrieve competitive data like: Competitors' categories & products Product titles & descriptions Product SKUs, model number, Product images & banners
  • Number of freelancers needed: 2
Skills: Web Crawler C# VB.NET
Fixed-Price - Expert ($$$) - Est. Budget: $75 - Posted
1) We need a good content writer with knowledge in medical domain to re-write the content on our website www.medicalanimation.in 2) Content of the website should be SEO friendly. 3) we do not want to change the structure/topic of the website 4) Any other suggestions are welcome..
Skills: Web Crawler Content Writing Web design