Web Crawler Jobs

69 were found based on your criteria {{ paging.total|number:0 }} were found based on your criteria

show all
  • Hourly ({{ jobTypeController.getFacetCount("0")|number:0}})
  • Fixed Price ({{ jobTypeController.getFacetCount("1")|number:0}})
Fixed-Price - Intermediate ($$) - Est. Budget: $300 - Posted
I need a Python script that will we are looking for someone that has experiences in Web \ Data Scraping. We are looking to extract certain type of info from about 26 sites. We are only looking for the last 90 days of data and be able archive what we choose for later. On the 91th day have it go into an archive section that we can search for later. The 2nd part would be pulling the contact info based off of the first half of scraping. We would need all the data in a CSV file or some type of online dashboard that we can review the info. We are looking for updated result daily or as soon as it available on the site. We will also need to have this code hosted on our sever (DIGITALOCEAN). We will also need an interface can we have a section where we can add/delete secured parties for the scraper. We have a site that can be searched or scarped to get the business owner name and number. Must have proxy to hide our IP / CAPTCHA blocker
Skills: Web Crawler Web Crawling Data mining Digital Ocean
Fixed-Price - Entry Level ($) - Est. Budget: $70 - Posted
Web/Data Social media post/metadata scraping, data mining specialist to build a Search Software on Desktop. The application will connect to the internet and gather information on a very specific Social Media site. This is just a kick-start. If the project is done excellently, the project and the chosen developer will be considered for the next big moves.
Skills: Web Crawler Data mining Desktop Applications Web scraping
Fixed-Price - Expert ($$$) - Est. Budget: $1,000 - Posted
Need to build a web crawler either in Python or Scala: 1. It should be Multi-threaded 2. Crawl Regular webpages 3. CrawlRSS Feeds 4. Should be able to Stop/Pause/Resume Crawl 5. Set Recrawl Periodicity 6. Define File type inclusion/exclusion 7. Javascript Links detection 8. Respect Robots.txt This project is to create the framework and one site script.
Skills: Web Crawler
Fixed-Price - Expert ($$$) - Est. Budget: $10,000 - Posted
Looking for a lead development team. Trying to find student housing locations in the U.S. and would be purchasing verified leads at a negotiated price per lead. Data for leads: - Property Name - Verified Phone - Primary Contact - Website - Verified Email - Address - Verified Student Housing Additional Questions: -Do you offer Individual leasing by the bed space? -How many beds/students lease at the property? -Who is the Management Group? -What software are you using for accounting? -How do you lease? School year, by semester, is it a bulk move in move out situation? -Do you offer roommate matching? Looking for 1,000 leads a month. All leads verified on the first 7 items before being accepted. We have lots of ideas but feel a group focussed on this would create a great partner.
Skills: Web Crawler Data Entry Data mining Lead generation
Fixed Price Budget - Intermediate ($$) - $100 to $200 - Posted
We want a program that scrapes/extract data from various websites. The program will go to a website and then put various inquires and search. Next it will look at the used price of the book. If the price of the book is greater than $40 then it should click on the link and extract the ISBN number. If it is less than $40 then the program should move on to the next book. It will continue this process until it reaches the end of the books/searches and put the information in an excel file. You will need to use a proxy list to continuously change the "location" of the program. The program will then use the ISBN numbers (in the excel file) to go to a website and use the ISBN numbers one by one to search for the prices. Next, the program will look at the lowest price and check the merchant/website. The program should compare the price for new and used. If new is cheaper then select the price based on the rating as well. However if used is cheaper then, the program should check the condition of the lowest price book. If the condition is "brand new, new, like new, mint, very good, or good" then the program should move on to the seller's comments available. (If the condition does not match then it should move on to the next result). However when the program is extracting the comments it should look for specific words in the description and if they are present then the program should move on to the next search result and repeat the process. If the next one is good (does not contain "blacklisted words") then the program should extract the information (price, condition, and the seller's comments) and then put the information in the same excel file as the ISBN numbers. Lastly, we will specify another website to scrap the sell price for the same ISBN. This is the whole process but the program interface should be made as follows: The program should be extremely simple. It should consist an option to search all subjects (biology, accounting, etc). We will provide you with a list of subjects for the program to search and also a text file with the "blacklisted words". The program will put the information from each of the subjects into the previously mentioned excel file. The program will then put the extracted information in individual tabs of the excel file. For each of the subjects the program will extract all of the ISBN numbers from the 1st site and the information for each of the ISBN numbers. The program will also consist of a button to run the program. I understand that the description seems complex but it is much simpler than it sounds. Please feel free to ask any questions you may have. I will provide additional information on the specifics of the program once you apply. We had been working with the programmer who has not been answering on a consistent basis and I will provide you with the old program when we hire you. Please note that this is a Perl based program but will need a few changes so you can use this as a guide to build a better program. A second condition is that you must be extremely responsive in the future. The program will require updates in the future and we will pay you for each update but if you take more than 48 hours to respond then we will hire someone else.
Skills: Web Crawler Web scraping
Fixed-Price - Entry Level ($) - Est. Budget: $15 - Posted
I need someone to extract large amounts of data from websites whereby the data is extracted and saved to a local file in your computer or to a database in table (spreadsheet) format. Please submit some of your past work experiences. Website http://wwwaliexpress.xyz/ Excel file of data that needs to be attracted attached. Please start scraping in the watch category. Please no discount prices. Please provide detailed categories. plain description, description with html like examples. parent and child variants. Please extract each variant with matching image url and with price. Please remove all No Longer Available products. Please extract biggest urls, if possible longest side 1001. Please provide if Category is from Mens, Womens or Children,etc. Please provide minimum order field. Please start scraping in the Watch Category. Price is Negotiable. Bonus if you can create UPC or Barcodes. In addition looking for someone who specializes in Pinterest Marketing. We needs to see fast results.
Skills: Web Crawler Administrative Support Amazon Web Services Bootstrap
Fixed-Price - Intermediate ($$) - Est. Budget: $50 - Posted
We need several data scrapers to get data from websites. We need the code and the result of the downloads. These websites are some of the most well known ecommerce websites. Technology to use is Scrapy with Python. Further details are explained via messages.
Skills: Web Crawler Data scraping Python Scrapy
Fixed-Price - Intermediate ($$) - Est. Budget: $60 - Posted
Seeking someone who can crawl and scrape the web to provide phone numbers and email addresses. I can provide some details (first name, last name and addresses) but need someone who can use that data to find the associated phone numbers and emails. I attached an excel sample of the data I can provide, and highlited the cells that need to be filled. Please let me know if you are capable of crawling/scraping the internet to fill in the blanks in this sample. If so, would like an estimate of how much it would cost to fill in the blanks for 1000 - 1200 names.
Skills: Web Crawler Web Crawling Data scraping Web scraping
Fixed-Price - Intermediate ($$) - Est. Budget: $50 - Posted
Simple web scraping or manual data entry: - visit http://www.thecampsite.org/directory - there are 789 contacts listed in the directory and, for each, we want you to grab: a) Membership level b) First name c) Last name d) Company/Organization e) Occupation f) Email g) Business Phone h) Chapter North The output is a Google Sheet with those data for all 789 contacts. This must be completed by 10am California time on Monday, May 9.
Skills: Web Crawler Data Entry Internet research Scrapy