Web Crawler Jobs

59 were found based on your criteria {{ paging.total|number:0 }} were found based on your criteria

show all
  • Hourly ({{ jobTypeController.getFacetCount("0")|number:0}})
  • Fixed Price ({{ jobTypeController.getFacetCount("1")|number:0}})
Hourly - Entry Level ($) - Est. Time: Less than 1 week, 10-30 hrs/week - Posted
We are looking for a software developer to create a webscraping script/program that checks if a website is responsive or not. The goal to check a large list of websites (>1000 /day) if there are responsive and mobile-responsive-friendly. The application/script should work either on windows or linux and can be programmed as a script (e.g. python) or windows application (e.g. c#). Also a RESTful service can be called to check this. Any ideas for an SaaS solution are welcome, although we prefer an inhouse-solution for this problem. The data should be stored either a Excel file, CSV, XML, MongoDB, MySQL, MS SQL Server. What flavor you prefer. Datasource can also be one of those sources or CLI argument. The process should be executed manually. Required skills: 1) Experienced developer (>3 years) 2) Good with crawling / webscraping 3) Good verbal and written English skill If you believe you absolutely have all of the above, please take some time to provide the followings with your cover letter: 1. Your resume 2. Your hourly rate 3. Your availability (how many hours per week and how early you can start) 4. An estimated time effort 5. Years of professional experience This position has the potential to get more web scraping projects in the future.
Skills: Web Crawler C# Data scraping Web scraping
Fixed-Price - Entry Level ($) - Est. Budget: $10 - Posted
Here is the youtube channel https://www.youtube.com/user/SACRvideos/videos I want to know the titles of the songs used in all the videos. I already tried but no tool could identify them properly, they are not well known. Before applying, try to identify at least one of the songs. If you are able to find one then apply to the job and mention the title.
Skills: Web Crawler Audio Editing "Extract, Transform and Load (ETL)" Google search
Fixed-Price - Intermediate ($$) - Est. Budget: $250 - Posted
Hi, Need youtube-dl extractor to be made for few sites such as ErosNow.com Looking for someone who has done this before. I want to do this for 1 site first, this contract is for that. If this is successful, I have 6 other sites in the pipeline that I want this done for.
Skills: Web Crawler MongoDB Web scraping
Fixed-Price - Expert ($$$) - Est. Budget: $5 - Posted
We are looking for an experienced programmer (parsehub or similar) to help us with extraction of hotel IDs. Step 1: load & use a master table provided by us with "hotel name", "city", "country", etc. for >100.000 hotels - replace "country" values that are equal to "United States" with "USA" Step 2: take each hotel and auto-enter the data (including city & country) into www.trivago.com's search field Step 3: Extract the hotel ID from the trivago-URL in case there is a positive match (=the first suggestion Trivago gives in its dropdown) or note that there was no exact match (=trivago indicates that this hotel cannot be found in the DB). Also extract the hotel-name exactly how trivago calls it (sometimes hotel names are slightly differently called!) Step 4: Use the extracted hotel names from Trivago (if there was a match, otherwise use the initial data provided by us) and auto-enter them with city into www.hotels.com and into www.booking.com and extract the respective hotel ID from their URLs & the hotel name Step 5: Collate a google sheet with columns: a) hotel name (original) b) hotel city (original) c) hotel country (original) d) hotel name (as of trivago) e) hotel city (as of trivago) f) hotel country (as of trivago) g) hotel ID from trivago-URL h) hotel name (as of hotel.com) i) hotel ID from hotels.com-URL j) hotel name (as of booking.com) k) hotel ID from booking.com-URL l) Status (OK = positive match and all IDs found // CHECK = no exact match of name // PARTLY MISSING = found on trivago but not on hotels.com or booking.com) Crawling task need to be run once but we will most likely need a regular paid update Please quote fix price and ask before in case you have questions! Budget depends on offfers Thanks Nathan
Skills: Web Crawler Web Crawling Data scraping "Extract, Transform and Load (ETL)"
Hourly - Expert ($$$) - Est. Time: 3 to 6 months, 30+ hrs/week - Posted
I need to do the following: 1. SMTP verify emails: https://blog.kickbox.io/what-is-email-verification/ 2. Be able to scale this out to hundreds of requests per minute in a web app. 3. Solve the problem that your IP address gets banned if you verify from the same location over and over :) In your proposal, please tell me: 1. how you would solve this problem. 2. What language you'd use 3. How you would scale this out. 4. Address the issue of IP address getting banned.
Skills: Web Crawler Application Programming haproxy Java
Fixed-Price - Intermediate ($$) - Est. Budget: $50 - Posted
Looking for excellent workers with experience in extracting large amounts of data from websites whereby the data is extracted and saved to a local file in your computer or to a database in table (spreadsheet) format. Pay is negotiable. Please tell us your fixed fee. We will not pay hourly wages for this job. Please prepare to write a code or script to collect some product details from website to confirm you can do the work. After you submit your application. We will give you website and category and template. Then you must add the product using automation. Seeking hard workers, dedicated, efficient, reliable, energetic, professional & motivated. We are looking for freelancers that have experience in Data Entry, Data Mining, Data Science, Data Scraping, Excel VBA, Microsoft Excel, Perl, Photo Editing, PhotoScape, PHP, Product management, Web Scraping, Data Scraping, Data Cleansing, Data Mining, Data Entry, Web / Internet Research MS Excel / Word Template Development, Database Administration, Document Processing, Administrative support. Please see attachment and submit some test results, at least l000 or more with variants. Please extract data from Aliexpress.com or Doba.com Save excel file as your name, store name or category. Submit file and We will confirm if test results are accurate. If test results are accurate we will confirm your fixed fee, delivery times and quantity Fields not Required A, D, O, P, R, S, U, V, W, X, Z and AA thru AS Fields Required Handle (Same as Title, No Dashes Requred) Title (Product Name) Short SEO Friendly Tile Body (Descriptions or Item Specifics) Variant SKU (you can develop or create) Type (you can find in sub sub sub sub categories) Tags (seo friendly product tags) Variant Price (No discount Prices) Please add one price per product Variant Inventory Qty (Stock) Image Src (Main Product Image Url) Image Alt Text (Add The Long Product Name) Option1 Name Option1 Value if possible image url should match colors. Color red blue white Option2 Name Option2 Value if possible size chart image url add a new field Size small meduim large Please add one price per product. Some sies have different prices. Gender (Female, Male or Unisex) Age Group ('newborn' infant' 'toddler' kids' 'adult')
Skills: Web Crawler Automation Data Entry Data mining
Hourly - Entry Level ($) - Est. Time: 1 to 3 months, 10-30 hrs/week - Posted
I want to build a web crawler which can crawl news articles and number of unique visitors and team member information (contact points) of the certain sites. For example, I have a database of "Alibaba" and I want to crawl all the related news in the world and want to know the weekly Unique visitors and team member information of "Alibaba"
Skills: Web Crawler Data mining Django Python