Scrapy Framework Jobs

35 were found based on your criteria {{|number:0 }} were found based on your criteria

show all
  • Hourly ({{ jobTypeController.getFacetCount("0")|number:0}})
  • Fixed Price ({{ jobTypeController.getFacetCount("1")|number:0}})
Fixed-Price - Expert ($$$) - Est. Budget: $5 - Posted
NOTE ABOUT BUDGET We don't know how much this will cost. Please give us your fixed bid. PROJECT OVERVIEW Write an application that will scrape information from profiles on a major social media website. (Site will be revealed to the freelancer who gets the job.) PROJECT REQUIREMENTS Your application must be written in a popular scripting language like Python or Ruby using standard libraries. It must be designed to run on a Unix platform. We prefer Python using the Scrapy library, but you may choose another language with our approval. PROJECT DETAILS See attached flowchart for a diagram of execution flow. The Scraper will be run from the command line by one User. The User will enter their site username and password into a configuration file. The User will set up search criteria on the Site using their web browser. Once the User has obtained the first page of search results, the User will enter that Result Page URL into the Scraper. There may be a queue of multiple Result Page URLs. Once a day, the Scraper will begin with the next Result Page URL in the queue and scrape information from all the profiles on that page. The Scraper will then advance to the next page of results and continue scraping until the maximum number of scrapes for the day has been met. Inbetween each profile scrape, the Scraper should pause for a random number of seconds. The pause between scrapes, as well as limiting the number of scrapes per day, is to prevent the Site from identifying the Scraper as a bot. The Scraper should output results to a CSV file.
Skills: Scrapy Python Web scraping
Fixed-Price - Intermediate ($$) - Est. Budget: $100 - Posted
I need someone to build a Excel vba based web crawler to extract large amounts of data from websites. Someone that has work experience of more than 3 years creating web crawlers/ scripts to extract data big websites and solve any security issues. Please see attached file for format needed. Site aliexpress . com ioffer . com etc.
Skills: Scrapy Data mining Data Science Data scraping
Hourly - Expert ($$$) - Est. Time: Less than 1 month, 10-30 hrs/week - Posted
We are looking for a Scrapy expert to help scrape 8 major retail sites (e.g., Amazon, Home Depot). We are a well-funded startup that is collecting and connecting data on retail household chemical products. This should be a straightforward job for any and all Scrapy experts. At first this will be a short-term job (less than 30 days). But if we work well together this could expand into a longer commitment, which we think would be an exciting opportunity for the right developer. You'll get the chance to work with a skilled and experienced team that includes someone who was named one of Computerworld's Top 100 Technology Leaders leveraging transformative technology! Please write a brief cover letter that covers your experience with Scrapy. Once you reach out to me, I'll provide more info on our data strategy and we can take it from there. Looking forward to hearing from you!
Skills: Scrapy
Hourly - Expert ($$$) - Est. Time: Less than 1 month, 10-30 hrs/week - Posted
We need to 1. Scrape and download images locally 2. Scrape and save images data (like alt text, page title) in a csv file We have a list of 30 sites so far and more to come. The sites are quite big with lots of images but the structure of them is very easy and 99% don't have any anti-scrape systems. We are looking for someone who can take care of them nice and easy, we can pay a low amount for each site given the fact this is bulk work and they're easy.
Skills: Scrapy Web Crawling Data scraping Python
Fixed-Price - Intermediate ($$) - Est. Budget: $50 - Posted
Hey Guys, I want text file of all the [StudentEmail, FullName \n ] for all universities with greaterthan 20,000 current student enrollment. List should read: Email, FullName \n StudentEmail, FullName \n StudentEmail, FullName \n StudentEmail, FullName \n ... List should have 20,000+ entries. Bonus for 45,000+ emails and names I want the text file and the source code for your scraper, I will run the scraper to verify the list is accurate. I will pay a fixed amount per scraper and list you provide me, and I am open to negotiation. It takes me about a day to build my own, but I need to collect these faster than I can work alone. You are free to use whatever tools you have in your knowledge base, but I must be able to verify the list for its accuracy. Hints: Some directories are difficult to tackle but I have found these tips help, and can get a good number. I have found that using python, selenium, and Beautifulsoup work really well as you can navigate the public directories of universities. Some are harder than others. I have found that some directories limit the results, but allow you to search names Search first names ‘aa’, ‘bb’, ‘cc’, … Search last names ‘aa’, ‘bb’, ‘cc’, … Be creative!!! and find a way that works… You will get duplicates performing the above technique, be sure you purge your list for duplicates as they do not count towards your 20,000+ entries. *** If you happen to be a student or have access to a student’s internal account, currently enrolled at a university that uses Enterprise Gmail for that universities email, then you are in luck, as I have a scraper that works for you… I will send you the scraper upon request; Just type in your info, run it, and send me the results. *** python, java, scraper, scraper box,, screen scraper IF YOU WANT TO START: Reference this list for the largest universities by enrollment: DO NOT SCRAPE the universities on this spreadsheet: Payment: MileStone 1: I will set a milestone for half what we negotiate, to receive that you must show me that the program works via Skype screenshare, and the completed list. I will pay in full after you send it to me.
Skills: Scrapy Data scraping scrapebox Web scraping
Fixed-Price - Intermediate ($$) - Est. Budget: $20 - Posted
Hello, I need a web developper to make me a small script that will: - scrap 12 fields of data on an HTML page for each of the urls (using xpath for example) - allow me to use proxies - give a .csv result It's better if you write that with Scrapy (Python) but i'm open to other languages as long as it's easy to run for me (so please nothing that will require complex server setups) As you'll see on my profile, i know what i want, i'm fast to answer and loyal to the good freelances. Let's make some good work. Yann
Skills: Scrapy Python Web scraping
Hourly - Entry Level ($) - Est. Time: 1 to 3 months, Less than 10 hrs/week - Posted
For automated data mining from social media and content posting. We need a junior developer with experience in: - Selenium - must - PhantomJS - must - Python - must - Splash - nice to have - Lua - nice to have Must have good communication skills and reliability. The first task is a paid "trial" task to post content specified in son into a Facebook group.
Skills: Scrapy PhantomJS Python Selenium