You've landed at the right place. oDesk is now Upwork. Learn about the new platform.

Scrapy Framework Jobs

18 were found based on your criteria

show all
show all
only
only
only
show all
only
only
only
only
only
show all
only
only
only
Hourly - Est. Time: 1 to 3 months, 30+ hrs/week - Posted
Hi, I need help in developing a Content Management System (CMS) based in Python, Django and Scrapy. The project has already started and we will need help in pushing it forward and improving it. Most of the data models already exist, and the project is already connected to Scrapy. We will need to improve the work with Scrapy to be able to: 1) scrape single recipes from various blogs by the click of a button 2) automatically add tags to different forms of content based on a pre-defined logic that will be included in one of the data models of the CMS 3) include new data models to be used in the future add functionality to automatically analyze the content using Natural-Language Processing (using our own library, so this will involve connecting the CMS to an existing library). 4) connect the CMS to batch scraping via Scrapy and Celery to keep the data up-to-date. 5) have a user-management system with various forms of credentials 6) more features will be developed in the future. I'm...
Fixed-Price - Est. Budget: $ 100 Posted
I need millions of data from a website. This is a Brazilian website: http://www.brasyp.com/browse-business-directory I need all the companies, web urls, and categories the company belong to... It has 899,069 Companies & 97780 web address ***I need it in a SQL file format. This website is quite sensitive & blocks IP. So proxy rotation might be needed for your task. 1. I need to know how long it will take for you to this work? 2. If you able do it what will be the total costing? I will appreciate your response with Sample. You can provide me these data or You can provide a powerful script/solution for me so that I can do this from myself. I have plenty of work later on if anyone can do this real quick using the multiple machines/power force. please let me know Regards Hassan R. Dhaka, Bangladesh
Fixed-Price - Est. Budget: $ 100 Posted
I need someone who could create a web based application (application will be hosted on a digital ocean droplet specifically created for this purpose) which wil be capable of extracting data from product page of taoao.com and tmall.com I am giving sample links , you may have a look https://item.taobao.com/item.htm?id=45428232575&spm=2014.21379799.0.0 https://detail.tmall.com/item.htm?id=43944007169&spm=2014.21379799.0.0 We will be supplying thousands of such URLs in each run and the scraper should be capable of going to all these URLs and extracting the data. The scraped data will be feed into a database hosted on the same server. Each individual job should go to one database table. Separate database table will be formed for separate jobs. There should be a simple UI to be able to download the database table as well as csv format of each table. Also a facility for renaming,deleting or adding new database table should be there. There should be a functionality to schedule the scraping...
Hourly - Est. Time: Less than 1 week, 30+ hrs/week - Posted
1, it can scrape about 100M products info daily or weekly. Each product have 300-1K bytes. If it is hard, scraping 10M product daily or weekly also ok. 2, so it seem a distributed crawl is a must. 3, use proxies to save money and can detect IP ban and rotation or slow scraping to avoid IP ban. 4, I found scrapy cluster project is great buy not try. At moment, scrapy(or custom crawl) + redis is a option. 5, save data to mysql or sql server or even oracle to query products.
Fixed-Price - Est. Budget: $ 8 Posted
I want to use proxy with scrapy but I seem to be getting error. If anyone knows how to use proxy with scrapy and able to explain how to use it, that would be great !
Skills: Scrapy
Hourly - Est. Time: Less than 1 month, 10-30 hrs/week - Posted
Two jobs: 1) We have a (nearly complete) python web scraping program for facebook and need your help finishing. 2) We have an online directory and can only access 150 contacts daily. We need to run a daily script to collect the information of 150 local contacts each day.
Fixed-Price - Est. Budget: $ 300 Posted
I need someone who can create a no-hassle software that can scrape data from different classified ad websites effectively and thousands of data at a time. The software must be able to rotate IP addresses or work in conjunction with VPN software. Security is an issue and I need to gather this data effectively, for a large research project I am doing. I would plan to use this tool once a week. I am interested in scraping and from there I would like the scraper to have a function to import all the data into excel. I'd like to get all the text into the file. Thanks!
Hourly - Est. Time: Less than 1 week, Less than 10 hrs/week - Posted
Hi We need someone able to build a script to programmatically help us use our linkedin with bing, to improve profile management. More details of the required script to the successful candidates. Successful candidates MUST have very good knowledge of Scrapy, and previous experience using it with Bing and Linkedin. Regards
Skills: Scrapy