Scrapy Framework Jobs

40 were found based on your criteria {{ paging.total|number:0 }} were found based on your criteria

show all
  • Hourly ({{ jobTypeController.getFacetCount("0")|number:0}})
  • Fixed Price ({{ jobTypeController.getFacetCount("1")|number:0}})
Hourly - Entry Level ($) - Est. Time: More than 6 months, 30+ hrs/week - Posted
Hey guys, thanks for checking our posting! We are an SEO team heavily focused on tech and data here in Odessa Ukraine, currently with a team of about 10. Our parent company potential.co has offices internationally including one in Bali Indonesia. We believe in making work fun and will be taking the current team to Bali, for 2 months of work-cation. We pay for all accommodations, rent scooters and plan some fun activities. This experience is available to all long term employees and tentatively we are planning them twice per year. Of course this is only an option and not a requirement for work. We are expanding out tech team quickly to do various projects: Data scraping for infographics: https://www.dealsunny.com/mobile-usage-statistics Automation of our marketing process - example email outreach etc. Web Analysis: Drawing various insights about the google search algorithm from our own data driven research and development. We are looking for people with technical expertise that also enjoy a fun and free culture. We assume you will likely not live in Odessa (but if so great, you can work with the team in office), if not then we will manage everything remotely. English is a plus, however our team can manage if your english isn't the best. We will do a 2 week paid trial period with some small tasks managed by our wonderful scrum master Alina. The majority of work will be done in python If you have any experience in php or wordpress databases this will be helpful for some small projects. Also any front-end technologies are greatly appreciated and always useful for data visualizations in the realm of web content. If you know anything about SEO, SMM as well please list that in the response :D Thanks all for your time! Looking forward to discussing the opportunity with you. Best, Josh
Skills: Scrapy Python
Fixed-Price - Intermediate ($$) - Est. Budget: $250 - Posted
General idea is to run crawler for previously added domains and collect all external links and save them into mysql database. This process needs to be repeatable for given or new domains on daily/weekly basis. Crawler needs to have a simple management panel based on known framework xcrud - to see all results and manage crawler options. Project Requirements Simple panel management to see and control crawling process Crawler needs to collect external links, has to have an option to collect all external links or links which are containing particular domains names taken from previously created list Save all links to mysql database Detect if site is not responding, if so, save this event into database Add / remove / edit new domains to crawl Run / stop crawler from management panel Able to start task form the last failure Able to see actual crawling process Able to send email notifications if crawler will suddenly stop Able to add domain to the domains list by url: http://mycrawler.com/add_domain.php?domain_name=www.newdomain.com (at this point only domain adding is required) - script should return result (added or display error if occurred ) Xcrud framework will be provided. Dev space will be provided. More details will be provided if you are interested in.
Skills: Scrapy MySQL Programming PHP Python