Upwork home
Sign up
Secondary

Hire the best Scrapy Developers in Germany

Check out Scrapy Developers in Germany with the skills you need for your next job.
Get started
  • $157 hourly
    • 5.0/5
    • (242 jobs)
    A true story: Their project was dead. Thousands of dollars sunk into a system that barely worked. Hidden gaps in the data popping up everywhere. Upset partners. Angry clients. And their CEO ready to “pull the plug” on their startup after another VC pitch was botched by demoing with incomplete and low quality data. Several other scraping experts (including two former G***le data scientists) had attempted to solve the problems and failed. They were truly frustrated. In a last ditch effort to salvage their startup they shared the complete story with me. Drawing on my 15+ years experience I put their system into a high stress test environment. Ferreted out all the errors. Plugged all holes where their system was leaking data. And filled the gaps in the existing data to get it ready for the next demo ... The result? Two exhausting weeks later success was back. VC’s impressed by a demo with complete data. Clients delighted by the latest update. And the CEO asking me to become their new CTO. Well, good for me I didn’t accept her offer. I’m too much in love with being a freelancer. Good for you, too. As I’m now here to help you. I truly hope you are far from a similar situation, so contact me to make sure you really are. And let me prove that I am more than just a firefighter. At the end of it all you, too, may end up posting such a “this guy rocks” review. I can’t solve every problem. Some challenges in certain projects are just too great to overcome. And while I sometimes can do the impossible, I would rather be known for taking care of all my clients from the beginning to the end of the project, providing a smooth and painless project experience that many call the absolute best experience in working with freelancers they have ever had. Like this guy: "Lots of people wrote "this guy rocks". And that is simply true. Ruediger is extremely smart and clear and the quality is extraordinary." - a Client @upwork That's why I can guarantee results: you are satisfied or you pay nothing. Period. What I can do for you: 1. Scraping unstructured data from websites, online directories, web archives etc for Research and Data Acquisition 2. Building systems for market intelligence that scan websites continuously for changes in price, product listings, etc … in almost any market imaginable (I’ve seen and built such projects for retailers, in real estate, building equipment and some more) 3. Extracting data for Lead Enrichment and Lead Generation - improve your marketing efforts with data that’s available on the web 4. Anything else data and web automation related that comes to your mind – invite me to your project and we can discuss what can be done. You need me, if: - the data source is difficult to scrape and others have failed already - you have complex data that needs intense restructuring and cleanup - you need the data prepared and imported into database structures ranging from MySQL to complex Hadoop or Spark clusters - the data is highly unstructured and requires the use of NLP (Natural Language Processing) and Deep Learning algorithms for cleanup and restructuring Your benefits of working with me: 1. Things will run smoothly like on autopilot. You won’t get another employee-like guy who needs babysitting, micromanagement and has to be pushed to work. 2. Open communication: you will receive project updates from me at least once a week, if necessary even daily. In the rare cases when problems come up I will tell you openly and you'll never see me disappear (I'm still astounded that this is a common problem with other freelancers) 3. Quick turnaround times and reliable, complete and accurate results ensured by me using Python and the scrapy framework (that's my strong recommendation if technology can be choosen freely) plus building everything in a test driven way (I’m the author of the scrapy_tdd testing library) 4. There's little that can stop me: anti scraping mechanisms (captchas, bandwidth throttling, IP banning) are easily overcome using a network of 3000+ proxy servers, the tor network plus my lateral thinking and well grounded education in computer science 5. Messy or strangely coded data is no problem as I'll clean it using my data mining experience plus OCR automation to handle data that's hidden in images (like emails in many business directories) 6. Top Secret: If required, I'll stay stealthy while scraping for you using my network of proxies or the infamous Tor network - as a result your scraping project won't ring your competitions' alarm bells and won't give away what your plans are Or to put it into a shorter version: “Ruediger has performed an outstanding job for us. He demonstrates a great deal of professionalism and technical excellence. I strongly endorse him without reservation!” - a Client @upwork So don’t wait to contact me. Invite me today and you might be posting such a happily raving review in a few weeks, too.
    Featured Skill Scrapy
    Data Science
    Statistics
    Business with 1-9 Employees
    Selenium WebDriver
    Selenium
    Data Mining
    Data Scraping
    Python
  • $75 hourly
    • 5.0/5
    • (20 jobs)
    # About me: - Over 18 years of experience in web development (Node.js, Python, PHP). - For the last 7 years, I have specialized in automation, bot, crawler development, and web scraping # Develop web crawlers that can: - Authenticate (sign in/sign out) - Bypass anti-bot services (Adscore, Incapsula, CloudFlare, Distil Network) - Bypass image CAPTCHA, Google reCAPTCHA v2, reCAPTCHA v3 (score 0.7-0.9) - Run JavaScript, send AJAX requests and use WebSockets - Scrape any webpage data (text, attributes, links, images, tables, files) - Click buttons, checkboxes, links, dropdown menu, move mouse and scroll webpage - Save data to files (CSV, JSON, XLSX) or databases (MongoDB, MySQL, PostgreSQL) # Crawler features: - Multithreaded - Browser fingerprints rotation - Proxy IP rotation (HTTP, SOCKS) - Keeps cookies and sessions - Runs JavaScript, sends AJAX, uses WebSockets - Uses delays and retries - Error and exception logging - REST API and webhooks support - Web interface for task monitoring and error tracking - Scalable and deployable across multiple servers
    Featured Skill Scrapy
    Bot Development
    Scripts & Utilities
    Web Development
    Web Crawling
    Data Scraping
    Selenium
    Node.js
  • $20 hourly
    • 4.9/5
    • (46 jobs)
    I can help u if u wanna build a nextjs app, (backend with python), AI app , emebd openai or lanchain in your web app or local script , create a chatbot for your web page, chat over a long document , handle query operations over vectordb (pinecone, weaviate, ...) , general python script 💥NLP Tasks💥 ✅ Text Generation and classification ✅ chatbot with chatgpt3,4 (Discord, ..) ✅Chatting over a specific files ✅Langchain (Agents, indexes) ✅Handling vectordb operations like Pincone ✅ Fine Tuning an open source Model 💥Web Development 💥 ⚡ FastAPI, Flask APIs, DRF ⚡ Streamlit and gradio web apps ⚡ Fullstack app (React, nextjs, firebase) 💥Programming Skills 💥 ✅HTML, CSS , JavaScript ✅ React ✅Bootstrap ✅FastAPI ✅Flask ✅Django 💥DataBase 💥 ⚡SQL ⚡ MySQL ⚡ PostgresSQL ⚡ Mongodb ⚡ Firebase 🚩Bots 🚩⚡ 💥 Discord 💥 Telegram
    Featured Skill Scrapy
    Beautiful Soup
    JavaScript
    OpenAI Codex
    HTML5
    CSS 3
    Arduino
    React
    YOLO
    Raspberry Pi
    ChatGPT
    Django
    Selenium
    Bootstrap
    Flask
    Algorithm Development
    Computer Vision
    Python
    API
  • $5 hourly
    • 5.0/5
    • (18 jobs)
    𝐈 𝐞𝐱𝐭𝐫𝐚𝐜𝐭 𝐯𝐚𝐥𝐮𝐚𝐛𝐥𝐞 𝐝𝐚𝐭𝐚 𝐟𝐫𝐨𝐦 𝐭𝐡𝐞 𝐰𝐞𝐛 𝐭𝐨 𝐟𝐮𝐞𝐥 𝐲𝐨𝐮𝐫 𝐬𝐮𝐜𝐜𝐞𝐬𝐬 🚀 𝗖𝗼𝗺𝗽𝗲𝘁𝗶𝘁𝗼𝗿 𝗽𝗿𝗶𝗰𝗶𝗻𝗴: Uncover your competitors' pricing strategie and get up-to-date pricing information. This will help you stay competitive and make sure you're not overcharging your customers. 𝗟𝗲𝗮𝗱 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻: Identify companies seeking products or services like yours. I can scrape websites to find potential leads for your business. This will help you increase your sales and grow your business. 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗹𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗗𝗮𝘁𝗮𝘀𝗲𝘁𝘀: Build powerful machine learning models with large, custom datasets scraped from the web. If you are looking for a web scraping expert, I am the right person for the job. I can help you gather the data you need to make better decisions, grow your business, and improve your bottom line. Contact me today to learn more about my services. I have experience in: Selenium, Requests, xpath, Pandas, regular expression, json, pandas, Seaborn, flask, html, beautifull soup, proxies, multithreading, appium, mobile app scraping, Automation Testing, Data Analysis, API Development and Integration,
    Featured Skill Scrapy
    Beautiful Soup
    Selenium WebDriver
    Database
    Data Mining
    Data Scraping
    Web Scraping
    Selenium
    Python-Requests
    Python
  • $85 hourly
    • 4.7/5
    • (221 jobs)
    Hello there 😇, I'm Suheyl Ünüvar, a versatile software developer hailing from Germany. With expertise spanning languages such as C#, Python, HTML, PHP, JavaScript, and SQL, I'm adept at navigating the intricate labyrinth of API communication and a plethora of other tech skills. Fitness and self-improvement aren't just hobbies—they're philosophies that shape my work ethic. I take great pride in maintaining a high standard of professional communication with my clients, ensuring that your needs are always met with precision and punctuality. 🎯 What I Offer: - Web Scraping/Data Mining - Automation Solutions - Data Extraction & Transformation - Any Format Data Delivery - My mission is to efficiently handle these tasks for you, liberating your time and energy to concentrate on other pivotal aspects of your enterprise. 🗣 Client Testimonial: "A stellar experience overall. Suheyl is not just hardworking but also inventive, devising creative solutions for complex problems. He excels in communication and is a rising star on Upwork. Highly recommended!" - Client Feedback 🤝 Who I'm Looking For: If you're a proactive and communicative business owner who values quality and time, I'd love to discuss how my skill set can align with your vision. 💼 My Work Principles: - Full-Time Freelancer: Available up to 16 hours per day, sans weekends and holidays. - Exclusive Focus: I undertake one project at a time to ensure undivided attention. - Quality Assurance: Delivery of top-tier solutions. - Swift Execution: Because time is money. - Customer-Centric: Your satisfaction is my ultimate metric. - Professional Dialogue: Clear and precise communication at all times. 🕒 Why Wait? Skip the freelancer queue and unlock rapid, reliable results. Your project could be completed within the next 24 hours! Looking forward to your message! 😊 Best regards, Suheyl Ünüvar 🔑𝐊𝐄𝐘𝐖𝐎𝐑𝐃𝐒: Design, UX/UI, UX, UI, User Interface Design, Landing Page Design, STARTUP, CJM, E-commerce app design, Web & Mobile Design, Responsive Design, Graphics & UI Design, User Experience Design, Website, Website wireframing, Wireframe, Wireframing, Google, Material Design, UI Research, Web Design, Design, Webdesign, Webflow Design, web designer, Animations creation, WordPress design, Design Thinking, Interaction Design, eCommerce app design, Animations creation, Principle GUI design, SaaS Platform Design, WordPress Development, SaaS platform, Graphic Design, Hire, Graphics, Mobile UI/UX Design, WordPress design, pitch deck presentation design, prototype design, Branding, InVision, Marvelapp, Webflow, Figma, Illustration creation, moodboard design, Branding, logo design, icon design, Graphic Design, Animation UI, eCommerce web design, Graphic design, User Flow, Testing, Long term, Responsive, Adobe, Platform Creation, Clickable prototype, Strategy consultation, Motion Design, MockUp design, pitch deck presentation design, eCommerce web design, Material Design, UX Design, User experience design, iOS App Design, Update, Application design, Desktop, Visual Design, Mobile Design, Designer, UI/UX Design, iOS guidelines, Android A/B Application, Guidelines, iOS, Logotype, NFT, Search, Digital, Promo, Redesign, Creation, UI Design, Adaptive, Design Expert, crypto, Art Director, Management, Head of Design, Website Prototyping, iOS and Android guidelines, Logo Design, Responsive Web Design, SaaS Project Design, Solutions, Mock-up, Creative, iPhone, App Design, Export Assets from Sketch, Platforms, SaaS product, SAAS, usability audit, UX Research, Graphic Design Creation, Agency, Productive, Testing, Advertising, Usability, Google Material Development, platform design, Identity, Android, Effective, Experienced, Sketch, App, Adobe Photoshop, Adobe Illustrator, Landing Page, Design Guru, Job, website design, wireframes design, prototype designI, Android and iOS app design, Design, iPad Design, iPhone Design, User Experience, Illustrator, Infographics design, Design Thinking, FinTech, Android app design, Consultation, Infographics design, Principle, Branding Design, portfolio website, Tailwind, WordPress implementation, sleek, Web, pitch deck presentation design, Mobile UX Design, Effective, Photoshop, Mock-up..
    Featured Skill Scrapy
    Machine Learning
    Artificial Intelligence
    AI Bot
    Web Development
    Web Crawling
    Data Mining
    ETL
    Data Extraction
    pandas
    Microsoft Excel
    Data Scraping
    html2text
    Selenium
    Python
  • $50 hourly
    • 5.0/5
    • (12 jobs)
    I am a tech-agnostic problem solver with a passion for creating value through product-oriented solutions. With extensive expertise in Data Engineering and Fullstack development, I am dedicated to building fast and robust software that drive innovation and real value. I thrive in exploratory environments and my startup experience helps me in crafting solutions that are both cutting-edge and practical. You have something interesting going on? Let's work together!
    Featured Skill Scrapy
    Midjourney AI
    LLM Prompt Engineering
    Selenium WebDriver
    Amazon S3
    NoSQL Database
    AWS Fargate
    AWS Lambda
    PostgreSQL
    ChatGPT API Integration
    JavaScript
    Amazon DynamoDB
    AWS Amplify
    React
    Python
  • $100 hourly
    • 5.0/5
    • (21 jobs)
    With over several years in tech, I bring a blend of start-up experience and freelancing versatility. Specialized in Full-Stack development and DevOps, I'm proficient in several Frameworks. Ready to take on dynamic challenges and deliver solutions. AI-Expert · Python · Django · Docker · GitHub · CI/CD · Linux · React.js · FastAPI · Go · TypeScript · ElectronJS · Kubernetes · Helm
    Featured Skill Scrapy
    API Development
    Off-Page SEO
    Outreach Strategy
    Local SEO
    Search Engine Optimization
    German
    Django
    Bot Development
    Flask
    Python
  • $20 hourly
    • 5.0/5
    • (3 jobs)
    I am a Senior Software Engineer living abroad. Always looking for new challenges and technologies to learn. My main focus is the backend aspect of the systems but whenever there is the need to solve frontend issues I am flexible enough to jump in. My main technology is Java but I can also develop on C, PL/SQL, JavaScript and Python. I am a well organised developer who will try to break the big problems into smaller to make it simpler to resolve.
    Featured Skill Scrapy
    Web Scraping
    NodeJS Framework
    Oracle PLSQL
    Spring Framework
    Web Crawling
    Data Scraping
    Python
    API
    CI/CD
    Java
    Spring Boot
    SQL
    PostgreSQL
  • $28 hourly
    • 5.0/5
    • (3 jobs)
    I like Python and PHP Javascript. I working with php - 15 years, python 5 years, vue js 5 years. Projects on php CMS Bitrix, Wordpress, project on python Django, Flask, FastApi. Create spyders and scrapy data, Speed Optimization & General Performance Improvements
    Featured Skill Scrapy
    Beautiful Soup
    Joomla
    PHP
    Django
    Python
    Flask
    WordPress
    MySQL
    HTML5
    JavaScript
  • $80 hourly
    • 0.0/5
    • (1 job)
    Extended expertise in cloud network engineering and infrastructures with a comprehensive understanding of advanced mobile core networks, and an updated outlook on relevant informatics and telecoms businesses. Motivated by a passion for cutting-edge technologies and satisfying customer experience.
    Featured Skill Scrapy
    Automation
    Web Crawler
    Network Planning
    API
    Data Collection
    Network Design
    Cloud Computing
    Data Engineering
    Machine Learning
    Data Cleaning
    Network Engineering
    Unity
    C#
    Python
  • $40 hourly
    • 0.0/5
    • (0 jobs)
    Passionate Data Scientist/Machine Learning Engineer with three years of experience managing and delivering Machine Learning projects to drive positive changes in organisations.
    Featured Skill Scrapy
    Beautiful Soup
    Web Scraping
    Bot Development
    CV
    Back-End Development
    Front-End Development
    Full-Stack Development
    Selenium
    GPT-4
    ChatGPT
    GPT-3
    BERT
    Machine Learning
    Natural Language Processing
  • $45 hourly
    • 0.0/5
    • (0 jobs)
    I specialize in designing and implementing robust, scalable solutions that streamline business processes and enhance efficiency. My expertise includes developing sophisticated automation workflows for tasks such as web scraping, bulk emailing, and network packet analysis. With a strong focus on delivering high-performance, scalable applications, I am adept at transforming complex challenges into automated solutions that drive productivity and innovation.
    Featured Skill Scrapy
    Browser Extension
    Google Chrome Extension
    Automated Workflow
    Reverse Engineering
    C++
    Email Management
    Network Analysis
    Automation
    Web Scraping
    Bulk Marketing
    Email Campaign
    Wireshark
    Selenium
    Python
  • $15 hourly
    • 0.0/5
    • (0 jobs)
    Need data from any website? I can help! I specialize in web scraping and automation, extracting valuable data using Python, BeautifulSoup, Scrapy, and Selenium. Whether you need to scrape job listings, product prices, or large-scale business data, I ensure accuracy, speed, and anti-bot bypassing. What I Offer: ✅ Web scraping from any website ✅ Data cleaning & structuring (CSV, JSON, Excel etc.) ✅ Regular communication to fulfill your need Let’s automate your data collection and save you time!
    Featured Skill Scrapy
    Beautiful Soup
    Python
    Selenium
    Web Scraping
    Mining
    Artificial Intelligence
    Data Extraction
    Machine Learning
    Data Mining
  • $25 hourly
    • 0.0/5
    • (0 jobs)
    I specialize in the web and desktop applications development. I love data scraping, API designing, back-end, database engineering, security problem solving and UNIX administration. But also do not mind of a front-end tasks. Most of my applications are designed for data scraping/processing and other automatic tasks in the Internet with avoiding of any type of protection(CAPTCHA, IP blocking, authentication etc.). Technologies familiar to me: -Python, Flask and Flask extensions; -Scrapy, requests, beautifulsoup4, asyncio, aiohttp, selenium for data scraping; -jQuery or pure Javascript, AJAX, HTML5, CSS3, Bootstrap and other front-end stuff; -PostgreSQL, MongoDB, SQLAlchemy, MySQL, SQLite3; -OAuth 2, Gmail API, Telegram API and other integration; You can see some of my projects here in portfolio.
    Featured Skill Scrapy
    API Integration
    Automation
    Puppeteer
    Selenium
    Beautiful Soup
    Web Scraping
    JavaScript
    RESTful API
    AWS Lambda
    Flask
    Python
    ExpressJS
    NestJS
    Node.js
  • $30 hourly
    • 4.5/5
    • (34 jobs)
    Greetings, As a seasoned Full Stack Developer and Senior Software Engineer, I specialize in building robust, scalable, and efficient software solutions that deliver real-world impact. With over seven years of experience, I've worked with cutting-edge technologies to create dynamic, high-performance applications across industries such as e-commerce, digital twins, and automation tools. My dedication to technical excellence and delivering measurable results makes me a valuable asset to any project. Technologies I Excel In - Backend: Node.js, Python (Flask, FastAPI), .NET Core - Frontend: Angular, Vue.js, React, TypeScript, JavaScript - Databases: MongoDB, MySQL, PostgreSQL, Redis, ElasticSearch - Cloud Platforms: AWS, Azure, GCP - DevOps: Docker, Kubernetes, Jenkins, GitHub Actions, CI/CD Pipelines - Real-time Systems: Socket.IO, Kafka, Firebase - Testing & Automation: Playwright, Cypress, Selenium - Web Scraping: Scrapy, Puppeteer, Cheerio Key Contributions - Digital Twin Platform for Kongsberg Digital: Led the development of features such as real-time calculation dashboards and chart components for the KogniTwin project, empowering clients in oil & gas to optimize operations. - BMW CLEA Data Insights: Enhanced cloud dataset management across AWS and Azure, developing dynamic views on AWS QuickSight and automating CI/CD pipelines with Node.js and Python for efficient deployments. - Scalable E-commerce Automation with Tree3: Built a data-mining solution that crawled millions of products daily from 1,000+ websites, leveraging Node.js, Python, and Redis for high efficiency and scalability. - Evolutionary Prediction Engine for MelonFrost: Integrated machine learning algorithms with real-time data streams using Python and Kafka, enabling advanced prediction capabilities for bioreactor clusters. - Automation with Krakbot: Developed a Robotic Processing Automation Tool to streamline repetitive engineering tasks for MSPs, utilizing Angular, TypeScript, and AWS EKS. Why Choose Me? - Proven Impact: My work has consistently driven success for clients, from improving QA efficiency by 30% with automated test cases to scaling e-commerce platforms for millions of users. - Technical Versatility: Whether it's microservices, real-time data systems, or CI/CD pipelines, I have the expertise to handle complex requirements with ease. - Collaboration Focused: I thrive in team environments, collaborating with cross-functional stakeholders to deliver solutions that align with business goals. - Commitment to Excellence: I’m passionate about delivering high-quality software, adhering to best practices, and continuously improving systems for long-term success. I am ready to bring my expertise, adaptability, and passion for impactful projects to help you achieve your goals. Let’s connect and create something amazing!
    Featured Skill Scrapy
    JavaScript
    CSS
    Angular
    Amazon Web Services
    Kubernetes
    Spring Boot
    Node.js
    React
    HTML
    Docker
    Web Development
    .NET Core
    Vue.js
    Python
    Data Mining
    Data Scraping
  • $25 hourly
    • 0.0/5
    • (2 jobs)
    Hi, Jaydeep Here, As a full-stack developer, I have experience with various programming languages, frameworks, and tools including Python, React, Django, TensorFlow, MongoDB, and Git. I mainly work with Machine learning and AI related field. I do have good scripting skills as well. Check out my work on Github. Don’t hesitate to reach out me.
    Featured Skill Scrapy
    Selenium
    JavaScript
    Machine Learning
    TensorFlow
    Python
    Deep Learning
  • Want to browse more freelancers?
    Sign up

How hiring on Upwork works

1. Post a job

Tell us what you need. Provide as many details as possible, but don’t worry about getting it perfect.

2. Talent comes to you

Get qualified proposals within 24 hours, and meet the candidates you’re excited about. Hire as soon as you’re ready.

3. Collaborate easily

Use Upwork to chat or video call, share files, and track project progress right from the app.

4. Payment simplified

Receive invoices and make payments through Upwork. Only pay for work you authorize.

Top cities for Scrapy Developers in Germany
Show more