Ahmad isn't taking new orders for this project right now. Here are some similar projects to explore.
You will get a custom Python script to fetch and structure API responses
Project details
You will receive structured, normalized, and validated data with consistent formatting and high data integrity. The output is ready for direct database storage or AI-agent processing.
Data Tool
PythonWhat's included
| Service Tiers |
Starter
$50
|
Standard
$150
|
Advanced
$500
|
|---|---|---|---|
| Delivery Time | 1 day | 3 days | 5 days |
Number of Revisions | 1 | 1 | 2 |
Frequently asked questions
7 reviews
(7)
(0)
(0)
(0)
(0)
This project doesn't have any reviews.
SR
Simon R.
Jul 27, 2026
banks project
Great experience with Ahman once again. A very good developer.
SR
Simon R.
Jun 2, 2026
scraping project
SR
Simon R.
Mar 17, 2026
tests websites
Good job on our scraping project done by Ahmad.
RS
Rehan S.
Mar 6, 2026
Automation Script Development for Invoice Downloading Using Playwright
ME
Mahmoud E.
Oct 31, 2025
Web scraping
About Ahmad
Web Scraping / Web Crawling / Data Mining / Browser Automation Expert
100%
Job Success
Tanta, Egypt - 11:45 am local time
I build the systems that don't do that.
Over the past 5+ years, I've built production data pipelines, backend services, and automation systems that process millions of records reliably. My work goes beyond extracting data—I design the infrastructure around it, including task queues, schedulers, APIs, databases, authentication flows, and monitoring, so the entire pipeline keeps running long after deployment.
One of my largest projects powers a pharmaceutical data provider, ingesting millions of products across 30+ vendors on a recurring schedule. The pipeline validates every run with automated quality gates that detect silent failures before corrupted data reaches downstream systems or warehouses.
## Real production problems I've solved
### Pharmaceutical Catalog Ingestion (NDA)
The existing scraper relied on HTML selectors and frontend pagination. During investigation I discovered the site silently returned corrupted data whenever a cookie and header combination became invalid. No exception was raised, only incorrect product counts.
I reverse engineered the underlying product API and rebuilt the pipeline around backend endpoints instead of the frontend.
Result
* HTML selectors eliminated
* Silent corruption removed
* Faster and significantly more reliable extraction
---
Another target triggered an anti bot challenge on nearly every request, rapidly consuming captcha solver credits.
After analyzing the request flow I discovered that a specific warm up request established a valid session and bypassed the challenge for the remainder of the IP session.
Result
* Solver usage reduced dramatically
* Throughput increased by roughly 3x
* Lower operating cost with higher stability
---
A third target exposed no public JSON endpoints and was protected behind a complex frontend.
By identifying the platform as Magento, I reconstructed the catalog directly through its GraphQL layer, bypassing the UI completely.
Result
* Structured data instead of HTML parsing
* Faster collection
* More resilient pipeline
---
### Insurance & Banking Portals (NDA)
Built a backend platform integrating multiple SAML and OAuth authenticated portals, each with different session lifecycles, validation rules, MFA requirements, and request formats.
The platform minimizes browser automation by reverse engineering APIs whenever possible, orchestrates authentication flows, classifies failures by severity, retries recoverable errors automatically, and prevents partial or corrupted data from propagating downstream.
A sample implementation covering an F5 BIG IP APM → ADFS → Cognito OAuth PKCE authentication chain is available on request.
## Beyond scraping
Many projects require much more than collecting data.
I also build the backend services that keep those systems running in production, including
* REST APIs for downstream consumers
* Celery workers for distributed processing
* Celery Beat scheduling
* Redis based task queues
* PostgreSQL data modeling
* Dockerized deployments
* Authentication and session management
* ETL and data normalization pipelines
* Monitoring and automated recovery
* Integration with existing internal systems
Whether the challenge is extracting the data, processing it, or delivering it reliably, I build the complete workflow.
## What you get
* Silent failure detection through automated validation of product counts, categories, brands, and other business rules before data reaches production
* Anti bot resilience using proxy rotation, fingerprint management, browser automation only when necessary, and challenge mitigation strategies for Cloudflare, DataDome, PerimeterX, and reCAPTCHA
* Reliable authenticated portal automation supporting complex enterprise login flows with intelligent retries and session recovery
* Production ready backend services built with scalable architectures rather than standalone scripts
* Clean, normalized delivery through CSV, JSON, PostgreSQL, REST APIs, or custom integrations
I don't write scripts that break in production.
I build systems that keep running for months.
If your current pipeline is unreliable, your data is stale, or your target site actively resists extraction, send me the URL and I'll explain exactly how I'd approach it.
---
**Specialized stack**
Python • Django • FastAPI • Playwright • SeleniumBase • Camoufox • curl_cffi • Celery • Celery Beat • Redis • PostgreSQL • Docker • Django Channels • REST APIs • GraphQL • ETL • Data Pipelines • Reverse Engineering • Web Scraping • Automation
**Enterprise authentication**
SAML 2.0 • OAuth2 • PKCE • ADFS • F5 BIG IP APM • AWS Cognito
Steps for completing your project
After purchasing the project, send requirements so Ahmad can start the project.
Delivery time starts when Ahmad receives requirements from you.
Ahmad works on your project following the steps below.
Revisions may occur after the delivery date.
discuss details about required fields
have a small discussion about what exactly you want from the website including hidden fields