Upwork home
Sign up
Project Catalog
DesignDesign
Video & AudioVideo & Audio
Development & ITDevelopment & IT
MarketingMarketing
Writing & TranslationWriting & Translation
MoreMore

You will get a production-ready REST API to ADLS Gen2 ingestion pipeline

Sanchit P.Status: Offline
Sanchit P.
Preview image

Let a pro handle the details

Buy Scripting services from Sanchit, priced and ready to go.
Sanchit P.Status: Offline
Sanchit P.
Preview image

Let a pro handle the details

Buy Scripting services from Sanchit, priced and ready to go.

Project details

I am a certified Azure Data Engineer (Databricks Certified, April 2026) with production experience building REST API ingestion pipelines at scale. I will deliver a complete Python pipeline that fetches data from your paginated REST API, handles authentication, transforms the payload using PySpark, and stores it in ADLS Gen2 with date partitioning and structured error logging. Every pipeline I deliver is production-ready — not a one-off script. You get clean, documented, working code with retry logic and schema enforcement built in.
Programming Languages
Python
Coding Expertise
Performance Optimization, Security

What's included $80

These options are included with the project scope.

$80
  • Delivery Time 4 days
  • Number of Revisions 1
  • Number of Sources Mined/Scraped 1
    • Install Script
    • Test Script
    • Task Automation
Optional add-ons You can add these on the next page.
Additional Revision
+$10
Install Script
+$15
Test Script
+$15

Frequently asked questions

Sanchit P.Status: Offline

About Sanchit

Sanchit P.Status: Offline
Azure Data Engineer | Databricks | PySpark | Delta Lake | ADF
New Delhi, India - 11:20 pm local time
I am a certified Azure Data Engineer (Databricks Certified Data Engineer Associate, April 2026) with production experience building end-to-end data pipelines on the Azure stack.
My core expertise:

PySpark & Delta Lake on Azure Databricks
Azure Data Factory (ADF) for orchestration
ADLS Gen2, Azure SQL for storage and serving
Medallion Architecture (Bronze/Silver/Gold) design and implementation

Recent project: Built a full REST API ingestion pipeline (pagination, JSON→Spark), Medallion layers on ADLS Gen2, ADF orchestration with failure alerting, and Power BI reporting — production-grade, end to end.
I also have hands-on experience with NL-to-SQL and event classification using Azure OpenAI, and RAG pipelines using LangChain and FAISS.
If you need clean, well-structured Azure data pipelines — I deliver production-quality work, not just scripts.

Steps for completing your project

After purchasing the project, send requirements so Sanchit can start the project.

Delivery time starts when Sanchit receives requirements from you.

Sanchit works on your project following the steps below.

Revisions may occur after the delivery date.

Review API structure

Analyse endpoint, auth method, pagination logic, and confirm ADLS Gen2 storage path and output format.

Build ingestion pipeline

Develop paginated Python ingestion script with retry logic, PySpark transformation, and partitioned ADLS Gen2 storage.

Review the work, release payment, and leave feedback to Sanchit.