Upwork home
Sign up
Secondary
Shiv A. Status: Offline
Mathura, India
100% Job Success
Top Rated Plus

Data Engineer | AWS | GCP | DevOps | Agents and Chatbot

Seeking challenging opportunities in the design and development of scalable backend infrastructure, High-Performance Computing (HPC), and AI-driven solutions. Expertise Domains: ETL Pipelines Data Engineering DevOps & Cloud Deployment (AWS & GCP) Machine Learning High-Performance Computing (HPC) AI Agents & AI Chatbots I primarily design and implement solutions in Python, leveraging 10+ years of extensive experience. My expertise spans a wide range of powerful frameworks and libraries, including Flask, Django, Pandas, Numpy, PyTorch, Scrapy, and more. ETL Pipelines: I deliver end-to-end data pipelines using AWS, GCP, and Custom Frameworks, with 7+ years of experience in this domain. I possess a strong command of Scrapy and have developed 300+ crawlers to date. Data Warehousing: With deep expertise in Google BigQuery and AWS RedShift, I have hands-on experience handling and analyzing millions of data points using cloud-based warehousing solutions. Serverless Applications: I have 5+ years of experience in designing and deploying Serverless Applications on AWS and GCP. I leverage a broad range of cloud services to deliver efficient and cost-effective solutions. High-Performance Computing (HPC): I design and optimize HPC infrastructure to accelerate data processing, complex simulations, and large-scale computations. I specialize in leveraging distributed computing and parallel processing to improve performance. AI Agents & Chatbots: I build intelligent AI agents that automate tasks, perform semantic search, and provide insights by extracting and processing large datasets. I have hands-on experience with developing conversational AI chatbots that enhance customer support and internal operations. Whether it's building scalable backend solutions, optimizing cloud infrastructure, or developing AI-driven applications, I strive to push the boundaries of innovation in every project I undertake.
Work history
Shiv A. has more jobs. Create an account to review them
Skills

Skills

Testimonials

Endorsements from past clients

"I highly recommend Shiv Agrawal for web-scraping and we have been working with their team for over 5 years. The communication and response time is typically within minutes, and no longer than 1 hour. If we have issues, the team is available and our questions are always addressed. The work received is clear, detailed and functional. Of the many projects we have asked Shiv to complete, we have rarely had any instances where they could not complete the tasks. "

Michael K. | System Manager
Data Aggregator Feb 2020

Verified

"Our company and I have been working with Shiv for the last 4 years and we couldn't be happier. It is very hard to find reliable freelancers online but with Shiv we have been in great hands. 4 years ago we needed a freelancer to build an in house SQL data base that is connected to 30+ automated scrapers hosted on an AWS server in order to keep an eye on our competition. Shiv designed, built and still maintains the scrapers and database. He is extremely reliable and responds in a very timely matter. We are looking forward to continuing working with Shiv for many years to come. Any company will value his commitment and strong working ethics. Thank you Shiv! Bastian Green"

Bastian G. | Project Manager
Data Aggregator Feb 2020

Verified

"My go to guy for all scrapping needs. One time had to call him mid night for an urgent problem - no issue. Very impressive guy."

Pradyumna G. | Co-founder at AVOCADO
Feb 2020

Verified

Browse similar freelancers