Upwork home
Sign up
Secondary

Hire the best Apache Spark Engineers in Lahore, PK

Check out Apache Spark Engineers in Lahore, PK with the skills you need for your next job.
Hire freelancers
Clients rate Apache Spark Engineers
Rating is 4.8 out of 5.
4.8/5
based on 107 client reviews
  • $60 hourly
    • 4.7/5
    • (12 jobs)
    Greetings! I am an experienced and talented Robotics and AI Engineer. I have 7+ years of experience in the software development industry, who is focused on developing large and scalable embedded systems, AI models and software solutions. My core skills are includes Robotics, AI, Machine Learning, IoT, Blockchain, Python, C++ and others. I develop not just a code but a solid and unique solution for your business. We are passionate about the field of IoT, Data handling particularly Machine Learning & Artificial Intelligence, robotics and the potential it holds to make the world a better and smarter place. Technologies which I use in my work: ☘️ ROS || ROS 2 ☘️ C | C++ | QT | Boost | STL ☘️ Python | PyTorch | NumPy | Pandas ☘️ TensorFlow | Scikit Learn | Keras ☘️ RTOS | Circuit Designing ☘️ Matlab | Simulink | OpenCV ☘️ Arduino | NVIDIA Jetson | Intel ☘️ MariaDB | MySQL | SQL | MongoDB | PostgreSQL ☘️ Unit Testing | Code Optimization ☘️ Unity Engine 4 | CAD | STL | URDF I am looking to cooperate with clients and companies to establish a strong, long-lasting relationship that we can both benefit from. I always strive to suggest the most efficient solutions for the project that will help it grow and move at the fastest pace, because my goal is to keep my clients 100% satisfied with the end result and the time it took to get there.
    Featured Skill Apache Spark
    OpenCV
    Data Science
    Artificial Intelligence
    Database Development
    Database Management System
    Electronics
    SQL Programming
    SQL
    Robot Operating System
    Oracle PLSQL
    C++
    Oracle Database
    React
    MongoDB
  • $50 hourly
    • 5.0/5
    • (39 jobs)
    Big Data engineer, AWS certified Developer and AWS DevOps Professional with excellent skills of coding in python, c++, java and c#. Has worked on different big data projects using amazon web services and open source tools. I have also completed three certifications, AWS Certified Big Data - Specialty, AWS Certified DevOps Professional and AWS Certified Developer Associate .
    Featured Skill Apache Spark
    Amazon Athena
    AWS Glue
    Data Mining
    Data Migration
    Data Visualization
    Big Data
    Amazon S3
    Amazon Redshift
    Amazon EC2
    AWS Lambda
    PostgreSQL
    Python
    Amazon DynamoDB
    Amazon Web Services
  • $30 hourly
    • 5.0/5
    • (16 jobs)
    Senior Data Engineer | ETL Specialist | Cloud & Big Data Expert Top Rated Plus | 100% Job Success Score | 1100+ Hours | $50K+ Earned Greetings! I'm a results-driven Senior Data Engineer with 5 years of experience transforming complex data challenges into strategic business advantages. My expertise lies in designing and implementing scalable, efficient data solutions that drive actionable insights and tangible ROI. Why Work With Me? 1. Proven Track Record: 14 successful projects, 100% client satisfaction, and consistent 5-star ratings. 2. Efficiency Maestro: Reduced model deployment time from 2 weeks to 3 days and slashed candidate matching time from 4 days to under 24 hours. 3. Cost Saver: Implemented cloud migrations resulting in $5,000 annual savings in capital expenses. 4. Performance Booster: Increased sales by 3% through advanced customer behavior analysis and boosted client satisfaction by 20% with optimized job matching. 5. Scalability Expert: Processed 50GB+ data daily, orchestrating high-performance pipelines for compute-intensive workloads. Core Competencies - ETL Pipeline Design & Optimization (Python, SQL, Spark) - Cloud Architecture (AWS, GCP) - Big Data Processing (Databricks, Redshift, BigQuery) - Workflow Orchestration (Airflow, Prefect) - CI/CD Implementation (GitLab, GitHub Actions) - Infrastructure as Code (Terraform) - Containerization & Orchestration (Docker, Kubernetes) Recent Impact Stories 1. E-commerce Giant: Engineered ETL pipelines processing 5GB+ daily data, resulting in a 3% sales increase through enhanced customer insights. 2. AI-Powered Recruitment Platform: Optimized job-candidate matching, reducing time-to-match by 75% and increasing client satisfaction by 20%. 3. FinTech Startup: Led cloud migration initiative, cutting annual IT costs by $5,000 while improving data processing capabilities. My Commitment to Your Success - Rapid response times and clear communication - Agile methodology for flexible, iterative development - Detailed documentation and knowledge transfer - Proactive problem-solving and continuous optimization Ready to transform your data into a strategic asset? Let's connect and discuss how I can drive efficiency, cut costs, and unlock insights for your organization.
    Featured Skill Apache Spark
    CI/CD
    ETL Pipeline
    BigQuery
    Google Cloud Platform
    Data Engineering
    Amazon Redshift
    Terraform
    Apache Airflow
    Git
    Amazon S3
    SQL
    Python
  • $20 hourly
    • 5.0/5
    • (6 jobs)
    I am a skilled data engineer who has 4 years’ plus experience in design, analysis and development of ETL solutions for various financial Institutions and retail Organizations. My skills as an ETL developer includes; data analysis, data profiling, designing the solution architecture, data conversion and development of ETL Pipelines. I have exposure to multiple ETL tools and technologies such as: Databricks, Python, Spark, DBT, SQL Server Integration Services, Azure Data Factory, Talend open studio. As a Data Engineer, I have handled the structured, un-structured and semi-structured data. I am expert in databases such as MS SQL, PostgreSQL and modern warehousing engines such as snowflake. In addition to that, I have deep understanding of queries execution plan and have optimized the Enterprise level queries.
    Featured Skill Apache Spark
    AWS Glue
    Oracle PLSQL
    Talend Data Integration
    Data Cleaning
    Data Extraction
    Data Scraping
    Amazon Redshift
    BigQuery
    ETL Pipeline
    Databricks Platform
    Data Engineering
    Snowflake
    Python
    SQL
  • $25 hourly
    • 5.0/5
    • (4 jobs)
    🏆 Top Talent of IT Industry 🔥 Worked with Fortune 500 companies ⭐️ Software Architect with 12+ years of experience ⭐️ Data Engineering ⭐️ AWS Expert ⭐️ GCP Expert I am Syed Waqas Faheem, a Software Architect with 12 year of experience. I am an experienced professional who can help your business to excel & reach to the billion dollar club. From custom software development including web & mobile application development I will help you build & transform your business with the highest level of customer satisfaction guaranteed. In the last 12 years, I have worked with many Fortune 500 companies and big brands such as IBM, Vodworks , Global Engineering Services to Startups like Five Rivers Technology & Eagersoft. I have completed more than 50 projects for companies over 34 countries. I have 100% track record of deliver projects on time. Not an inch passed the deadline, that’s my personal credo. If it’s a yes, it would definitely be a long-term & profitable partnership for both of us. Let’s Chat !
    Featured Skill Apache Spark
    Apache Airflow
    Python
    Core Java
    FinTech Consulting
    AWS Development
    ETL Pipeline
    Java
    Continuous Integration
    Database
    Apache Kafka
    Spring Boot
    Kubernetes
    Microservice
    Automation
    Big Data
    Data Engineering
  • $30 hourly
    • 4.7/5
    • (31 jobs)
    🟩 Ranked top 10% of all Upwork talent 🟪 𝟐𝟔 Happy Customers✍ 🟦 5-star client ratings 📢 𝙄𝙛 𝙢𝙮 𝙬𝙤𝙧𝙠 𝙙𝙤𝙚𝙨𝙣’𝙩 𝙢𝙚𝙚𝙩 𝙩𝙝𝙚 𝙢𝙖𝙧𝙠, 𝙮𝙤𝙪 𝙜𝙚𝙩 𝙖 100% 𝙧𝙚𝙛𝙪𝙣𝙙! Hi, I am Taha , a Senior data engineer with 𝟕+ 𝐲𝐞𝐚𝐫𝐬 of experience in the domain of data warehousing, data modelling, ETL development and reporting. In my professional career, I have worked with many 𝐦𝐮𝐥𝐭𝐢-𝐛𝐢𝐥𝐥𝐢𝐨𝐧 𝐝𝐨𝐥𝐥𝐚𝐫 worth USA based companies which include Regeneron and Inovalon along with some startups like Impel and perch insights. I am certified in below technologies: ✔️ AWS Cloud Certified ✔️ Snowflake Certified ✔️ Power BI Certified ✔️ Python, Pyspark Certified 💎 Key Skills: I have hands-on expertise in below tools and technologies : 🌟 AWS Cloud: 1- Proficient AWS Data Engineer with expertise in Redshift, Glue, Lambda, Athena, S3, RDS, EC2, Step functions, Cloud formation. 🌟 ETL & Integration tools: 1-Excellent command on DBT, AWS GLUE, Microsoft SSIS, Fivetran 🌟 Programming Languages: 1-Hands on experience working with Python (PySpark, pandas) , SQL , JavaScript 🌟 Datawarehouse and Database: 1-Competent in working with Snowflake, AWS Redshift, RDS, SQL Server. 🌟 Reporting tools 1-Extensive experience in working with Power BI, Metabase. 𝐈𝐦𝐩𝐨𝐫𝐭𝐚𝐧𝐭 ❗ I will take full responsibility for the final result and finding solutions to your complex problems.
    Featured Skill Apache Spark
    Amazon Web Services
    Data Engineering
    Metabase
    Apache Airflow
    Fivetran
    dbt
    PySpark
    Microsoft Power BI
    AWS Glue
    AWS Lambda
    Amazon Redshift
    Snowflake
    SQL
    Python
  • $20 hourly
    • 5.0/5
    • (4 jobs)
    I am a seasoned Web Developer and Python Programmer with a wealth of experience in crafting robust and scalable applications. My expertise spans across a range of technologies including AWS, Flask, Django, and React. With a passion for delivering top-notch software solutions, I bring a proactive approach to problem-solving and a keen eye for detail to every project. My journey as a senior Python developer has equipped me with a deep understanding of creating efficient and effective applications. I thrive on transforming complex ideas into elegant, functional, and user-friendly solutions. My proficiency in Python and its associated frameworks allows me to architect applications that are not only reliable but also easily maintainable. One of my core strengths lies in my ability to harness the power of AWS services, ensuring seamless deployment, scalability, and security of applications. Whether it's designing RESTful APIs using Flask, building robust web applications with Django, or creating dynamic user interfaces with React, I am well-versed in crafting solutions tailored to the unique needs of each project. What sets me apart is my commitment to delivering excellence. I believe in open communication, and I actively collaborate with clients to understand their vision and requirements. This collaborative approach allows me to transform concepts into reality while maintaining transparency throughout the development lifecycle. My track record speaks for itself, having successfully delivered a diverse array of projects ranging from e-commerce platforms and content management systems to real-time web applications. Whether it's optimizing code for peak performance or ensuring a seamless user experience, I bring a holistic perspective to development that encompasses both functionality and aesthetics. If you're looking for a senior Python developer who can leverage cutting-edge technologies to bring your ideas to life, while adhering to best practices and industry standards, I'm here to help. Let's embark on a journey to create exceptional software that leaves a lasting impact. Looking forward to collaborating on your next project.
    Featured Skill Apache Spark
    Website
    Vue.js
    Computer Vision
    Odoo
    Python
    Selenium
    Scripting
    RESTful API
    Flask
    Java
    Python Script
    Django
    Data Scraping
  • $40 hourly
    • 4.6/5
    • (14 jobs)
    I am a diligent and passionate full stack developer with extensive experience in developing web and mobile applications using different technology stacks including Java/Angular, MEAN, React and React native. I am skilled in following tools/technologies/frameworks: Front-end : Angular, React, React native, Angular Material, Bootstrap, jQuery, AJAX, Html, JavaScript, CSS Back-end : Nodejs, Spring Boot Databases: MongoDB, PostgreSQL, Oracle, SQL Server, MySQL Unit/integration tests: JUnit, Mockito, Jasmine, Karma, NUnit Others Skills: 508 compliance/web accessibility, internationalization/localization.
    Featured Skill Apache Spark
    React Native
    Mobile App
    Flutter
    Spring Framework
    React
    TypeScript
    Angular
    Spring Boot
    Node.js
  • $45 hourly
    • 5.0/5
    • (3 jobs)
    I'm a Senior Data Scientist with 7+ years of experience wrangling Big Data using Machine Learning, Deep Learning, NLP and Generative AI. I've helped Fortune 500s and startups unlock hidden insights and build awesome data science solutions that solve real-world problems. I don't just speak data, I translate it into actionable strategies you can actually use!
    Featured Skill Apache Spark
    PyTorch
    Python Scikit-Learn
    Databricks Platform
    Azure Machine Learning
    Amazon SageMaker
    Time Series Analysis
    Natural Language Processing
    Generative AI
    Deep Learning
    Machine Learning
    SQL
    R
    Python
  • $70 hourly
    • 5.0/5
    • (27 jobs)
    🏅 Top 1% Expert Vetted Talent 🏅 5★ Service, 100% Customer Satisfaction, Guaranteed FAST & on-time delivery 🏆 Experience building enterprise data solutions and efficient cloud architecture 🏅 Expert Data Engineer with over 13 years of experience As an Expert Data Engineer with over 13 years of experience, I specialize in turning raw data into actionable intelligence. My expertise lies in Data Engineering, Solution Architecture, and Cloud Engineering, with a proven track record of designing and managing multi-terabyte to petabyte-scale Data Lakes and Warehouses. I excel in designing & developing complex ETL pipelines, and delivering scalable, high-performance, and secure data solutions. My hands-on experience with data integration tools in AWS, and certifications in Databricks ensure efficient and robust data solutions for my clients. In addition to my data specialization, I bring advanced proficiency in AWS and GCP, crafting scalable and secure cloud infrastructures. My skills extend to full stack development, utilizing Python, Django, ReactJS, VueJS, Angular, and Laravel, along with DevOps tools like Docker, Kubernetes, and Jenkins for seamless integration and continuous deployment. I have collaborated extensively with clients in the US and Europe, consistently delivering high-quality work, effective communication, and meeting stringent deadlines. A glimpse of a recent client review: ⭐⭐⭐⭐⭐ "Abdul’s deep understanding of business logic, data architecture, and coding best practices is truly impressive. His submissions are invariably error-free and meticulously clean, a testament to his commitment to excellence. Abdul’s proficiency with AWS, Apache Spark, and modern data engineering practices has significantly streamlined our data operations, making them more efficient and effective. In conclusion, Abdul is an invaluable asset – a fantastic data engineer and solution architect. His expertise, dedication, and team-oriented approach have made a positive impact on our organization." ⭐⭐⭐⭐⭐ "Strong technical experience, great English communications skills. Realistic project estimates." ⭐⭐⭐⭐⭐ "Qualified specialist in his field. Highly recommended." ✅ Certifications: — Databricks Certified Data Engineer Professional — Databricks Certified Associate Developer for Apache Spark 3.0 — CCA Spark and Hadoop Developer — Oracle Data Integrator 12c Certified Implementation Specialist ✅ Key Skills and Expertise: ⚡️ Data Engineering: Proficient in designing multi-terabyte to petabyte-scale Data Lakes and Warehouses, utilizing tools like Databricks, Spark, Redshift, Hive, Hadoop, Snowflake. ⚡️ Cloud Infrastructure & Architecture: Advanced skills in AWS and GCP, delivering scalable and secure cloud solutions. ⚡️ Cost Optimization: Implementing strategies to reduce cloud infrastructure costs significantly. ✅ Working Hours: - 4AM to 4PM (CEST) - 7PM to 7AM (PDT) - 10PM - 10AM (EST) ✅ Call to Action: If you are looking for a dedicated professional to help you harness the power of AWS and optimize your cloud infrastructure, I am here to help. Let's collaborate to achieve your technological goals.
    Featured Skill Apache Spark
    Amazon Web Services
    Apache Hive
    Apache Hadoop
    Microsoft Azure
    Snowflake
    BigQuery
    Apache Kafka
    Data Warehousing
    Django
    Databricks Platform
    Python
    ETL
    SQL
  • $50 hourly
    • 4.9/5
    • (12 jobs)
    - A hardworking and motivated professional having Master’s degree in Computer Science with 10+ years of experience in software development, Expertise in analysis, design and development of efficient software applications and general problem solving. The Skills and Services are as follows (not limited to) SKILLS: - Database Migration - Database Design and Optimisations - ETL - Data warehousing - Relational / Non Relational Databases - Python - Node.js - SQL - API Development - Serverless Framework - Web Scrapping - Data Lake formation - Apache Spark (PySpark) AWS: (Hands on 50+ Services) - IAM, VPC, API Gateway, AppSync - S3, KMS, EC2, Auto Scaling, ELB - EBS, EFS - SFTP - Route53, Cloudfront, Lambda - Glue, Athena, DynamoDB - Redshift, Redshift Spectrum, RDS, Aurora - DMS, EMR, Data Pipeline - Step Function, System Manager, Cloudwatch - Elastic Search, Textract, Rekognition - Transcribe, Transcode, Lex - Connect, Pinpoint, SNS - SQS, Cognito - Cloudformation, Code pipeline, Code Deploy - Hands on experience of working on Enterprise applications and AWS solutions - Proactively support team building and on boarding efforts via mentoring contributions - Proven track record of professional and hardworking attitude towards job and always focused on delivery. - Participate in agile development process, including daily scrum, sprint planning, code reviews, and quality assurance activities - Believe in one team model and always provide assistance when required.
    Featured Skill Apache Spark
    Amazon Redshift
    Amazon S3
    AWS Lambda
    Tableau
    Amazon EC2
    Amazon Cognito
    Amazon Web Services
    AWS Glue
    PostgreSQL
    ETL Pipeline
    Data Migration
    Python
    SQL
  • $45 hourly
    • 5.0/5
    • (21 jobs)
    🏅 Expert-Vetted | 🏆 100% Job Success Rate | ⭐ 5-Star Ratings | 🕛 Full Time Availability | ✅ Verifiable projects | ❇️ 7,000+ Hours Introduction 🎓 I am a seasoned Product Developer with over a decade of experience in Automation, Data Science, and Big Data domains. Specializing in Generative AI projects, SaaS products, and leading teams of multiple developers, I have a unique expertise in converting LLM-based MVPs to production-grade applications. Utilizing event driven asynchronous programming and introducing retrying mechanisms, I strive to make pipelines robust and reliable, innovating and excelling in the industry. Technical Expertise 💻 👉 For Generative AI🤖: I specialize in creating cutting-edge Generative AI solutions, leveraging the latest frameworks and technologies. Vector Databases: Pinecone: Utilizing Pinecone for large-scale vector search and similarity scoring. Chroma: Implementing Chroma, the open-source embedding database, for efficient vector operations. Milvus: Leveraging Milvus for hardware-efficient advanced indexing, achieving a 10x performance boost in retrieval speed. Supabase, PgVector: Employing these databases for real-time management and PostgreSQL vector operations. Frameworks: Langchain: At its core, Langchain is a framework built around LLMs, used for chatbots, Generative Question-Answering (GQA), summarization, and more. It allows chaining together different components for advanced use cases around LLMs. Auto-GPT: An Autonomous GPT-4 Experiment, an open-source application showcasing GPT-4 capabilities, chaining together LLM. Llama Index, BabyAGI, SuperAGI: Utilizing these frameworks for indexing, early-stage AGI development, and advanced AGI solutions. Dolly 2.0: Working with Dolly 2.0, a 12B parameter language model based on the EleutherAI pythia model family, for creative content generation. Platforms like Hugging Face, Replicate.com: Collaborating on these platforms for model sharing, version control, and collaboration. Converting LLM-based MVPs, LLAMA 2, Amazon Polly, Speech to Text, OpenAI, RAG Approach, Chain of Thoughts, Optimizing LLM Memory, Generative AI-based Course Generator, ChatBot Builder Project 👉 For Big Data📊: I have extensive experience in handling large-scale data, ensuring efficiency and accuracy in processing and analysis. Expertise in building machine learning and ETL pipelines from scratch Expertise in Kafka, Apache Spark, Spark Streaming, MapReduce, Hadoop GeoSpatial Analysis, Machine Learning techniques, VAS Applications in Telco Environment Experience with ELK stack, Cloud Environments: Proficient in AWS, GCP, Azure 👉 For Web Development💻: I offer comprehensive web development solutions, focusing on scalability, user experience, and innovative technologies. Languages: Proficient in Python, Java, Scala, NodeJS Frontend Frameworks: Mastery in React and other modern frontend technologies Asynchronous Programming, Backend Development, Search Technology, CI/CD Tools, Cloud Environments: Heroku, AWS, Azure, GCP Specialization in Building SaaS Products 🚀 I have a strong background in designing and developing Software as a Service (SaaS) products, ensuring scalability, reliability, and innovation. My experience ranges from backend development to deploying complex systems end-to-end. My portfolio reflects a blend of cutting-edge innovation and practical application. My specialized knowledge in Generative AI, Big Data, Web Development, and SaaS products highlights my proficiency in various domains. If you're seeking a versatile and results-driven engineer with a strong innovative track record, I would love to hear from you
    Featured Skill Apache Spark
    AI Chatbot
    Elasticsearch
    Amazon Web Services
    ETL
    Data Visualization
    Salesforce CRM
    Big Data
    Web Development
    React
    ChatGPT
    Tableau
    Data Science
    Machine Learning
    Python
  • $45 hourly
    • 5.0/5
    • (67 jobs)
    I have completed 60+ jobs, earned $200K (on Upwork), and have a stellar 5/5 feedback. ⭐️⭐️⭐️⭐️⭐️ $145K+ - Golang/Python Microservices "Having worked with several overseas teams in the past, I have to say that I struck gold finding Fahad and his team. They really became an extension to my team, are very disciplined about understanding what needs to happen, contributing at the highest levels. Fahad is an honest guy, smart, and always helpful in meeting deadlines and pleasing customers. I look forward to continuing to work with Fahad and his team as they are a true asset and highly productive development group." ⭐️⭐️⭐️⭐️⭐️ $28K+ - Python/Flask "Fahad knows his stuff and has integrated very well into our team. This is due to his communication skills and motivation. I highly recommend him" And many more! My experience represents countless hours spent mastering skills and solving complex problems, ensuring you don't have to navigate these challenges yourself. Hire me if you: ✅ Want a SWE with strong technical skills ✅ Need a Python, Go, or a Rust developer ✅ Seek to leverage AI for predictive analytics, enhancing data-driven decision-making ✅ Require AI-based optimization of existing software for efficiency and scalability ✅ Wish to integrate AI and machine learning models to automate tasks and processes ✅ Need expert guidance in selecting and implementing the right AI technologies for your project ✅ Desire a detail-oriented person who asks questions and figures out things on his own ✅ Even have a requirement in your mind but are not able to craft it into a technical format ✅ Want advice on what tools or tech you want to implement in your next big project ✅ Are stuck in a data modeling problem and need a solution architect ✅ Want to optimize a data pipeline Don't hire me if you: ❌ Have a huge project that needs to be done overnight ❌ Have academic work to be done About me: ⭐️ A data engineer with proven experience in designing and implementing big data solutions ⭐️ Skilled in integrating AI technologies to solve complex problems, improve efficiency, and innovate within projects ⭐️ A Go developer specialized in creating microservices ⭐️ A certified Data Engineer on AWS technologies ⭐️ Will optimize your code in every single commit without even mentioning or charging extra hours ⭐️ Diverse experience with start-ups and enterprises taught me how to work under pressure yet work professionally
    Featured Skill Apache Spark
    Web Scraping
    Microservice
    ETL Pipeline
    Big Data
    AI Bot
    OpenAI API
    Artificial Intelligence
    Generative AI
    Large Language Model
    Golang
    Python
  • $45 hourly
    • 5.0/5
    • (18 jobs)
    I’ve been automating and containerizing multi-tier workloads and developing solutions to migrate applications from on-prem to AWS/Cloud. I’ve also streamlined deployments for multiple clients using CI/CD pipelines like Jenkins. Recently, I’ve been working with serverless technologies to handle big data and ETL workloads. Following are the 𝐭𝐞𝐜𝐡𝐧𝐢𝐜𝐚𝐥 𝐯𝐞𝐫𝐭𝐢𝐜𝐚𝐥𝐬 along with tools that I've been working on: 𝐈𝐀𝐂: - CloudFormation - Terraform 𝐌𝐞𝐭𝐫𝐢𝐜𝐬 𝐚𝐧𝐝 𝐥𝐨𝐠𝐠𝐢𝐧𝐠 / 𝐀𝐥𝐞𝐫𝐭𝐬 / 𝐌𝐨𝐧𝐢𝐭𝐨𝐫𝐢𝐧𝐠: - CloudWatch Logs - CloudWatch Events / Event Bridge - Datadog - SNS - AWS end-end platforming for monitoring, logging, and alerting 𝐏𝐫𝐨𝐠𝐫𝐚𝐦𝐦𝐢𝐧𝐠 / 𝐒𝐜𝐫𝐢𝐩𝐭𝐢𝐧𝐠: - Python - Bash - SQL 𝐋𝐢𝐧𝐮𝐱 𝐒𝐲𝐬𝐭𝐞𝐦 / 𝐃𝐞𝐯𝐎𝐩𝐬: - Amazon AMIs - Ubuntu/Centos/RHEL - Bash 𝐂𝐨𝐧𝐭𝐚𝐢𝐧𝐞𝐫𝐬: - Docker - Kubernetes - AWS - ECS/EKS - AWS Fargate (Serverless) 𝐂𝐈/𝐂𝐃: - Jenkins, - AWS Code pipeline - AWS CodeBuild - AWS CodeDeploy - Azure DevOps - GitHub Actions 𝐂𝐨𝐧𝐟𝐢𝐠𝐮𝐫𝐚𝐭𝐢𝐨𝐧 𝐌𝐚𝐧𝐚𝐠𝐞𝐦𝐞𝐧𝐭 & 𝐒𝐞𝐜𝐮𝐫𝐢𝐭𝐲: - AWS VPC - AWS SSM - AWS Secrets Manager - AWS IAM 𝐒𝐞𝐫𝐯𝐞𝐫𝐥𝐞𝐬𝐬 𝐓𝐞𝐜𝐡𝐧𝐨𝐥𝐨𝐠𝐢𝐞𝐬: - AWS Lambda, AWS Glue, AWS Fargate (containers), AWS Athena, AWS S3 𝐁𝐢𝐠𝐃𝐚𝐭𝐚/𝐄𝐓𝐋: - Apache Airflow - Apache Spark - AWS EMR - AWS Glue - AWS S3 - AWS Athena - AWS Transfer for SFTP
    Featured Skill Apache Spark
    CI/CD
    Deployment Automation
    Terraform
    Amazon Web Services
    Jenkins
    Docker
    AWS CloudFormation
    Kubernetes
  • $25 hourly
    • 4.8/5
    • (19 jobs)
    Certified AWS Professional, AWS Data Engineer and AWS Expert with 10+ years of experience in scalable data pipelines, ETL workflows, data lakes, and cloud architectures using AWS (Redshift, Glue, S3, Athena) and Apache Airflow. Skilled in LLMs and Generative AI, leveraging SageMaker, Bedrock, and LangChain for AI-driven applications. Also proficient in backend development and serverless APIs, designing RESTful and event-driven architectures using Lambda, API Gateway, and DynamoDB. ✅ Top 3% Expert-Vetted on Upwork | 😊 18 Happy Customers | ⭐ Top Rated Plus | 💯 Job Success Score AWS Certified: 🚀 AWS Certified Solutions Architect – Associate (CSAA) 🚀 AWS Certified Solutions Architect – Professional (CSAP) 🚀 AWS Certified Data Analytics – Specialty (CDAS) Proven track record in optimizing cloud solutions, reducing IT costs by 40%, and improving operational efficiency by 50%. ⚡️ 10+ years of experience | Big Data | AI-driven solutions | Backend | REST APIs | GenAI | LLMs ⚡️ Scalable Data Pipelines | Data lake | Data Warehousing | DWH | Data Security | Data Quality ⚡️ SCD | Incremental Load | Data Migration | Database Design | Data Modeling | ERD ⚡️ AWS | AWS Glue | Apache Spark | Apache Airflow | Redshift | RDS | S3 | Athena | Segment | Databricks | Snowflake ⚡️ SageMaker | Bedrock | Claude | Llama | Titan | Mistral | LangChain | RAG | Fine Tuning ⚡️ Amazon Lambda | API Gateway | DynamoDB | Serverless Framework | SAM ⚡️ NodeJS + Sequelize + RDS (PostgreSQL & MySQL). A "𝐁𝐈𝐆 𝐘𝐄𝐒" to those who value ✅ best practices and scalable, secure and governed Data Pipeline from the very start (from MVP) ✅ secure REST APIs/Private APIs on AWS Cloud ✅ power of LLMs, Chatbots and AI Powered Solutions ✅ open to design suggestions that could save infrastructure cost while having operational excellency ✅ prompt and transparent communication ✅ quick feedbacks and turn arounds If you will work with me you will get 👉 Normalised Database designs for transactional Databases 👉 flow diagrams, ERD, and source code for APIs 👉 architecture diagram and source code. 👉 project delivery as I have 99.99% success rate of delivering top notch services in my career. 👉 quick and prompt answers in less than 15 minutes unless I am sleeping. 👉 transparency and daily updates with every work log. Here are few of my client testimonial that I usually see when I am feeling down in my life. 🌟 "𝘈𝘴𝘩𝘢𝘴 𝘪𝘴 𝘢 𝘳𝘦𝘢𝘭𝘭𝘺 𝘥𝘦𝘥𝘪𝘤𝘢𝘵𝘦𝘥 𝘣𝘢𝘤𝘬𝘦𝘯𝘥 𝘥𝘦𝘷𝘦𝘭𝘰𝘱𝘦𝘳 𝘏𝘦 𝘨𝘪𝘷𝘦𝘴 𝘤𝘰𝘯𝘴𝘵𝘳𝘶𝘤𝘵𝘪𝘷𝘦 𝘴𝘶𝘨𝘨𝘦𝘴𝘵𝘪𝘰𝘯𝘴 𝘢𝘯𝘥 𝘱𝘦𝘳𝘴𝘦𝘷𝘦𝘳𝘦𝘴 𝘢𝘯𝘥 𝘵𝘢𝘬𝘦𝘴 𝘵𝘩𝘦 𝘭𝘦𝘢𝘥 𝘏𝘦 𝘥𝘪𝘥 𝘢 𝘨𝘰𝘰𝘥 𝘫𝘰𝘣 𝘧𝘰𝘳 𝘶𝘴 𝘐𝘯 𝘵𝘦𝘳𝘮𝘴 𝘰𝘧 𝘴𝘬𝘪𝘭𝘭 𝘭𝘦𝘷𝘦𝘭 𝘈𝘴𝘩𝘢𝘴 𝘪𝘴 𝘴𝘵𝘪𝘭𝘭 𝘨𝘳𝘰𝘸𝘪𝘯𝘨 𝘢𝘯𝘥 𝘥𝘦𝘷𝘦𝘭𝘰𝘱𝘪𝘯𝘨 𝘣𝘶𝘵 𝘩𝘢𝘴 𝘢 𝘨𝘳𝘦𝘢𝘵 𝘢𝘵𝘵𝘪𝘵𝘶𝘥𝘦." (𝐔𝐩𝐖𝐨𝐫𝐤) 🌟 "𝘈𝘴𝘩𝘢𝘴 𝘪𝘴 𝘢 𝘳𝘦𝘢𝘭 𝘱𝘳𝘰 𝘪𝘯 𝘩𝘪𝘴 𝘧𝘪𝘦𝘭𝘥 𝘷𝘦𝘳𝘺 𝘦𝘢𝘴𝘺 𝘵𝘰 𝘸𝘰𝘳𝘬 𝘸𝘪𝘵𝘩 𝘢𝘯𝘥 𝘲𝘶𝘪𝘤𝘬 𝘤𝘰𝘮𝘮𝘶𝘯𝘪𝘤𝘢𝘵𝘪𝘰𝘯 𝘢𝘯𝘥 𝘵𝘶𝘳𝘯𝘢𝘳𝘰𝘶𝘯𝘥!" (𝐔𝐩𝐖𝐨𝐫𝐤) 🌟 "𝘸𝘢𝘴 𝘢 𝘨𝘳𝘦𝘢𝘵 𝘱𝘢𝘳𝘵 𝘰𝘧 𝘵𝘩𝘦 𝘵𝘦𝘢𝘮 𝘐 𝘭𝘰𝘰𝘬 𝘧𝘰𝘳𝘸𝘢𝘳𝘥 𝘵𝘰 𝘸𝘰𝘳𝘬𝘪𝘯𝘨 𝘢𝘨𝘢𝘪𝘯 𝘪𝘯 𝘵𝘩𝘦 𝘧𝘶𝘵𝘶𝘳𝘦" (𝐔𝐩𝐖𝐨𝐫𝐤) 🌟 "𝘈𝘴𝘩𝘢𝘴 𝘪𝘴 𝘷𝘦𝘳𝘺 𝘸𝘪𝘭𝘭𝘪𝘯𝘨 𝘵𝘰 𝘩𝘦𝘭𝘱 𝘰𝘶𝘵 𝘢𝘯𝘥 𝘩𝘢𝘴 𝘢 𝘥𝘪𝘷𝘦𝘳𝘴𝘦 𝘴𝘬𝘪𝘭𝘭𝘴𝘦𝘵 𝘸𝘩𝘪𝘤𝘩 𝘩𝘢𝘴 𝘦𝘯𝘢𝘣𝘭𝘦𝘥 𝘮𝘦 𝘵𝘰 𝘣𝘶𝘪𝘭𝘥 𝘢𝘯𝘥 𝘥𝘦𝘱𝘭𝘰𝘺 𝘮𝘺 𝘢𝘱𝘱 𝘶𝘴𝘪𝘯𝘨 𝘷𝘦𝘳𝘺 𝘤𝘰𝘴𝘵 𝘦𝘧𝘧𝘦𝘤𝘵𝘪𝘷𝘦 𝘴𝘦𝘳𝘷𝘦𝘳𝘭𝘦𝘴𝘴 𝘪𝘯𝘧𝘳𝘢𝘴𝘵𝘳𝘶𝘤𝘵𝘶𝘳𝘦" (𝐔𝐩𝐖𝐨𝐫𝐤) 𝑭𝑬𝑬𝑳 𝑭𝑹𝑬𝑬 to message me, I am just a one message away for all your AWS projects,
    Featured Skill Apache Spark
    Databricks Platform
    Data Engineering
    Amazon Athena
    ETL Pipeline
    Apache Airflow
    Amazon Bedrock
    PySpark
    Solution Architecture
    Amazon Redshift
    Amazon S3
    Amazon API Gateway
    AWS Glue
    AWS Lambda
    Amazon Web Services
  • $30 hourly
    • 5.0/5
    • (4 jobs)
    I have been working as a Senior Data Engineer (remotely) for a firm in the UK. There I have worked on projects from various companies, a few of them are: ● BBC ● NatWest ● MyEnergy ● Ministry of Justice... My work normally involves a lot of Pyspark, any data integration service (typically Glue) and SQL. I have also extensively worked on setting up CICD using AWS CodePipeline, CodeBuild and CodeDeploy. One of the major recent projects that I have done is setting up the MLOps foundations for BBC’s Iplayer recommendations team. I used AWS CDK and CloudFormation to create the necessary infrastructure stacks, the backbone of the foundations was Sagemaker studio which allowed Data Scientists to create notebooks and pipelines for experimentations. This foundation supported SageMaker project templates which were really beneficial for replicating model promotion strategies across different models. I previously worked as a Data scientist and Machine Learning Engineer for a Singapore based firm. Not long ago I realised that you cannot build good AI solutions unless you have strong and mature data foundations. Which led me to pursue Data Engineering so that I could help clients achieve data maturity at first and then work on AI use-cases which comes at the peak of the pyramid of needs for Data Science. I hold a Mphil. degree in Data Science (Graduated with the highest distinction) from Information Technology University (ITU) and a MicroMasters in the same from the University of California San Diego (UCSD). I love to work on cloud based services as these eliminate our dependencies on the hardware and network teams.
    Featured Skill Apache Spark
    Amazon SageMaker
    AWS CloudFormation
    AWS CodePipeline
    AWS Glue
    Data Visualization
    Cloud Computing
    Big Data
    SQL
    Data Science
    Machine Learning
    Natural Language Processing
  • $15 hourly
    • 5.0/5
    • (9 jobs)
    Proficient data engineer experienced in big data pipeline development and designing data solutions for retail, healthcare, etc. I've designed and implemented multiple cloud-based data pipelines for companies located in Europe and the USA. I'm Experienced in designing enterprise-level data warehouses, have Good analytical and communication skills, team player, and am hard working. Experiences: - More than 4+ years of experience in data engineering. - Hand-on experience in developing data-driven solutions using cloud technologies. - Designed multiple data warehouses using Snowflake and Star schema. - Requirement gathering and understanding business needs, to propose solutions. Certified: - Databricks Data Engineer Certified. - Microsoft Azure Associate Data Engineer. Tools and tech: - Pyspark - DBT - Airflow - Azure Cloud - python - Data factory - Snowflake - Databricks - C# - Aws - Docker - CI/CD - Restful API Development
    Featured Skill Apache Spark
    AWS Lambda
    PySpark
    Microsoft Azure
    Databricks MLflow
    dbt
    Snowflake
    API Development
    Data Lake
    ETL
    Databricks Platform
    Python
    Apache Airflow
  • $50 hourly
    • 4.6/5
    • (62 jobs)
    DataOps Leader with 20+ Years of Experience in Software Development and IT Expertise in a Wide Range of Cutting-Edge Technologies * Databases: NoSQL, SQL Server, SSIS, Cassandra, Spark, Hadoop, PostgreSQL, Postgis, MySQL, GIS Percona, Tokudb, HandlerSockets (nosql), CRATE, RedShift, Riak, Hive, Sqoop * Search Engines: Sphinx, Solr, Elastic Search, AWS cloud search * In-Memory Computing: Redis, memcached * Analytics: ETL, Analytic data from few millions to billions of rows and analytics on it, Sentiment analysis, Google BigQuery, Apache Zeppelin, Splunk, Trifacta Wrangler, Tableau * Languages & Scripting: Python, php, shell scripts, Scala, bootstrap, C, C++, Java, Nodejs, DotNet * Servers: Apache, Nginx, CentOS, Ubuntu, Windows, distributed data, EC2, RDS, and Linux systems Proven Track Record of Success in Leading IT Initiatives and Delivering Solutions * Full lifecycle project management experience * Hands-on experience in leading all stages of system development * Ability to coordinate and direct all phases of project-based efforts * Proven ability to manage, motivate, and lead project teams Ready to Take on the Challenge of DataOps I am a highly motivated and results-oriented IT Specialist with a proven track record of success in leading IT initiatives and delivering solutions. I am confident that my skills and experience would be a valuable asset to any team looking to implement DataOps practices. I am excited about the opportunity to use my skills and experience to help organizations of all sizes achieve their data goals.
    Featured Skill Apache Spark
    Python
    Scala
    ETL Pipeline
    Data Modeling
    NoSQL Database
    BigQuery
    Sphinx
    Linux System Administration
    Amazon Redshift
    PostgreSQL
    ETL
    MySQL
    Database Optimization
    Apache Cassandra
  • $30 hourly
    • 5.0/5
    • (8 jobs)
    I'm an experienced 𝐌𝐚𝐜𝐡𝐢𝐧𝐞 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫 and 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐀𝐈 specialist with a background in 𝐃𝐚𝐭𝐚 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 with extensive experience in designing and implementing end-to-end AI and ML solutions. My expertise spans various industries, and I've worked with multi-national teams to deliver innovative solutions. 🌎🏆 Are you looking to optimize your data engineering pipelines or leverage advanced machine learning for real-world impact? Need assistance in designing and deploying robust AI and ML solutions in your business environment? Let's connect and transform your ideas into actionable outcomes. 🔧💻✨ ✔️Core Services ✅ Generative AI and Large Language Models (LLMs)🤖 • Proficient in AutoGen and GPT-3.5-turbo for a range of generative AI applications, from code generation to task automation. • Experienced in creating multi-agent frameworks, conducting reinforcement learning with human feedback (RLHF), and integrating document processing and analysis with tools like ChatGPT and Langchain. • Skilled in designing complex workflows, implementing custom prompts, and exploring parameter-efficient fine-tuning techniques to optimize LLM performance. ✅ Machine Learning and Predictive Analytics📊 • Built ML models for sales forecasting, financial analysis, and other predictive tasks. • Strong background in PySpark and ML algorithms like Prophet and SARIMAX. • Used Google BigQuery, Google Dataproc, and Apache Airflow for orchestration in various projects. ✅ Data Engineering and ETL Pipelines 🔄 • Specialize in designing, optimizing, and migrating ETL pipelines using Azure Data Factory, Databricks, Google Cloud Platform (GCP), and more. • Extensive experience in large-scale data transformation and efficient data flow. ✅ Chatbot Development💬 • Design and deploy intelligent chatbots integrated with various data sources or APIs to enhance customer engagement and streamline business processes. ✅ Custom Python Scripting and APIs 🐍 •Develop custom Python scripts and APIs to interact with databases, AI models, and other software systems, enabling seamless automation and integration with existing workflows. 𝐔𝐧𝐢𝐪𝐮𝐞 𝐂𝐨𝐦𝐩𝐞𝐭𝐞𝐧𝐜𝐢𝐞𝐬: 𝐏𝐚𝐫𝐚𝐦𝐞𝐭𝐞𝐫-𝐄𝐟𝐟𝐢𝐜𝐢𝐞𝐧𝐭 𝐅𝐢𝐧𝐞-𝐓𝐮𝐧𝐢𝐧𝐠 (𝐏𝐄𝐅𝐓) ⚙: I have expertise in advanced LLM techniques, including fine-tuning, chain-of-thought prompting, and reinforcement learning with human feedback (RLHF). 𝐀𝐈-𝐁𝐚𝐬𝐞𝐝 𝐁𝐮𝐬𝐢𝐧𝐞𝐬𝐬 𝐀𝐮𝐭𝐨𝐦𝐚𝐭𝐢𝐨𝐧 ⚡: I can help you automate business processes and boost efficiency using AI and ML techniques. 𝐑𝐞𝐬𝐞𝐚𝐫𝐜𝐡, 𝐂𝐨𝐧𝐬𝐮𝐥𝐭𝐚𝐭𝐢𝐨𝐧 𝐚𝐧𝐝 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐦𝐞𝐧𝐭 🔬: I offer expert guidance and hands-on development in AI and ML, focusing on delivering practical solutions to real-world challenges. 𝐋𝐞𝐭'𝐬 𝐂𝐨𝐧𝐧𝐞𝐜𝐭: 💡 If you're interested in exploring the potential of AI, data engineering, or machine learning for your business, I'd love to hear from you. Let's discuss your requirements and create tailored solutions to meet your unique needs. Together, we can drive innovation and transform your vision into reality.
    Featured Skill Apache Spark
    Retrieval Augmented Generation
    LangChain
    LLM Prompt Engineering
    Generative AI
    Microsoft Azure
    CI/CD
    Google Cloud Platform
    PySpark
    Apache Airflow
    ETL Pipeline
    Python
    Machine Learning
    MLflow
    Databricks Platform
  • $25 hourly
    • 4.8/5
    • (54 jobs)
    Microsoft certified Data engineer with over 8 years of experience in design and development of Data and cloud Projects Hello, my name is Muddaser Talal, and I am an Azure certified data engineer with over 8 years of experience. I have completed around three full-scale enterprise solutions on Microsoft cloud platforms. My expertise spans DataBricks, DataFactory, and more. I specialize in the orchestration and management of data processes using modern tools and have developed robust solutions in .NET C#, Java, and Python. My professional journey has enabled me to master the nuances of cloud data services, enhancing efficiency and productivity. Furthermore, my capabilities extend to web scraping and data mining, leveraging tools like Selenium and Python. I have extensive experience with scraping data from e-commerce sites and social platforms, providing actionable insights and support. 💻 Azure Certified Data Engineer I am skilled in leveraging Azure certifications to deploy, orchestrate, and optimize data solutions on the cloud, ensuring reliability and scalability. 🌐 API Development Developing API-driven architectures and DataFactory orchestrations to streamline the integration and management of data processes. 🚀 Databricks Orchestration Expert in creating, managing, and optimizing Databricks jobs and clusters to handle large-scale data processing tasks efficiently. 🔄 DataFactory Proficient in using Azure DataFactory to automate and orchestrate data transformation and movement processes across platforms. 🔍 Stream Analytics Experience in implementing real-time data processing solutions using Stream Analytics for data streaming and analytics at scale. 🏢 EventHubs Skilled in using EventHubs for capturing, transforming, and storing streaming data, ensuring reliable and efficient data pipelines. 🗂️ DataLake Store Proven ability to manage and utilize Azure DataLake Store for effective data storage and retrieval, ensuring data is accessible and secure. 📊 Big Data Tools Hands-on experience with tools like Apache Spark, Scala, and Kafka, enhancing big data solutions’ processing power and capabilities. 👨‍💻 Web Scraping Leveraging Python and Selenium for comprehensive web scraping projects, extracting data from various sites, and providing structured end results. 📈 Data Mining Utilizing advanced data mining techniques to derive meaningful insights from large datasets, enhancing data-driven decision-making. Let’s take your data projects to the next level! Contact me today to discuss how my expertise can contribute to your success.
    Featured Skill Apache Spark
    Google Cloud Platform
    Apache Airflow
    Data Management
    Microsoft Azure
    Snowflake
    Big Data
    Selenium
    Data Scraping
    Python
  • $20 hourly
    • 0.0/5
    • (2 jobs)
    🚀 Unlock the Power of Your Data with an Expert! With extensive experience in cloud platforms and data engineering, I specialize in delivering robust and scalable data solutions. Here's what I bring to the table: 🌩️ Cloud Expertise: Proficient in AWS and Azure, I design and implement efficient cloud-based data architectures. 🔧 Data Engineering: Skilled in building and managing data pipelines, ensuring seamless data integration and transformation. 📊 Reporting Solutions: Experienced in creating insightful and actionable reports using advanced data visualization tools like Tableau and Power BI. 🧠 Data Science: Leveraging Python, R, and SQL, I perform complex data analyses to drive strategic decision-making. 🔒 Data Security: Committed to maintaining data integrity and security, I implement best practices to safeguard sensitive information. I thrive on solving complex data challenges and helping businesses unlock the full potential of their data. Let's work together to transform your data into actionable insights! 🌟
    Featured Skill Apache Spark
    ASP.NET
    React
    Data Lake
    Data Cloud
    ETL Pipeline
    AWS Server Migration
    Cloudera
    AI Data Analytics
    SAP Warehouse Management
    Snowflake
    Cloud Engineering
    ETL
    API Integration
    Laravel
  • $21 hourly
    • 0.0/5
    • (0 jobs)
    As a Data Engineer with 4+ years of experience, I specialize in designing scalable data pipelines, ETL workflows, and data storage solutions. My expertise spans across key technologies including: • Programming & Analysis: Python (Pandas, NumPy, Seaborn, Matplotlib), SQL, PostgreSQL • Cloud Platforms: AWS (IAM, S3, Glue, EMR, DynamoDB, RDS, API Gateway, Lambda) • Data Engineering Tools: Apache Spark, Apache Airflow, Snowflake, DBT • BI & Visualization: Microsoft Power BI • Core Strengths: Data modeling, query optimization, and building scalable data architectures I deliver efficient, high-performance solutions that align with business goals across diverse industries. I'm not only technically skilled but also a strong communicator and collaborator—able to bridge the gap between technical teams and business stakeholders. Driven by curiosity and committed to quality, I aim to build robust data solutions that generate real business value. Let’s work together to transform your data infrastructure and unlock deeper insights.
    Featured Skill Apache Spark
    BigQuery
    Data Ingestion
    AWS Lambda
    Microsoft Azure
    Data Modeling
    AWS Glue
    Microsoft Power BI
    PostgreSQL
    Data Warehousing
    Apache Airflow
    Snowflake
    Python
    MySQL
    Data Engineering
  • $20 hourly
    • 5.0/5
    • (1 job)
    Hello! I'm Abdullah, a dedicated and enthusiastic data science student with a solid foundation in data analysis, machine learning, and statistical modeling. I have hands-on experience with Python, R, SQL, C++ and various data visualization tools like Tableau and Matplotlib. Strengths and Skills: Programming Languages: Proficient in C++. Moreover Python and R for data manipulation and analysis. Data Analysis: Expertise in using Pandas and NumPy for data cleaning and preprocessing. Machine Learning: Skilled in applying machine learning algorithms using Scikit-learn and TensorFlow. Data Visualization: Adept at creating insightful visualizations with Matplotlib and Seaborn. Databases: Experience with MySQL for database querying and MongoDB for handling NoSQL data. Tools: Familiar with Jupyter Notebook for interactive data analysis and Git for version control.
    Featured Skill Apache Spark
    Jupyter Notebook
    GitHub
    C++
    Data Structures
    Object-Oriented Programming
    Machine Learning
    Apache Kafka
    Apache Hadoop
    Python
    Data Visualization
    Data Analysis
    Data Science
  • $20 hourly
    • 5.0/5
    • (1 job)
    ✅ BI & Analytics Consultant ✅ 🚀 Driving Data-Driven Transformations for Business Success! 🚀 With Visionet Systems, Inc since October 2020, I've been instrumental in spearheading multiple projects aimed at revolutionizing data management and analytics. Leveraging advanced tools and technologies, I've consistently delivered impactful solutions to drive business growth and efficiency. 🌟 Projects & Roles: Halcyon Still Waters (May 2021 - September 2021) Role: Consultant – BI & Analytics Leveraged PySpark within the Databricks environment for data exploration and transformation, ensuring optimal data structuring and usability. Engineered robust Azure Data Factory pipelines for seamless ETL processes, facilitating efficient data ingestion, transformation, and orchestration. FIFA Fan Registration (October 2021 - December 2021) Role: Consultant – BI & Analytics Managed data ingestion from MongoDB sources and designed optimized Azure Data Factory pipelines for ETL orchestration. Developed data warehousing solutions using SQL Server and implemented comprehensive reporting mechanisms. Mattress Firm (February 2022 – June 2022) Role: Consultant – BI & Analytics Led data management initiatives in Azure Purview, ensuring data lineage, cataloging, and glossary establishment for enhanced data governance. Implemented data quality measures within Azure workflows and leveraged Delta Live Tables for improved data quality assurance practices. TAC BOP (October 2022 – Present) Role: Techno Functional (Business Analyst & Consultant Data Analyst) Orchestrated meticulous documentation of Function Specification Documents (FSD) and optimized data organization for efficient analysis and reporting. Played a key role in configuring and parameterizing the TAC (Temenos Advance Collection) application, ensuring seamless integration and maximum functionality alignment with business needs. 🎯 Core Competencies: Expertise in PySpark, Python, SQL, Azure Data Factory, Azure Purview, and Delta Live Tables. Proficient in data warehousing, ETL processes, and BI reporting tools. Strong analytical skills with a focus on data quality assurance and governance. Proven track record of collaborating with cross-functional teams and providing comprehensive training and support. 🛠️ Technical Toolkit: PySpark | Python | SQL | Azure Data Factory | Azure Purview | Delta Live Tables Databricks | MongoDB | SQL Server | Crystal Reports | Temenos Advance Collection (TAC) 🌟 Elevate Your Data Strategy with Expert BI & Analytics Consultation! 📞 Reach out to explore how we can drive your business towards data-driven success. ✅ Click the "Invite" button to initiate our collaboration for transformative data solutions!
    Featured Skill Apache Spark
    Databricks Platform
    Data Lake
    Analytics
    Microsoft Azure
    Database
    Data Warehousing
    Information Analysis
    ETL
    Machine Learning
    Data Science
    Data Analysis
    Big Data
    Query Development
  • $20 hourly
    • 0.0/5
    • (3 jobs)
    Are you looking to turn your raw data into powerful business insights? I'm Hammad Saleem, a Data Scientist and Machine Learning Expert with strong expertise in predictive analytics, AI-driven solutions, and interactive dashboards. I specialize in: • Data Cleaning & Preprocessing • ML Models (Classification, Clustering, Forecasting) • Business Intelligence (Power BI, Tableau, Excel, KNIME) • NLP & Sentiment Analysis • Python, R, SQL, Pandas, TensorFlow I help businesses make smarter decisions through clear visualizations and result-driven analysis. Whether it's building a model, automating reports, or solving real-world data challenges — I'm here to help. Let’s connect and grow your business with data!
    Featured Skill Apache Spark
    MATLAB
    SQL
    Microsoft Excel
    Tableau
    Microsoft Power BI
    KNIME
    Python
    Big Data
    Deep Learning Modeling
    Machine Learning
    Microsoft Power BI Data Visualization
    Data Annotation
    Data Science
    Data Analysis
  • $5 hourly
    • 0.0/5
    • (0 jobs)
    C++ Development with SFML: I have extensive experience in C++ development, particularly with the SFML (Simple and Fast Multimedia Library) framework. I can create interactive and visually appealing applications, games, simulations, and multimedia projects using SFML. Machine Learning: I specialize in machine learning techniques and algorithms, leveraging libraries such as TensorFlow, Scikit-learn, and PyTorch in Python. I can develop machine learning models for a wide range of applications, including classification, regression Furthermore, I possess expertise in SQL, proficient in designing and optimizing databases for efficient data management and retrieval
    Featured Skill Apache Spark
    Data Science
    Data Analysis
    MongoDB
    Apache Kafka
    Apache Hadoop
    C++
    SFML
  • $25 hourly
    • 0.0/5
    • (0 jobs)
    I’m an AI Engineer with experience in building machine learning solutions and data pipelines for businesses of all sizes. Whether you’re aiming to leverage AI for decision-making, automate your data workflows, or ensure data integrity, I can help. • Expertise: Training ML models, developing automated data pipelines with Apache Airflow, cleaning and processing data using Hadoop and Spark, and creating data visualizations with Apache Superset. • Skills: TensorFlow, Scikit-Learn, Python, SQL, R, Apache Airflow, Hadoop, Apache Spark, Apache Superset. • Project Management: Offering full project management from start to finish to ensure timely and quality delivery. Regular communication is important to me, so let’s stay in touch to achieve your project goals together.
    Featured Skill Apache Spark
    Artificial Intelligence
    SQL
    LLM Prompt Engineering
    TensorFlow
    Flask
    Python
    Apache Superset
    Apache Kafka
    Apache Airflow
    Apache Hadoop
  • Want to browse more freelancers?
    Sign up

How hiring on Upwork works

1. Post a job

Tell us what you need. Provide as many details as possible, but don’t worry about getting it perfect.

2. Talent comes to you

Get qualified proposals within 24 hours, and meet the candidates you’re excited about. Hire as soon as you’re ready.

3. Collaborate easily

Use Upwork to chat or video call, share files, and track project progress right from the app.

4. Payment simplified

Receive invoices and make payments through Upwork. Only pay for work you authorize.

Trusted by

How do I hire a Apache Spark Engineer near Lahore, on Upwork?

You can hire a Apache Spark Engineer near Lahore, on Upwork in four simple steps:

  • Create a job post tailored to your Apache Spark Engineer project scope. We’ll walk you through the process step by step.
  • Browse top Apache Spark Engineer talent on Upwork and invite them to your project.
  • Once the proposals start flowing in, create a shortlist of top Apache Spark Engineer profiles and interview.
  • Hire the right Apache Spark Engineer for your project from Upwork, the world’s largest work marketplace.

At Upwork, we believe talent staffing should be easy.

How much does it cost to hire a Apache Spark Engineer?

Rates charged by Apache Spark Engineers on Upwork can vary with a number of factors including experience, location, and market conditions. See hourly rates for in-demand skills on Upwork.

Why hire a Apache Spark Engineer near Lahore, on Upwork?

As the world’s work marketplace, we connect highly-skilled freelance Apache Spark Engineers and businesses and help them build trusted, long-term relationships so they can achieve more together. Let us help you build the dream Apache Spark Engineer team you need to succeed.

Can I hire a Apache Spark Engineer near Lahore, within 24 hours on Upwork?

Depending on availability and the quality of your job post, it’s entirely possible to sign up for Upwork and receive Apache Spark Engineer proposals within 24 hours of posting a job description.

Looking to hire for long-term or full-time assignments? See how Full-time hiring on Upwork simplifies the process.