Hire the best Apache Spark Engineers in London, ENG

Check out Apache Spark Engineers in London, ENG with the skills you need for your next job.
  • $45 hourly
    Data engineer with extensive commercial experience in designing and building cloud native data solutions. Available for long-term collaboration. Experience & Skills Python application development ✅ Developing ETL and ELT applications ✅ Reading and writing files ✅ Data manipulation using Pyspark and Pandas ✅ Developing applications on Docker ✅ Writing pytest test cases SQL/Data Warehousing ✅ Snowflake, BigQuery, RDS (PostgreSQL, MySQL, Aurora) ✅ Data warehousing and modelling ✅ Writing complex queries and metrics ✅ Creating dbt models Infrastructure ✅ Serverless architecture design ✅ Event driven architecture design using SNS and SQS ✅ AWS Glue applications with event trigger or cron schedule, including crawlers and Athena table integration ✅ AWS ECS tasks and services to run dockerized applications ✅ AWS Batch jobs to run dockerized applications ✅ AWS Lambda functions with event trigger or cron schedule ✅ Static website hosting on S3 with CDN ✅ AWS EMR to run Apache Spark applications ✅ All infrastructure is provisioned using Terraform Monitoring and alerting ✅ Job monitoring and alerting using Cloudwatch metrics and Grafana CI/CD ✅ CircleCI, GoCD, GitLab CI/CD Version Control ✅ GitHub, GitLab
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    AWS Lambda
    Terraform
    Snowflake
    Data Ingestion
    Grafana
    SQL
    AWS Glue
    Amazon ECS
    Python
    dbt
    CI/CD
    Data Modeling
    Apache Hadoop
  • $40 hourly
    Leading 4 programs as lead big data architect and lead data architect and many project as technical lead in Banking for 14 years. I have end to end big data, DW architecture and implementation experience with stakeholder management. - Lead/architect consultancy for data lake & analytics in Sainsbury’s, TUI, Collinson group, ... - AWS solution architecture, AWS EMR, S3, RDS, CloudFormation, VPC, IAM with GDPR PII/PCI compliance. - Azure solution architecture. Azure Synapse, Azure Gen2/Blob data lake, Data factory, sql pools, Databricks, Jupyter Notebooks, Logic apps. - AWS Glue, Athena, AWS Lambda serverless - Big Data (Spark, Hadoop, YARN, Nosql, Hive, Sqoop, Flume, Kafka, R) 3+ years. - In depth knowledge in dimensional&relational data modeling and data warehousing, 13 years - Data Science with MSC degree (Weka, Python, Pandas, numpy, nltk) 5+ years - Analysis, design and implementation of DW&mining system - 11+ years. - Designing end to end in-house CRM 360 degree customer view, Salesforce systems integration as solution architect - 5 years. - Planning and coordinating data migration of a banking system. - Designing banking migration infrastructure, configuring migration simulation environment, sql, pl/sql tuning migration, coordinating end of day batches of bank. - Data mining, customer segmentation, propensity modeling, churn modeling, - Oracle OFSA, datamart design, data quality, ODS, financial reporting. Skills: - Hadoop, Oracle Nosql, Apache SOLR, Pig,Hive,Spark. - Python: flask, django rest, numpy, dask - Oracle ODI (Sunopsis/10g/11g/12C),PL/SQL - Reporting: OBIEE, - Shell scripting, XML,XSD - Oracle: PL/SQL, SQL, DB Tuning - IBM DB2, SQL Server, AS/400, SYBASE, MySql
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Data Management
    ETL Pipeline
    Database Architecture
    Data Science
    Big Data
    Amazon S3
    AWS Glue
  • $30 hourly
    Hi there! I have over a year of experience as a Data Engineer cum Data Analyst in my current role. I use Python as my daily driver, and I regularly work with technologies and frameworks like SQL, Apache Spark / PySpark, Pandas, and Google Data Studio. I can help you with tasks like Data Extraction, Data Cleaning, Data Transformation, Data Analysis and Data Visualisation. Feel free to reach out if you'd like to discuss your project with me! Languages - Python, SQL Data Processing, Transformation and Analysis - Apache Spark, PySpark, Pandas Data Visualisation - Google Data Studio, Plotly Data Storage Formats - CSV, Microsoft Excel, Google Sheets Others - Jupyter Notebook, ipynb
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Algorithm Development
    Data Management
    Java
    Data Analysis
    Data Structures
    Resume
    Interview Preparation
    Candidate Interviewing
    Machine Learning
    Data Science
    Career Coaching
    PySpark
    Python
    SQL
  • $80 hourly
    This is Amber Ameer, professionally providing IT services based on my skills and experience. I have 9+ years of experience building Data Lake, Data Warehousing & Business Intelligence solutions. Hands-on DevOps engineer & ETL developer. Development team lead experience from design, documentation, coding, testing & review, through to release & support. Experienced in data warehouse (Redshift) optimization projects. Good communication & networking skills. Adaptable, resourceful, and conscientious. Happy to work both independently and as part of a team. Core Strengths • Python • Certified Cloud Specialist (Amazon Web Services) • DevOps Specialization (Cloudformation, Codecommit, Codepipeline, CodeBuild, Code Deploy, Jenkins) • Dockers (Amazon ECS ECR, Fargate, Kubernetes) • Data Transformation (Glue, Spark, ETL) • Solving Problems • Continuous Learning & Improvement • Sharing Knowledge • Passionate about IT
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Data Analysis
    Data Science Consultation
    Database
    ETL Pipeline
    Data Migration
    ETL
    Big Data
    DevOps
    PostgreSQL
    Docker
    Java
    Amazon Web Services
    Python
  • $12 hourly
    I am a highly qualified Data Scientist experienced working with vast data sets to break down information, gather relevant points, and solve advanced business problems. Skilled in data cleaning, data analytics, predictive modelling, data mining, and hypothetical testing. Offering expertise in advanced data pre-processing, data visualization, feature engineering, machine learning, and data analytical techniques. I guarantee I will provide high-quality professional work on the given timeline at a very low cost.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Database
    Data Cleaning
    Data Mining
    Apache Hadoop
    Apache Pig
    Data Science
    Data Visualization
    Data Analysis
    Big Data
    Apache Hive
  • $45 hourly
    I'm an experienced data professional with a strong background in machine learning, data science, data engineering and project management. I hold a Master's degree in Artificial Intelligence from Queen Mary University of London. Whether you're are startup/enterprise looking to build next-generation Machine Learning solutions or even create Data Science and Analytics solutions that scale to your needs - I can help! Some of my strong points include but not limited to, • Experienced in communicating your requirements for the work. • Cross-functionally work as a consultant to provide actionable insights for increased ROI. • Project manage from initiation to delivery, ensuring regular communication. • Working in collaboration with all departments. • End-to-end Data Science and Machine Learning micro-service solutions. I also take on Data Visualization and Analytics dashboard projects to help you get the most out of your business! All you have to do is send me a message and I will help you with a solution that suit your needs.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Data Analysis
    API Development
    Data Engineering
    Microsoft SharePoint Administration
    Artificial Intelligence
    Game Testing
    Software Architecture & Design
    Cloud Services
    MLOps
    Python
    Machine Learning
    Data Science
    Amazon SageMaker
    Computer Vision
  • $35 hourly
    Experienced data engineer with several years of experience in designing, building and implementing scalable and reliable high-performance systems for business-critical data. Experienced with AWS cloud and on-premise systems in big data technologies and data pipelines. effective communicator, accustomed to working in agile environments with excellent problem-solving skills, always committed to delivering quality work within a fast-paced and challenging environment.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Agile Project Management
    CI/CD
    Data Modeling
    Interactive Data Visualization
    Serverless Computing
    Microservice
    Data Warehousing & ETL Software
    NoSQL Database
    Database
    AWS Development
    SQL
    Apache Hadoop
    PySpark
    Python
  • $750 hourly
    With 18+ years in Computer Science Engineering, I'm a proactive problem-solver known for innovation and analytical skills. Expert in Presales and Data Integration, I bridge departments and drive collaboration. My SAAS software expertise enables me to drive innovation and exceed customer expectations. I am eager to leverage my expertise as a Solutions Engineer to deliver transformative solutions and drive business success. I love big data technologies. I love API platforms. Reader discretion advised: my enthusiasm is contagious!
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Microsoft SQL Server
    Amazon Redshift
    Pricing
    Data Cloud
    Solution Architecture
    Jupyter Notebook
    RESTful API
    Amazon Web Services
    Databricks Platform
    Python
  • $150 hourly
    A full-stack engineer with a background that lends itself to helping companies stay lean and connected whilst scaling up their customers and services. With 7 year's experience in providing DevOps solutions and services to Finance, Web3.0 and Data Analytics - I have been heavily involved in building scalable platforms for microservices on Kubernetes, migrating infrastructure and services to the cloud and creating build environments for growing teams of developers. Skills: Cloud migrations - AWS, Azure, GCP, DigitalOcean, on-premise, Hetzner Container orchestration - Kubernetes (k8s) Rancher, Docker Swarm, OpenShift Infra-as-code - Pulumi, Terraform, CloudFormation, Sceptre Continuous delivery/integration - Jenkins, DroneIO, Helm, Kubernetes, GoCD, Tilt, Earthly Database - Elasticsearch, MongoDB, MySQL, MSSSQL, Postgres Applications - Docker, Nginx, LAMP, CoreOS, Terraform, Tableau, MS Exchange, Nutanix, VMWare Horizon/vCenter, Kafka, Atlassian Jira/Confluence, Microsoft SQL Server, Microsoft Exchange CloudFormation, Hugo Networking: DNS, DHCP, VLANs, NAT, Cisco Switch/Firewall Languages: Strong - PowerShell, Bash, Python, YAML, JSON Intermediate - GoLang, NodeJS, JavaScript, HTML, CSS, C#, TSQL Basic - Haskel, OCaml, Rust Achievements (in the last 2 years): - Re-engineered SAAS architecture - migrating all production to microservices on Kubernetes reducing the company's total software expenditure by 40% - Developed Terraform templates to make an automated multi-cloud disaster recovery solution - Implemented build pipelines to allow developers to work with isolated and identical versions of dev, test, and prod - Product Owner and Scrum Master of an Agile software development project for iPhone app - Advocate the need for a transparent business vision by employing OKRs and helped to align cascading team OKRs down through the organisation - Automated the provisioning of on-premise Kubernetes clusters and build pipelines using Matchbox, Bash and Helm templating Qualifications and education: 2020 - Kubernetes Certified Applications Developer 2020 - Kubernetes Certified Administrator 2018 - AWS Certified Developer Associate 2018 - Agile Certified Practitioner 2008 - 1st class degree in Electronic Engineering and Cybernetics Please get in touch to if you think my background can be helpful to you.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Grafana
    Amazon ECS
    Kubernetes
    Docker Compose
    Amazon ECS for Kubernetes
    Continuous Integration
    Docker
    Jenkins
    Amazon Web Services
    DevOps
    Terraform
    Microsoft Azure
  • $13 hourly
    Aspiring Data Scientist looking for a role that challenges my ability and push me harder towards my goals. #python #datascience
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    ETL Pipeline
    Anaconda
    Database
    Core Java
    ABAP
    Jupyter Notebook
    Object-Oriented Programming
    Data Science
    PyTorch
    ETL
    Python
    Java
  • $19 hourly
    Your go-to professional for all things data. Accurate, efficient, and reliable for all your needs. Just ask!
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    pandas
    Web Scraping
    Big Data
    PySpark
    Data Engineering
    SQL
    Python
  • $35 hourly
    I'm highly driven with an outstanding track record of supporting various client with various capacity to design and implement a scalable cloud native data solution. I have been into the data engineering space for more than 18 years and developing Azure based data solution for more than 9 years. I primarily dealt with below skills but not limited to: - Azure, Python, Spark, SQL, TSQL, Databricks, DevOps, Terraform and ARM Template Let's chat if you have any specific problem or requirement in your mind. We can make this happen!
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Microsoft Azure SQL Database
    Databricks Platform
    Microsoft SQL Server
    Azure DevOps
    Terraform
    Kusto Query Language
    Python Script
    SQL Programming
    SQL
    Microsoft Azure
    Python
  • $50 hourly
    I have Twenty Four years of experience in designing and delivering complex, distributed systems and data platforms. I am a Data Engineer, technology leader and hands-on polyglot programmer, with proficiency in multiple programming languages. Detail-oriented, result-driven professional with experience in areas of big data engineering, performance programming and system architecture. I am deeply passionate about clean code, refactoring, flexible product, frameworks, performance and scalability.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Microsoft Power BI
    Data Visualization
    Data Analysis
    Google Cloud Platform
    Kubernetes
    Greenplum
    Amazon Web Services
    Apache Airflow
    Snowflake
    Apache Hadoop
    Data Engineering
  • $25 hourly
    Senior Data Engineer 8 Years of experience working in IT for banking ,manufacture and energy industry. Having strong background in designing, building and maintaining large scale data systems and pipelines on GCP, Hadoop, Azure Responsibilities as Data engineer in my past roles --------------------------------------------------------------- 1:- Design, build and manage of data pipelines to collect metrics from heterogenous data sources including batch and live stream, store and process large volume of datasets ,Ensure quality/secure data delivery to end users by following data governance principles. 2:- Designing, building, and maintaining the data infrastructure and pipelines that enable organizations to collect, store, process, and analyse large amounts of data. 3. Developing and maintaining data architectures, data pipelines, and datalake and data warehouse 4. Managing data security and privacy, ensuring compliance with relevant regulations 5. Collaborating with data scientists, analysts, and other stakeholders to define data requirements and ensure data quality 6.Monitoring data infrastructure performance and pipelines and troubleshooting issues as they arise 7. Keeping up-to-date with the latest data technologies and best practices and incorporating them into the organization's data infrastructure. 8:- Statistical analysis of large/complex datasets and Troubleshooting complex data-related issues. 9:- Building frameworks to bring to life cutting-edge data ingestion and Analytics solutions that solves the secure data delivery, data privacy/compliance problems in an Modern Evolving Cloud for a Large financial services Organization. 6:- Build and test CICD pipelines to deploy and run containerized big data spark workloads ,Jupiter-hub and Kafka steam data pipelines on Kubernetes for banking project. Skill set and Expertise --------------------------- GCP , Azure Hadoop Apache Spark PySpark Kafka Hive Cloud Storage Dataproc DataFlow Pub/SUB Big query Databricks Data Factory Java , Python , Scala Sql , NoSql Jenkins Docker Kubernetes
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Data Lake
    PySpark
    Data Engineering
    Google Dataflow
    Google Cloud Platform
    Agile Software Development
    Kubernetes
    Docker
    Hive
    Apache Kafka
    Apache Hive
    Apache Cassandra
    Databricks Platform
    Apache Hadoop
  • $15 hourly
    Results-driven Data Engineer with a demonstrated history of optimising database systems and automating processes. Skilled in ETL/ELT procedures, Python scripting, SQL database management, and dbt (data build tool). Proven expertise in financial data analysis and reporting, complemented by advanced teaching skills in database management. Adept at collaborating effectively in cross-functional teams, including experience working in agile teams, with a commitment to continuous learning and professional development.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Database Design
    Data Analytics
    dbt
    Data Cleaning
    ETL Pipeline
    Microsoft Excel
    pandas
    Google Cloud Platform
    Microsoft Power BI
    Snowflake
    BigQuery
    Scala
    SQL
    Python
  • $15 hourly
    🚀 Data Professional Greetings! I'm Hassaan, a seasoned Data Professional with 8+ years of hands-on experience in data engineering, architecture, data modelling, software quality assurance, machine learning, business intelligence, data reporting and application development. 💡 Certified Data Engineer with 8+ Years of Experience 💡 With over a decade of expertise in propelling businesses forward through data-driven strategies, I specialise in providing comprehensive data consultancy and services aimed at unlocking the full potential of your data. My primary focus is on empowering businesses to make strategic decisions and enhance operational efficiency. 🔍 Forward-Thinking Analytical Approach 🔍 With a creative and analytical mindset, I excel at translating business challenges into technical solutions. My diverse skill set is focused on driving growth and improving ROI, ensuring that your data initiatives yield tangible results. 📊 Research Data Analyst Expertise 📊 I specialise in conducting historical and diagnostic data analyses to uncover meaningful insights and trends. With a keen eye for detail and a deep understanding of data patterns, I translate complex data into actionable recommendations for informed decision-making. 💡 Let's Transform Your Data Landscape 💡 Whether you're looking to optimise your data infrastructure, harness the power of predictive modelling, or gain deeper insights through advanced analytics, I'm here to help. With a track record of success in data engineering, analytics, and research, I'm committed to driving your business forward through data-driven innovation. 💼 Data Professional with a Personal Touch 💼 Are you searching for top-notch data services to elevate your business to new heights? Look no further! With a comprehensive range of personal skills and expertise, I offer tailored solutions to meet your data needs and drive success. 🛠️ Skills and Expertise 🛠️ 🥇 Domain Experience ⚡ Insurance / Healthcare / Banking / Telecom / Microfinance / Fintech / Retail / E-commerce ✅ Amazon Web Services ⚡ Redshift / RDS / S3 / Athena / Glue / Lambda / QuickSight / EC2 ✅ Microsoft Azure ⚡ Azure Data Lake / Azure Databricks / Azure Data Factory / Synapse ✅ Big Data Stack ⚡ Apache Spark / Apache Hadoop / Apache Airflow / Apache Hive / Apache Kafka / Cloudera / Databricks ✅ Data Visualization ⚡ MS PowerBI / Tableau / Azure Synapse / AWS QuickSight / MS Excel / Google Data Studio / IBM Cognos ✅ Languages & Libs ⚡ SQL / Python / R / Java / PL/SQL / C++ / .Net / JavaScript / JSON / Node.js / PySpark / Pandas / Numpy / Matplotlib / ReactJs / HTML / CSS / Django / ScikitLearn / Tensorflow ✅ ETL & Other Tools ⚡ Teradata / Talend Data Integration / IBM Datastage / Informatics / Alteryx / Apache Airflow / Docker / Jenkins / Confluence / Jira ✅ Databases ⚡ Vertica / IBM DB2 / Teradata / Greenplum / PostgreSQL / Oracle / MySQL / MS SQL / Oracle Goldengate / Cassandra 💡 Why Choose Me ? With a proven track record of success, a passion for innovation, and a commitment to excellence, I'm your go-to partner for all your data needs. Let's streamline your data infrastructure, harness the power of business intelligence, and unlock new opportunities through data-driven insights.
    vsuc_fltilesrefresh_TrophyIcon Apache Spark
    Big Data
    Tableau
    Data Science
    ETL
    Data Visualization
    PySpark
    Database
    Python
    SQL
    Data Management
    Data Analysis
    Amazon Web Services
    Microsoft Azure
    Business Intelligence
  • Want to browse more freelancers?
    Sign up

How hiring on Upwork works

1. Post a job (it’s free)

Tell us what you need. Provide as many details as possible, but don’t worry about getting it perfect.

2. Talent comes to you

Get qualified proposals within 24 hours, and meet the candidates you’re excited about. Hire as soon as you’re ready.

3. Collaborate easily

Use Upwork to chat or video call, share files, and track project progress right from the app.

4. Payment simplified

Receive invoices and make payments through Upwork. Only pay for work you authorize.

Trusted by

How do I hire a Apache Spark Engineer near London, ENG on Upwork?

You can hire a Apache Spark Engineer near London, ENG on Upwork in four simple steps:

  • Create a job post tailored to your Apache Spark Engineer project scope. We’ll walk you through the process step by step.
  • Browse top Apache Spark Engineer talent on Upwork and invite them to your project.
  • Once the proposals start flowing in, create a shortlist of top Apache Spark Engineer profiles and interview.
  • Hire the right Apache Spark Engineer for your project from Upwork, the world’s largest work marketplace.

At Upwork, we believe talent staffing should be easy.

How much does it cost to hire a Apache Spark Engineer?

Rates charged by Apache Spark Engineers on Upwork can vary with a number of factors including experience, location, and market conditions. See hourly rates for in-demand skills on Upwork.

Why hire a Apache Spark Engineer near London, ENG on Upwork?

As the world’s work marketplace, we connect highly-skilled freelance Apache Spark Engineers and businesses and help them build trusted, long-term relationships so they can achieve more together. Let us help you build the dream Apache Spark Engineer team you need to succeed.

Can I hire a Apache Spark Engineer near London, ENG within 24 hours on Upwork?

Depending on availability and the quality of your job post, it’s entirely possible to sign up for Upwork and receive Apache Spark Engineer proposals within 24 hours of posting a job description.