Hire the Best Big Data Engineers
in Canada

Struggling with slow, unreliable data pipelines, rising cloud costs, or a data stack that can't keep up with your AI initiatives? I help businesses fix exactly that. I'm a Senior Data Engineer with hands-on experience helping businesses design, build, and scale reliable data infrastructure across Azure, GCP, and AWS. I specialize in end-to-end ETL/ELT pipelines, real-time data processing, and cloud-native data architecture using PySpark, Python, SQL, Databricks, and BigQuery. Across 21+ completed projects on Upwork, I've delivered more than $100K in data engineering solutions, including systems processing over 10 million records with performance improvements of up to 40%. My work spans automated ingestion frameworks, CRM and analytics integrations, RAG-based LLM pipelines, and cloud cost optimization without sacrificing performance. Clients consistently point to my clear communication, reliability, and commitment to quality, reflected in my 100% Job Success score and Top Rated Plus status. Whether you need a pipeline built from scratch, an existing data stack optimized, or an ongoing data engineering partner, I bring the technical depth and communication needed to get projects delivered on time and on budget. If you have a data challenge, let's connect and discuss how I can help move your project forward.

Robin Huang CFA, FRM VB, VBS, VBA, VCBasic, SQL, Access 7 years Word, Excel, PowerPoint, Outlook 7 years HTML, CSS, JavaScript, JQuery, Python 4 years C++, Ruby, Swift, Xcode, Parse 1 year Spark, Anaconda, Jupyter, Linux, Unix 3 years Visual Studio, Report Builder, Publisher 4 years XML, Android Studio, Java, PHP 3 years Financial Modeling, Financial Engineering 7 years Power Pivot, Power Query, Power BI 5 years IBM OnDemand 4 years JIRA, Confluence, Drools, SSAS, SSIS 1 year IEX, FactSet, Interactive Brokers (IB) 2 years Good knowledge of SWIFT and Corporate Actions Good knowledge of DevOps, Jenkins and xMatters Good knowledge of Hadoop, HDFS, Hive, Impala Bloomberg, Thomson Reuters Eikon 4 years MS SQL Server, SDLC, ETL 7 years Capital Markets 7 years SSRS, SSMA, Vertica, SharePoint 3 years .Net, Xamarin, C#, XAML 4 years IBM Cognos TM1/ EPM, MDX Query 3 years Business Intelligence, Business Analysis 7 years VMware, MATLAB, Photoshop, Tableau 4 years STATA, R, SAS, Lean Six Sigma 3 years Data Science, Data Mining 4 years Oracle Business Intelligence, PeopleSoft 2 years Credit Risk, Market Risk 2 years Mainframe, OutsideView, Extra! X-treme 1 year Google GCP, Amazon AWS 2 years

AI Automation • Cloud Data Platforms • Agentic Workflows • Streaming Pipeline . Real-Time Analytics Major Clients & Key Projects: Ballys Interactive USA (Online Gaming): Online gaming streaming for game and props odds. Kafka, Airflow, Pyspark, Google Bigquery for realtime streaming. Walmart USA (Retail): High-Throughput Supply Chain Analytics: Re-engineered core inventory management pipelines using PySpark and BigQuery. Transitioned batch processing to a streaming-first design, reducing end-to-end processing times by 60% for real-time stock and logistics tracking. Rogers Communications Canada(Telecom): Real-Time Telemetry Pipeline: Architected decoupled, horizontally scalable streaming pipelines using Kafka/MSK and AWS Glue to process massive telecom network data. Implemented partitioning and backpressure strategies to stabilize SLAs and reduce latency by 70%. Bank of Nova Scotia Canada (Banking & Insurance): Cloud Modernization & Governance: Led the migration of legacy financial and risk data systems to a multi-cloud Snowflake and BigQuery environment. Integrated robust data tiering and access controls, lowering cloud OpEx by 20% while hitting strict compliance targets. Enterprise Clients USA(Construction): Operational AI Automation: Built low-code/no-code agentic workflows using n8n and LLMs to automate multi-vendor invoice data extraction, asset tracking, and project metadata ingestion into centralized PostgreSQL and Elasticsearch engines. Principal Data Engineer and Architect with extensive experience building cloud-scale data ecosystems and AI-driven automation frameworks. Proven track record transitioning legacy legacy architectures to modern, high-throughput, streaming-first environments across Telecom, Banking, Insurance, Retail, and Construction. Expert in optimizing multi-cloud unit economics (AWS, GCP, Azure) to accelerate time-to-insight while maintaining rigid governance standards. Core Tech Stack & Architecture Data Engineering & Orchestration: AWS Glue, Amazon Athena, MSK (Kafka), Apache Airflow, PySpark, real-time ETL/ELT lifecycles. Cloud Data Warehousing: Snowflake, Google BigQuery, Amazon Redshift; specialized in storage tiering, clustering optimization, and cross-platform governance. AI & Workflow Automation: Production LLM/RAG integration, n8n, Zapier, Claude Code; automated operational metadata and content pipelines. Modern Data Stack: PostgreSQL, Redis, Elasticsearch for high-performance caching, state management, and real-time streaming.

Transforming Raw Data into Insights That Drive Business Success With years of hands-on experience in data engineering and analysis, I specialize in building robust data pipelines, optimizing workflows, and uncovering actionable insights that power data-driven decisions. I’ve collaborated with businesses across various industries to: - Build automated ETL pipelines, reducing data processing time by 30% and improving operational efficiency. - Develop dynamic dashboards for real-time decision-making using Power BI and Tableau, helping clients visualize key performance indicators. - Optimize SQL queries and database structures, cutting query execution times by up to 50%. My Expertise 🔑 Data Engineering: Design and maintain efficient, scalable ETL processes and data pipelines. 📊 Data Analysis & Visualization: Extract insights and present data through compelling dashboards and reports using Power BI, Tableau, and Python. 📂Database Management: Optimize database performance and design relational databases that scale with business growth. ⚡ Cloud Integration: Leverage cloud platforms like AWS to build scalable, cost-effective data solutions. 📈 Big Data Tools: Harnessing the power of Apache Spark and Kafka to process massive datasets efficiently. What I Can Do for You - Data Engineering: Designing scalable ETL pipelines and managing end-to-end data workflows to streamline operations and improve data accessibility. - Data Analysis: Turning complex datasets into insights that drive business growth, focusing on actionable results that align with business objectives. - Data Visualization: Creating intuitive, user-friendly dashboards and reports that simplify decision-making and foster data-driven culture. - Cloud Integration: Utilizing platforms like AWS to build scalable, cost-effective solutions, ensuring reliability and performance in cloud environments. My Process I combine design thinking, data-driven strategies, and cutting-edge tools to ensure that my solutions not only meet technical requirements but also align with broader business goals. Here’s how I work: ◆Discovery: I start by understanding your business needs, your target audience, and the challenges you're facing. ◆Execution: I create and optimize data pipelines, dashboards, and reports tailored to your needs, ensuring they are scalable and effective. ◆Results: I deliver insights that empower your teams to make informed decisions, improving key business outcomes such as conversion rates, operational efficiency, and cost savings. Why Work With Me? 🔥 Results-Oriented: I focus on delivering measurable improvements in efficiency and insight. ⚡ Technical Expertise: With proficiency in SQL, Python, Apache Spark, and cloud platforms like AWS, I bring technical precision to every project. 📈 Business-Driven Approach: I align every solution with your strategic goals to maximize ROI. 🎯 Dependable & Efficient: I’m committed to delivering quality work on time and within budget. Tools & Technologies I Use 💻 Python, SQL, PySpark, Apache Kafka 🌐 AWS (S3, Redshift, EC2), Azure 📊 Power BI, Tableau 📂 PostgreSQL, MySQL Availability 🕘 9 am – 7 pm (GMT), Monday – Friday ⚡ Open to urgent work and long-term collaborations. Let’s collaborate to harness the power of your data and drive your business forward.

With over 15+ years of hands-on experience, I am a highly skilled and certified Enterprise Application and Product Engineering professional. I'm 1x Databricks certified and 2x Google certified Professional Data Engineer. I am adept at leveraging the power of Google Cloud, AWS, and Databricks to create innovative and efficient solutions. LICENSES & CERTIFICATIONS ● Databricks Certified Professional Data Engineer Credential: credentials.databricks.com/b3da0de6-1efd-47b2-bf5b-d1b8a83bc1a5 ● Professional Data Engineer - Google Cloud Credential: google.accredible.com/a9d0ae0d-a5e5-433d-b8f4-330f6147c725 ● Professional Cloud Database Engineer - Google Cloud Credential: google.accredible.com/287a4094-e445-401c-9f37-e50915a7ae42 EXPERIENCE Project 1 - Senior Data Engineer (Consultant) ● web3 solution built using Google Cloud Platform (GCP) and Databricks combines proprietary data with patented algorithms to deliver unique insights, increase productivity, and enable fast, actionable decision-making within the crypto economy ● Built Extract pipeline using PySpark, Databricks workflow, Databricks Jobs, and Databricks Delta Tables to fetch Bitcoin, Ethereum, Arbitrum, and Polygon transaction data into the Databricks platform. ● Built Transformation pipelines to apply required business logic to convert the ingested data into meaningful information ● Built load pipeline to store data in Google BigQuery for use in analytics tools. ● Tech Used: GCP, DataBricks, Spark, Python, Apache Airflow, BigQuery Project 2 - Senior Data Engineer (Consultant) ● Project 01: Design and develop a streaming big data ETL pipeline using GCP services ● Built ETL pipeline using Apache Beam, Dataflow, Composer, and Java to read messages from Kaa and Pub/Sub and store the transformed message in BigQuery and Google Cloud Storage (GCS) ● Configured an Apache Kafka cluster in GCP to consume incoming messages ● Use Google Secret Manager to manage passwords and other important info ● Building a streaming Extract, Transform, Load (ETL) pipeline using Dataflow to consume data from the Kafka topics ● Design and build data warehouses using Google Big Query ● Store the data from Dataflow to BigQuery Data lake Project 3 - Senior Data Engineer (Consultant) ● Design and Implement batch pipelines to sync data from MongoDB to Databricks Delta Tables. ● Built ETL pipeline using Unity Catalog, Workflow, Jobs, Python, and PySpark to read messages from MongoDB and store the transformed message in Delta Tables ● Implement scheduled Jobs to run the batch pipeline every 12 hours. ● Other Technology Used: Spring boot, Java 8, Python, Git, Maven, Github, IntelliJ, Eclipse, JUnit, Google Cloud Platform. Project 4 - Senior Data Engineer (Consultant) ● Project 01: Design and Implement batch and streaming Big Data transformation pipelines using GCP Services. ● Built and maintained batch pipelines using Dataflow, Pub/Sub, Storage bucket, Composer ● Built and maintained real-time pipelines using GKE, Kubernetes, Compute Engine, Container Registry ● Used Firestore, MemoryStore, and Redis to store intermediate data ● Implement RESTful applications using Spring boot ● Built user-facing application using Spring MVC ● Other Technology Used: Spring boot, Java 8, Mockito, ELK, Tomcat, Git, Maven, GitHub, Bitbucket, IntelliJ, Eclipse, JUnit, Google Cloud Platform, Cloud SQL, Jenkins. Project 5 - Senior Cloud Engineer (Consultant) ● Credit Card Customer Analyzer ● Built ETL pipeline using Spark, and Python to read messages from Credit history and generate insights on potential customers based on their card usage ● Implement RESTful applications using Spring Boot ● Other Technology Used: Spring boot, Java 8, Python, Git, Maven, Github, IntelliJ, Eclipse, JUnit, Google Cloud Platform.

Most growing teams don't need a full-time data engineer — they need the right one, part of the time. I embed with SaaS and tech companies as a fractional data engineer, acting as a senior technical partner to build reliable pipelines, clean up messy data infrastructure, and deliver systems your team can actually maintain. ✓ 60% infrastructure cost reduction for a global nonprofit ✓ 3× faster data access for Carrefour (Fortune 500) ✓ First production data platforms built from scratch for 3 startups ✓ Zero technical debt left behind with past clients (dbt + Metabase) ------ This works well if you: — Have multiple data sources that need integrating — Are spending 10+ hours/week on manual data work — Are weighing a full-time hire but not ready to commit ------ Services Fractional Data Engineering — Ongoing embedded engagement. I act as your senior data engineer part-time, owning pipelines, architecture decisions, and knowledge transfer. Pipeline Build & Automation — Production-grade ETL/ELT on AWS, BigQuery, or Snowflake using Airflow, dbt, Lambda, and Glue. Platform Architecture — End-to-end data platform design for startups and scaling teams, with stack selection and cost optimization baked in. Audit & Roadmap — I assess your current setup, find what's draining time and budget, and deliver a prioritized 90-day plan. ------ Stack AWS (S3, Glue, Lambda, MWAA, Athena, Step Functions) · BigQuery · Snowflake · Python · SQL · Airflow · dbt · Spark · Terraform · Docker · Metabase · Looker Studio ------ Why clients hire me 6+ years delivering production data platforms · business-first mindset · predictable delivery, no surprises · everything I build is yours to own and maintain