Hire the Best Big Data Engineers
in Canada

Struggling with slow, unreliable data pipelines, rising cloud costs, or a data stack that can't keep up with your AI initiatives? I help businesses fix exactly that. I'm a Senior Data Engineer with hands-on experience helping businesses design, build, and scale reliable data infrastructure across Azure, GCP, and AWS. I specialize in end-to-end ETL/ELT pipelines, real-time data processing, and cloud-native data architecture using PySpark, Python, SQL, Databricks, and BigQuery. Across 21+ completed projects on Upwork, I've delivered more than $100K in data engineering solutions, including systems processing over 10 million records with performance improvements of up to 40%. My work spans automated ingestion frameworks, CRM and analytics integrations, RAG-based LLM pipelines, and cloud cost optimization without sacrificing performance. Clients consistently point to my clear communication, reliability, and commitment to quality, reflected in my 100% Job Success score and Top Rated Plus status. Whether you need a pipeline built from scratch, an existing data stack optimized, or an ongoing data engineering partner, I bring the technical depth and communication needed to get projects delivered on time and on budget. If you have a data challenge, let's connect and discuss how I can help move your project forward.

I build data systems that actually work in real-world conditions — not just clean demo pipelines. Most datasets are messy, incomplete, delayed, or inconsistent. I specialize in designing pipelines that handle that reality: cleaning, validating, and transforming data so it becomes usable for analytics and decision-making. I’ve worked on projects involving large-scale data processing, streaming pipelines, and financial data analysis, where accuracy and reliability matter more than anything. What I can help you with: • Build end-to-end data pipelines using Python, PySpark, and SQL • Design ETL workflows for messy, real-world datasets • Set up streaming systems using Kafka + Spark • Data cleaning, validation, and anomaly detection systems • Web scraping and API-based data collection • Financial data analysis and automation • Dashboard-ready data modeling (for BI tools) Tech Stack: • Python, PySpark, SQL • Databricks, Delta Lake • Kafka (Streaming Pipelines) • BigQuery (Analytics) • Airflow (Basic orchestration) • APIs, Web Scraping • Streamlit (Dashboards) I focus on building systems that are: Reliable • Scalable • Practical • Not over-engineered If you need someone who understands both the data and the system behind it — I can help.

Robin Huang CFA, FRM VB, VBS, VBA, VCBasic, SQL, Access 7 years Word, Excel, PowerPoint, Outlook 7 years HTML, CSS, JavaScript, JQuery, Python 4 years C++, Ruby, Swift, Xcode, Parse 1 year Spark, Anaconda, Jupyter, Linux, Unix 3 years Visual Studio, Report Builder, Publisher 4 years XML, Android Studio, Java, PHP 3 years Financial Modeling, Financial Engineering 7 years Power Pivot, Power Query, Power BI 5 years IBM OnDemand 4 years JIRA, Confluence, Drools, SSAS, SSIS 1 year IEX, FactSet, Interactive Brokers (IB) 2 years Good knowledge of SWIFT and Corporate Actions Good knowledge of DevOps, Jenkins and xMatters Good knowledge of Hadoop, HDFS, Hive, Impala Bloomberg, Thomson Reuters Eikon 4 years MS SQL Server, SDLC, ETL 7 years Capital Markets 7 years SSRS, SSMA, Vertica, SharePoint 3 years .Net, Xamarin, C#, XAML 4 years IBM Cognos TM1/ EPM, MDX Query 3 years Business Intelligence, Business Analysis 7 years VMware, MATLAB, Photoshop, Tableau 4 years STATA, R, SAS, Lean Six Sigma 3 years Data Science, Data Mining 4 years Oracle Business Intelligence, PeopleSoft 2 years Credit Risk, Market Risk 2 years Mainframe, OutsideView, Extra! X-treme 1 year Google GCP, Amazon AWS 2 years

AI Automation • Cloud Data Platforms • Agentic Workflows • Streaming Pipeline . Real-Time Analytics Major Clients & Key Projects: Ballys Interactive USA (Online Gaming): Online gaming streaming for game and props odds. Kafka, Airflow, Pyspark, Google Bigquery for realtime streaming. Walmart USA (Retail): High-Throughput Supply Chain Analytics: Re-engineered core inventory management pipelines using PySpark and BigQuery. Transitioned batch processing to a streaming-first design, reducing end-to-end processing times by 60% for real-time stock and logistics tracking. Rogers Communications Canada(Telecom): Real-Time Telemetry Pipeline: Architected decoupled, horizontally scalable streaming pipelines using Kafka/MSK and AWS Glue to process massive telecom network data. Implemented partitioning and backpressure strategies to stabilize SLAs and reduce latency by 70%. Bank of Nova Scotia Canada (Banking & Insurance): Cloud Modernization & Governance: Led the migration of legacy financial and risk data systems to a multi-cloud Snowflake and BigQuery environment. Integrated robust data tiering and access controls, lowering cloud OpEx by 20% while hitting strict compliance targets. Enterprise Clients USA(Construction): Operational AI Automation: Built low-code/no-code agentic workflows using n8n and LLMs to automate multi-vendor invoice data extraction, asset tracking, and project metadata ingestion into centralized PostgreSQL and Elasticsearch engines. Principal Data Engineer and Architect with extensive experience building cloud-scale data ecosystems and AI-driven automation frameworks. Proven track record transitioning legacy legacy architectures to modern, high-throughput, streaming-first environments across Telecom, Banking, Insurance, Retail, and Construction. Expert in optimizing multi-cloud unit economics (AWS, GCP, Azure) to accelerate time-to-insight while maintaining rigid governance standards. Core Tech Stack & Architecture Data Engineering & Orchestration: AWS Glue, Amazon Athena, MSK (Kafka), Apache Airflow, PySpark, real-time ETL/ELT lifecycles. Cloud Data Warehousing: Snowflake, Google BigQuery, Amazon Redshift; specialized in storage tiering, clustering optimization, and cross-platform governance. AI & Workflow Automation: Production LLM/RAG integration, n8n, Zapier, Claude Code; automated operational metadata and content pipelines. Modern Data Stack: PostgreSQL, Redis, Elasticsearch for high-performance caching, state management, and real-time streaming.

As a seasoned Data Engineer, I excel in crafting and optimizing data pipelines and systems that drive business success through data-driven insights. My technical prowess spans a wide array of technologies, ensuring streamlined data processing, storage, and integration that cater to individual business requirements. What I Bring to the Table Data Engineering Expertise: Adept in SQL, Python, PySpark, SparkSQL, and cutting-edge data frameworks for streamlined processing and transformation. Cloud Solutions Mastery: With extensive experience in Azure services such as Azure Data Factory, Azure Synapse Analytics, and Azure Blob Storage, I build scalable cloud-based solutions. Data Warehousing & Modeling: Skilled in developing high-performance data warehouses and implementing effective data models to support analytics and reporting. End-to-End Workflow Automation: Proficient in orchestrating workflows, automating ETL/ELT processes, and integrating diverse data sources into unified systems. Why Work With Me? Proven track record of delivering scalable, reliable, and secure data solutions aligned with business objectives. Commitment to clean, efficient, and maintainable code. Strong analytical and problem-solving skills to tackle complex data challenges. Transparent communication and dedication to timely, quality results. Services I Offer Design and development of data pipelines and workflows. Implementation of ETL/ELT processes for data integration and transformation. Cloud-based data engineering with Azure tools and platforms. Performance optimization for data processing jobs and systems. Development and management of data warehouses and models. Let’s Collaborate Whether you're embarking on a new data solution or enhancing existing systems, I can assist you in realizing the full potential of your data. Together, we can create impactful, efficient, and reliable data solutions that foster insights and growth for your business.

Most growing teams don't need a full-time data engineer — they need the right one, part of the time. I embed with SaaS and tech companies as a fractional data engineer, acting as a senior technical partner to build reliable pipelines, clean up messy data infrastructure, and deliver systems your team can actually maintain. ✓ 60% infrastructure cost reduction for a global nonprofit ✓ 3× faster data access for Carrefour (Fortune 500) ✓ First production data platforms built from scratch for 3 startups ✓ Zero technical debt left behind with past clients (dbt + Metabase) ------ This works well if you: — Have multiple data sources that need integrating — Are spending 10+ hours/week on manual data work — Are weighing a full-time hire but not ready to commit ------ Services Fractional Data Engineering — Ongoing embedded engagement. I act as your senior data engineer part-time, owning pipelines, architecture decisions, and knowledge transfer. Pipeline Build & Automation — Production-grade ETL/ELT on AWS, BigQuery, or Snowflake using Airflow, dbt, Lambda, and Glue. Platform Architecture — End-to-end data platform design for startups and scaling teams, with stack selection and cost optimization baked in. Audit & Roadmap — I assess your current setup, find what's draining time and budget, and deliver a prioritized 90-day plan. ------ Stack AWS (S3, Glue, Lambda, MWAA, Athena, Step Functions) · BigQuery · Snowflake · Python · SQL · Airflow · dbt · Spark · Terraform · Docker · Metabase · Looker Studio ------ Why clients hire me 6+ years delivering production data platforms · business-first mindset · predictable delivery, no surprises · everything I build is yours to own and maintain