Hire the Best Apache Spark Engineers
in Vietnam

I am a highly motivated and passionate data engineer. I often work with Python, Scala, and Java and use the latest big data technologies to solve problems, making tools to improve my and others' work productivity. I am currently certified with AWS Solution Architect and SnowPro. Check out my blog for my work and newsletter lam-tran.dev

Top 1% on Upwork — Top Rated Plus, 100% Job Success, 5.0★ (100+ reviews). I build production AI agents (LangGraph, RAG, MCP), data pipelines & web scraping. Founder of Van Data Team: 20+ AI products shipped, 9 of them live public platforms, for clients across the US, EU, and APAC. 🚀 WHAT I'VE SHIPPED (live products, real numbers) - Enterprise AI assistant — LangGraph Supervisor routing sub-agents across 70+ MCP tool integrations; web, PWA, iOS & Android from one React codebase - AI cybersecurity platform — fine-tuned Mistral 7B on 424k training samples; Neo4j knowledge graph + RAG - EdTech RAG platform — raised answer accuracy from ~80% to 95%+ (pgvector, LangChain, Claude) - Multi-tenant enterprise AI suite — 5-agent architecture with persistent memory, running at 94.7% platform health - Sales-KPI AI dashboard — cut LLM cost by 92% with Claude Haiku - My own SaaS — multi-tenant AI content agents, personalized per tenant (live product, see Portfolio section) 💬 AI AGENTS, CHATBOTS & LLM APPS (Full-Stack) - AI Agent Development: autonomous agents with tool calling, multi-step reasoning, and decision-making (LangGraph, LangChain, MCP) - Multi-Agent Systems: supervisor patterns, agent-to-agent communication, task delegation - AI Chatbots: customer-service bots, document Q&A, and conversational AI grounded in your business data (RAG, no hallucinations) - Custom MCP servers: turn your internal tools into capabilities for Claude, Cursor, and other LLM clients - Models & APIs: Claude (Anthropic), GPT-4/4o (OpenAI), Gemini, DeepSeek, Llama 3, Hugging Face; AWS Bedrock, Azure OpenAI, Vertex AI - Vector databases: Pinecone, Milvus, ChromaDB, pgvector, FAISS + Neo4j graphs - Observability & evals: LangSmith, Langfuse; fine-tuning for domain-specific tasks - Frontend: Next.js, React, TypeScript, Tailwind — Backend: FastAPI, Flask, Django, NestJS, Node.js/Express, GraphQL Perfect for: support automation, sales chatbots, document Q&A, intelligent search, business-process automation. ⚡ DATA ENGINEERING & ETL PIPELINES Production infrastructure processing millions of records: - Pipelines: Apache Spark, Airflow, Dagster, dbt • Streaming: Kafka - Batch: AWS Glue, Google Dataflow, Azure Data Factory - Warehouses: BigQuery, Snowflake, Redshift, Athena — optimized schemas, HIPAA-style audit & lineage - Zero-downtime migrations to cloud warehouses 🕷️ WEB SCRAPING & DATA EXTRACTION - Platforms: Facebook, TikTok, LinkedIn, Shopify, HubSpot, QuickBooks, SaaS portals - Tools: Scrapy, Playwright, Selenium, Puppeteer, BeautifulSoup, Crawl4AI - Anti-bot: Cloudflare, CAPTCHA, rate limiting — solved with proxy rotation & monitoring - Output: clean CSV/JSON, SQL databases, or a ready-to-use API ☁️ CLOUD & DEVOPS AWS (Lambda, ECS Fargate, Glue, Athena, Redshift, SQS, EventBridge, EMR) • GCP (BigQuery, Cloud Run, Dataflow, Dataproc) • Azure (Data Factory, Databricks) • Docker, Kubernetes, Terraform, GitHub Actions & GitLab CI/CD • Monitoring: CloudWatch, Datadog, Prometheus, Grafana ✅ WHY CLIENTS PICK ME - Founder-led, senior-only execution — you work with me directly, no agency layer - AI-native workflow (Claude Code, Cursor, Copilot): 2–3x faster delivery, production quality - 6+ years across AI engineering, data engineering, and full-stack development - End-to-end ownership: architecture → code → deployment → monitoring - Clear communication: regular updates, transparent workflow, overlap with US/EU hours Let's build your AI solution — send an invite or message me, and I'll reply with a concrete plan within hours.

⏰ Available 24/7 – Long-term & High-impact Projects Hi, I’m Nghi, a Senior Data Engineer and Data Architect with a strong backend foundation, now focused on building high-performance analytics platforms, explainable data pipelines, and production-grade cloud architectures. I help companies transform unreliable, slow, or opaque data systems into scalable, well-documented, and business-trustworthy platforms. 🧠 WHAT I SPECIALIZE IN 🏗️ Data Architecture & Platform Design - Designing modern lakehouse & warehouse architectures - dbt-first analytics engineering with testing, freshness & lineage - Event-driven and batch hybrid pipelines - Data quality frameworks & SLA monitoring - Customer-facing data explainability systems Tools: dbt, Dagster, Airflow, Spark, Kafka, Snowflake, BigQuery, Redshift, PostgreSQL, DuckDB, ClickHouse ⚡ Database Performance Engineering - If your queries are slow, costs are high, or dashboards lag, This is my zone - Query plan analysis & index strategies - Warehouse cost optimization (Snowflake, BigQuery, Redshift) - OLTP & OLAP performance tuning - High-concurrency workload design 🔄 Reverse ETL & Operational Analytics - Syncing analytics back to CRMs & internal tools - Building real-time metrics pipelines - Feature-store style transformations 🕷️ Enterprise-grade Web Data Extraction - I don’t just scrape pages, I build durable data acquisition systems: - Complex ASP.NET, JS-heavy, authenticated & paginated systems - Anti-bot bypassing & failure-recovery pipelines - Headless browser automation + async scraping - Real-estate, finance, campaign-finance & marketplace platforms ☁️ Cloud Infrastructure - AWS | Azure | GCP - EMR / Dataproc / Glue / Dataflow / Synapse / BigQuery / Redshift - Terraform-based deployments - Cost-aware architectures - Kubernetes + Dockerized data services 🧪 What You Get Working With Me ✔️ Production-ready pipelines ✔️ Clean, testable dbt models ✔️ Well-documented architecture diagrams ✔️ Transparent data logic for non-technical stakeholders ✔️ Systems that scale beyond MVP ✔️ Honest advice and not over-engineering 🏆 Ideal Projects 👍 Data warehouse migrations 👍 Broken pipelines that need debugging & stabilization 👍 Analytics platforms that lack trust or explainability 👍 Performance bottlenecks costing thousands per month 👍 Long-term data platform ownership ❣️ Why Clients Stay Long-Term 🍀Clear communication 🍀 Business-first thinking 🍀 No black-box systems 🍀 I build systems others can maintain 🇻🇳🇻🇳🇻🇳🇻🇳 If your data platform feels fragile, slow, or impossible to explain to customers, I can fix that. Let’s make your data system something you can confidently stand behind.

I have experience with Data Scraping, RDBMS, NoSQL DB, building ETL pipeline, Big Data - Data Scraping with Python with Scrapy, Selenium, Python Request. - Expert SQL with Oracle, MySql. - Expert AWS in Data Engineering field - Experience with NoSQL: ElasticSearch, HBase, Cassandra. - Experience with Minio, AWS, Kafka, Terraform, Redis - Expert with Spark, HDFS Looking for long term contract

I have 6 years of working experience in the software industry, and I am confident most as a Backend Developer and Data Engineer. For now, I want to spend more time on the freelance path, to help others solve their problem happily. As a backend developer: - I build microservices using Java and Go - I design a Database schema for features I work on. - I write REST API and work with the frontend team for integration. As a data engineer: - I do ETL tasks. - I write batch jobs and real-time jobs for streaming over 50 GB of data per day. - I store processing data into HBase, MySQL. - I visualize data to make it meaningful to others. Besides, I also spend time playing with frontend stuff such as NextJS, VueJS, CSS. Availability: My time zone is GMT+7, but I am flexible on working time, even at midnight. If you want to work with me for more than 30 hours, just say it, I will consider working more. Let’s work together to turn your idea into reality!

- Build & maintain ETL pipelines for small to medium sized companies - Visualize data using Power BI - Preprocess data using regex or other techniques - Custom Python scripts