Work as tech lead & chief architect for a web company serving 4m+ users. Collect and store user behavior data & transaction data, perform daily and ad hoc ETL on the data, find useful pattern and provide BI visualization; design & develop server backend and providing apis to end users. Able to help a middle sized tech firm to setup OLAP(Online Analytical Processing) & OLTP(Online Transactional Processing) platform from zero. Has rich working experience in Spark, MongoDB, Fastapi, Clickhouse, Pandas, Numpy, Terraform, Agent development, AWS environment(Lambda, Athena, DynamoDB, Glue, EMR, Bedrock, Cognito, QuickSight, Sagemaker, etc). Rich coding experience in Python/Scala/Java. Rich experience in Machine Learning especially NLP. Can work as part-time job, 10~20 hours per week.

I'm a Senior Data Engineer (6+ years) who helps teams build data platforms that are fast, reliable, and cost-efficient and keep them that way in production. My focus is on analytics-ready systems: ClickHouse (OLAP), Lakehouse/Delta Lake, ETL/ELT, Python/SQL, Spark/Polars, orchestration (e.g., Temporal), and delivery to BI tools (e.g., Apache Superset). I care about practical trade-offs: performance vs cost, normalized vs flat models, and simplicity vs over-engineering. What I deliver (Data Engineering - primary) * Scalable Lakehouse pipelines (Bronze/Silver/Gold): batch + incremental processing, dependable datasets * Strong analytics foundations: Star Schema/Data Vault/DWH modeling for real product use cases * Performance & cost optimization: schema/partitioning, ingestion patterns, and query tuning * Production reliability: retries, monitoring/observability, clear ownership, fewer incidents How I help (Consulting - secondary)

13+ years of full-stack experience Stack: Ruby, JavaScript, Python, Go DevOps Skills: AWS, Docker, Kubernetes Favorite Tech: Rails, React, Elastic, PostgreSQL, ClickHouse, Kafka, MongoDB Areas of Expertise: Proficient in developing large-scale, reliable applications for high-load environments. Experienced in redesigning and refactoring legacy projects, including transitioning to microservices and optimizing performance. Skilled at creating custom APIs and integrating with existing ones, including Payment, Social, EDI, and Ticketing. Well-versed in video broadcasting solutions. Adept at implementing complex search systems. Industry Experience: Online gaming Trading platforms Payment systems Marketing platforms Video platforms User review and ticketing websites Recent Projects: (Links are prohibited, but you can easily google it) Earth-2 (Game) Scompler (Marketing Platform) Show-Score (User Reviews and Tickets)

𝗠𝗼𝘃𝗶𝗻𝗴 𝗔𝗜 𝗳𝗿𝗼𝗺 "𝗖𝗼𝗼𝗹 𝗗𝗲𝗺𝗼𝘀" 𝘁𝗼 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻-𝗥𝗲𝗮𝗱𝘆 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 Stop wasting budget on hallucinating chatbots and slow prototypes. I architect high-performance multi-agent workflows and enterprise RAG pipelines that deliver measurable ROI not just code. Most AI projects fail because of high latency, low accuracy, and poor coordination. I bridge that gap by building production-grade systems that are cost-efficient, lightning-fast, and battle-tested for real-world complexity. 𝗠𝘆 𝗖𝗼𝗿𝗲 𝗘𝘅𝗽𝗲𝗿𝘁𝗶𝘀𝗲 𝗛𝗶𝗲𝗿𝗮𝗿𝗰𝗵𝗶𝗰𝗮𝗹 𝗠𝘂𝗹𝘁𝗶-𝗔𝗴𝗲𝗻𝘁 𝗦𝘆𝘀𝘁𝗲𝗺𝘀 I design autonomous agent teams using CrewAI and LangGraph where coordinators manage workflow orchestration while specialized sub-agents handle research, analysis, and execution. These systems solve complex problems like real-time meeting intelligence, customer support routing, and automated development without constant supervision. 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲-𝗚𝗿𝗮𝗱𝗲 𝗥𝗔𝗚 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 Building hallucination-resistant knowledge engines using LangChain, LlamaIndex, and ClickHouse for ultra-fast vector search. My implementations handle 10,000+ documents with sub-second latency through hybrid search, semantic chunking, and intelligent re-ranking. 𝗙𝘂𝗹𝗹-𝗖𝘆𝗰𝗹𝗲 𝗟𝗟𝗠𝗢𝗽𝘀 & 𝗣𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗜𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲 Engineering robust MLOps pipelines with Docker, Kubernetes, and NVIDIA Triton. I implement circuit breakers, evaluator patterns, and real-time monitoring using ClickHouse for unparalleled query speed and observability. Every system includes token tracking, cost alerts, and failure prevention. 𝗠𝗼𝗱𝗲𝗹 𝗙𝗶𝗻𝗲-𝗧𝘂𝗻𝗶𝗻𝗴 & 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Customizing open-source models (Llama 3.1, Mistral, Granite 4.0) using LoRA/PEFT and federated fine-tuning for domain-specific performance. I optimize inference pipelines with vLLM and TensorRT-LLM, achieving up to 23x throughput improvements. 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲 𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆 & 𝗚𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 Implementing multi-factor authentication, role-based access control, and ISO 42001 compliance frameworks. My systems include audit trails, encrypted data handling, and cryptographic signing for model provenance. 𝗜𝗻𝗱𝘂𝘀𝘁𝗿𝗶𝗮𝗹 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝗩𝗶𝘀𝗶𝗼𝗻 Applying CNNs and YOLOv8 for real-time object detection and vision-to-decision pipelines in specialized use cases. 𝗧𝗲𝗰𝗵𝗻𝗶𝗰𝗮𝗹 𝗦𝘁𝗮𝗰𝗸 𝗠𝗼𝗱𝗲𝗹𝘀: 𝗚𝗣𝗧-𝟰𝗼, 𝗖𝗹𝗮𝘂𝗱𝗲 𝟯.𝟱, 𝗟𝗹𝗮𝗺𝗮 𝟯.𝟭, 𝗠𝗶𝘀𝘁𝗿𝗮𝗹, 𝗚𝗲𝗺𝗶𝗻𝗶 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸𝘀: 𝗖𝗿𝗲𝘄𝗔𝗜, 𝗟𝗮𝗻𝗴𝗚𝗿𝗮𝗽𝗵, 𝗟𝗮𝗻𝗴𝗖𝗵𝗮𝗶𝗻, 𝗟𝗹𝗮𝗺𝗮𝗜𝗻𝗱𝗲𝘅 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻: 𝘃𝗟𝗟𝗠, 𝗧𝗲𝗻𝘀𝗼𝗿𝗥𝗧-𝗟𝗟𝗠, 𝗡𝗩𝗜𝗗𝗜𝗔 𝗧𝗿𝗶𝘁𝗼𝗻, 𝗠𝗼𝗱𝗲𝗹 𝗤𝘂𝗮𝗻𝘁𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗜𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲: 𝗖𝗹𝗶𝗰𝗸𝗛𝗼𝘂𝘀𝗲, 𝗤𝗱𝗿𝗮𝗻𝘁, 𝗣𝗶𝗻𝗲𝗰𝗼𝗻𝗲, 𝗗𝗼𝗰𝗸𝗲𝗿, 𝗞𝘂𝗯𝗲𝗿𝗻𝗲𝘁𝗲𝘀, 𝗔𝗪𝗦/𝗚𝗖𝗣 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗼𝗻: 𝗻𝟴𝗻, 𝗭𝗮𝗽𝗶𝗲𝗿, 𝗣𝘆𝘁𝗵𝗼𝗻 (𝗙𝗮𝘀𝘁𝗔𝗣𝗜/𝗙𝗹𝗮𝘀𝗸) 𝗪𝗵𝘆 𝗪𝗼𝗿𝗸 𝗪𝗶𝘁𝗵 𝗠𝗲 Production Focus: I build for scale with systems that maintain 99.9% accuracy under real-world load. Every solution includes error handling, retry logic, and comprehensive monitoring. 𝗖𝗼𝘀𝘁 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻: Reduce token costs by 40-70% through intelligent caching, prompt compression, and strategic model selection. Full transparency with consumption tracking and budget alerts. Coordination Complexity Solved: I handle the hardest part of multi-agent systems ensuring agents communicate effectively, avoid redundant work, and maintain state consistency across distributed workflows. 𝗗𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝗮𝘁𝗶𝗼𝗻 𝗙𝗶𝗿𝘀𝘁: Every build includes clear architecture maps, runbooks, and repeatable processes so your team can maintain and scale what I build. 𝗣𝗿𝗼𝘃𝗲𝗻 𝗥𝗢𝗜: My clients see 12-18 month competitive advantages through early adoption of sophisticated multi-agent architectures that are rapidly becoming industry standard. 𝗥𝗲𝗮𝗱𝘆 𝘁𝗼 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁 𝗮 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻 𝗔𝗜 𝘀𝘆𝘀𝘁𝗲𝗺 𝘁𝗵𝗮𝘁 𝗱𝗲𝗹𝗶𝘃𝗲𝗿𝘀 𝗺𝗲𝗮𝘀𝘂𝗿𝗮𝗯𝗹𝗲 𝗯𝘂𝘀𝗶𝗻𝗲𝘀𝘀 𝘃𝗮𝗹𝘂𝗲? 𝗟𝗲𝘁'𝘀 𝗱𝗶𝘀𝗰𝘂𝘀𝘀 𝘆𝗼𝘂𝗿 𝗿𝗼𝗮𝗱𝗺𝗮𝗽 𝗶𝗻 𝗮 𝟭𝟱-𝗺𝗶𝗻𝘂𝘁𝗲 𝘀𝘁𝗿𝗮𝘁𝗲𝗴𝘆 𝗰𝗮𝗹𝗹.

"Work with David was excellent. He knows his stuff!"" I've been helping businesses large and small gain insights from their data for over 20 years. I enjoy pushing technical limits, solving difficult problems, writing clean, efficient queries and building intuitive and beautiful data visualizations for my clients.

I build production-grade data and AI infrastructure. If you need someone who can architect a RAG pipeline, set up observability for a platform processing millions of events, or build a knowledge graph, that's me. What I bring: At Shopify, I spent 4 years building the Logging & Distributed Tracing Analytics platform that had ClickHouse clusters ingesting 15M+ spans/sec, managing 2+ PB/week of infrastructure data. I designed the Query Builder, Schema Explorer, and implemented TraceQL from the OpenTelemetry spec. I also built a GPT-powered chatbot for trace-based developer support. I'm the author of CogDB, an open-source graph database with 338+ GitHub stars and 151K+ PyPI downloads. It's used by a Swiss university for geodata knowledge graphs and in Big Data curriculum at Bellevue University. Before Shopify, I built analytics pipelines at scale - Protocol Buffers APIs with 4x performance gains, graph-based ad targeting systems, and distributed file storage at Oracle. What I can help you with: → RAG pipeline architecture & implementation (embedding pipelines, vector stores, retrieval optimization, knowledge graph-augmented generation) → Observability & monitoring infrastructure (ClickHouse, OpenTelemetry, Prometheus, Grafana, distributed tracing) → Knowledge graph design & implementation (graph databases, entity extraction, relationship modeling, graph queries) → AI/LLM integration (OpenAI API, chatbots, AI-powered analytics, prompt engineering for production systems) → Data pipeline engineering (Kafka, Spark, S3, Parquet, ETL/ELT, real-time streaming) Tech stack: Python, Go, TypeScript, Java | ClickHouse, BigQuery, Elasticsearch, Redis, Postgres, Supabase, Firebase | Kubernetes, GCP, AWS | React, Next.js Stanford Graduate Certificate in Mining Massive Datasets (ML, Deep Learning for NLP, Information Retrieval). I work fast, communicate clearly, and ship production-quality code. Happy to discuss your project.

I am a seasoned database professional with more than 15 years of specialized experience, primarily focused on MySQL and MariaDB, both replication as well as Galera Cluster and InnoDB Cluster. Those skills are complemented by proficiency in PostgreSQL, ClickHouse, Redis, and MongoDB. My expertise lies in database performance analysis and optimization, as well as the design and implementation of robust, high-availability architectures. I possess in-depth knowledge of load balancing technologies, including ProxySQL, MaxScale and HAProxy. On PostgreSQL side I've been working with pgBouncer and pgPool. I have successfully managed complex systems in both AWS and on-premises environments, ensuring exceptional reliability and efficiency.

Most automation freelancers can wire up n8n or Make. Few can also build the backend the automation depends on, the API integrations, the search and data layer, the pipelines that don't fall over at scale. I do both. I'm a data & AI engineer with 5+ years building production systems: workflow automation (n8n, Make), LLM/agent integrations, Elasticsearch/ELK, Kafka pipelines, and data engineering on enterprise-scale datasets. What I deliver: - AI automation that holds up - n8n/Make workflows with LLM agents for extraction, classification, routing, and decisioning, built with real error handling and batch orchestration, not happy-path demos. - The hard integrations - REST/SOAP APIs, webhooks, Elasticsearch, Kafka, SQL/NoSQL - so your automation connects to the systems that matter, not just the no-code-friendly ones. - Search & data layer - Elasticsearch index design, ingestion pipelines, query tuning, including Arabic-language search and NLP (a rare specialization). Recent work: an n8n + vision-LLM pipeline that auto-classified ~50,000 passport images by issuing country; a bidirectional Arabic–English name-translation pipeline updating Elasticsearch via point-in-time pagination; AI reporting automation that cut processing time ~40%. Top Rated · 100% Job Success · 1,600+ hours. I lead with the recommendation, flag trade-offs, and don't disappear mid-project.

Hello I'm Jay, I have 10+ years of hands-on experience building data solutions for fintech, telecom, and media, working with high-volume, real-time data in complex production environments. Whether you need an end-to-end cloud migration, delta lake architecture, or query performance tuning, I deliver clean, production-ready solutions designed for long-term growth. Core Services & Capabilities: ✔ Data Warehousing & Architecture: Hive SQL, Synapse Analytics, Medallion Architecture (Delta Lake) ✔ Advanced Analytics & Python: Data transformation, PySpark, API integrations, web scraping ✔ Business Intelligence: Executive Power BI dashboards, Tableau ✔ Database Migration & Optimization: Legacy SQL Server to Azure, PostgreSQL Technologies I Work With Daily: • Cloud & Big Data: Databricks, Delta Lake • Databases: PostgreSQL, MySQL, MSSQL, Oracle, Hive, Impala, MariaDB, TiDB , Iceberg • Languages & Automation: Java, Python, Scala, Bash • Analytics & BI: Power BI, Tableau, Microsoft Fabric dashboards • Cloud & Infrastructure: Docker, AWS (S3, EC2, RDS, Lambda, SQS, Glue) • Streaming & Orchestration: Apache Kafka, Apache Flink (Java / Python /Scala / SQL), Apache Spark / Spark Streaming, Apache Airflow, Kafka Connect, Apache Doris, Clickhouse, Apache Oozie, Kettle)

I am an AI Automation & Data Engineer specializing in building scalable agentic workflows, orchestrated ETL pipelines, and high-volume web scraping systems. I bridge the gap between full-stack Python development and modern AI integration to deliver production-grade automation that optimizes business processes and unlocks the value of data. Core Expertise Web Scraping & Data Extraction: Over 5 years of engineering highly resilient, distributed web crawling frameworks using Scrapy, Selenium, and Playwright. Deeply experienced in advanced data mining, anti-bot bypass mechanisms, proxy rotation, and e-commerce data extraction. AI & Agentic Workflows: Designing intelligent automation, autonomous agents, and RAG systems using LangChain and OpenAI APIs for advanced data enrichment. Data Engineering & ETL: Building and managing robust data pipelines using Prefect to orchestrate data warehousing across ClickHouse, Snowflake, and PostgreSQL. Full-Stack & API Integrations: Developing backend automation tools using Flask, RESTful APIs, and async workers to seamlessly sync data with HubSpot, Apollo, and Google APIs. Databases & Cloud: Managing diverse data layers (PostgreSQL, MySQL, MongoDB, Redis) deployed across cloud infrastructures utilizing AWS and Docker.