Data Analyst/Data Engineer for DatabaseEmailer.com

Data Analyst / Data Engineer — Large-Scale B2B & Consumer Contact Data (700M+ Rows, ClickHouse + Postgres) Posting type: Hourly · Long-term · 20–40 hrs/week · Remote (worldwide) About the work DatabaseEmailer.com is a B2B contact data platform. We license, clean, enrich, and deliver large consumer and business contact datasets to customers who use them for outreach and audience building. Our warehouse currently holds 700M+ rows in ClickHouse Cloud, with Postgres (Neon) running the transactional side of the product and a scheduled sync between the two. The data we license arrives messy: mixed vendor enrichment codes, malformed date columns, header rows leaking into the body, inconsistent field semantics between vendor drops, and duplicate identities across sources. Someone has to make it trustworthy — and then build the pipelines so it stays trustworthy on every refresh. That's this role. It's roughly an even split between analyst work (profile it, find what's broken, quantify it, fix it) and engineering work (turn the fix into a repeatable, monitored pipeline). What you'll actually do Profile large tables and produce clear data-quality reports: null rates, cardinality, format violations, duplicate clusters, distribution drift between vendor drops Write and run cleaning and normalization jobs (Python/SQL, Jupyter or scripts) across hundreds of millions of rows without melting the cluster Build and maintain ingestion pipelines for new vendor files — parsing, validation, staging, dedupe, load Own and improve the Neon Postgres → ClickHouse sync: correctness, incrementality, schedule, failure handling Design and tune ClickHouse schemas — table engines, ORDER BY/primary keys, partitioning, materialized views, projections — so customer-facing queries stay fast Write reusable data-quality checks that run automatically on every load and alert when something regresses Build ad-hoc analyses and internal reporting for the founders: coverage by segment, match rates, enrichment lift, deliverability signals Document what you build clearly enough that the next person doesn't have to reverse engineer it Required 3+ years working with real production data at scale (hundreds of millions of rows, not 10k-row CSVs) Strong SQL. Window functions, CTEs, query plans, and a genuine instinct for why a query is slow Strong Python for data work — pandas/Polars/DuckDB, plus comfort writing plain, memory-conscious scripts when a dataframe won't fit Hands-on experience with a columnar/analytical database — ClickHouse strongly preferred; BigQuery, Snowflake, Redshift, or Druid experience transfers if you can speak to the trade-offs Postgres experience, including schema design and bulk-load patterns Practical data-cleaning judgment: you can look at a column and tell us what's wrong with it, how much of it is wrong, and what it costs to fix Clear written English — most of our collaboration is async and written Reliable overlap with US Eastern hours (at least 3–4 hours/day) Nice to have ClickHouse Cloud specifically, and experience tuning it for high-cardinality string data dbt, Dagster, Airflow, or similar orchestration Kubernetes / containerized job experience (our stack runs on K8s) Experience with contact, identity, or marketing data — email validation, address normalization, identity resolution, suppression and consent handling Git-based workflow and code review as a normal habit Our stack ClickHouse Cloud · Neon Postgres · Python · SQL · Jupyter · Kubernetes · Git Details Engagement: hourly, long-term, ongoing. Starting around 20 hrs/week with room to grow to full-time for the right person Location: anywhere, as long as you have the EST-hours overlap Start: immediately How to apply Generic proposals get skipped. Please include: A specific example of a large, messy dataset you cleaned or a pipeline you built — what was broken, what you did, and what changed as a result. Numbers are welcome. How you'd approach this: you're handed a 700M-row table from a new data vendor with no documentation, and you have two days before it has to be queryable by customers. What do you check first, and what do you refuse to ship without? Your ClickHouse (or comparable columnar DB) experience — largest table you've worked with, and one performance problem you solved. Your available hours per week and your time zone. Start your proposal with the word "ORDER BY" so we know you read the whole post. Shortlisted candidates will be paid for a short, scoped test task on a sample of real data before any long-term commitment.