Data Architect Consultation: Document Database for Historical Change Tracking

Data Architect Consultation: Document Database for Historical Change Tracking We are seeking experienced data architects and engineers for a paid discovery engagement. We have accumulated monthly snapshots of a large administrative dataset spanning several years and are exploring approaches for converting these periodic extracts into historical entity records using a document database approach. Our goals include: • Preserving record-level changes over time • Maintaining flexible historical records as source data changes • Identifying meaningful changes in status, attributes, locations, and relationships • Supporting future prediction, experimentation, and alerting workflows At this stage, we are not looking for someone to immediately build the system. Instead, we are looking for practitioners who can review the problem, ask questions, and propose potential architectures, technologies, and implementation approaches. Selected candidates will be invited to a paid consultation where we will share additional details about the dataset, existing analyses, and project objectives. ## Examples Perhaps you have designed systems such as: • A healthcare platform that reconstructs patient histories from periodic exports of electronic health records and preserves changing attributes over time. • A bank or fintech platform that maintains historical customer profiles, detects meaningful changes in account behavior, and flags accounts at risk of fraud, churn, or closure. • A customer data platform that ingests periodic CRM exports and maintains evolving customer profiles, addresses, subscriptions, and engagement histories. • An insurance system that tracks policyholders over multiple years, including coverage lapses, address changes, claims history, and renewal risks. • A logistics or supply chain platform that reconstructs shipment, vendor, or inventory histories and generates alerts when patterns change. • A public sector system that maintains historical records of residents, permits, licenses, benefits recipients, or other administrative records. • A census, demographic, or population research project that follows individuals or households over time to study mobility, participation, or attrition. • A university system that tracks students longitudinally, identifies those at risk of dropping out, and triggers interventions from advisors. If you have worked on systems like these, especially using document databases or flexible schema approaches, we would be interested in hearing how you approached data modeling, identity resolution, historical change tracking, and alert generation. ## To Apply Please tell us: 1. Briefly describe your experience designing document databases or flexible-schema systems that preserve historical changes over time. 2. What industries or domains have you worked in? Examples might include healthcare, finance, insurance, customer data platforms, logistics, higher education, public sector, civic technology, or scientific research. 3. If you were handed monthly snapshots of a large dataset spanning several years, what questions would you ask before proposing an architecture? 4. What technologies would you initially consider and why? Examples might include MongoDB, Couchbase, Elasticsearch/OpenSearch, BigQuery, PostgreSQL JSONB, object storage, dbt, Airflow, Python, or hybrid architectures. 5. Have you designed systems involving any of the following? • Document databases • Flexible or evolving schemas • Identity resolution • Historical record reconstruction • Temporal data modeling • Change Data Capture (CDC) • Event sourcing • Master Data Management (MDM) • Alerting systems ## Engagement Structure We expect to speak with several candidates, compensate them for their time, and then determine whether to engage an individual, firm, or team for a larger implementation effort. Compensation for the discovery consultation will be discussed with selected candidates.