Data Catalog & Governance Implementation

We are seeking an experienced professional to implement a data catalog and governance solution using Microsoft Purview and Azure Databricks Unity Catalog. The role involves designing and deploying a data catalog, ensuring data quality, and implementing data governance policies. The ideal candidate will have experience in data management and a strong understanding of Microsoft Azure services. ROLE REQUEST Data Engineer Contractor - Data Catalog & Governance Microsoft Purview / Azure Databricks Unity Catalog Implementation 1. Engagement Overview We have recently stood up an enterprise data lake on Databricks and are implementing Microsoft Purview as our organization-wide data catalog and governance layer. We are seeking an experienced contractor to lead the technical build-out, working alongside internal data platform and governance stakeholders. Architecture context: Databricks Unity Catalog is the system of record for technical metadata, lineage, and access enforcement within the lake. Purview is being layered on top as the org-wide discovery, business glossary, classification, and stewardship workflow tool, scanning Unity Catalog and other enterprise sources into a single catalog. 2. Role Summary The Data Catalog & Governance Contractor will lead the configuration, connection, and rollout of Microsoft Purview as the enterprise data catalog, with Azure Databricks Unity Catalog as the primary source system. This includes source onboarding, collection and glossary design, classification setup, lineage validation, and stewardship enablement. The contractor will work closely with the data platform team (Databricks/Unity Catalog owners), data domain owners, and security/compliance stakeholders. 3. Key Responsibilities Source Onboarding & Connectivity • Register and configure the Azure Databricks Unity Catalog connector in Purview, including managed identity / service principal setup and required Unity Catalog permissions • Onboard additional enterprise sources as needed to extend lineage beyond the lake boundary • Define and implement scan rules, scopes, and recurring scan schedules; troubleshoot scan failures and permission issues Catalog Structure & Governance Design • Design the Purview collection hierarchy mapped to business domains (e.g., Finance, Risk, Operations etc.), in partnership with internal stakeholders • Define collection-level access control and assign/onboard collection admins and data stewards • Establish naming conventions, metadata standards, and classification taxonomies for consistency across domains Business Glossary & Metadata • Build out the initial business glossary structure (terms, definitions, term owners/experts) • Link glossary terms to underlying technical assets (tables, columns) discovered via scans • Partner with domain owners to populate business-friendly descriptions for priority datasets Classification & Sensitivity Labeling • Configure built-in and custom classifiers for sensitive data (PII, financial, confidential) during scans • Align Purview classification/sensitivity labels with existing Microsoft Purview Information Protection labels where applicable • Document whether classification is informational only or feeds back into Unity Catalog enforcement (tags, masking, row filters), and implement accordingly Lineage Validation • Validate end-to-end lineage from source systems through Databricks/Unity Catalog into downstream BI tools (e.g., Power BI) • Identify and close lineage gaps at system boundaries Rollout, Enablement & Documentation • Execute a phased rollout: pilot domain(s) first, then expand source-by-source and domain-by-domain • Produce setup, configuration, and operating documentation (runbooks) for internal teams to maintain post-engagement • Train and enable internal data stewards on glossary upkeep, tagging, and ongoing catalog maintenance • Define adoption/usage metrics and provide a handover plan for sustained ownership after the contract ends 4. Required Qualifications • Hands-on experience implementing Microsoft Purview in a production enterprise environment (not just exposure/training) • Experience configuring the Purview connector for Azure Databricks Unity Catalog, including managed identity / service principal permissions • Solid understanding of Databricks Unity Catalog: catalogs, schemas, grants, tagging, and lineage • Experience designing collection hierarchies, glossaries, and classification taxonomies in a catalog tool • Familiarity with Azure IAM/RBAC as it relates to scanning and connector permissions • Strong stakeholder communication skills — able to work directly with both technical platform teams and business domain owners 5. Preferred Qualifications • Prior experience standing up a catalog from greenfield (not just maintaining an existing one) • Experience with Microsoft Purview Information Protection / sensitivity labels across M365 and data estate • Background in data governance program design (stewardship models, data domain ownership frameworks) • Experience with Power BI lineage integration • Relevant certifications (e.g., Microsoft Azure Data Engineer, Databricks certifications) 6. Deliverables Deliverable Databricks Unity Catalog connector live, initial scan complete Collection hierarchy + access model defined and approved Pilot domain(s) fully catalogued: descriptions, glossary, lineage Classification/sensitivity labeling configured and validated Org-wide rollout across remaining priority domains Runbooks, stewardship training, and handover documentation 7. Evaluation Criteria for Candidates • Can speak in specific detail about a prior Purview + Databricks/Unity Catalog implementation (not just general Purview experience) • Can explain the difference between catalog-level governance (Purview) and platform-level enforcement (Unity Catalog), and how they should divide responsibilities • Has a clear point of view on collection design and glossary ownership models, not just tool configuration steps • Demonstrates experience driving adoption, not just technical setup (stewardship enablement, training, documentation)