Title:  Senior Data Platform Engineer

Responsibilities

• Develop and maintain modular ingestion patterns supporting batch, API-based, and event-driven data sources (e.g., JDBC, SaaS/ERP connectors, Kafka, CDC).
• Establish reusable ingestion frameworks (metadata-driven when possible) that enable rapid and consistent onboarding of new data sources.
• Build and optimize Databricks-based data pipelines (Spark, Delta Live Tables/Lakeflow) across the medallion architecture (Bronze, Silver, Gold).
• Implement and evolve data governance capabilities, including access controls, lineage, cataloging, auditing, data classification, and privacy compliance.
• Define and enforce data quality standards, data contracts, monitoring, and alerting across ingestion and transformation layers.
• Implement data protection and privacy solutions, including masking, tokenization, pseudonymization, and other appropriate anonymization techniques to meet regulatory requirements (e.g. GDPR).
• Build, own, and evolve data products aligned to a canonical data model, ensuring clear ownership, SLAs and documentation.
• Collaborate with domain teams to integrate source data into the canonical model while preserving domain-specific nuance.
• Champion a Data as a Product approach: discoverability, self-service, versioning, and consumer-facing contracts. 
• Build and maintain data consumption and serving layers, including BI/analytics datasets, APIs (REST/GraphQL), and use case-driven extensions such as streaming outputs or knowledge graph representations.
• Drive technical standards and platform direction across CI/CD, Infrastructure as Code, testing, observability, and cost optimization
• Mentor engineers and partner with governance, security, architecture, and business stakeholders.

Requirements

• Around 6+ years in data engineering, with 2+ years designing/operating Databricks-based Lakehouse platforms at production scale.
• Hands-on expertise with Apache Spark and Delta Lake; experience with Delta Live Tables/Lakeflow Declarative Pipelines is a strong plus.
• Practical experience with Unity Catalog (or equivalent) for access control, lineage, and cataloguing.
• Demonstrated experience implementing data anonymisation/pseudonymisation techniques in a regulated environment.
• Solid understanding of canonical/conformed data modelling (e.g., data vault, dimensional modelling, domain-driven design for data).
• Track record exposing data via APIs (REST/GraphQL) - designing schemas, contracts, and versioning for external/internal consumers.
• Proficient in Python and/or Scala, SQL, and infrastructure-as-code (Terraform preferred).
• Strong grasp of data governance frameworks, data privacy regulation (GDPR/CCPA), and data security practices.
• Experience with cloud platforms (Azure, AWS, or GCP) and their native data/event services.

Nice to have

• Exposure to Structured Streaming or event-driven ingestion (Kafka, CDC tools such as Debezium) - not a core requirement, but useful as the platform matures.
• Familiarity with knowledge graph concepts (e.g., RDF/property graphs, Neo4j, graph layers on Spark) - a plus rather than an expectation.
• Experience with data mesh or federated data product architectures.
• Experience with metadata management/catalog tools beyond Unity Catalog (e.g., OpenMetadata, Collibra, Purview).
• Exposure to MLOps or feature store patterns on Databricks.
• Relevant certifications (Databricks Data Engineer Professional, cloud architecture certs).