Senior Data Engineer

• Data Platform Management: Maintain the availability, performance, and cost efficiency of the Group’s data platform and pipelines through proactive monitoring and optimisation.
• Data Engineering & Pipelines: Design and operate robust ETL/ELT pipelines and lakehouse workflows that deliver trusted, well-modelled data to analytics and business consumers.
• Data Governance & Quality: Ensure data quality, lineage, cataloguing, security, and access control across all data assets, maintaining audit readiness.
• AI & Analytics Enablement: Enable AI and machine-learning workloads by providing curated datasets, feature pipelines, and the supporting platform capabilities.
• Security & Compliance: Ensure the data estate meets cybersecurity, patching, and data-protection standards with zero critical vulnerabilities.
• Automation & Continuous Improvement: Develop and implement automation and improvement initiatives that enhance operational efficiency and platform reliability.
• Collaboration & Stakeholder Engagement: Build strong relationships with the Development team, Cybersecurity, and business stakeholders to ensure alignment and effective delivery.
• Reporting & Documentation Discipline: Produce accurate monthly reports and maintain up-to-date technical documentation for all data platforms and pipelines.

Key Responsibilities

• Design, build, and maintain scalable data pipelines and lakehouse solutions using Databricks, Azure Data Factory, Azure SQL, PostgreSQL, and Redis.
• Develop and optimize data warehouses, data marts, and ETL/ELT processes.
• Monitor platform performance, automate workflows, and maintain technical documentation.

• Deliver trusted datasets to support reporting, analytics, and AI/ML initiatives.
• Collaborate with development teams to integrate data solutions into business applications.
• Ensure data platforms are scalable, performant, and cost-effective.

• Own data governance, security, and compliance across the Group's data platform.
• Ensure compliance with POPIA, GDPR, and internal IT governance standards.
• Manage data access, lineage, cataloguing, encryption, and audit readiness.
• Partner with Cybersecurity on risk management, vulnerability remediation, and compliance reviews.

• Act as the primary data engineering contact within Group IT.
• Work closely with regional IT teams, R&D, and business stakeholders.
• Prepare operational reports, manage risks, and drive continuous improvement initiatives.

Qualifications & Experience

• Bachelor's degree in Computer Science, Data Engineering, Information Technology, or a related field.
• 5+ years' experience in data engineering, data platform development, or a similar role.
• Strong hands-on experience with Databricks, Azure Data Factory, Azure SQL, and PostgreSQL.
• Experience designing and supporting data warehouses, data marts, and ETL/ELT pipelines.
• Knowledge of Lakehouse architecture, Spark, and cloud-based data platforms.
• Understanding of data governance, data security, and compliance with POPIA/GDPR.
• Experience with Power BI, Microsoft Fabric/Synapse, or the Azure data ecosystem is advantageous.
• Exposure to AI/ML data pipelines, Azure OpenAI, or MLOps is beneficial.
• Familiarity with CI/CD, Infrastructure-as-Code (Bicep/Terraform), and Agile delivery methodologies is an advantage.
• Relevant Microsoft Azure or Databricks certifications are highly desirable.