Lead Data Engineer (Databricks, MLops, AWS)

Role Overview 

We are seeking a Senior/Lead Platform Engineer who will take ownership of the design, implementation and operation of our core data, analytics and ML infrastructure. This role spans across platform architecture, DevSecOps, DataOps, and ML infrastructure, and requires a combination of strategic thought leadership and hands-on execution. You will build, integrate and operate platforms on AWS and Databricks, enabling scalable, secure, production-grade ML/AI solutions. 

Key Responsibilities 

• Architect and implement end-to-end data and ML platforms: data lakes, warehouses, streaming and batch pipelines, model training/deployment infrastructure, on AWS + Databricks. 

• Lead DevSecOps and DataOps practices: infrastructure as code (IaC), CI/CD pipelines for data & ML workflows, secure multi-account/multi-region cloud operations. 

• Integrate AWS services (e.g., S3, Redshift, Kinesis, Lambda, EKS/ECS) with Databricks runtime, Delta Lake, Unity Catalog etc to build scalable, performant pipelines. 

• Build and operate ML infrastructure: training clusters, model versioning, MLOps toolchain (e.g., MLflow), model monitoring and observability, automatic retraining wo