Data Engineer

The Wow

This role will contribute in designing and developing our platform to fulfil the business needs and to also improve our systems. Within this capacity you will have the chance to design new data pipelines, maintain platforms hosted on data streams for batch and real-time loading and real-time visualisations.

 

A taster of what you will be involved with: 

 

• Maintaining and evolving our existing data platform;
• Building processes to ingest data from Kafka, APIs and Databases using AWS MSK Connect.
• Development and Maintenance of real-time data processing applications using frameworks like Spark Structured Streaming and Kafka Streams.
• Applying transformations to data streams;
• Being involved in data modelling following standards e.g. Inmon, Kimball, Data Vault;
• Ensuring data quality by verifying data consistency and accuracy;
• Keeping up to date on research and development of new technologies and techniques to enhance our data platform;
• Have an investigative mindset to be able to troubleshoot – thinking outside the box wh