Job Purpose
The role holder will be responsible for designing, developing, and maintaining our data infrastructure and data pipeline architecture. They will spend time optimizing data flows and systems in support of cross functional teams. They will be responsible for designing and maintaining data pipelines, ensuring that data platforms, production schedules, and self-service tools are operating efficiently and always available. The holder will be required to work with analysts and developers to design and maintain data infrastructure to support business requirements.
Key Accountabilities
Understand, maintain, and support bank data infrastructure
Design and implement ETL pipelines into the data warehouse and KAFKA
Collaborating with other members of the data engineering team and business stakeholders to ensure data accuracy, completeness, and consistency.
Staying up to date with the latest data engineering technologies and tools, including Hadoop, Spark, and cloud platforms.
Holding team accountable to engineering best practices.
Design systems to track data quality and consistency for both on premise and cloud data.
Ensure both on premise and cloud data infrastructure are always available.
Build and maintain logical data models, data marts and multidimensional models
Automate data pipelines for Data Science team in support of ML and AI automation
Manage use of DevOps/Jira etc to document, assign, action, and track tasks to completion
Develop pipelines to integrate data from on SharePoint, PowerApps, Dataverse, AWS, Python, external and internal APIs.
Mentoring junior data engineers
Leading junior engineers and assign tasks where necessary for execution.
Ensure Governance requirements are met in all engineering tasks on Absa infrastructure
Implement standards set by the data governance team in development of data artefacts and BI solutions
Track consumption of analytics resources and optimise performance.
Actively challenge status quo and offer ideas to improve operations and existing solutions deployed by colleagues
Work in sprints with multidisciplinary teams including Analysts, data scientists, product managers, agile delivery managers, to scope, plan and deliver data driven insight
Qualifications
Bachelor's degree in IT, technology, data science, business analytics or math focused fields is preferred (or equivalent on-the-job experience and personal analytics projects).
Certified in SQL, AWS or Hadoop infrastructure support
Agile technology project management certification
Preferred Experience
Minimum 5 years technical experience
Experience in working within a large complex organization with multiple stakeholders
Knowledge & Skills
Strong analytical and diagnostic skills
Ability to work in remote teams
Ability to lead a team and communicate effectively with business stakeholders.
Knowledge of technology project management tools such as JIRA, Planner, DevOps