Job Title: Data Engineering Manager
Location: Hybrid (Must be able to travel to London once a week)
Employment: Contract (outside of IR35)
Duration: 6 Months
About YLD:
Everything we do is to empower our clients to move forward. Our purpose is to help our clients develop the capabilities they need to outperform their competitors. Our success is a consequence of ensuring our clients are successful. Our company is built on extraordinary people; we aim to attract, inspire, develop and retain them. YLD is a software engineering and design consultancy; we create digital capabilities for our clients that last beyond our engagement. We have offices in London, Lisbon and Porto.
Job Summary:
We are seeking a highly skilled and experienced Data Engineering Manager to join our team. As a Data Engineering Manager, you will be responsible for overseeing and leading a data engineering team to design, develop, and maintain the data infrastructure and systems. You will collaborate closely with cross-functional teams, including data engineers, data scientists, analysts, and business stakeholders, to ensure data quality, availability, and usability.
Responsibilities:
- Lead and manage a team of data engineers, providing guidance, mentorship, and performance feedback.
- Collaborate with cross-functional teams to understand data requirements and translate them into scalable data solutions.
- Design and implement robust data pipelines, ETL processes, and data integration workflows.
- Ensure data integrity, quality, and security by implementing effective data governance and data management practices.
- Identify and evaluate new technologies, tools, and frameworks to optimize data engineering processes and improve efficiency.
- Support on architecture, design, and scale data services.
- Monitor and optimise data storage and processing systems to ensure optimal performance and scalability.Manage relationships with external vendors, contractors, and partners
- Stay up-to-date with industry trends and best practices in data engineering, and recommend innovative solutions to enhance our data infrastructure.
Requirements:
- Proven experience in data engineering or a similar role, with demonstrated progression into management.
- Strong proficiency in programming languages such as Python, Java, or Scala.
- Confident user of R or other statistical software.
- Experience with distributed storage processing engines (e.g. Apache Hadoop/Spark).
- Extensive experience with workflow management/batch schedulers, such as Airflow, Prefect, and Dagster.
- Strong SQL skills: extract, manipulate, transform and analyse large volumes of data with a high level of accuracy
- Solid understanding of database management systems, data warehousing concepts, and cloud-based data platforms (e.g., AWS, Azure, GCP).
- Expert in Continuous Integration and Delivery systems and patterns.
- Familiarity with cloud native constructs and orchestration technologies (e.g. Kubernetes, Docker, Terraform, and Ansible)
- Experience with data modelling, schema design, and performance tuning.
- Excellent leadership and people management skills, with the ability to motivate and inspire a team.
- Strong problem-solving and analytical abilities, with a keen attention to detail.
- Exceptional communication and collaboration skills, with the ability to effectively present technical concepts to both technical and non-technical stakeholders.
Preferred Qualifications:
- Advanced degree in Data Science, Computer Science, or a related field.
- Familiarity with machine learning and data science concepts.
- Experience with agile software development methodologies.
We live and breathe our values, and know you will too:
- Growing every day
- Including everyone
- Relationships built on honesty and ethics
- Inspiring solutions
- Winning together
We’re an equal-opportunity employer and value diversity of all kinds. We don’t discriminate based on race, religion, colour, national origin, gender, sexual orientation, pregnancy or maternity, age, marital status, or disability status. Likewise, we also offer a remote-first working environment, meaning that flexible working and work-life balance come as standard for all employees.
