The Data Engineer will design and develop data pipelines, manage data processing solutions, ensure data quality, and collaborate with team members to create scalable data assets.
UBDS Group is seeking skilled Data Engineers to design, build, and optimise modern data platforms. This role is suited to engineers with strong experience in data pipelines, cloud technologies, and distributed data processing, who can deliver scalable, high-quality data solutions.
You will work as part of a multidisciplinary team to develop reliable data assets that support a range of operational and analytical use cases.
Key Responsibilities- Design, develop, and maintain data pipelines and transformation processes using modern data engineering tools
- Build and manage batch and real-time data processing solutions
- Develop curated, reusable data assets to support downstream analytics and reporting
- Ensure data quality, integrity, and consistency across multiple data sources
- Work with data ingested via APIs, file-based ingestion (e.g. SFTP), and event streams (e.g. Kafka)
- Optimise data pipelines for performance, scalability, and cost efficiency
- Contribute to the development of data platform components and reusable engineering patterns
- Work within established architecture and engineering standards
- Collaborate with DevOps engineers, testers, and architects to deliver end-to-end data solutions
- Support onboarding of new data sources and integration into existing platforms
- Contribute to knowledge sharing and continuous improvement within engineering teams
- AWS (primary platform)
- S3, EMR, Glue
- Kafka for event-driven data processing
- Python and PySpark for data engineering and transformation
- Terraform (Infrastructure as Code)
- GitHub (version control and CI/CD)
- Azure data platforms
- Databricks and/or Microsoft Fabric
- Multi-cloud architectures and tooling
- Experience in data engineering, building and maintaining data pipelines
- Strong hands-on experience with Python and PySpark
- Experience working with AWS data services (e.g. S3, EMR, Glue)
- Experience with event-driven architectures, particularly Kafka
- Understanding of batch and real-time data processing
- Experience working with large-scale datasets and distributed systems
- Familiarity with data quality, validation, and schema management
- Experience using Terraform and GitHub within a DevOps environment
- Ability to work within defined engineering standards and best practices
- Experience with Azure, Databricks, or Microsoft Fabric
- Experience in regulated environments
- Exposure to data migration from legacy systems
- Understanding of analytics, reporting, or data product use cases
- Experience working in collaborative, cross-functional teams
UBDS Digital Manchester, England Office
24 Mount Street, Industrious, Windmill Gree, Manchester, United Kingdom, M2 3NX
Similar Jobs
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Design and build scalable, secure data pipelines (streaming and batch), drive CI/CD and automation, participate in cloud migration and architecture discussions, collaborate with clients, and mentor internal teams to deliver production-grade data solutions.
Top Skills:
AirflowAWSBigQueryCircleCIContainerisationEltETLGraph DatabasesHadoopJavaJenkinsKafkaKinesisPythonRedshiftScalaSnowflakeSparkSpark StreamingSQL
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
Design and build cloud-based data pipelines using AWS tools, collaborate with clients, ensure data governance, and mentor junior engineers.
Top Skills:
AWSCi/CdDevOpsHadoopJavaKafkaKinesisPythonScalaSparkSpark Streaming
Fintech • Professional Services • Consulting • Energy • Financial Services • Cybersecurity • Generative AI
The role involves designing and building data pipelines using AWS technologies, leading projects, and mentoring junior engineers in a data engineering context for financial services.
Top Skills:
AWSAws Well-Architected PrinciplesHadoopJavaKafkaKinesisPythonScalaSparkSpark Streaming
What you need to know about the Manchester Tech Scene
Home to a £5 billion digital ecosystem, including MediaCity, which consists of major players like the BBC, ITV and Ericsson, Manchester is one of the U.K.'s top digital tech hubs, at the forefront of advancements in film, television and emerging sectors like as e-sports, while also fostering a community of professionals dedicated to pushing creative and technological boundaries.
