Build and maintain Microsoft Fabric data pipelines, PySpark and SQL transformations, dimensional models, data quality checks, and CI/CD workflows. Integrate data from Snowflake, SQL Server, files, APIs, and third-party sources; support monitoring, troubleshooting, documentation, infrastructure-as-code, and Power BI datasets for financial, risk, portfolio, and asset analytics.
We are seeking highly skilled and experienced Azure Data Engineers to join a newly formed group concentrating on Data. Within this role you will be a key member of the team, working on a complex and challenging project within the Financial Services/Capital Markets industry. The primary focus of the role would be on building resilient, reusable Data Pipelines to extract, load, and transform raw data into a relational data model. The successful candidate will work across complex, multi-source datasets including loan servicing systems, property and valuation platforms, collections systems, and third-party data providers, delivering reliable and auditable data at scale.
Key Responsibilities
- Build and maintain ingestion and transformation pipelines on Microsoft Fabric (Data Pipelines, PySpark notebooks, Dataflows Gen2) across the Landing, Curate and Enrich layers of the medallion architecture.
- Onboard new source systems by setting up the team’s reusable ingestion patterns (Fabric mirroring from Snowflake and SQL Server, change data capture, file and API loads). Do this through configuration rather than one-off code.
- Write and tune PySpark and T-SQL for Lakehouse and Warehouse workloads, including fact and dimension models in the enterprise data warehouse.
- Implement data quality rules and validation pipelines, and help investigate and fix data issues found by DQ checks and reconciliations.
- Work through the team’s engineering workflow: Git-based development in Azure DevOps, pull requests and peer review, and CI/CD promotion from Dev to UAT to Production.
- Contribute to the infrastructure-as-code (Terraform) and the orchestration config database (SQL projects / DACPAC) as needed.
- Work with Data Stewards, Data Analysts and the Data Architect to understand source data and turn mapping specs and business rules into working transformations.
- Keep documentation current, including source-to-target mappings, transformation rules and runbooks, so the work stays auditable and supportable.
- Support the platform day to day: monitor pipeline runs, triage failures and escalate where needed.
- Help prepare trusted datasets and Power BI semantic models for portfolio analytics, asset strategy, finance and risk.
Our tech stack- Platform: Microsoft Fabric (Lakehouse, Warehouse, Notebooks, Data Pipelines, Dataflows Gen2, Mirroring, OneLake)
- Languages: Python / PySpark, Spark SQL, T-SQL
- Sources: Snowflake, SQL Server, files and third-party feeds
- Azure: Azure SQL Database, ADLS Gen2, Key Vault, Data Factory, Entra ID (service principals and managed identities)
- Engineering: Azure DevOps (Repos, Pipelines, Boards), Git, fabric-cicd, Terraform / HCP Terraform, SQL Database Projects
- Governance and reporting: Microsoft Purview (data quality), Power BI (semantic models)
About you
- Experience building data pipelines as a Data Engineer or in a similar role
- Strong Python/PySpark and SQL skills
- Hands-on experience with Microsoft Fabric, or with Databricks / Azure Synapse / Azure Data Factory and a willingness to move to Fabric
- Comfortable with Git, pull requests and CI/CD. Terraform or other infrastructure-as-code experience is a plus
- Understanding of data modelling (dimensional / medallion) and data quality practices
- Works well in a team, takes feedback in code review, and is keen to learn and develop
- Strong analytical, verbal and written communication skills
- Degree in Computer Science, Information Systems, Data Science or a related field preferred
- Experience in financial data domains (loan servicing, transactions, reference data, IBOR/ABOR) or in Real Estate, Credit, Banking or NPL Asset Management is desirable
- Microsoft certification (e.g. DP-700 Fabric Data Engineer) a plus
About
Great talent comes in many forms, and we’re committed to building a diverse and inclusive team. Whilst a number of our roles do require specific qualifications and experience, and industry knowledge, we also value potential, unique perspectives, and transferable skills. If you’re excited about this opportunity but don’t meet every requirement, we’d still love to hear from you. We occasionally collaborate with recruitment agencies to fill niche or specialist roles. However, we do not accept agency terms or pay fees for speculative CVs submitted directly to our hiring managers or outside our Applicant Tracking System. If you are a recruitment agency interested in partnering with us for candidate supply, please reach out to [email protected]
Arrow Global Group Manchester, England Office
Belvedere, 12 Booth Street, Manchester, United Kingdom, M2 4AW
Similar Jobs
Digital Media • Gaming • Software • Esports • Automation
Lead the modernization of regulatory data systems by designing scalable BigQuery and Dataform pipelines on GCP. Build and support ETL/ELT workflows, reporting systems, data-quality controls, monitoring, and operational safeguards. Drive technical designs, delivery planning, architecture simplification, CI/CD, testing, and documentation while mentoring engineers and using AI-assisted development tools to improve engineering quality.
Top Skills:
Ci/CdClaude CodeCloud ComposerCloud FunctionsCloud StorageDataformEltETLGoogle BigqueryGoogle Cloud PlatformPub/SubSQLSQL Server
Digital Media • Gaming • Software • Esports • Automation
Lead the modernization of legacy SQL Server reporting systems into scalable Google Cloud architectures. Design and maintain BigQuery and Dataform pipelines, implement data-quality and reconciliation controls, improve observability and engineering standards, and support regulatory reporting. The role includes technical design leadership, delivery planning, mentoring engineers, reducing technical debt, and using AI-assisted development tools to improve quality and efficiency.
Top Skills:
Ci/CdClaude CodeCloud ComposerCloud FunctionsCloud StorageDataformEltETLGoogle BigqueryGoogle Cloud PlatformPub/SubSQLSQL Server
Digital Media • Gaming • Software • Esports • Automation
Develop and maintain regulatory data reporting systems across SQL Server and GCP BigQuery. Build ETL/ELT pipelines, migrate legacy solutions to cloud-native platforms, and implement data modeling, validation, monitoring, alerting, QA, and documentation. The role requires advanced SQL, BigQuery, GCP services, orchestration, Terraform, CI/CD, and version control experience, while supporting accurate, traceable, and performant regulatory submissions.
Top Skills:
Ci/CdClaude CodeCloud ComposerCloud FunctionsCloud StorageEltETLGitGitlabGoogle BigqueryGoogle Cloud PlatformPub/SubSQLSQL ServerTerraform
What you need to know about the Manchester Tech Scene
Home to a £5 billion digital ecosystem, including MediaCity, which consists of major players like the BBC, ITV and Ericsson, Manchester is one of the U.K.'s top digital tech hubs, at the forefront of advancements in film, television and emerging sectors like as e-sports, while also fostering a community of professionals dedicated to pushing creative and technological boundaries.

