Designs and maintains scalable data pipelines and real-time streaming applications using Java, Kafka, AWS, and Python. Develops batch and streaming ETL processes, optimizes low-latency data flows, integrates cloud services, and collaborates with engineers, analysts, and data scientists. The role also requires documentation, security and compliance adherence, production support, and use of authorized AI capabilities for data engineering and operational workflows.
As a skilled Senior Lead Software Engineer at JPMorgan Chase, within the Commercial and Investment Bank you will design, develop, and maintain scalable data pipelines and real-time streaming solutions, leveraging cloud technologies and distributed systems. You will collaborate with cross-functional teams to deliver high-quality data solutions that drive business insights and operational efficiency across the execution platform.
Job Responsibilities
- Design and develop robust, scalable data pipelines and streaming applications using Java and Apache Kafka, forming the backbone of real-time execution and market data flows.
- Integrate with the cloud by implementing and managing data solutions on AWS, including services such as S3, Lambda, EC2, RDS, and Glue.
- Process data at scale by developing ETL processes for both batch and real-time data ingestion, transformation, and storage.
- Optimize performance by monitoring, troubleshooting, and tuning data flows to ensure reliability and efficiency in a low-latency environment.
- Collaborate cross-functionally with data scientists, analysts, and fellow engineers to understand requirements and deliver production-grade solutions.
- Document thoroughly, maintaining clear records of data flows, architecture, and operational procedures.
- Uphold security and compliance, ensuring all data solutions adhere to the firm's security, privacy, and regulatory standards.
- Uses enterprise-authorized AI capabilities within the work environment to accelerate data platform and design analysis and technical documentation, validating outputs and handling data according to sensitivity and security requirements.
- Applies reuse-first, AI-assisted practices within delivery and operational routines (e.g., validation automation and access control review support), ensuring traceability/auditability and alignment to resiliency and security expectations.
Required Qualifications, Capabilities, and Skills
Candidates should hold a Bachelor's or Master's degree in Computer Science, Engineering, Mathematics, or a related technical field, and bring experience in data engineering and software development roles. The role demands strong proficiency in Java (core and advanced concepts), hands-on experience with Apache Kafka (producers, consumers, and stream processing), and a solid working knowledge of AWS cloud services including S3, Lambda, EC2, RDS, and Glue. Proficiency in Python is also required.
Beyond core languages and platforms, the ideal candidate will have practical experience with the following:
- ETL tools and data pipeline frameworks
- SQL and NoSQL databases
- Data modelling, data warehousing, and big data technologies
- CI/CD practices and version control (e.g., Git)
- Strong problem-solving and analytical skills, together with excellent communication and teamwork abilities, are essential for success in this collaborative, high-stakes environment.
Preferred Qualifications, Capabilities, and Skills
The following experience would distinguish a candidate, though it is not required:
- Containerization technologies (Docker, Kubernetes)
- Exposure to additional programming languages (Scala)
- Data governance and data quality best practices
- AWS certification or related credentials
- Electronic trading and market data feeds
- Familiarity with the Equities trading stack and Futures & Options market structure
- Trading data or FIX-adjacent data pipelines
- Demonstrated experience using enterprise-authorized AI capabilities within the work environment to support data engineering workflows with strong validation habits and awareness of data sensitivity.
We recognize that our people are our strength and the diverse talents they bring to our global workforce are directly linked to our success. We are an equal opportunity employer and place a high value on diversity and inclusion at our company. We do not discriminate on the basis of any protected attribute, including race, religion, color, national origin, gender, sexual orientation, gender identity, gender expression, age, marital or veteran status, pregnancy or disability, or any other basis protected under applicable law. We also make reasonable accommodations for applicants’ and employees’ religious practices and beliefs, as well as mental health or physical disability needs. Visit our FAQs for more information about requesting an accommodation.
Similar Jobs
Artificial Intelligence • Cloud • HR Tech • Information Technology • Productivity • Software • Automation
Build and deploy production-grade, LLM-enabled applications spanning backend services, data orchestration, vector search, and frontend interfaces. Work directly with strategic customers in agile sprints, integrate legacy systems, debug and optimize distributed systems, and create reusable SDKs, prompts, libraries, and scaffolds. Influence platform and product strategy through field-tested implementations while delivering secure, scalable enterprise AI solutions.
Top Skills:
AngularAWSAzureCi/CdContainersDistributed SystemsGCPGraphQLInfrastructure As CodeJavaLangchainLlmsMonitoringNode.jsPrompt ChainingPythonReactRestSemantic KernelVector DatabasesVector Search
Fintech • Financial Services
Design, develop, test, and deploy Python-based solutions for finance regulatory reporting platforms (Axiom ControllerView). Improve data pipelines and ETL processes, collaborate with business and tech partners, identify service-quality improvements, and follow security, compliance, and Agile/DevOps practices.
Top Skills:
Agent FrameworksAPIsAutomated TestingAxiom ControllerviewCi/CdCloudCommand-Line ToolsData EngineeringDevOpsETLGitLarge Language ModelsLinux/UnixMachine LearningPythonScriptingSdksSQL
Fintech • Financial Services
Manage the full commercial loan closing lifecycle for Fund Finance, coordinating pre-closing/closing and booking activities with Credit, Legal, Relationship Management, and Loan Operations. Ensure accurate systems data, regulatory and policy compliance, resolve implementation issues, and deliver timely, risk-aware execution while building stakeholder relationships.
Top Skills:
LoaniqExcelMicrosoft OutlookMicrosoft PowerpointMicrosoft WordNcinoSalesforceStars
What you need to know about the Manchester Tech Scene
Home to a £5 billion digital ecosystem, including MediaCity, which consists of major players like the BBC, ITV and Ericsson, Manchester is one of the U.K.'s top digital tech hubs, at the forefront of advancements in film, television and emerging sectors like as e-sports, while also fostering a community of professionals dedicated to pushing creative and technological boundaries.
