Senior Data Engineer
Job title: Senior Data Engineer in Iselin, NJ at Brains Workgroup
Company: Brains Workgroup
Job description: Our client, a major bank in Central, NJ, is looking for Senior Data Engineer.
Hybrid commute, 2 days on-site in Central NJ Locations and 3 day remote per week.This is a permanent FT career opportunity, with base salary range 170 - 190K DOE, plus around 20% bonus and great benefits package.Looking for Senior Data Engineer in banking/financial industry.Major Responsibilities:
- Building and maintaining data pipelines to ingest data from source systems into Databricks
- Designing, Building and maintaining generic frameworks to ingest data through the medallion layers
- Working with the core data platform team to integrate the framework with reference architecture
- Assisting the data mesh teams with best practices and on-boarding to the new framework
- 10+ years of data engineering experience in building OLTP and OLAP databases, both on-prem and in the cloud
- 5+ years of experience with large scale data processing using SQL, Python, Pandas and Pyspark
- 5+ years of experience building robust and scalable data pipelines in Databricks Lakehouse platform on any key cloud platform provider, preferably Azure cloud
- Experience with implementing Medallion architecture for data processing
- Experience with implementing Streaming data ingestion using Kafka
- Test driven development experience
- Proven experience in solution design and applying best design practices
- Data modeling and ER/UML diagram skills
- Experience with orchestration/enterprise scheduler platforms like Airflow, Tidal etc..
- Experience with implementing API based data integration pipelines
- Experience with source control and DevOps for CI/CD
- Experience working in Agile/scrum methodologies
- Experience with Jira, confluence for project work and documentation
- Experience with Service now for incident management
- Finance domain knowledge in Investment Banking (Securities, Capital markets) and Security reference data knowledge
- Nice to have ADF and SSIS experience
- College degree required.