Lead Software Engineer Pyspark Data Bricks Data Engineer

Year    KA, IN, India

Job Description

JOB DESCRIPTION




Join us as we embark on a journey of collaboration and innovation, where your unique skills and talents will be valued and celebrated. Together we will create a brighter future and make a meaningful difference.


As a Lead Data Engineer at JPMorgan Chase within the Compliance Technology Team which is aligned to Corporate Technology division , you are an integral part of an agile team that works to enhance, build, and deliver data collection, storage, access, and analytics solutions in a secure, stable, and scalable way. As a core technical contributor, you are responsible for maintaining critical data pipelines and architectures across multiple technical areas within various business functions in support of the firm's business objectives.

Job responsibilities



Provides recommendations and insight on data management, governance procedures, and intricacies applicable to the acquisition, maintenance, validation, and utilization of data Designs and delivers trusted data collection, storage, access, and analytics data platform solutions in a secure, stable, and scalable way Defines database back-up, recovery, and archiving strategy Generates advanced data models for one or more teams using firm wide tooling, linear algebra, statistics, and geometrical algorithms Approves data analysis tools and processes Creates functional and technical documentation supporting best practices Advises junior engineers and technologists Evaluates and reports on access control processes to determine effectiveness of data asset security Adds to team culture of diversity, equity, inclusion, and respect

Required qualifications, capabilities, and skills



Formal training or certification on Data Engineeringconcepts and 5+ years applied experience Demonstrated leadership abilities, including leading and mentoring a team of data engineers. Expertise in designing and implementing scalable data architectures and pipelines. Strong proficiency in programming languages such as Python and PySpark. Experience with AWS cloud platform including their data services. In-depth knowledge of data warehousing solutions and ETL processes. Ability to work collaboratively with cross-functional teams, including data scientists, analysts, and business stakeholders. Experience with data governance and ensuring data quality and integrity. Proficiency in Big Data technologies, with a strong focus on Performance Optimization using best practices Excellent communication skills, with the ability to convey complex technical concepts to non-technical audiences. Experience with version control systems like Git and CI/CD pipelines for data engineering workflows.

Preferred qualifications, capabilities, and skills



Experience with Databricks using PySpark Solid understanding of the Parquet file format. Experience with ETL tools, and experience in ETL process Familiarity with machine learning frameworks and integrating them into data pipelines.

ABOUT US

Beware of fraud agents! do not pay money to get a job

MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.


Job Detail

  • Job Id
    JD4096901
  • Industry
    Not mentioned
  • Total Positions
    1
  • Job Type:
    Full Time
  • Salary:
    Not mentioned
  • Employment Status
    Permanent
  • Job Location
    KA, IN, India
  • Education
    Not mentioned
  • Experience
    Year