Job Description

Job Summary

As an Lead Data Engineer with a focus on Palantir PySpark and Python you will play a pivotal role in designing and implementing scalable data solutions. With a hybrid work model you will collaborate with cross-functional teams to drive innovation and efficiency. Your expertise will contribute to the companys mission of delivering cutting-edge technology solutions that have a significant impact on society.

Responsibilities

Design and implement scalable data architectures using Palantir Spark and SparkSQL to meet business requirements. Collaborate with cross-functional teams to understand data needs and provide innovative solutions. Develop and optimize Python scripts to enhance data processing and analysis capabilities. Ensure data quality and integrity through rigorous testing and validation processes. Lead the development of data models and pipelines to support analytics and reporting. Provide technical guidance and mentorship to junior team members to foster growth and development. Oversee the integration of data from various sources to create a unified data platform. Monitor system performance and implement improvements to enhance efficiency and reliability. Stay updated with the latest industry trends and technologies to drive continuous improvement. Communicate effectively with stakeholders to align data solutions with business objectives. Document technical specifications and processes to ensure knowledge sharing and continuity. Contribute to the companys purpose by delivering data solutions that drive informed decision-making. Impact society by enabling data-driven insights that lead to innovative solutions. Qualifications Possess extensive experience in designing data architectures with Spark and SparkSQL. Demonstrate proficiency in Python programming for data processing and analysis. Exhibit strong problem-solving skills and the ability to work collaboratively in a hybrid environment. Have a solid understanding of data modeling and pipeline development. Show experience in ensuring data quality and integrity through testing and validation. Display excellent communication skills to interact with stakeholders and team members. Stay abreast of industry trends and advancements in data technologies.

Certifications Required

Certified Spark Developer Python Programming Certification