Job Description

Enable Data Incorporated is seeking a highly skilled Senior Spark Engineer to join our dynamic team. As a leading provider of advanced application, data, and cloud engineering services, we leverage modern solutions to drive increased value across our clients\' business ecosystem. As a Senior Spark Engineer, you will be responsible for overseeing all aspects of our clients\' Spark data processing pipelines, including development, implementation, and testing. In addition, you will work closely with our clients to identify opportunities to optimize their data processing pipelines and leverage Spark to achieve their goals. This is an excellent opportunity for a talented Spark Engineer who is looking to advance their skills and work with cutting-edge technologies in the Information Technology and Services industry. Responsibilities Design, develop, implement, and maintain Spark data processing pipelines for our clients Work with clients to identify opportunities to optimize their data processing pipelines and leverage Spark to achieve their goals Collaborate with our team of engineers to design, develop, and maintain data solutions that meet client requirements Conduct code reviews and ensure that best practices are being followed in the development and maintenance of Spark data processing pipelines Participate in the full software development lifecycle including analysis, design, implementation, testing, and maintenance Develop and maintain technical documentation, including system design documents, user manuals, and operational procedures Keep up-to-date with emerging trends and technologies in the Spark ecosystem and make recommendations for future improvements Requirements Bachelor\'s or Master\'s degree in Computer Science or a related field 10+ years of hands-on Software Development experience using Java or Scala Strong experience developing Spark data processing pipelines Strong hands-on experience in at least one major public cloud platform (Azure or AWS preferred) Experience with Python and Scala programming languages Exceptional knowledge of Spark architecture, including Spark Core, Spark SQL, and Spark Streaming Proficiency with distributed computing systems and big data technologies, such as Hadoop and HDFS Expertise in designing and developing scalable, fault-tolerant, distributed data processing systems Proven hands on experience with Databricks (certification preferred) Strong analytical and problem-solving skills Excellent communication and interpersonal skills, with the ability to work collaboratively with clients and team members

foundit