Job Description

APE DATA ENGINEER
Contract Period: ~ 12 month
Work Location: Chennai, India
The data engineer will be responsible for the end-to-end design, build, monitoring, and supporting Cloud Big Data architecture including platform and applications
SUMMARY OF ESSENTIAL JOB FUNCTIONS

• Ensure deployment of modern data structures and models to enable reliable and scalable data products and feature stores

• Develop and manage end-to-end data pipeline and application stack (Hadoop, Vertica, Tableau, Hue, Superset etc) and lead/provide end-to-end operation support for all applications.

• Develop processes for automating, testing, and deploying the work

• Identify risks and opportunities of potential logic and data issues within the data environment

• Perform RCA and resolution activities for complex data issues.

• Develop and maintain documentation relating to all assigned systems and projects

• Ability to independently perform proof of concepts for new tools and frameworks and present to leadership

• Work in an agile environment. Develop and drive multiple cross-departmental projects.

• Establish effective working relationships across disparate departments to deliver business results.

• Collaborate effectively with the global team and ensure day-to-day deliverables are met

• 5 to 10 years of Strong Experience with big data tools and data processing: Hadoop, Spark, Scala, Kafka, Yarn cluster, Java, etc.

• Strong development experience with EMR and SPARK is a must-have

• Strong experience with object-oriented/object function scripting languages: SQL, Python, Java, Scala, etc.

• Must have 5+ years of working experience with AWS

• Experience with GCP is nice to have

• Experience with SQL/NoSQL databases like Vertica, Postgres, Cassandra, etc.

• Experience with data modeling concepts is desired

• Experience with streaming /event driven technologies work such as Lambda, Kinesis, Kafka, etc.

• Nice to have but not required - exposure to ML (frameworks like pytorch/tensorflow), model management and servi,ng, containerizing and application development experience with Talend, Tableau, Snowflake, Redshift etc.

• Prior experience as a senior data architect, technical lead, system architect, or similar is required

• Excellent verbal and written communication skills.