Job Description

Job Category: Data and Business Intelligence
:
POSITION SUMMARY: We are seeking an experienced Data Engineer to lead a small team of data engineers in leveraging AWS native technologies such as Lambda and Glue to build a data Lakehouse environment. The ideal candidate will have a strong background in data engineering, a deep understanding of AWS services, and proven leadership skills
JOB FUNCTIONS AND RESPONSIBILITIES

• Lead and mentor a team of data engineers, fostering a collaborative and innovative environment.
• Design, develop, and maintain scalable data pipelines using AWS native technologies including AWS Glue, Lamda etc.
• Implement best practices for data management, including data quality, data governance, and data security.
• Ensure high standards by reviewing architecture, design, and code.
• Collaborate with cross-functional teams to deliver and operate large scale, distributed services in the cloud
• Optimize data workflows for performance and cost-efficiency.
• Own system architecture, scalability, reliability, and performance
• Stay updated with the latest advancements in AWS technologies and data engineering practices

EDUCATION / EXPERIENCE

• BE/B Tech/MSc/MCA - with Specialization in Computer Science/Information Systems.
• 5+ years of non-internship professional software development experience
• Experience with AWS components to build and expand lake house in AWS
• Experience building Python programs across Glue ETL jobs and Lambda functions
• Design & Build S3 buckets, tiers, lifecycle policies, as strategic storage layer for Data Lake
• Experience building applications based on data in both Relational databases (e.g. MySQL), NoSQL databases (e.g. DynamoDB, MongoDB) and filesystems (e.g. S3, HDFS)
• Require 3+ hands-on experience on AWS S3, Spark,Glue ETL & Catalog, Lamba Functions, Athena & Kafka
• 2-3 years of experience in writing SQL Queries.
• Certified on one of - Solution Architect, Data Engineer or Data Analytics Specialty by AWS

WORK SCHEDULE OR TRAVEL REQUIREMENTS
No travel requirements. Mid Shift (2PM - 11PM)

Skills Required