Job Description

Job Title: Lead Data Engineer
Experience: 7 - 9 years
Location: Hyderabad, INDIA.OSI Digital is looking for a Lead Data Engineer to work on enterprise scale implementations, upgrade and support ongoing projects. At OSI good performers will be rewarded with our well-designed recognition programs.Responsibilities:

• Hands-on technical person with rich experience on the above mentioned skills & managing a small team of 5-10 members and providing oversight on the project.

• Communicating with senior management and customers verbally and in writing.

• Architect, Build & Monitor data systems and pipelines.

• Provide Solutioning as need arises.

• Develop Data Frames using PySpark

• Conduct complex data analysis and report on results.

• Combine raw information from different sources.

• Explore ways to enhance data quality and reliability.

Required Skills:

• 7-9 years of data engineering experience loading, analyzing and interpreting complex datasets.

• Min 5+ years of rich experience with AWS Glue, Lambda, Step Functions, Python, PySpark, Redshift, S3,
• Kinesis, IAM, VPC, EC2, Security Roles/Policies, Cloud Watch, Load Balancers.

• Experience with Airflow / any other orchestration frameworks.

• Experience with distributed systems such as Cloudera/Horton Works.

• Experience with data modeling, warehousing and building ETL pipelines on Cloud Platforms.

• Experience working on and delivering end to end projects independently & managing a small team of 5-10 members and providing oversight on the project.

• Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases).

• 5+ years of developing and operating large-scale data structures for business intelligence analytics (using ETL/ELT processes) experience, such as ETL Glue etc.,

• Experience in at least one modern scripting or programming language, such as Python, Java, Scala, or NodeJS and reasonable knowledge/experience writing Cloud Formation Templates (CFT) / Terraform.

We are not looking for someone who is

• Not interested in working from the office