Job Description

Job Responsibilities

• Responsible for administering big data systems like EMR, HDFS, YARN Spark, Hive, Oozie and others.
• Work closely with development teams at the design phase of big data jobs and pipelines to improve operational efficiency.
• Troubleshoot big data systems for capacity, bottlenecks of memory, CPU, OS, storage, and network.
• Performance tuning of Hadoop clusters, Jobs, workflows and pipelines.
• Work with other operations teams to improve big data infrastructure to guarantee high availability.
• Work with AWS support on need basis.
• Fast learner, able to pick up new ideas and approaches quickly.
• Will be on-call rotation along with other team members for critical incidents.

• Bachelor's or master's degree in computer science or equivalent.
• 5 years of experience working on big data platforms and related technologies like EMR, Hadoop, Spark, Cassandra, etc.
• Hands on experience on AWS cloud, certification on AWS cloud is preferred.
• Experience in using Terraform, Ansible and scripting language like python and shell.
• Strong understanding of distributed data processing concepts like data partitioning, bucketing, distributed joins and aggregation, Map/Reduce, etc.
• Strong understanding of Hadoop internals and Linux operating system.
• Should be able to support 24X7 support for critical system.
• Good understanding of programming languages like Java, Python, etc.
• Good communication skills, knowledge of English.