to manage, monitor, and optimize our Hadoop ecosystem. The ideal candidate will have hands-on experience in maintaining
HDFS, Hive, and Spark
clusters, ensuring smooth data flow and system reliability across distributed environments.
Key Responsibilities:
Perform daily monitoring, maintenance, and troubleshooting of Hadoop clusters (HDFS, Hive, Spark).
Manage cluster upgrades, patching, and performance tuning.
Ensure data reliability, availability, and security within the Hadoop ecosystem.
Diagnose and resolve cluster issues, node failures, and performance bottlenecks.
Implement data replication, backup, and recovery strategies.
Work closely with data engineering teams to optimize Hive queries and Spark jobs.
Monitor and maintain system health using tools like Ambari, Cloudera Manager, or similar.
Develop scripts for automation and system improvements (using Shell, Python, etc.).
Document processes, configurations, and incident resolutions.
Required Skills & Experience:
2-5 years of experience in Hadoop ecosystem management.
Strong knowledge of
HDFS architecture
and file management.
Expertise in
Hive
(data warehousing, query optimization) and
Spark
(batch/stream processing).
Proficiency in
Linux/Unix systems
and shell scripting.
Familiarity with
Hadoop ecosystem tools
like YARN, Oozie, Sqoop, or Kafka (preferred).
Understanding of cluster resource management and tuning.
Experience with
monitoring tools
and
log analysis
.
Preferred Qualifications:
Experience with
Cloudera
or
Hortonworks
distributions.
Knowledge of
cloud-based Hadoop deployments
(AWS EMR, Azure HDInsight, GCP Dataproc).
Strong analytical and troubleshooting skills.
Excellent communication and documentation abilities.
Education:
Bachelor's degree in Computer Science, Information Technology, or related field.
Job Types: Full-time, Permanent
Work Location: In person
Beware of fraud agents! do not pay money to get a job
MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.