Job Description

Key Responsibilities:
Build up data pipelines for consumption by the data science team.
Clear understanding and experience with Python and PySpark.
Experience in writing Python programs and SQL queries.
Experience in SQL Query tuning.
Build and maintain data pipelines in Pyspark with SQL and Python.
Knowledge of Cloud (Azure/AWS) technologies is additional. Suggest and implement best practices in data integration.
Split the planned deliverables into tasks and assign them to the team.
Good oral, written and presentation skills.

Skills & Experience Required:
Degree in Computer Science, IT, or a similar field. Hands-on experience with Python and Pyspark
3-5 years of hands-on experience in Python, PySpark, and SQL.
Great numerical and analytical skills.
Proven expertise in building and optimizing data pipelines using Spark and Hadoop ecosystem tools.
Experience working on cloud platforms like Azure or AWS.
Should be able to collaborate and coordinate in a remote environment. Be a problem solver and be proactive to solve the challenges that come his way.

Skills Required