Develop, maintain, and monitor data pipelines, ensuring reliability and prompt resolution of failures as they occur.
Collaborate with stakeholders to gather requirements, provide updates, and participate in design discussions.
Design and implement custom ETL processes and automation scripts using AWS, Databricks, Python, and PySpark.
Prepare architecture and design documentation for data engineering projects.
Utilize expertise in cloud-based big data integration and infrastructure, with a strong focus on Databricks and the Apache Spark framework.
Demonstrate proficiency in Python and other scripting languages for data engineering tasks.
Leverage strong SQL and PL/SQL skills to write complex procedures and queries for dataframes and notebooks.
Maintain a data engineering mindset to ensure quality, reliability, and efficiency across data systems and processes.
Requirements-
4+ years of relevant professional experience in data engineering or related fields.
At least 2 years of hands-on experience with Databricks, Spark, Databricks workflows, and Autoloader.
Proficiency in additional scripting languages such as Scala and JavaScript, as well as experience with API development.
Familiarity with data orchestration and ETL tools, including Alteryx, Snowflake, and Azure Data Factory.
* Experience with business intelligence tools such as Tableau, Power BI, or QuickSight.
Beware of fraud agents! do not pay money to get a job
MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.