Job Description

"Responsibilities
Architect and build: Design and implement scalable data architectures and pipelines on Azure, utilizing services like Azure Data Factory and Azure Synapse Analytics.
Develop and optimize: Create and maintain data pipelines, including ETL/ELT processes, with a focus on performance and scalability, often using Python/PySpark and Databricks.
Ensure data quality: Implement processes and checks to maintain data quality, integrity, and security across all data solutions.
Collaborate: Work with data scientists, analysts, and other stakeholders to understand data needs and deliver effective solutions.
Troubleshoot and monitor: Monitor data systems, troubleshoot performance issues, and ensure the smooth operation of data pipelines.
Mentor and lead: Provide technical leadership and mentorship to junior data engineers.
Document: Create and maintain comprehensive documentation for data workflows, processes, and best practices.
Qualifications
Education: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
Experience: 5+ years of experience in data engineering or a similar role.
Azure expertise: Proven experience with Azure data services including Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake Storage.
Databricks expertise: Advanced proficiency with Databricks, including cluster management, notebook development, and Delta Lake features.
Technical skills: Strong knowledge of SQL, NoSQL databases, and big data technologies like Apache Spark (PySpark, Scala).
Programming: Proficiency in programming languages like Python.
Other skills: Experience with data warehousing, data modeling, and performance tuning. Excellent problem-solving and communication skills.
Sr Azure Data Engineer Template - Expertia AI
Responsibilities. Design and develop scalable data architectures on Azure. Implement and optimize data pipelines using Azure Data ...
"

Skills Required