Develops and maintains scalable data pipelines to support continuing increases in data volume and complexity.
Collaborates with analytics and business teams to improve data models that feed business intelligence tools, increase data accessibility, and fostering data-driven decision making across the organization.
Implement processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it
Writes unit/integration tests, contributes to engineering wiki, and documents work.
Designs and develops data integrations and data quality framework
Bachelor of Engineering or equivalent degree in Computer Science or a related technical field
At least 7+ years of experience in the Data engineering field.
6+ years of experience in Big Data, Hadoop Ecosystem, Spark, Python, SQL, Hive,
Proficiency in relational SQL and NoSQL databases.
In-depth skills in developing and maintaining ETL/ELT data pipelines
Experience in data modelling techniques such as Kimball star schema.
Hands-on experience with Azure Cloud services such as Azure Data Factory (ADF), Databricks, and Cloud Storage.
Proficient In Airflow configuration and creating dependencies .
Experience in workflow management tools such as Airflow, Data streaming App such as Kafka, and CI/CD tools
Excellent communication and collaboration skills, with the ability to work effectively in a team environment
MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.