Job Description

Engineered a scalable data pipeline that seamlessly moved data from diverse sources, including REST APIs, on-premise databases, and SharePoint, into Snowflake using Azure Data Factory, significantly improving data integration efficiency. o Led successful requirement gathering sessions, translating complex business needs into actionable technical specifications, which directly contributed to the project's successful delivery. o Implemented robust CI/CD pipelines for Azure Data Factory components and Azure Databricks notebooks, enabling smooth and reliable deployments to higher environments using Azure DevOps. o Automated the data ingestion process,eliminating 90% of manual data processing, drastically improving operational efficiency and reducing human error. o Developed an enterprise-wide data model,consolidating data from multiple sources and ensuring consistent, organization-wide data access, leading to improved decision-making capabilities. o Optimized data processing capabilities,allowing the system to handle 2GB of data every 2 hours with high performance and reliability, ensuring the infrastructure met business needs. o Implemented parallelism in data pipelines,reducing pipeline execution time significantly, boosting performance, and enhancing data flow efficiency. o Designed and documented a comprehensive architecture, detailing data transformation logic and Snowflake schema design, establishing a clear framework for future scalability and maintenance. o Collaborated effectively with business users,ensuring data accessibility, gathering critical feedback, and providing ongoing support that enhanced user satisfaction and system usability.