Job Description

• Own and deliver enhancements associated with Data platform solutions.
• Maintains and enhances scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity.
• Enhance/Support solutions using Pyspark/EMR, SQL and databases, AWS Athena, S3, Redshift, AWS API Gateway, Lambda, Glue and other Data Engineering technologies.
• Write Complex Queries and edit them as required for implementing ETL/Data solutions.
• Measure performance and environment of application with system & application log tools and act to improve accordingly.
• Implement solutions using AWS and other cloud platform tools, including GitHub, Jenkins, Terraform, Jira, and Confluence.
• Follow agile development methodologies to deliver solutions and product features by following DevOps, Data Ops and Dev Sec Ops practices.
• Propose Data load optimizations and continuously implement to improve the performance of the Data loads
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep the data separated and secure across through multiple data centers and AWS regions.
• Be available and participate in on-call schedule to address critical operational incidents and business requests

• Bachelor's degree in Computer Science with at least 5 years of Data Engineering experience using AWS services, Pyspark/EMR.
• Having Certifications like AWS Certified Data Analytics, CCA Spark and Hadoop Developer or CCP Data Engineer is highly desirable.

• 8+ Years of Experience in Data Lake, Data Analytics & Business Intelligence Solutions and at least 3+ as AWS Data Engineer
• Full life cycle project implementation experience in AWS using Pyspark/EMR, Athena, S3, Redshift, AWS API Gateway, Lambda, Glue and other managed services
• Strong experience in building ETL data pipelines using Pyspark on EMR framework
• Hands on experience in using S3, AWS Glue jobs, S3 Copy, Lambda and API Gateway.
• Working SQL experience to troubleshoot SQL code. Redshift knowledge is an added advantage.
• Strong experience in DevOps and CI/CD using Git and Jenkins, experience in cloud native scripting such as CloudFormation and ARM templates
• Hands-on with system & application log tools like Datadog, CloudWatch, Splunk etc.
• Experience working with Python, Python ML libraries for data analysis, wrangling and insights generation
• Experience using Jira for task prioritization and Confluence and other tools for documentation.
• Experience in Python and common python libraries.
• Strong analytical experience with database in writing complex queries, query optimization, debugging, user defined functions, views, indexes etc.
• Experience with source control systems such as Git, Bitbucket, and Jenkins build and continuous integration tools.
• Strong understanding of AWS Data lake and data bricks.
• Exposure to Kafka, Redshift, Sage Maker would be added advantage
• Exposure to data visualization tools like Power BI, Tableau etc.
• Functional Knowledge in the areas of Sales & Distribution, Material Management, Finance and Production Planning is preferred

• Full life cycle implementation experience in AWS using Pyspark/EMR, Athena, S3, Redshift, AWS API Gateway, Lambda, Glue and other managed services
• Experience with agile development methodologies by following DevOps, Data Ops and Dev Sec Ops practices.
• Manage life cycle of ETL Pipelines and other cloud platform tools, including GitHub, Jenkins, Terraform, Jira, and Confluence.
• Excellent written, verbal and inter-personal and stakeholder communication skills.
• Ability to analyze trends associated with huge datasets.
• Ability to work with cross functional teams from multiple regions/ time zones by effectively leveraging multi-form communication (Email, MS Teams for voice and chat, meetings)
• Excellent prioritization and problem-solving skills.
• Action Oriented: Have a sense of urgency, high energy and enthusiasm in managing Systems and Platforms
• Drives Results: Consistently achieving results, even under tough circumstances.
• Global Perspective: Takes a broad view when approaching issues; using a global lens.
• Learn and train other team members
• Communicates Effectively: Provide timely and consistent updates and recommendations on BI Operational issues and improvements to stakeholders.
• Drive to meet and exceed BI Operational SLAs for Service Now incidents, Major Incidents, xMatters alerts, Employee Experience Metrics and BI application /process availability metrics.