Job Description

:

Role: Data Engineer - AWS
Job Location: Gurgaon/Bangalore
Experience: 3+ years

Position Summary

We are seeking a highly motivated and experienced AWS Engineer, to join our team at the EXL. This position requires an individual with AWS cloud experience and ambition to continually keep up with best practices when it comes to cloud development. The successful candidate must be able to seek out requirements and create best in class cloud-native solutions. The engineer must always create solutions that are repeatable, scalable, and well-governed. They will deploy and rigorously test solutions to ensure they are robust and secure. The engineer will be responsible for creating and maintaining diagrams associated with any solutions that are deployed into production.

Responsibilities:

• Create and maintain scalable, robust AWS architecture.
• Collaborate with technical teams on modern architectures (Microservices, REST APIs, DynamoDB, Lambda, API Gateway).
• Develop API-based, CDC, batch, and real-time data pipelines for structured and unstructured datasets.
• Enable integration with third-party systems as needed.
• Ensure solutions are repeatable and scalable across the organization.
• Work with development teams to gather requirements, develop solutions, and deploy them.
• Provide robust solution documentation for a wide audience.
• Collaborate with data professionals to bring applications to life, meeting business needs.
• Prioritize data protection and cloud security in all deliverables.

Qualifications (Required):

• Experience building serverless applications in AWS.
• Degree in Data Science, Statistics, Computer Science or other related fields or an equivalent combination of education and experience.
• 3+ years of SQL, Python & PySpark programming experience with good command on all of them.
• 2-3 years of experience with AWS tech stack (Glue, Redshift, Kinesis, Athena, CloudTrail, CloudWatch, Lambda, API Gateway, Step functions, SQS, S3, IAM roles, Secrets Manager).
• Good experience in Airflow, MWAA
• Experience working with ETL Tools like Glue, Fivetran, Talend, Matillion, etc
• Good exposure and hands on knowledge on Datawarehouse / Data Lake solutions
• Very good knowledge on Data Quality Management and ability to perform data validation
• Working knowledge of DevOps methodologies, including designing CI/CD pipelines
• Experience with comprehensive unit testing methodologies and frameworks, ensuring the reliability and performance of AWS-based data solutions.
• Proficiency in utilizing unit testing tools and practices to validate ETL processes, data pipelines, and other components of data engineering workflows.
• Good working experience of communicating with the stakeholders and collaborate effectively with the business team for data testing.

• Strong communication, problem solving and analytical skills with the ability to do time management and multi-tasking with attention to detail and accuracy.

• Strong business acumen & demonstrated aptitude for analytics that incite action.

Qualifications (Preferred):

• AWS Cloud Certifications.
• 1-2 years of experience in DBT with Data Modeling, SQL, Jinja templating, packages/macros to build robust, performant, and reliable data transformation and feature extraction pipelines
• 1-2 years of experience in Airbyte building ingestion modules for streaming, batch, API sources including building CDC mechanism for database sources
• Good experience building Real-Time streaming data pipelines with Kafka, Kinesis etc.
• Knowledge of Jinja/YAML templating in Python is a plus.
• Knowledge and experience in designing and developing RESTful services.
• Experience building distributed architecture-based systems, especially handling large data volumes and real-time distribution.
• Initiative and problem-solving skills when working independently.
• Familiarity with Big Data Design Patterns, modeling, and architecture.
• Exposure to NoSQL databases and cloud-based data transformation technologies.
• Understanding of object-oriented design principles.
• Knowledge of enterprise integration patterns.
• Experience with messaging middleware, including queues, pub-sub channels, and streaming technologies.
• Expertise in building high-performance, highly scalable, cloud-based applications.
• Experience with SQL and No-SQL databases.

EXL Service