Job Description

:

Job description (Data Engineer)

Job Location: Gurgaon/Bangalore

Experience: 4+ years

Role:

We are seeking a highly motivated and experienced AWS Engineer, to join our team at the NFL. This position requires an individual with AWS cloud experience and ambition to continually keep up with best practices when it comes to cloud development. The successful candidate must be able to seek out requirements and create best in class cloud-native solutions. The engineer must always create solutions that are repeatable, scalable, and well-governed. They will deploy and rigorously test solutions to ensure they are robust and secure. The engineer will be responsible for creating and maintaining diagrams associated with any solutions that are deployed into production.

• Create and maintain scalable, robust AWS architecture.
• Collaborate with technical teams on modern architectures (Microservices, REST APIs, DynamoDB, Lambda, API Gateway).
• Develop API-based, CDC, batch, and real-time data pipelines for structured and unstructured datasets.
• Enable integration with third-party systems as needed.
• Ensure solutions are repeatable and scalable across the organization.
• Work with development teams to gather requirements, develop solutions, and deploy them.
• Provide robust solution documentation for a wide audience.
• Collaborate with data professionals to bring applications to life, meeting business needs.
• Prioritize data protection and cloud security in all deliverables.

Required Qualifications:

• Experience building serverless applications in AWS.
• 3-4 years of SQL & Python programming experience.
• 2-3 years of experience with AWS tech stack (Glue, Redshift, Kinesis, Athena, CloudTrail, CloudWatch, Lambda, API Gateway, Step functions, SQS, S3, IAM roles, Secrets Manager).
• 1-2 years of experience in DBT with Data Modeling, SQL, Jinja templating, packages/macros to build robust, performant, and reliable data transformation and feature extraction pipelines
• 1-2 years of experience in Airbyte building ingestion modules for streaming, batch, API sources including building CDC mechanism for database sources
• Experience working with ETL Tools like Glue, Fivetran, Talend, Matillion, etc
• Good exposure and hands on knowledge on Datawarehouse / Data Lake solutions
• Very good knowledge on Data Quality Management and ability to perform data validation
• Working knowledge of DevOps methodologies, including designing CI/CD pipelines.

Preferred Qualifications (What will help you stand out):

• AWS Cloud Certifications.
• Good experience in PySpark programming
• Good experience in Airflow, MWAA
• Good experience building Real-Time streaming data pipelines with Kafka, Kinesis etc.
• Knowledge of Jinja templating in Python is a plus
• Knowledge and experience in designing and developing RESTful services.
• Experience building distributed architecture-based systems, especially handling large data volumes and real-time distribution.
• Initiative and problem-solving skills when working independently.
• Familiarity with Big Data Design Patterns, modeling, and architecture.
• Exposure to NoSQL databases and cloud-based data transformation technologies.
• Understanding of object-oriented design principles.
• Knowledge of enterprise integration patterns.
• Experience with messaging middleware, including queues, pub-sub channels, and streaming technologies.
• Expertise in building high-performance, highly scalable, cloud-based applications.
• Experience with SQL and No-SQL databases.

EXL Service