Job Description

Epergne Solutions is looking for AWS Data Engineers across PAN India. Currently role has flexibility to work from home however if the need arises candidate must be willing to go to any offices located in Indore, Gurgaon, Bhubaneshwar, Chandigarh, Hyderabad, Delhi, Pune, Bangalore, ChennaiJob Title: Data EngineerRole & Responsibilities:

• Design, develop, and maintain end-to-end data pipelines on AWS, utilizing serverless architecture.
• Implement data ingestion, validation, transformation procedures using AWS services such as Lambda, Glue, Kinesis, SNS, SQS, and CloudFormation.
• Write orchestration tasks within Apache Airflow.
• Develop and execute data quality checks using Great Expectations to ensure data integrity and reliability.
• Collaborate with other teams to understand mission objectives and translate them into data pipeline requirements.
• Utilize PySpark for complex data processing tasks within AWS Glue jobs.

• Bachelor\'s degree in Computer Science, Engineering, or related field.
• Strong proficiency in Python programming language.
• Hands-on experience with AWS services.
• Experience with serverless architecture and Infrastructure as Code (IaC) using AWS CDK.
• Proficiency in Apache Airflow for orchestration of data pipelines.
• Familiarity with data quality assurance techniques and tools, preferably Great Expectations.
• Experience with SQL for data manipulation and querying.
• Strong communication and collaboration skills, with the ability to work effectively in a team environment.
• Experience with Data Lakehouse, dbt, Apache Hudi data format is a plus.

Epergne Solutions