Job Description

Role:

Data Engineer - (Django Must- Big Data Platform)

Company:

PayU

Location:

Bangalore/Mumbai/Gurgaon

Job responsibilities:

Building, maintaining, and monitoring sustainable infrastructure with a focus on developing and maintaining high-volume, low-latency data platforms using latest opensource frameworks, highly available RESTful services, and back-end Cloud based/open source systems

Execute day to day operation of the data platform to ensure top-notch performance of data platform systems for customer-facing applications

Work on features to make the data Platform more robust and resilient

Drive automation of application deployment for production environments

Identify and support areas for process and efficiency improvement within platform operations

Taking ownership of your projects, and have the work in small cross functional teams to drive projects to closure

Highly collaborative with a strong belief in continuous improvement.

Ability to stay focused and calm in a fast-paced environment.

Requirements to be successful in this role:

BE/B-Tech/BS/Meng/MSc in computer science or a related technical field from top-tier colleges or equivalent practical experience

2.5-5 years of experience working on platform infrastructure based on Cloud and open-source systems and automation.

Must be very good with programming lang :

Python

Django/Flask (preferred, must be experienced in building APIs)

Databases:

SQL querying & db understanding (RDBMS: mysql, postgresql; Nosql: mongodb, elasticsearch, dynamodb)

Familiarity with big data tools & distributed systems (spark, redshift, kafka)

Cloud Computing (Any of AWS/Azure/GCP), preferred:

AWS

Orchestration:

airflow, docker

Good to Have:

Monitoring/Logging frameworks (ELK, Grafana)

Scalable Compute solutions (Kafka, Snowflake)

Workflow management libraries and tools

Container orchestration systems like Kubernetes

Code-as-Infrastructure/Pipeline-as-Code (Terraform/Chef)

Understanding and functional experience in Big Data Engineering practices: Data

Warehousing, ingestion pipelines, ELT, data migration, QC