Job Description

At the heart of everything we do is data and this team. Our premium data assets empower the team to drive desirable outcomes for leading brands across industries. Armed with high volumes of transactional data, digital expertise and unmatched data quality, the team plays a key role in improving all our product offerings. Our data artisans are keen on embracing the latest in technology and trends, so theres always room to grow and something new to learn here.
Why we are looking for you
You have hands on experience in - Kafka, Flume, Spark, Java/Scala, Hadoop, HDFS, Hive,SQL
to work with Epsilon Market place team
You have hands on experience in coding languages like Python & Scala .
You have hands on experience in fine tuning Spark Jobs .
You are very good in handling & working closely with key stakeholders of the Project, you must be able to communicate and keep them informed on overall health on projects impacting Data platforms.
You have exposure on Airflow, Docker container, noSQL databses like Hbase.
What you will enjoy in this role
As part of Data Pipeline team will be processing Billions of data per day from multiple region/ data centers.
Working on processing Ad-server data into Storage layer where further analytics will be done
Working on Bigdata technologies like Flume, Kafka, Spark, and loads the aggregated/processed data into HDFS.
Working on identifying a key area of ownership the team has, pipelining data.
Working on intraday (hourly, 5 minute ,15 minute) aggregations using Spark Structured Streaming to jobs performed.
Working on the data assets which will be further used in performance measurement and efficacy of the defined solution, as well as feeding business analytics and data mining.
What you will do
End to end development of automated receipt of anonymized data
End to end development of processing of logs data
Data center to data center replication
Data processing using Flume, Kafka , Spark jobs, Airflow,Docker etc
Migration of production data assets to downstream consuming systems
Disaster Recovery and Business Continuity implementations
Workload Performance Management and Tuning
Ensuring that coded solutions meet functional business requirements for ad serving and measurement
Application specific controls and scheduling
Custom solution building for syndicated and third party datasets

Qualifications
Bachelors Degree in Computer Science or equivalent degree is required
3 5 years of data engineering experience around database marketing technologies and data management, and technical understanding in these areas.
Experience in coding using Java or Scala
Experience with scheduling applications like Airflow, Oozie with complex interdependencies
Good hands on experience in opensource components like Kafka, Flume, Spark, Hadoop, HDFS.
Participate in code reviews providing constructive feedback and accepting feedback on own code.
Experience in No SQL Databases (ex: Hbase) ,Docker containers a plus
Excellent written and verbal communication skills.
Excellent Analytical and problem solving skills
Ability to diagnose and troubleshoot problems quickly
Education: Any Graduate
Industry: Advertising/PR/Event Management

Skills Required