Job Description

& Qualifications

• Expert of the data and its application by users
• Understands data landscape and environments: sources, elements, update freq, completeness, stewards/contacts, platforms
• Manages ETL: uses programming and tools for data ingestion, configures pipelines, applies transformations and decoding, integrates and fuses data, moves and securely deliver
• Translates business requirements to build repeatable, sustainable, efficient, coded processes that can be productionized by Software Engineers and readily modified by other Data Engineers
• Creates POC processes in Dev/QA and works with Software Engineers to productionize and align best coding and process practices that maximize efficiency, speed, stability, system resources and capabilities
• Leverages frameworks in place with big data tools: Hadoop, Spark, Python, Kafka, etc.
• Experienced with relational SQL and NoSQL databases
• Awareness of and compliance with: data privacy, security, legal and contractual guidelines
• Incorporates data quality and privacy checks/alerts to minimize bad data being consumed by end users, models and dashboards, and to protect customer data; revises checks for new data issues
• Validates data products and pipelines are functioning as expected following system or application upgrades, source changes, etc.
• Responsible for data architecture including sources, table structures, physical models
• Works closely with Architects to align systems, tools and applications being utilized with business use case and performance requirements
• Communicates with end users to set expectations and ensure alignment around data accuracy, completeness, timeliness and consistency
• Provides data product support and maintenance
• Establishes, tracks and monitors KPIs related to specific data products and deliverables

Education Qualifications

B.E., B.Tech., M.Sc, MCA

No. of years of experience

5 to 8 years

Area of professional exposure (technical skills)

5+ years of PCF/Python/Spark/Scala/Pyspark and big data processing experience
2+ years of GCP, AWS like cloud technologies

Skill Team

Global Data Insights & Analytics