Job Description

Data Engineer - BEAT

We are looking for a senior data engineer who will help us to source, validate and create rule based data ingestion system using state of art data mining algorithms. Responsibility includes leveraging latest data mining techniques, designing validation engines, Integrating with MDS API, Snowflake and DataBrick, error monitoring and reviewing for RCA, doing statistical analysis on our datasets and create robust data pipeline for data consumption.

Responsibilities

1. Building efficient models for large scale data platforms (Snowflake)
2. Design, construct, install, test and maintain the data management systems.
3. Strong knowledge of API integrations (MDS providers and Internal applications)
4. Build high-performance predictive data models, and prototypes for existing and new datasets
5. Ensure that all systems meet the business/company requirements as well as industry practices.
6. Develop set processes for data Integration, data modeling, and data production.
7. Research new uses for existing data.
8. Collaborate with members of your team (like, data architects, IT team, data scientists etc.) on the project's goals.
9. Install/update disaster recovery procedures and manage the data security requirements
10. Recommend different ways to constantly improve data reliability and quality.
11. Designing and building validation engines for monitoring of sourced data
12. Processing, cleansing, and verifying the integrity of data used for analysis
13. Enhancing data collection procedures to include information that is relevant for building analytic systems

Skills and Qualifications

1. Graduate in Engineering, Technology, applied mathematics, physics statistics along with good business skills
2. Intellectual curiosity to find new and unusual ways of how to solve data management issues.
3. Snowflake Certification will be added advantage.
4. 3+ years of experience administering enterprise data platforms, with at least 1 year hands-on experience with Snowflake.
5. Ability to approach data organization challenges while keeping an eye on what's important.
6. Excellent understanding of data sourcing and validation algorithms
7. Experience of working in Cloud based infrastructure with exposure to Data Ops. or ML Ops. Would be added advantage.
8. Good applied statistics skills, such as distributions, statistical testing, regression, etc.
9. Good scripting and programming skills in Python and NLP
10. Experience with data visualization tools, such as Grafana, GGplot, etc.
11. Experience with NoSQL databases, such as MongoDB, Cassandra
12. Good communication skills
13. Good problem solving and negotiation skills