Job Description

Data Engineer - BEAT

We are looking for a senior data engineer who will help us to source, validate and create rule based data ingestion system using state of art data mining algorithms. Responsibility includes leveraging latest data mining techniques, designing validation engines, error monitoring and reviewing for RCA, doing statistical analysis on our datasets and create robust data pipeline for data consumption.

Responsibilities

1. Building efficient models for large scale data extraction from websites
2. Design, construct, install, test and maintain the data management systems.
3. Build high-performance algorithms, predictive models, and prototypes for existing and new datasets
4. Ensure that all systems meet the business/company requirements as well as industry practices.
5. Develop set processes for data mining, data modeling, and data production.
6. Research new uses for existing data.
7. Collaborate with members of your team (like, data architects, IT team, data scientists etc.) on the project's goals.
8. Install/update disaster recovery procedures and manage the data security requirements
9. Recommend different ways to constantly improve data reliability and quality.
10. Designing and building validation engines for monitoring of sourced data
11. Processing, cleansing, and verifying the integrity of data used for analysis
12. Enhancing data collection procedures to include information that is relevant for building analytic systems

Skills and Qualifications

1. Graduate in Engineering, Technology, applied mathematics, physics statistics along with good business skills
2. Intellectual curiosity to find new and unusual ways of how to solve data management issues.
3. Ability to approach data organization challenges while keeping an eye on what's important.
4. Excellent understanding of data sourcing and validation algorithms
5. Experience of working in Cloud based infrastructure with exposure to Data Ops. or ML Ops. Would be added advantage.
6. Good applied statistics skills, such as distributions, statistical testing, regression, etc.
7. Good scripting and programming skills in Python and NLP
8. Experience with data visualization tools, such as Grafana, GGplot, etc.
9. Experience with NoSQL databases, such as MongoDB, Cassandra
10. Good communication skills
11. Good problem solving and negotiation skills