Responsibilities
Direct Responsibilities
Work on the stages from data ingestion to analytics, encompass integration, transformation, warehousing and maintenance
The Data Engineer designs architecture, orchestrate, deploys and monitors reliable data processing systems.
Implement Batch and streaming data pipelines to ingest data into the data warehouse
Perform undercurrent activities (Data architecture, Data management, DataOps, Security)
Perform data transformation and modeling, to convert data from OLTP to OLAP to speed up data querying, and best align with business needs
Serve downstream stakeholders across the organization, whose improved access to standardized data will make them more effective at delivering use cases, building dashboards and guiding decisions
Technical & Behavioral Competencies
Master Data engineering fundamentals concepts (Data warehouse, Data Lake, Data Lakehouse)
Master Golang, Bash, SQL, Python
Master of HTTP and REST API Best practices
Master batch and streaming datapipeline using Kafka
Master code versioning with Git and best practices for continuous integration & delivery (CI/CD)
Master writing clean and tested code following software engineering best practices (Readable, Modular, Reusable, Extensible)
Master data modeling (3NF, Kimball, Vault)
Knowledge of data orchestration using Airflow or Dagster
Knowledge to self-host and manage tools like Metabase, DBT
Knowledge of cloud principals and infrastructure management (IAM, Logging, Terraform, Ansible)
Knowledge of data abstraction layers (Object Storage, Relational, NoSQL, Document, Trino, and Graph databases)
Knowledge with Containerization and workload orchestration with (Docker, Kubernetes, Artifactory)
Background in working in an agile environment (knowledge of the methods and their limits)
Skills Referential
Behavioural Skills:
Communication skills - oral & written
Attention to detail / rigor
Adaptability
Ability to synthetize / simplify
Transversal Skills:
Ability to develop and adapt a process
Ability to understand, explain and support change
Analytical Ability
Ability to set up relevant performance indicators
Ability to anticipate business / strategic evolution
Education Level:
Bachelor Degree or equivalent
Experience Level
At least 5 years
MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.