Job Title: Data EngineerExperience Level: 5+ YearsLocation: Hyderabad
Job Summary
We are looking for a seasoned and innovative Senior Data Engineer to join our dynamic data team. This role is ideal for professionals with a strong foundation in data engineering, coupled with hands-on experience in machine learning workflows, statistical analysis, and big data technologies. You will play a critical role in building scalable data pipelines, enabling advanced analytics, and supporting data science initiatives. Proficiency in Python is essential, and experience with PySpark is a strong plus.
Key Responsibilities
Data Pipeline Development: Design and implement scalable, high-performance ETL/ELT pipelines using Python and PySpark.
ML & Statistical Integration: Collaborate with data scientists to integrate machine learning models and statistical analysis into data workflows.
Data Modeling: Create and optimize data models (relational, dimensional, and columnar) to support analytics and ML use cases.
Big Data Infrastructure: Manage and optimize data platforms such as Snowflake, Redshift, BigQuery, and Databricks.
Performance Tuning: Monitor and enhance the performance of data pipelines and queries.
Data Governance: Ensure data quality, integrity, and compliance through robust governance practices.
Cross-functional Collaboration: Partner with analysts, scientists, and product teams to translate business needs into technical solutions.
Automation & Monitoring: Automate data workflows and implement monitoring and alerting systems.
Mentorship: Guide junior engineers and promote best practices in data engineering and ML integration.
Innovation: Stay current with emerging technologies in data engineering, ML, and analytics.
Required Qualifications
Bachelor's or Master's degree in Computer Science, Data Science, Engineering, or a related field.
5+ years of experience in data engineering with a strong focus on Python and big data tools.
Solid understanding of machine learning concepts and statistical analysis techniques.
Proficiency in SQL and Python; experience with PySpark is highly desirable.
Experience with cloud platforms (AWS, Azure, or GCP) and data tools (e.g., Glue, Data Factory, Dataflow).
Familiarity with data warehousing and lakehouse architectures.
Knowledge of data modeling techniques (e.g., star schema, snowflake schema).
Experience with version control systems like Git.
Strong problem-solving skills and ability to work in a fast-paced environment.
Excellent communication and collaboration skills.
MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.