Data Engineer with Databricks - 7 Years of Experience
Experience Required:
7+ Years of overall experience in Data Science, Advanced Analytics, and Machine Learning, with at least 2+ years hands-on experience using Databricks.
Key Responsibilities:
Design, develop, and deploy machine learning models and advanced analytics solutions using Databricks and Azure/AWS cloud services.
Perform exploratory data analysis (EDA), feature engineering, and statistical analysis to uncover insights from large datasets.
Collaborate with data engineers to build data pipelines using Databricks notebooks, Delta Lake, and Spark.
Optimize and scale machine learning workflows using distributed computing frameworks (PySpark, SparkML, MLflow).
Develop production-grade data science solutions integrating with business processes and cloud-based environments.
Communicate findings, insights, and recommendations to stakeholders through visualizations, reports, and presentations.
Collaborate with cross-functional teams including Data Engineers, Product Managers, and Business Leaders to solve complex business problems.
Ensure best practices in code quality, version control (Git), and model lifecycle management (MLOps).
Required Skills and Qualifications:
? 7+ years of experience in Data Science, Advanced Analytics, or related fields.
? Strong programming skills in
Python
and experience with machine learning libraries (scikit-learn, XGBoost, TensorFlow, etc.).
? 2+ years of hands-on experience working with
Databricks
platform (preferably on Azure or AWS).
? Proficiency in
PySpark
or
Spark SQL
for large-scale data processing and distributed computing.
? Solid understanding of statistical modeling, machine learning algorithms, and data mining techniques.
? Experience building data pipelines and performing data transformations using Databricks notebooks.
? Familiarity with
Delta Lake
, data versioning, and performance optimization techniques.
? Experience with
MLflow
or similar tools for experiment tracking and model management.
? Strong SQL skills for data querying and analysis.
? Experience with visualization tools (Power BI, Tableau, or Databricks visualizations).
? Understanding of cloud platforms:
Azure
,
AWS
, or
GCP
.
Preferred Qualifications:
Databricks Certification (e.g.,
Databricks Certified Data Engineer Associate
or
Databricks Certified Machine Learning Professional
).
Experience with Azure Data Lake, Azure Data Factory, or AWS equivalent services.
Knowledge of MLOps principles and CI/CD pipelines for model deployment.
Experience with real-time data processing or streaming data.
Background in business domains such as Finance, Healthcare, Retail, or Manufacturing is a plus.
Soft Skills:
Excellent problem-solving and analytical thinking skills.
Ability to translate business requirements into technical solutions.
Strong communication and collaboration skills across technical and non-technical teams.
Self-starter with the ability to work independently in a fast-paced environment.
Education:
Bachelor's or Master's degree in Computer Science, Data Science, Statistics, Mathematics, Engineering, or related field.
Job Types: Full-time, Contractual / Temporary, Freelance
Contract length: 12 months
Pay: ?80,000.00 - ?90,000.00 per month
Benefits:
Health insurance
Schedule:
Monday to Friday
Work Location: Remote
Beware of fraud agents! do not pay money to get a job
MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.