Job Description

Comprehensive Rehab Consultants focuses on partnering with skilled nursing facilities to meet their individual needs. We are passionate about healthcare technology, innovation, and delivery. CRC is widely considered as visionaries in the post-acute space, designing new care models to improve efficiency, decrease hospitalizations, and improve clinical outcomes.

We operate a modern Azure Lakehouse platform built on Databricks that powers enterprise analytics, machine learning, and AI-driven care models. Our platform integrates multiple enterprise systems including our internal EHR, along with external monthly and quarterly CMS and payer datasets.

We are searching for a Senior Databricks Engineer to join our team. Your primary responsibility will be to own, maintain, and evolve CRC's Azure Databricks Lakehouse platform and Medallion Architecture (Bronze, Silver, Gold). You will work closely with another data engineer, a machine learning engineer, and a team of healthcare analysts to ensure our platform is reliable, scalable, secure, and production-grade.

General Requirements

Strong communication and documentation skills Native/Fluent English, spoken and written Able to work independently and collaboratively in a fast-moving environment Available M-F 7:00 AM to 3:00 PM (Central Standard Time) and some weekends Able to own production systems with accountability for reliability, performance, and cost

Roles and Responsibilities

Own and operate CRC's Azure Databricks Lakehouse platform and supporting infrastructure Maintain and optimize Databricks clusters, jobs, workflows, and Unity Catalog Design, build, and extend automated ingestion pipelines from enterprise systems including the internal EHR and external CMS and payer datasets Automate ingestion of new data sources and implement scalable ingestion frameworks Develop and maintain Bronze, Silver, and Gold layers of the Medallion Architecture Build Python and PySpark transformation frameworks for data standardization, validation, and enrichment Develop Silver-layer feature tables for machine learning workloads Build Gold-layer analytical models to support Power BI dashboards Partner with the Machine Learning Engineer to support production ML pipelines Partner with analysts to ensure reliable, performant, and trusted data Implement data quality checks, reconciliation, and audit controls Support nightly and near-real-time refresh pipelines Implement security, access controls, and data governance best practices Provide redundancy with the Data Engineer to ensure no single point of failure Document pipelines, schemas, data contracts, and dependencies Participate in rotational QA of pipelines and platform

Education & Experience Requirements

Bachelor's degree in Computer Science, Engineering, or Information Technology preferred but not required Equivalent certifications and proven enterprise project work accepted in lieu of a formal degree

Minimum 7+ years of professional data engineering experience required

Minimum 4+ years of hands-on Databricks engineering experience required

Demonstrated experience operating production-grade data platforms Strong experience building Medallion Architecture (Bronze, Silver, Gold) Experience integrating multiple enterprise data sources (EHR, claims, CMS, APIs, operational systems) Experience supporting analytics and machine learning workloads Healthcare data experience strongly preferred

Required Certifications (Must Be Current)

At least

two

of the following certifications are required:

Databricks Certified Data Engineer Professional Databricks Certified Data Engineer Associate Microsoft Certified: Azure Data Engineer Associate (DP-203) Azure Administrator (AZ-104) Azure Solutions Architect (AZ-305) Databricks Lakehouse Fundamentals
Candidates without current certifications will not be considered.

Preferred Skills

Expertise with Azure Databricks, Delta Lake, and Unity Catalog Strong Python and PySpark for data engineering Strong SQL and Spark SQL performance tuning Experience building scalable ingestion frameworks Experience with Azure Data Lake Storage Gen2 Familiarity with MLflow and feature engineering pipelines Experience supporting Power BI semantic models Experience with CI/CD for data pipelines (GitHub, Azure DevOps) Understanding of healthcare data models (EHR, claims, RCM, CMS, HIE) Familiarity with HIPAA, PHI security, and healthcare compliance
Job Type: Full-time

Pay: ?2,709.00 - ?4,064.00 per hour

Experience:

professional data engineering: 7 years (Required) hands-on Databricks engineering: 4 years (Required)
Work Location: Remote