At Innover, we endeavor to see our clients become connected, insight-driven businesses. Our integrated Digital Experiences, Data & Insights and Digital Operations studios help clients embrace digital transformation and drive unique outstanding experiences that apply to the entire customer lifecycle. Our connected studios work in tandem to reimagine the convergence of innovation, technology, people, and business agility to deliver impressive returns on investments. We help organizations capitalize on current trends and game-changing technologies molding them into future-ready enterprises.
Take a look at how each of our studios represents deep pockets of expertise and delivers on the promise of data-driven, connected enterprises.
About the Role:
As part of our
Innovation Team
, we are seeking a
C
ertified Senior Databricks Engineer / Tech Lead
with 7-8 years of hands-on experience in building scalable data platforms. This role will focus on designing, building, and operationalizing data solutions on the
Databricks platform
to accelerate advanced analytics and AI use cases.
Key Responsibilities:
Architect, develop, productionize and maintain end to end solutions in
Databricks
Implement and optimize
ETL/ELT processes
for structured and semi-structured data
Leverage
Delta Lake
for ACID transactions, data versioning, and time-travel features
Drive adoption of the
Lakehouse architecture
to unify data warehousing and AI/ML workloads
Implement
CI/CD pipelines
using
Databricks Repos
,
Asset Bundles
, and integration with
DevOps tools
Configure and enforce
Unity Catalog
for secure, governed access to data assets
Design and implement
data quality and validation frameworks
to ensure trusted data
Lead performance tuning and optimization efforts for
Spark jobs and queries
Integrate with
external systems
such as
Kafka
,
Event Hub
, and
REST APIs
for real-time and batch processing
Collaborate with data scientists and business stakeholders to build feature-rich datasets and reusable assets
Troubleshoot and debug complex data workflows in development and production environments
Guide junior engineers and contribute to best practices in data engineering and platform usage
Ensure platform
security, access controls
, and compliance with enterprise data governance standards.
Required Skills:
Expertise in Apache Spark and Databricks platform
Experience with Databricks Lakehouse architecture
Delta Lake concepts
Proficient in PySpark, SQL, and Delta Lake
Strong knowledge of Data Engineering concepts
Experience with data ingestion, ETL/ELT pipelines
Familiarity with Unity Catalog and data governance
Hands-on with Databricks Notebooks and Jobs
CI/CD automation with Databricks Repos and DevOps, Asset Bundles
Databricks Asset Bundle implementation knowledge
Strong understanding of performance tuning in Spark
Data quality and validation framework implementation
Experience in handling structured, semi-structured data
Proficient in debugging and troubleshooting
Collaboration with data scientists and analysts
Good understanding of security and access control
Experience with
Mosaic AI
or Databricks ML capabilities
Exposure to
streaming pipelines
using Structured Streaming
* Familiarity with
data observability
and
lineage tools.
Beware of fraud agents! do not pay money to get a job
MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.