Optimize Spark jobs (memory management, execution plan tuning, resource allocation).
Analyze metrics/logs to identify bottlenecks; implement partitioning, caching, and data layout optimizations to reduce latency and cost.
Advanced Pipeline Development & Data Architecture
Design/build complex ETL workflows in Databricks/Spark for large-scale security telemetry.
Implement high-performance data architectures (Delta/Parquet, indexing, partitioning).
Collaborate on data models enabling efficient querying, enrichment, and analytics.
Troubleshooting & Continuous Improvement
Diagnose and resolve complex performance issues; create preventative playbooks.
Deliver documentation and knowledge transfer to internal teams.
Collaboration & Stakeholder Engagement
Work cross-functionally with engineering, data science, and analyst teams.
Communicate optimization strategies and dashboard insights to technical and non-technical stakeholders.
Technical Skills (Databricks, Spark, SQL, PowerBI), Stakeholder Management, Excellent communication
Responsibilities:
End - to - End ETL pipeline development independently
Testing, production deployment
documentation
stakeholder management
Skills:
4 to 6 years in the Data Engineering domain
Databricks, Spark, SQL, Tableau.
We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, colour, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status.
Job Snapshot
Updated Date
09-09-2025
Job ID
J_4108
Location
Chennai, Tamil Nadu, India
Employee Type
Permanent
Beware of fraud agents! do not pay money to get a job
MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.