Job Description

5 - 8 Years
1 Opening
Trivandrum


###

Role description

We are seeking a Data Engineer with Spark & Streaming skills builds real-time, scalable data pipelines using tools like Spark, Kafka, and cloud services (GCP) to ingest, transform, and deliver data for analytics and ML.

Responsibilities:

As a Senior Data Engineer, you will



Design, develop, and maintain ETL/ELT data pipelines for batch and real-time data ingestion, transformation, and loading using Spark (PySpark/Scala) and streaming technologies (Kafka, Flink).





Build and optimize scalable data architectures, including data lakes, data warehouses (BigQuery), and streaming platforms.





Performance Tuning: Optimize Spark jobs, SQL queries, and data processing workflows for speed, efficiency, and cost-effectiveness





Data Quality: Implement data quality checks, monitoring, and ing systems to ensure data accuracy and consistency.

Required Skills & Qualifications:

Programming: Strong proficiency in Python, SQL, and potentially Scala/Java. Big Data: Expertise in Apache Spark (Spark SQL, DataFrames, Streaming). Streaming: Experience with messaging queues like Apache Kafka, or Pub/Sub. Cloud: Familiarity with GCP, Azure data services. Databases: Knowledge of data warehousing (Snowflake, Redshift) and NoSQL databases. Tools: Experience with Airflow, Databricks, Docker, Kubernetes is a plus.

Experience Level

Total IT Experience - Minimum 6 years


###

Skills

data engineering,python,sql,kafka,gcp,spark sql,data warehouse,



###

About UST

UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world's best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients' organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact--touching billions of lives in the process.