Job Description

The health and safety of our employees and candidates is very important to us. Due to the current situation related to the Novel Coronavirus (2019-nCoV), we’re leveraging our digital capabilities to ensure we can continue to recruit top talent at the HSBC Group. As your application progresses, you may be asked to use one of our digital tools to help you through your recruitment journey. If so, one of our Resourcing colleagues will explain how our video-interviewing technology will be used throughout the recruitment process and will be on hand to answer any questions you might have. Some careers shine brighter than others. If you’re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further.
HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions.
We are currently seeking an experienced professional to join our team in the role of Senior Consultant Specialist.
In this role, you will:

• Work with stakeholders including Business, IT and Design teams to assist with data pipeline setup and infra needs.
• Integrate massive datasets from multiple data sources for data modelling using Python, Pyspark, Apache Beam, DataProc and Airflow.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using BigData and Cloud technologies
• Build and Maintain Streaming or Batch dataflow, dataproc pipelines using Pyspark, Apache Beam, Python scripts.
• Design, create and maintain optimal data pipelines and architectures for data processing
• Implement methods for Devops automation of all parts of the build data pipelines to deploy from development to production

Requirements
To be successful in this role, you should meet the following requirements:

• Good hands on experience of Python, Pyspark and Apache Beam programming for building Data Pipelines on Cloud
• Rich experience of creating streaming data pipeline using PySpark, Apache Beam frameworks.
• Rich experience of working on beam runner like Apache Spark, Apache Flink, GCP dataflow, AWS EMR etc.
• Experienced in designing and building production data pipelines from data ingestion, transformation to consumption within Cloud.
• Experience with data pipeline and workflow management tools: Airflow, Composer, EMR etc.
• Experience with object-oriented/object function scripting languages: Pyspark, Python, Beam etc.
• Experienced in designing, building and operationalizing large-scale enterprise data solutions and applications using one or more of GCP data and analytics services in combination with 3rd parties - Spark, Hive, Beam, Python, EMR etc.

You’ll achieve more when you join HSBC. www.hsbc.com/careers
HSBC is committed to building a culture where all employees are valued, respected and opinions count. We take pride in providing a workplace that fosters continuous professional development, flexible working and opportunities to grow within an inclusive and diverse environment. Personal data held by the Bank relating to employment applications will be used in accordance with our Privacy Statement, which is available on our website. Issued by – HSBC Software Development India