Data Engineering Senior Consultant (data Architecture, Data Governance, Pyspark)

Year    Bangalore, Karnataka, India

Job Description


Company Description

Visa is a world leader in digital payments, facilitating more than 215 billion payments transactions between consumers, merchants, financial institutions and government entities across more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable and secure payments network, enabling individuals, businesses and economies to thrive.

When you join Visa, you join a culture of purpose and belonging - where your growth is priority, your identity is embraced, and the work you do matters. We believe that economies that include everyone everywhere, uplift everyone everywhere. Your work will have a direct impact on billions of people around the world - helping unlock financial access to enable the future of money movement.

Join Visa: A Network Working for Everyone.



Job Summary:

Data Engineering Senior Consultant role for Asia Pacific Region is based out of Bangalore. The role will work with both internal VISA Consulting/Market data science organizations and with external clients of VISA. This position requires strong technical background in modern data architecture, evaluating and recommending suitable technology stack, building scalable, reliable, and secured data ecosystem and good appreciation of data governance and data management practices.

With external clients, the position is responsible for data engineering advisory, design and implementation of comprehensive data platforms and solutions that meets the need of aspiring data organizations. The role is expected to work and collaborate with cross-functional teams within Visa, external clients, and vendors to develop and implement data solutions that support the organization\'s data-driven initiatives and improve business outcomes while maintaining robust data solutions that align with industry best practices and compliance standards.

Internally, the role will support design, development, and delivery of data engineering solutions for the client facing market data science teams. The role will take responsibility for building and running data pipelines, designing our local data warehouse, data marts and data frameworks, and catering to different data presentation techniques based on client requirements.

Principal Responsibilities

  • Act as Data technology SME for both VISA\'s external clients and internal market teams for design and implementation of scalable, secure, robust, and interoperable data solutions that support advanced analytics, reporting, and decision-making processes.
  • Lead Development and implementation of data solutions, data integration, data warehousing, data lakes, data governance and quality, analytics platforms, and reporting tools for both internal and external clients.
  • Create and maintain optimal data pipelines and data exchange channels (Such as APIs, Secured File Transfer, other custom data exchange capabilities provided by VISA) between VISA and clients.
  • Evaluate and select appropriate technologies, tools, and frameworks to support data solution development and ensure data security, privacy, and compliance alignment with market regulations.
  • Explore new technology trends leveraging them to modernize our client data ecosystem. Our clients are Banks, Fintechs and Merchant in the region that we operate.
  • Collaborate with partners & stakeholders at varying levels both internally and externally to provide best in class solutions to VISA\'s clients.
  • Strong execution skills in a fast-paced environment using agile methodologies.
  • Experience in Event Driven Architecture, Massive Parallel and Stream processing, Microservices, ELT and micro-batches.
  • Advanced experience in Hadoop, Python, Spark, PySpark and cloud-based data technologies.
  • Hands on experience with public cloud like Azure, GCP, AWS
  • Driving Innovation and guiding the teams to implement solutions with future thinking.
This is a hybrid position. Hybrid employees can alternate time between both remote and office. Employees in hybrid roles are expected to work from the office 2-3 set days a week (determined by leadership/site), with a general guidepost of being in the office 50% or more of the time based on business needs.

Qualifications

Basic Qualifications
  • Bachelor\'s or Master\'s degree in engineering, Computer Science, Software Engineering, Information Technology, or technical field required.
  • 11+ years of experience in data architecture, data engineering, design, development and delivery of large-scale data solutions with 2+ years of experience in AWS/MA/GCP.
Technical Expertise
  • 11+ years of experience in data architecture, data engineering, design, development and delivery of large-scale data solutions with 2+ years of experience in AWS/MA/GCP.
  • Demonstrated technology and personal leadership in architecting, designing, and building highly scalable data ecosystems.
  • Deep knowledge and experience in architecting modern data extraction and ingestion frameworks using open source and emerging data architecture designs/patterns.
  • Certification in Hadoop, Apache Spark and cloud technologies preferred.
  • Hands-on experience of Hadoop ecosystem and associated technologies such as Hive, Apache Spark, PySpark, MLlib, GraphX.
  • Advanced experience in writing and optimizing efficient SQL queries and Python/PySpark scripts.
  • Good appreciation of ML engineering and Operationalization.
  • Experience with APIs, container-based software deployments.
  • Experience with DevOps, Continuous Integration and Continuous Delivery technologies.
  • Experience in data governance (process & standards for Metadata Management, Data Lineage, Master/Reference Data Management, Data Quality Management and associated tools/platforms) will be an added advantage.
  • Experience in Data Visualization (platforms such as power BI and tableau) and data security practices will be an added advantage.
  • Good appreciation of NoSQL and in-memory data stores
  • Good appreciation of machine learning algorithms, feature engineering, validation, prediction, recommendation, and measurement.
  • Understanding of Payments, Banking and Financial services domain will be a plus.
Leadership Competencies
  • Demonstrates integrity, maturity, and a constructive approach to business challenges.
  • Serves as a role model for the organization by implementing core Visa Values
  • Shows respect for individuals at all levels in the workplace.
  • Strives for excellence and extraordinary results.
  • Uses sound insights and judgments to make informed decisions in line with business strategy and needs.
  • Able to allocate tasks and resources across multiple lines of businesses and geographies.
  • Able to influence senior management both within and outside Data Science
  • Successfully persuading internal stakeholders to commit to best-in-class solutions, when required.
  • Leverages change management leadership as required.
Additional Information

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Visa

Beware of fraud agents! do not pay money to get a job

MNCJobsIndia.com will not be responsible for any payment made to a third-party. All Terms of Use are applicable.


Related Jobs

Job Detail

  • Job Id
    JD3231347
  • Industry
    Not mentioned
  • Total Positions
    1
  • Job Type:
    Full Time
  • Salary:
    Not mentioned
  • Employment Status
    Permanent
  • Job Location
    Bangalore, Karnataka, India
  • Education
    Not mentioned
  • Experience
    Year