Job Description

:

• Design and implement scalable and efficient data processing pipelines using Data Factory as a Service.
• Collaborate with data engineers and architects to ensure optimal data flow and integration.
• Implemented and maintained data quality checks using AWS Glue, contributing to the identification and resolution of data inconsistencies, leveraging Collibra data quality system for effective data governance.
• Engaged in data transformation and analysis using DataBricks, improving data processing efficiency and supporting data-driven decision-making.
• Integrate diverse data sources and formats into cohesive and usable datasets.
• Create and maintain comprehensive documentation for data pipelines, workflows, and integration processes. Implement robust error handling mechanisms to ensure the reliability of data pipelines.
• Create and maintain comprehensive documentation for data pipelines, workflows, and integration processes.
• Responsible for handling Release Management, raising Change Request and scheduling for PROD implementations.

Qualifications:

• Degree holder in Computer Science, Information Technology, Computer Engineering, Information System or equivalent
• Minimum 5 years of experience in designing, implementing, and optimizing data processing pipelines and workflows.
• Proven experience in data engineering, ETL processes, and Data ingestion frameworks (real-time & Batch processing)
• Expertise in using Data Factory as a Service, particularly within the Datawarehouse ecosystem.
• Strong programming and scripting skills (e.g., Java/Python/Scala).
• Familiarity with data modelling, Visualization, schema design, and database technologies.
• Data extraction tools experience (SaaS solutions, APIs, Open-source libraries, Proxies, and datasets)
• Worked on Big Data technologies (Spark, Pyspark, Hadoop, HDFS, Hive, Oozie, Sqoop and Hue)
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills

About Us:

Business Profile Description

Beyondsoft International (Singapore) Pte. Ltd. was set up in 2007 and established as the regional headquarters for the Southeast Asia (SEA) and European markets in September 2015. Based on our vision of "Using technology to promote social progress, economic development and become a global customer preferred partner" and our concept of "Beyond your expectations", Beyondsoft is committed to provide our customers in countries along the "Belt and Road" with comprehensive solutions and products and creating commercial value for customers to realizing continuous businesses development.

Our core business includes: * IT development services Providing customers with IT consulting, software research and development, software and hardware testing, system integration and operation and maintenance, data analysis and other services;

• New retail solutions and productsThrough intelligent products, helping small and medium-sized enterprises (SMEs) realize the digital transformation of their daily operations;
• Internet of Things (IoT) platform and solutions Comprehensive use of IoT, artificial intelligence, big data, cloud computing and other technologies to provide IoT solutions for intelligent upgrades in cities, parks, buildings and industries, to create a smart future.

For more information, please visit www.beyondsoft.com

Beyondsoft