Job Description

:

• Work with business to understand the project requirements
• Design, develop and implement data pipelines involving data fetched from diverse sources and housed in tables and views in the semantic layers
• Responsible for ETL processes and tools as well as database loading and manipulation to ensure that the data needs are met throughout the course of the projects and support.
• Ensure designs are in compliance with specifications
• Prepare and produce releases of software components
• Liaise with business and a team of developers and make sure, the best of the solutions are delivered with in time and budget
• Collaborate with other developers, business/data analysts and leads to design, develop and test project assignments

Qualifications:

• Bachelor in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
• Minimum 5 years of in Data Engineer role in Big Data platform by using Hadoop concepts and framework
• Hands on experience on Spark Core, Spark-SQL, Scala-Programming and Streaming datasets
• Expertise in HDFS architecture, Hadoop framework - Map Reduce, Hive, PIG, Sqoop, Flume & data warehouse concepts
• CI/CD experience (Jenkins, GitHub) is a must
• Proficient in Hadoop MapReduce programming and Hadoop ecosystem
• Good understanding of RDBMS principles, Shared nothing, MPP architecture
• Hands on performance tuning, query optimization, file and error-handling, restart mechanism
• Hands on experience on UNIX Shell Scripting and Python
• Knowledge of Teradata, SAS, QlikView reporting etc.
• Able to understand the complex transformation logic and translate them to Spark-SQL queries
• Worked on Cloudera distribution framework, Oozie Workflow (or any Scheduler)
• Familiar with Data Warehouse concepts and Change Data Capture (CDC and SCD Types)
• Core Java skillset is an added advantage
• Airflow is an added advantage
• Good Knowledge and experience in any database (Teradata or Data Lake or Oracle or SQL Server) are a plus

About Us:

Beyondsoft International (Singapore) Pte. Ltd. was set up in 2007 and established as the regional headquarters for the Southeast Asia (SEA) and European markets in September 2015. Based on our vision of "Using technology to promote social progress, economic development and become a global customer preferred partner" and our concept of "Beyond your expectations", Beyondsoft is committed to provide our customers in countries along the "Belt and Road" with comprehensive solutions and products and creating commercial value for customers to realizing continuous businesses development.

Our core business includes: * IT development servicesProviding customers with IT consulting, software research and development, software and hardware testing, system integration and operation and maintenance, data analysis and other services;

• New retail solutions and productsThrough intelligent products, helping small and medium-sized enterprises (SMEs) realize the digital transformation of their daily operations;
• Internet of Things (IoT) platform and solutionsComprehensive use of IoT, artificial intelligence, big data, cloud computing and other technologies to provide IoT solutions for intelligent upgrades in cities, parks, buildings and industries, to create a smart future.

For more information, please visit www.beyondsoft.com.

Beyondsoft