Join our Team

About this opportunity:

Responsible for designing, building, and maintaining scalable data pipelines to support our data-driven initiatives. Collaborate with cross-functional teams to understand their data needs and develop solutions to ensure reliable and efficient data processing.

What you will do:

· Data integration: global/local system integration with IDAP.

· Data vault build up according to local/global system DV definitions.

· Design and implement data pipelines to collect, process, and store large volumes of data from various sources.

· Work closely with data scientists and analysts to understand data requirements and provide the necessary infrastructure and tools for data analysis and reporting.

· Optimize data infrastructure for performance, reliability, and cost-effectiveness.

· Develop and maintain data quality and governance processes to ensure the accuracy and integrity of the data.

· Stay current with industry trends and best practices in data engineering and contribute to the continuous improvement of our data infrastructure.

· Data quality/data reliability design

· Raw/Standard data preparation according to business understanding.

The skills you bring:

· Proven experience in building and maintaining large-scale data pipelines and data warehouses.

· Database knowledge: SQL or no-SQL data base; view creation; data query

· Knowledge and experience on data vault

· Knowledge and experience on Snowflake and azure ADF

· Experience on data pipeline setup, data ETL, and better have experience on data preparation.

· SQL command knowledge is must have

· Knowledge on batch/streaming processing experience

· Knowledge on data processing knowledge: Store procedure, task

· Knowledge and experience on big data platform: such as Kafka, airflow, Flink, Flask.