数据工程师 Data Engineer

1. Design, build and maintain batch or real-time data systems and pipelines.

2. Develop ETL (extract, transform, load)processes to help extract and manipulate data from multiple sources

3. Maintain and optimize the data infrastructure required for accurate extraction, transformation and loading of data from a wide variety of data sources.

4. Automate data workflows such as data ingestion, aggregation, and ETL processing.

5. Prepare raw data in OLAP databases into a consumable dataset for both technical and non-technical stakeholders.

REQUIREMENT:

1. Bachelor degree or above. Computer science, Information Systems or a related field.

2. Advanced SQL skills & experience in relational databases and database design.

3. Strong proficiency in data pipeline and workflow management.

4. Experience building and deploying machine learning models.

5. Experience working with Kubernetes container platform.

6. Great numerical, analytical and problem-solving skills.

7. Excellent communication and organizational skills.

8. Proven ability to work independently and with a team.

9. Experience working with data streaming related platforms (e.g.,Apache Kafka, ksqlDB, Apache Pino, etc.)

10. Experience working with large data sets and distributed computing (e.g., Hadoop/Spark, Presto/Trino. Superset, etc.)

11. Proficiency in programming languages, e.g., Python, Java, Go, etc.