Big Data Engineer 大数据开发工程师
宏利金融全球服务中心 查看所有职位ChengduUpdate time: July 25,2019
Job Description
成都市 高新区

Responsibilities:

Responsibility:

Modern Data Architecture: Providing operational support to the strategic rollout, selection and delivery of proven as well as emerging tooling to ensure that data pipelines are scalable, repeatable and secure serving multiple users within the organization. The incumbent is responsible for supporting highly complex, multi-faceted big data initiatives associated the following functions areas:

  • Provide operational and technical requirements inputs into detailed design and high performing capabilities initiation

  • Lead the design and build/use of tools and applications supporting and/or optimizing operational processes and procedures;

  • Ensure the timely delivery to meet project timeline by automating development and deployment tasks;

  • Effectively manage operational risk and change with a continuous focus on process improvements

  • Governance and Controls: Responsible for ensuring all data management processes conform to standard operating procedures and aligns with how a modern data architecture aligns to enterprise technology standards and policies; contribute and provide guidance around data quality, metadata management, data stewardship and security, and access controls.

  • Innovation and Continuous Improvement: Evaluate/conduct PoC of new data technologies and capabilities that enable, maintain and streamline administration, deployment, configuration, installation, monitoring and security processes and procedures;

  • Platform Administration and Support: Responsible for ensuring the health of enterprise data lake platform as well as day-to-day operations.

  • Install and maintain enterprise data lake platforms; monitor patch releases and design various patch installation strategies and maintain systems according to Manulife security standard

  • Administer and monitor server performance and availability; Design and maintain system tools, scripts and automation processes

  • Provide guidance for application teams and users to onboard enterprise data lake platforms; Implement and deploy application code and components to the system; Perform troubleshoot and identify and analyze the reported issue in a timely manner based on SLA


Requirements

  • Demonstrated 3-7 years professional experience in big data/data management including a university degree in Engineering, Computer Science or equivalent program;

  • Extensive expertise with Linux (RHEL derivatives preferred)

  • Extensive expertise in data technologies and the use of data to support software development, advanced analytics and reporting. Particular focus on Cloud (Azure) and Hadoop-based technologies (HDP/HDF) and programming or scripting languages like Java, Scala, Linux, C++, PHP, Ruby, Python, R and SAS. Also expert knowledge should be present regarding different (NoSQL or RDBMS) databases such as MongoDB, Cassandra or Hbase; Working experience with modern data streaming process with Kafka, Apache Spark and data ingestion framework NiFi, Hive, Pig, etc.

  • Experience with security/data protection solutions, Kerberos, Active Directory, HDFS Access Control, OAuth2, OpenID, LDAP

  • Experience with network layer security design in VPN, firewall, Loadbalancer

  • Experience with Application Configuration Management Chef and Ansible

  • Experience and capability in translating non-technical user requests into complex technical specifications and solutions that meet these requirements.

  • Excellent organizational and time management skills, with ability to multi-task. Ability to work with minimal or no supervision while performing duties; has the ability and initiative to organize various functions necessary to accomplish department activities or goals and be a strong team player.

  • Additional knowledge of the following would be a plus:


  • Data mining and analysis using package such as R, Spark, Tableau, Qlik, SpotFire, Splunk.

  • Administration and deployment via containers such as Docker

  • Continuous Integration and Continuous Deployment frameworks, practices, and tools such as Jenkins, Maven, etc.


职能类别: 软件工程师

微信分享

联系方式

上班地址:高新区天府软件园C11 六楼

Get email alerts for the latest"Big Data Engineer 大数据开发工程师 jobs in Chengdu"