The Chinese University of Hong Kong’s (CUHK) Faculty of Engineering and Tencent Technology (Shenzhen) Co. Ltd. (Tencent) signed a Memorandum of Understanding (MOU) to step up efforts to cultivate technology talents and strengthen the University’s links with corporations.

The MOU marks a collaboration between the academic and business sectors to foster thought leadership and nurture future generations of innovation and technology leaders. The memorandum marked the implementation of jointly establishing internship opportunities and the exploration of a framework for cooperation in the field of recruitment activities, so as to facilitate further scientific research collaboration.

Under the MOU, Tencent offers internship and talent programmes that give CUHK Engineering students the chance to gain first-hand experience of business and industry knowledge. As part of this strengthened cooperation between academia and industry, students from the fields of computer technology, software engineering, artificial intelligence, cyberspace security and other related majors are the potential participants.

The MOU was signed by Professor Tsang Hon-ki, Dean of CUHK’s Faculty of Engineering, and Mr. Kenny Luo Haibo, Tencent Global Head of Recruitment and The Human Resources Director of Tencent’s Technology and Engineering Group.

Professor Tsang Hon-ki remarked, “We are delighted and honoured to collaborate with Tencent to nurture future R&D talents. The Faculty of Engineering has always been committed to R&D development and talent cultivation. We have always provided internship opportunities for our students, enabling them to solve practical challenges in R&D by applying the knowledge and skills they have learnt in class. This strategic partnership with Tencent, a leading internet and technology company in Mainland China, marks another new milestone for the Faculty of Engineering. This elevates our efforts to nurture more R&D talents for Hong Kong and the broader Greater Bay Area.”

Mr. Kenny Luo Haibo added, “Tencent has been committed to recruiting and nurturing talents in tertiary institutions, especially on the introduction and development of scientific and technological talents. We treasure the opportunity to learn and grow with these outstanding young talents at The Chinese University of Hong Kong. This collaboration will provide students with a rich and diversified corporate internship and practical opportunities as well as hands-on problem solving of the cutting-edge technological topics. Through this collaboration, we look forward to building a broader platform for talents acquisition and promoting the development of scientific and technological innovation in the Greater Bay Area for the fulfilment of our mission.”

A multidisciplinary research team from The Chinese University of Hong Kong (CUHK)’s Faculty of Engineering and Faculty of Medicine (CU Medicine) has developed new artificial intelligence (AI)-powered surgical robot automation techniques, successfully completing the world’s first multi-task surgical automation tests on a live animal. The research has been published in the prestigious multidisciplinary research journal Science Robotics.

Embodied intelligence technology leads to breakthroughs in surgical robot automation

Surgical robots have performed millions of minimally invasive procedures worldwide. Autonomy is envisaged for next-generation surgical robots, enhancing operational efficiency and consistency, while alleviating pressure on medical resources.

Professor Dou Qi, Assistant Professor from CUHK’s Department of Computer Science and Engineering, who led the study, said: “Traditional surgical automation approaches often relied on additional sensors or predefined models, which limited their clinical applicability. We used innovative AI techniques to create a brand-new embodied intelligence framework for surgical robot automation, contributing a data-driven and purely vision-based solution that is the first of its kind globally.”

This surgical embodied intelligence framework can analyse endoscopic images in real time, without additional sensors. The framework integrates advanced visual foundation models, reinforcement learning and visual servoing techniques to achieve accurate, efficient and safe automation of various surgical tasks. Its foundation-model-based visual perception allows it to robustly perform surgical scene understanding and depth estimation in practice. The reinforcement learning-based control policy was trained using SurRoL, an embodied AI simulator that the team developed, and the simulation-trained policy can be directly deployed in real-world robots via zero-shot sim-to-real transfer. In this research, the developed AI system has been seamlessly integrated into the Sentire® Surgical System which has distinctive AI-readiness and AI-friendly characteristics. This data-driven paradigm eliminates task-specific engineering, providing a general-purpose solution for versatile surgical autonomy through embodied AI, accelerating the translation from concept to pre-clinical testing.

In vivo testing validates AI-powered multi-task autonomy and human-robot collaboration

The research team conducted in vivo testing of the AI system using a live animal model that replicated clinical surgical conditions. The system successfully performed multiple autonomous surgical tasks, including tissue retraction, gauze picking and blood vessel clipping – actions that surgeons regularly perform during operations.

Dr Yip Hon-chi, Assistant Professor from Department of Surgery at CU Medicine, who led the animal testing, said: “This represents a breakthrough in AI-powered surgical robot automation, validated across diverse tasks and environmental conditions. Our system demonstrates remarkable generalisability, maintaining stable performance despite environmental changes such as different tissue appearances and varying lighting conditions.”

The technology has the potential to enable the automated robotic arm to function as a surgeon’s third hand, providing assistance during complex procedures. By automating routine tasks with an AI assistant, the system can potentially significantly reduce surgeon workload, improve overall surgical efficiency and shorten procedure time for patients.

InnoHK Multi-Scale Medical Robotics Center (MRC) provides an international platform for high-impact research

The InnoHK Multi-Scale Medical Robotics Center (MRC) played a pivotal role in this groundbreaking research. SurRoL was developed through a strategic collaboration between CUHK and Johns Hopkins University (JHU) in the United States, fostered by the MRC’s international network. The research team open-sourced the surgical embodied AI software infrastructure to the global surgical robotics research community in 2021, and it has since been adopted by numerous prestigious research institutions worldwide.

Professor Samuel Au Kwok-wai, Co-director of MRC and Professor from CUHK’s Department of Mechanical and Automation Engineering, said: “This work exemplifies the exceptional innovations that can emerge from international collaborations cultivated by the MRC. The research has achieved pioneering advancements in AI-powered surgical robot automation,”

The live animal experiments were conducted in the MRC’s hybrid operating room, which provided professional support for pre-clinical evaluation. This environment allowed the surgeon to rigorously test the newly developed AI algorithms under conditions that closely resemble actual surgical settings. Professor Philip Chiu Wai-yan, Co-director of MRC and Dean of CU Medicine, said: “The MRC creates a unique synergy of engineering innovation and surgical expertise, significantly accelerating the journey from laboratory concepts to pre-clinical studies. This engineer-clinician collaborative research showcases the transformative potential of AI co-pilots in robotic surgery, positioning CUHK at the forefront of the global advancement of surgeon-AI-robot partnerships.”

The work was supported by the InnoHK initiative of the Hong Kong government’s Innovation and Technology Commission, the Hong Kong Research Grants Council, and the National Natural Science Foundation of China.

Video: In vivo testing
Video: Presentation and demonstration

Source: https://www.cpr.cuhk.edu.hk/en/press/cuhks-newly-created-embodied-intelligence-platform-successfully-completes-the-worlds-first-multi-task-surgical-automation-tests-on-a-live-animal/