CUHK develops VLM with spatial intelligence to improve AI robotic manipulation in complex tasks

Date: 
2026-05-01
Thumbnail: 
Body: 

A research team from The Chinese University of Hong Kong (CUHK)’s Faculty of Engineering has developed a Vision-Language Model (VLM) integrated with spatial intelligence. This breakthrough enables robots to comprehend 3D spatial information like humans do, featuring scalability for visuo-tactile fusion[1], allowing them to autonomously perform complex, long-horizon manipulation tasks with various objects and further enhancing AI’s analytical capabilities. The findings have been published in the renowned international journal Science Robotics.

Although current VLMs allow robots to accurately understand human language instructions, they still lack a deep understanding of the 3D spatial relationships among objects, making it difficult to generate accurate plans for long-horizon manipulation tasks. To enhance the spatial understanding of VLMs, the CUHK team proposed a novel method called Retrieval-augmented Manipulation (RAM). This approach allows robots to simultaneously answer two critical questions during the planning process: what action to take at each step and how such actions can be executed feasibly in 3D space.

The team constructed a structured 3D object knowledge base for the robot, cataloguing the 3D geometries, stable placement configurations and graspable affordances of a variety of everyday objects. When generating a manipulation plan, the VLM retrieves relevant geometric and manipulation records from the knowledge base in real time. It evaluates physical feasibility to determine action sequences and intermediate states, while grounding abstract instructions in explicit spatial constraints. This equips the AI robots with the capability to handle long-horizon task manipulation.

The research deeply integrates vision-driven spatial intelligence with the long-horizon task planning capabilities of VLMs. By constructing a structured 3D object knowledge base, the VLM can dynamically retrieve the geometric and manipulation records of objects when planning long-horizon operations. This approach effectively extends the VLM’s language-level understanding and reasoning capability to complex 3D physical manipulation scenarios.

Professor Dou Qi, Associate Professor from CUHK’s Department of Computer Science and Engineering, who led the study, said: “Spatial intelligence is key to unlocking long-horizon manipulation, and visual perception is a crucial pathway to achieving it. Our method marks a breakthrough in bringing spatial understanding together with VLM reasoning.”

Professor Dou added that the proposed robot spatial intelligence technology scales effectively across tasks and platforms. In 14 manipulation tasks requiring spatial perception and covering 31 different objects, RAM enabled robots to accurately follow spatial language instructions, reason about 3D spatial relationships and perform adaptive manipulation conditioned on the scene’s physical context. RAM works seamlessly with leading VLMs and can be readily deployed on general-purpose humanoid robot platforms for fine-grained long-horizon manipulation.

Furthermore, CUHK’s newly developed system features scalability for visuo-tactile fusion, leveraging tactile feedback for more adaptive manipulation. Professor Liu Yun-hui, Choh-Ming Li Professor of Mechanical and Automation Engineering at CUHK, and Director of the Hong Kong Centre for Logistics Robotics (HKCLR), said: “This research demonstrates the potential of AI to advance robot manipulation, with promising applications across scenarios from industrial to household settings, which will ultimately help to improve human life.”

This research was supported by the HKCLR. Founded by CUHK, the centre is driven by a research team comprising professors from CUHK and the University of California, Berkeley. It is funded by the Innovation and Technology Commission of the HKSAR Government under the InnoHK Research and Development Platform. Its mission is to advance robot intelligence across perception, interaction, manipulation and mobility. Working closely with academic and industry partners in Hong Kong, the Greater Bay Area and the Chinese Mainland, the centre helps to translate cutting-edge AI and robotics research into real-world applications.

For the full research, please visit:  https://www.science.org/doi/10.1126/scirobotics.aea2092

[1] Visuo-tactile fusion is the process of combining visual data and tactile data to create a comprehensive understanding of an environment or object, enabling robots to perform complex, contact-rich manipulation tasks with human-like dexterity.

 

More details: CUHK develops VLM with spatial intelligence to improve AI robotic manipulation in complex tasks | CUHK Communications and Public Relations Office

A research team from CUHK’s Faculty of Engineering has developed a VLM integrated with spatial intelligence, allowing robots to autonomously perform complex, long-horizon manipulation tasks with various objects, and enhancing the analytical capabilities of AI.

Professor Liu Yun-hui (3rd left), Professor Dou Qi (2nd right) and research team members.

An illustration of the VLM technology with spatial intelligence.

 

The proposed framework demonstrates scalability across diverse tasks and platforms. It enables the precise execution of spatial instructions and manipulation (1st and 3rd left), facilitates dexterous manipulation on humanoid robot platforms (2nd left), and leverages tactile feedback to achieve adaptive visuo-tactile grasping capabilities (1st right).

 

Filter: Dept: 
Faculty
CSE
MAE
Media Release

CUHK develops molecular coating to enhance lithium metal battery stability A key technological breakthrough for the electric vehicle industry

Date: 
2026-05-03
Thumbnail: 
Body: 

A research team from The Chinese University of Hong Kong (CUHK) has developed a novel interface engineering strategy by designing and assembling a tailored molecular layer on the surface of battery positive electrode materials, successfully modulating the chemical environment at the electrode-electrolyte interface. This strategy significantly enhances the stability and cycling performance of high-voltage lithium metal batteries, offering a new pathway toward safer, more stable, and higher-energy lithium metal batteries for the future. It also helps enhance the safety and driving range of electric vehicles and energy storage devices. The research findings have been published in the leading journal Nature Nanotechnology.

The hidden threat of high-voltage batteries: the interface we cannot see

Lithium metal batteries are widely regarded as a promising technology for next-generation electric vehicles and energy storage systems because of their exceptionally high energy density. Compared to current lithium-ion batteries, they offer the potential for longer driving ranges and lighter battery packs. However, when these batteries operate at high voltages, a hidden challenge emerges: what happens in the thin, invisible region where the electrode meets the electrolyte often determines whether the battery can function reliably. At this interface, electrolyte molecules can undergo oxidative decomposition, and the resulting byproducts gradually accumulate, leading to rapid performance decay and even battery failure. This challenge is difficult to address because it cannot be solved by improving electrode materials alone or by adjusting the electrolyte composition alone.

Smart molecular coating changes the game

The research team, led by Professor Lu Yi-chun of the Department of Mechanical and Automation Engineering at CUHK’s Faculty of Engineering, proposed a novel approach: rather than passively enduring interfacial reactions, they chose to actively reshape the chemical environment at the interface. They assembled an ultrathin yet functionally precise molecular layer on the surface of the battery positive electrode material. This molecular coating changes how electrolyte molecules behave as they approach the surface. Some molecules in this layer are more welcoming and attract electrolyte molecules closer, while others are more reserved and keep them away. By tuning these molecular characteristics, the researchers can regulate the interfacial chemical environment, much like adjusting temperature. Through this delicate balance of attraction and repulsion, the team identified the optimal condition, forming a protective layer that suppresses harmful side reactions without interfering with the normal operation of the battery interface.

In experiments, the modified positive electrode maintained 80% of its initial capacity after 200 cycles under harsh high-voltage and elevated-temperature conditions (60°C), while unmodified electrodes showed much faster performance degradation. This improvement does not come from increasing the complexity of the electrode or electrolyte system but from a precise and controllable chemical modification at the interface. This suggests that the method could be integrated into existing battery manufacturing processes without requiring a complete system overhaul.

Professor Lu said: “This research reveals the molecular-level mechanisms at the electrode-electrolyte interface. We not only provide new scientific insights but also demonstrate a new pathway for interface design. Although the current validation has been carried out in laboratory-scale coin cells, in principle this method should be applicable to larger-scale battery systems. We hope this work will provide scientific guidance for the development of next-generation lithium metal batteries with both high energy density and high stability, accelerating their practical application and driving the electric vehicle and energy storage industries into a new phase of development.”

About Professor Lu Yi-chun

Professor Lu Yi-chun received her PhD in Materials Science and Engineering from the Massachusetts Institute of Technology (MIT) in 2012. She was in the first cohort of recipients of Excellent Young Scientists Fund in 2019 and the Young Scientist Fund (Type A) in 2025 from the National Natural Science Foundation of China. Her goal is to develop a stable, low-cost, scalable, rechargeable energy storage system.

Professor Lu is a Fellow of the Royal Society of Chemistry (RSC) and Associate Editor of Journal of Materials Chemistry A, published by the RSC. She is also a Founding Member of the Young Academy of Science of Hong Kong. She has been conferred various CUHK and international research and teaching awards, including the Battery Division M. Stanley Whittingham Mid-Career Award by the Electrochemical Society, Tajima Prize by the International Society of Electrochemistry; the Xplorer Prize from the Tencent Foundation; the Falling Walls Science Breakthroughs of the Year in Engineering and Technology, for which she was among the top 10 winners; the Early Career Award from the Research Grants Council; and the Research Excellence Award and the Vice-Chancellor’s Exemplary Teaching Award from CUHK.

More details: CUHK develops molecular coating to enhance lithium metal battery stability A key technological breakthrough for the electric vehicle industry | CUHK Communications and Public Relations Office

 

Professor Lu Yi-chun, Department of Mechanical and Automation Engineering, CUHK.

Molecular coating regulates the electrode-electrolyte interface and improves high-voltage lithium metal battery stability.

a. Schematic illustration of the electrical double layer with an ultrathin molecular coating anchored on the NMC811 positive electrode surface. The molecular layer is positioned close to the electrode surface and modulates the local chemical and electrostatic environment where electrolyte molecules approach the high-voltage electrode. Blue and pink circles represent electrolyte anions and cations, respectively, while white circles denote solvent molecules.

b. Long-term cycling performance of lithium metal coin cells at a high cut-off potential of 4.7 V. Compared with the pristine electrode, the molecular-coating-modified electrode shows markedly improved cycling stability and maintains 80% of its initial capacity after 200 cycles, demonstrating the effectiveness of interfacial molecular design in suppressing harmful side reactions.

 

Dr Wang Huwei, first author of the study and Postdoctoral Researcher in the Department of Mechanical and Automation Engineering, CUHK.

 

Filter: Dept: 
Faculty
MAE
Media Release

BME Undergraduate Wins Best Paper Award at ICMVA 2026

Date: 
2026-04-30
Thumbnail: 
Body: 

Mr. Sui Ip Ng, a 3rd year undergrad from the Department of Biomedical Engineering at CUHK, has won the Best Paper Award at the 9th International Conference on Machine Vision and Applications (ICMVA 2026). The conference took place from 24 to 26 April in Nanjing, China. Fewer than 6% of the submissions received this prestigious recognition, demonstrating the exceptional quality of the awarded research.

Award-Winning Research

The award recognises the work of the BME team: undergraduate researcher Mr. Sui Ip Ng (first author), Postdoctoral Fellow Dr. Xin Shu (mentor), and Prof. Renjie Zhou (supervisor), for their paper titled "An FPGA-Based Real-Time Artificial-Intelligence-Enabled Hematology Analyzer with Reagent-Free Imaging."

In this work, Sui Ip demonstrated the deployment of a compact neural network on a low-cost (US$129) FPGA platform for reagent-free white blood cell classification, achieving 94.3% classification accuracy with deterministic real-time inference. By eliminating the need for chemical reagents and leveraging efficient on-device AI acceleration, the system offers a promising pathway toward portable, low-cost, and real-time hematology analyzers for point-of-care diagnostics.

Sui Ip joined Prof. Renjie Zhou's lab as a student helper in March 2025 to gain research experience in his spare time. Under the mentorship of Dr. Shu, Sui Ip carried out all the experiments and data analysis, and wrote the manuscript. His journey exemplifies how early and dedicated involvement in research can empower an undergraduate to make impactful contributions to cutting-edge science.

About the Conference

ICMVA is an annual international forum that brings together researchers, scientists, and engineers working in machine vision, image processing, pattern recognition, artificial intelligence, and related areas. This year’s event attracted participants from across the globe to present cutting-edge research and foster academic exchange in the rapidly evolving area of intelligent vision systems. The conference was chaired by Prof. Wolfgang Osten (University of Stuttgart, Germany), a world-renowned authority in optical metrology and digital holography and the recipient of the 2019 Emmett N. Leith Medal from Optica.


 

Fig.1. Sui Ip Ng with his award certificate. Photo taken in the Laser Metrology and Biomedicine Lab at CUHK.

Fig.2. Dr. Xin Shu (mentor) at the Award Ceremony in Nanjing, China.

 

 

 

 

 

Filter: Dept: 
Faculty
BME
Media Release

中電夥中大研數碼培訓教材 培育電力專才

中華電力與香港中文大學簽署合作備忘錄,以培育工程人才為共同目標,建立策略性合作關係。今次合作將結合中電在電力工程培訓及營運實務上的電能專業,以及中大在學術與創科研究的優勢,透過應用創新科技,共同推動電力工程人才的數碼化培訓發展與學習體驗,為本地電力行業培育專才。

Date: 
Wednesday, April 22, 2026
Media: 
hkej

中電與中大合作培育電力工程人才 攜手研發創新數碼教材和工具

中華電力有限公司與香港中文大學周二(21日)簽署合作備忘錄,建立策略性合作關係。是次合作將結合中華電力在電力工程培訓及營運實務上的電能專業,以及中大在學術與創科研究的優勢,透過應用創新科技,共同推動電力工程人才的數碼化培訓發展與學習體驗,為本地電力行業培育專才。

Date: 
Thursday, April 23, 2026
Media: 
HK01

CUHK and CLP Power sign MoU to advance digital practical training and nurture future power talent

Date: 
2026-04-22
Thumbnail: 
Body: 

LThe Chinese University of Hong Kong (CUHK) and CLP Power Hong Kong Limited (CLP Power) signed a Memorandum of Understanding (MoU) on 21 April to establish a strategic partnership with the shared goal of nurturing engineering talent. By leveraging CUHK’s strengths in academic excellence, innovation and technological research, and CLP Power’s expertise in power engineering training and operational practice, the collaboration will advance digital training development and learning experiences for power talent through the application of innovative technologies, cultivating future professionals for the local power industry.

The MoU was signed by Professor Tsang Hon-ki, Dean of Engineering at CUHK, and Mr Eric Cheung, Chief Operating Officer of CLP Power. Under the MoU framework, the two parties will jointly develop innovative digital training materials and tools to further enhance training effectiveness for CLP Power’s engineering trainees, while also enabling CUHK students to better understand and acquire the latest skills and knowledge in the power industry, furthering the development of power talent.

Professor Tsang Hon-ki said: “This MoU marks an important milestone in the collaboration between CUHK’s Faculty of Engineering and CLP Power, as both parties work together to drive talent development and innovation in Hong Kong’s power and energy sector. We look forward to demonstrating how academia and industry can join forces to nurture engineering talent, promote innovation and deliver meaningful impact for Hong Kong.”

Mr Eric Cheung said: “CLP Power is pleased to join hands with CUHK in this partnership. By combining our respective professional strengths, we look forward to introducing greater application of technology and innovative learning models to further enhance training effectiveness. Our goal is to nurture a new generation of engineering talent with both practical skills and an innovative mindset, thereby strengthening operational safety, supply reliability and environmental performance to meet industry development and society’s needs.”

The power industry places a major emphasis on professional expertise and hands-on operational capability. Tasks such as low-voltage live-line operation and high-voltage switchgear operation involve stringent procedures and a high degree of precision. As such, engineering trainees typically require close, one-to-one supervision by instructors during practical training, which must often be conducted within a defined time frame.

CUHK and CLP Power will integrate innovative technologies into training programmes by jointly developing training systems and tools that incorporate immersive virtual reality (VR) and augmented reality (AR) technologies, to meet the growing demand for flexible, digital learning among new generations. Through technology-enhanced training, trainees can follow structured virtual guidance to master correct operating procedures and practise workflows as required. Instructors can provide more targeted guidance and support based on individual learning progress. This approach enhances interactivity and self-initiated learning, while enabling engineering trainees to gain a more comprehensive understanding of power equipment operation and related knowledge.

CLP Power will continue to improve and systematise its self-initiated learning model to enhance the overall learning experience and training effectiveness, in support of ongoing development of the power grid and evolving operational needs. CUHK will leverage its academic research expertise to provide scholarly advice on the development of digital training materials, supporting the continuous optimisation of engineering training for mutual benefit.

 

More details: https://www.cpr.cuhk.edu.hk/en/press/cuhk-and-clp-power-sign-mou-to-advance-digital-practical-training-and-nurture-future-power-talent/

Professor Tsang Hon-ki (right), Dean of Engineering at CUHK, and Mr Eric Cheung (left), Chief Operating Officer of CLP Power, sign the MoU to jointly advance the digitisation of power engineering training and talent development.

Professor Tsang Hon-ki (front row, right), Dean of Engineering at CUHK, Mr Eric Cheung (front row, left), Chief Operating Officer of CLP Power, and guests mark the establishment of the collaboration framework.

Mr Eric Cheung, Chief Operating Officer of CLP Power, says he looks forward to exploring further opportunities to apply innovative technologies in training through close exchange and collaboration between CLP Power and CUHK.

Professor Calvin Chan Chun-kit, Associate Dean (Education) of the Faculty of Engineering at CUHK delivers a speech, stating that the University would leverage its academic research expertise to provide scholarly advice on developing digital training materials and support the continuous optimisation of engineering training.

Mr Eric Cheung, Chief Operating Officer of CLP Power (right), visits a laboratory in the Department of Mechanical and Automation Engineering at CUHK to learn about its latest research projects.

 

Filter: Dept: 
Faculty
MAE
Media Release

中文大學3學者獲中銀香港科技創新獎 表彰科研成就推動轉化貢獻 原文網址: 中文大學3學者獲中銀香港科技創新獎 表彰科研成就推動轉化貢獻

中文大學三位學者榮膺第四屆「中銀香港科技創新獎」,包括化學病理學講座教授兼化學病理學系系主任陳君賜、機械與自動化工程學系教授盧怡君、以及信息工程系副教授兼交叉學科人工智能研究所所長林達華教授。三人各獲港幣200萬元獎金,以表彰其在尖端科研成就及推動科技成果轉化方面的卓越貢獻。

 

Date: 
Tuesday, April 21, 2026
Media: 
HK01

Two Engineering scholars awarded BOCHK Science and Technology Innovation Prize 2026

Date: 
2026-04-22
Thumbnail: 
Body: 

Two scholars from the Faculty of Engineering of The Chinese University of Hong Kong (CUHK) have been awarded the BOCHK Science and Technology Innovation Prize 2026 (STIP). They are Professor Lu Yi-chun, Professor in the Department of Mechanical and Automation Engineering, and Professor Lin Dahua, Associate Professor in the Department of Information Engineering, Medicine and Therapeutics and Director of CUHK Interdisciplinary Artificial Intelligence Research Institute (IAIRI). Each awardee receives HK$2 million in prize money in recognition of their outstanding contributions to cutting-edge research and technology translation.

 

Professor Lu Yi-chun: Breakthroughs in Aqueous Battery Technology for New Materials and New Energy

Professor Lu Yi-Chun's pioneering contributions in the field of aqueous flow batteries include developing molecular catalysts and charge-reinforced ion-selective membrane, enabling the commercialisation of sulfur-based flow batteries. Her work addresses the bottlenecks in safety and cost for large-scale energy storage, accelerating the global transition to green energy. Professor Lu Yi-Chun is also the founder and Chief Scientist of Luquos Energy.

 

Professor Lin Dahua: Multimodal Intelligence Breakthroughs in Artificial Intelligence and Robotics

Professor Lin Dahua has made systematic innovations in the fields of computer vision and multimodal intelligence, achieving breakthroughs in areas such as image and video understanding, multimodal large-scale models, and spatial intelligence. He has promoted independent innovation in key core technologies and the construction of an open-source ecosystem, facilitating the high-level application of AI research results to industry, and making significant contributions to the advancement of AI technology and industrial upgrading in China.

 

About the BOCHK Science and Technology Innovation Prize

The BOCHK Science and Technology Innovation Prize (STIP), established by the Hong Kong Alliance of Technology and Innovation (HKATI) and sponsored by the Bank of China (Hong Kong) Limited, is a non-governmental, merit-based award. The Prize aims to reward outstanding scientists and research teams who have made significant contributions to scientific and technological innovation, as well as the transformation of scientific and technological achievements in Hong Kong.

 

More details: https://stip.hk/en/

Professor Lu Yi-chun, Professor in the Department of Mechanical and Automation Engineering

Professor Lin Dahua, Associate Professor in the Department of Information Engineering

 

Filter: Dept: 
Faculty
IE
MAE
Media Release

Professor Chen Shih-Chi secures funding for 3D Nano-Fabrication in third round of ITC’s RAISe+ Scheme

Date: 
2026-04-21
Thumbnail: 
Body: 

The project “Ultrafast Multi-material 3D Nano-fabrication Platform Based on Two-photon Projection Technology” led by Professor Chen Shih-Chi from Department of Mechanical and Automation Engineering has secured funding in the third round of the Research, Academic and Industry Sectors One-plus Scheme (RAISe+), initiated by the Innovation and Technology Commission (ITC) of the HKSAR Government.

Advanced manufacturing is driven by engineering innovations that drastically lower production costs and enable the fabrication of structures and devices previously impossible. Currently, the $180 billion precision manufacturing market is stifled by a “speed-precision tradeoff” where nanoscale resolution is too slow and costly for mass production. To address this, Professor Chen Shih-chi and his team at CUHK have developed the Femtosecond Projection NanoPrinter (FP NanoPrinter), a breakthrough that significantly advances the envelope of micro-additive manufacturing.

By exploiting spatiotemporal focusing and digital holography, the project utilises ground-breaking methods to parallelise the 3D printing process, setting new world records for printing rates (400mm³/hour) and resolution (20–100 nm) – resulting in a 95% reduction in manufacturing costs. Unlike conventional platforms limited to single-material plastics, this process enables the high-speed 3D integration of over 20 material classes – including metals, semiconductors, and ceramics.

These capabilities allow for the mass production of functional nanostructures, such as optical metamaterials and flexible electronics, which are critical for the future of electric transportation, healthcare, and telecommunications.

The solution is being commercialised by Astra Optics Limited, a venture-backed CUHK spin-off, now supported by the Raise+ award, focused on high-precision components ranging from AR/VR optics and photonic interconnects for AI data centres to next-generation 3D photonic computing chips.

 

About the RAISe+ Scheme

Launched in October 2023, the RAISe+ Scheme aims to unleash the potential of local universities to transform and commercialise R&D outcomes, and facilitate collaboration among the government, industry, university and research sectors. The RAISe+ Scheme funds, on a matching basis, research teams in eight universities supported by the University Grants Committee that have the potential to become successful start-ups. Up to HK$100 million in funding support is provided to each project.

 

More details: https://www.cpr.cuhk.edu.hk/en/press/cuhk-secures-funding-for-ai-surgical-robot-cancer-nanomedicine-3d-nano-fabrication-in-third-round-of-itcs-raise-scheme/

Professor Chen Shih-Chi from Department of Mechanical and Automation Engineering

 

Filter: Dept: 
Faculty
MAE
Media Release

深圳先进院- 2026 校園招聘宣講會暨PI分享交流

Venue
Room706 William M W Mong Engineering Building (ERB 706)
Date: 
Thursday, April 23, 2026
Time
Thursday, April 23, 2026
e_title: 
深圳先进院- 2026 校園招聘宣講會暨PI分享交流
Not Available
Allow Regsiter: 

Pages