Optoelectronic device mimics human vision for diversified in-sensor computing

To make sense of the world, most humans rely in great part on their vision. Recent research suggests that the human visual system is hierarchical, meaning that it processes information on different levels, ranging from the low-level processing of sensory stimuli to the high-level processing associated with more advanced cognitive abilities.

Computer scientists have recently been trying to develop systems that mimic the hierarchical structure of the human visual system to tackle different levels of information processing effectively. One proposed approach to achieve this is in-sensor computing, which entails the integration of sensing, memory and processing functions into a single device.

Researchers at Tsinghua University recently developed a new promising optoelectronic device for diversified in-sensor computing. This device, introduced in a paper published in Nature Nanotechnology, is based on a fully integrated array of optoelectronic memristors (OEMs), hardware components that can both process and store information.

“Large-scale monolithic integration of in-sensor computing based on emerging devices with complementary metal–oxide–semiconductor (CMOS) circuits remains challenging, lacking functional demonstrations at the hardware level,” wrote Heyi Huang, Xiangpeng Liang and their colleagues in their paper.

“We report a fully integrated 1-kb array with 128 × 8 one-transistor one-optoelectronic memristor (OEM) cells and silicon CMOS circuits, which features configurable multi-mode functionality encompassing three different modes of electronic memristor, dynamic OEM and non-volatile OEM (NV-OEM).”

The device presented by the researchers leverages both electricity and light to concurrently process information and store data, which is essential for in-sensor computing applications. The OEMs have a layered structure with various materials (Pd/TiO_x/ZnO/TiN) placed on top of each other.

Notably, the team’s fully integrated OEM array has different configurable modes of operation. These modes allow the system to mimic the human visual system’s hierarchical information processing.

“These modes are configured by modulating the charge density within the oxygen vacancies via synergistic optical and electrical operations, as confirmed by differential phase-contrast scanning transmission electron microscopy,” wrote Huang, Liang and their colleagues.

So far, the researchers have evaluated their OEM-based device in a series of initial experiments, using it to run computer vision algorithms. Their findings were highly promising, as the OEM array enabled good accuracies on all the three visual tasks it was tested on, while also consuming less power.

“Using this OEM system, three visual processing tasks are demonstrated: image sensory pre-processing with a recognition accuracy enhanced from 85.7% to 96.1% by the NV-OEM mode, more advanced object tracking with 96.1% accuracy using both dynamic OEM and NV-OEM modes and human motion recognition with a fully OEM-based in-sensor reservoir computing system achieving 91.2% accuracy,” wrote Huang, Liang and their colleagues.

“A system-level benchmark further shows that it consumes over 20 times less energy than graphics processing units.”

The recent study by this team of researchers introduces a new cost-effective optoelectronic platform that could prove advantageous for the realization of a variety of in-sensor computing applications. As part of their next studies, Huang, Liang and their colleagues could further optimize the performance of their system, for instance by using transparent materials on the OEM’s top electrode to increase their light absorption rate.

© 2024 Science X Network