In recent years, thanks to advances in the immense processing capacity and parallelism of modern graphics processing units (GPUs), deep learning based on convolutional neural networks (CNN) has developed rapidly, leading to effective solutions for a variety of problems in artificial intelligence applications. However, the huge amounts of data involved in vision processing limit the application of CNNs to that portable, energy-efficient and computationally efficient hardware for processing the data on site.
Numerous studies have been conducted in the field of optical computing to overcome the challenges of electrical neural networks. Optical computing has many interesting advantages, such as optical parallelism, which can greatly improve processing speed, and optical passivity can reduce energy costs and minimize latency. Optical Neural Networks (ONNs) provide a way to increase processing speed and overcome electrical drive bandwidth bottlenecks. However, ONNs require a coherent laser as a light source for calculation and can hardly be combined with a mature machine vision system in natural light scenes. Hence, optoelectronic hybrid neural networks have been proposed, where the front-end is optical and the back-end is electrical. These goal-based systems increase the difficulty of use in edge devices, such as autonomous vehicles.
In a new article published in Light science and application, a team of researchers, led by Professor Hongwei Chen of the Beijing National Research Center for Information Science and Technology (BNRist), Department of Electronic Engineering, Tsinghua University, China, developed an optoelectronic neural network architecture (LOEN) lensless for machine vision activities that use a passive mask inserted into the imaging light path to perform convolution operations in the optical field and have faced the challenge of processing inconsistent and broadband light signals in natural scenes. In addition, the optical link, image signal processing and back-end network are seamlessly combined to achieve joint optimization for specific tasks in order to reduce computational effort and power consumption across the entire pipeline. .
Compared to the hardware architecture in conventional computer vision, this paper proposes an optical mask that closes to the imaging sensor to replace the lenses. According to the theory of geometric optics according to which light propagates in a straight line, scenes can be considered as sets of point light sources and the optical signal is spatially modulated by the mask to perform the convolution operation of displacement and superposition on the sensor of image. It has been verified that optical masks can replace convolutional layers of neural networks for the extraction of features in the optical domain.
For object classification tasks such as recognizing handwritten digits, a lightweight, real-time recognition network is created to verify the performance of optical convolution in the architecture. Using a single convolution kernel, recognition accuracy can reach 93.47%. When the multichannel convolution operation is implemented by arranging multiple kernels in parallel on the mask, the classification accuracy can be improved up to 97.21%. Compared with traditional computer vision links, it can save about 50% of power consumption.
Furthermore, by expanding the size of the optical mask, the image is channeled into the optical domain and the sensor acquires an alias image unrecognizable to the human eye, which can naturally encrypt private information without computational consumption. The performance of optical encryption has been verified in the facial recognition business. Compared to the random MLS pattern, the accuracy of mask recognition jointly optimized by an end-to-end network has been improved by more than 6%. At the same time as the encryption of privacy protection, it basically achieved the same accuracy performance as method recognition without encryption.
This work proposes an extremely simplified system for computer vision tasks, which not only performs the computation of the optoelectronic neural network in natural scenes, but also opens the entire optoelectronic link to complete joint optimization to obtain the best results for a task of specific vision. In combination with the non-linear materials, the natural light neural network will be realized. The new architecture will have numerous potential applications in many real-world scenarios, such as autonomous driving, smart homes and smart security.
Light science and applications
Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of press releases published on EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.