Can Artificial Neural Networks be Trained to Judge the Subjective Quality of Images?

■ Engineering & Technology /

Can Artificial Neural Networks be Trained to Judge the Subjective Quality of Images?

Researchers present an overview of the current challenges of automatic image quality assessment and how artificial neural networks could be the key to success.

Deep convolutional neural networks can be trained to recognize images easily; however, training them to assess the quality of pictures and images poses a much harder challenge.

Digital images and pictures represent an enormous portion of the total visual media consumed every year, always increasing as digital camera technology, video streaming services, and social media applications continue to grow and expand. Pictures, however, sometimes become distorted from the moment they are captured to the time in which users see them, and much work is being carried out on automatic algorithms or schemes to allow computers to assess the subjective quality of images without human intervention. Such programs can be used, for example, to ensure the quality of the images that the end-users see, which would certainly improve the overall user experience.

Therefore, a team of researchers from Yonsei University led by Prof. Sanghoon Lee have written an overview of the main challenges of making image quality assessment (IQA) algorithms and how researchers have dealt with them. “While many successful image quality assessment models have been devised, the problem is hardly solved, and there remains significant scope for improvement,” explains Prof. Lee.

Deep convolutional neural networks (CNNs) are digital systems that loosely resemble the structure of biological neuron networks and can be trained with an initial dataset in order to make them learn to perform a certain task automatically. For example, deep CNNs have been used successfully for image recognition and computer vision tasks in the past. Unfortunately, training such machine learning systems for IQA tasks has been challenging. As pointed out by the team, this is mainly due to the lack of sufficient training data, which in turn is caused by the inherent difficulty of making databases with information on image quality. Prof. Lee explains: “Creating databases for image quality assessment requires time-consuming and expensive subjective studies, which must be conducted under controlled laboratory conditions.”

In their overview, the team explains the different strategies that have been made using a variety of CNN structures and other machine-learning techniques, along with their strengths and weaknesses. CNN is an attractive option because the optimal features of the input images that the algorithm should look into are automatically learned by the system, as well as any regression steps that would improve the outcome. In other words, in the deeper layers of the CNN, the system learns to detect features that contain abstract information that captures relationships between image distortions and human perceptions of them.

Additionally, the team goes over various existing image databases specifically made for IQA and explains their limitations. They underline the need for larger sets with images with multiple authentic distortions instead of images with synthetic distortions. Finally, the team compares the performance of the abovementioned strategies for each of the image databases and analyzes their results.

This comprehensive summary and comparison of the existing CNN methods and databases for IQA is an excellent read for understanding the current challenges that are being faced in this field. Hopefully, the information compiled and the team’s insight on this problem will aid researchers to create deep CNN methods to ultimately enhance the experience of end-users in the future.

Updated in July 2018

Find Out More

Title of original article: : Deep Convolutional Neural Models for Picture-Quality Prediction
Journal: IEEE Signal Processing Magazine
DOI: 10.1109/MSP.2017.2736018
Contact corresponding author: Prof. Sanghoon Lee ( )