Intellectual Property Office

Non-Confidential Disclosures

"Automatically Predicting Image Quality and Attractiveness"

PSU Invention Disclosure No. 3210

Field of the Invention:

Image retrieval; photo aesthetics; digital libraries

Links:

http://www.stat.psu.edu/~jiali/
http://wang.ist.psu.edu/

Inventors:

R. Datta, J. Li, J. Wang

Background:

There is a clear need for automatic systems that can help shoot aesthetically pleasing pictures, choose among a set of pictures those that are most visually appealing, and/or help in aesthetically agreeable design of posters and packaging. The major difficulty in addressing this need is that judging beauty and other aesthetic qualities of photographs is a highly subjective task and there are no unanimously agreed upon standards for measuring aesthetic value. It is no surprise then that (to the best of our knowledge) there exists no computational system to automatically predict the visual/aesthetic quality of photographs.

Invention description:

In spite of the ambiguous definition of aesthetics, we have shown that there exist certain visual properties which make photographs, in general, more aesthetically pleasing. We treated the challenge of automatically inferring aesthetic quality of pictures using their visual content as a machine learning problem, with a peer-rated online photo sharing Website as data source. We extracted certain visual features based on the intuition that they can discriminate between aesthetically pleasing and displeasing images. These features include a number relevant to photographic quality such as a low depth-of-field indicator, a colorfulness measure, a shape convexity score and a familiarity measure. Automated classifiers were built using support vector machines and classification trees. Linear regression on polynomial terms of the features was also applied to infer numerical aesthetics ratings which we have shown correlates very well with human ratings. The resulting system has shown a 70-80% agreement with public ratings. Figure 1 shows the accuracy of the system compared to human ratings for 3581 photographs.

The invention can help assist in photography and visual design in an automated manner, reducing the need for service of humans. Potential applications include content-based image retrieval and digital photography. More specifically two possible applications that can result from the invention include: (1) camera subsystems that provide suggestions for quality photos and (2) software that helps select or rank visually appealing pictures.

Contact:

Mr. Bradley A. Swope
Sr. Licensing Officer
Intellectual Property Office
The Pennsylvania State University
113 Technology Center
University Park, PA 16802-7000
Phone: (814) 863-5987
Fax: (814) 865-3591
E-mail: bradswope@psu.edu