Tracing the colors of clothing in painting with image analysis

Abstract

In this thesis, we propose an automated methodology to study the trend of color preferences for male and female subjects of paintings over several hundred years. This study is part of a larger project on understanding how sex is defined and described in the Western culture, by tracing the transformation of gender representations in culture, literature, and arts from 17th to 20th century. The data are collected using digital images of paintings from Rijksmuseum Amsterdam. We proposed an approach to extract the dominant color in clothes of the sitters of portrait paintings. The resulting application of this study could provide a useful tool for the digital humanities scholars. Artworks used in this study consists of different artifacts and objects. We ran a face detection algorithm on the Rijksmuseum dataset for the portrait painting collection. Following that, the portraits were classified into their perceived sex by an algorithm, trained on photographic images. Three different face image databases were employed and compared to measure the impact of varied training set conditions on perceived sex classification from paintings. To concentrate on the color information of the sitter, clothing segmentation of the sitter is a necessity. Hence, a simple, yet robust algorithm that uses the location of the face as a landmark to identify a region of interest to represent the clothing in portrait paintings is proposed. This region is used to extract the color distribution, and one dominant color. We contrasted four color extraction methods for this purpose. An interactive interface, where the results of the approach can be viewed and analyzed by an individual is designed. It provides a full overview of the color trends on a temporal axis, thus making it possible to study color preferences in different eras, as well as the changes in color connotation. The interface is designed as a visualization tool for curators or researchers and makes it possible to receive feedback.