, , , , , , , , ,

How would you define a promenade in tango? Or an impressionist painting? In these cases like in many artistic domains, concepts are identified by qualitative and rather abstract definitions that leave a significant amount of personal interpretation. Thus it can be quite difficult for different people to discuss about a given concept with different interpretations in mind. The recent emergence of computer sciences has opened new perspectives to define those concepts: through large-scale data analysis and modeling, we are now able to study artistic disciplines in a quantitative way. In this context, we will discuss 3 interesting projects that attempted to use new technologies in order to offer a new insight to human behavior, in very different domains.

The ARTeFACT project aimed to provide a computer tool use natural language processing (NLP) in the universe of dance. It is divided into two main parts. The initial work was the development of methods for computer identification of dace movements in 3D. Therefore, a library of codified dance steps was created (along with relationships between steps). Then, each step of the library was performed by professional dancers and collected by a motion capture system (cameras, reflecting markers), from which several relevant features were extracted, such as foot-ground contact, knee and hip angles. From there, data analysis and classification was performed: each codified step was associated with a certain set of features. Finally, the classification model was tested on another set of codified steps in order to assess its reliability, showing very good results (97.3% correct step classification after model re-adjustment). The second work consisted of developing a parallel library matching physical features with “abstract movements” as they can sometimes be defined in dance: “struggle”, “victory”, “attack”… A similar procedure was implemented, in order to identify movement patterns in a variety of dance works. Although this particular work is still ongoing, it already shows some confident results. Hence, the overall project can be used in order to identify dance steps in a 2D dance film, but also extends to other movement-based disciplines.

The VisualPage project focused on the global understanding of poetry of the Victorian era. It started from the assumption that a poem cannot be summarized to plain text only, and that many other characteristics of the poem’s printed page should be taken in account. These additional features could then be used to identify undiscovered similarities and evolutions in poems of the Victorian culture. Therefore, the project was divided into three tasks. The feature extraction module performs image processing of the poem’s page and extraction of relevant features such as typeface size, margin size, spacing of text lines, before gathering these all features in a library. The pattern recognition module was designed in order to find relationships within the collection of poems in terms of the studied features. Finally, the analysis module provides a data visualization an exploration interface, where new queries can be defined and assessed. Although this project is still at the “proof-of-concept” stage, it could allow identifying significant patterns in the graphical design of Victorian books, as well as the evolution of these features during the Victorian period and their differences across different authors.

The final project consisted of a deep analysis of character networks in a large set of theater plays and movies, in order to discover similarities in literature and movies across genres and over time. Therefore, the first task was to develop methods for automatically extracting character interactions from scripts of movies/plays.  Then these interactions were analyzed and gathered into networks, using four different algorithms, each of them defining character interaction in a special way (number of scenes of common appearance, total number of words exchanged…). Next, several properties were computed for each network, in order to be able to compare them: relative importance of top/main characters, centrality of a main character, strength of relationships, number of storylines and number of characters. These network properties were used to determine characteristics of movies/plays for various media aspects (type, date, rating and critics, genre, author). Finally these assignments were tested on additional data using regression classifiers and decision trees. The results showed significant differences in those networks by comparing plays versus movies (plays usually display a central character having all important relationships, whereas movies use several main characters), different dates (older plays tend to have more disjoint group of characters and more distinct storylines than the new ones) genres (e.g. horror movies with only a few characters and a simple storyline) and authors. Hence, this project developed a classification tool in order to classify movies and plays according to several general characteristics.

As a conclusion, one could say that although the three presented projects operate on very different domains, they all provide a way of defining an abstract concept through concrete aspects. If dance was defined by humans with the help of a lot of metaphoric moves, the ARTeFACT project breaks down this wide artistic discipline into small but well-defined elements. Similarly, if a poem’s layout is more of a visual signal that usually comes along with a text to emphasize its effect, the VisualPage project defines it through quantitative geometrical properties. Finally, if a movie aspect is rather vaguely defined, the character network project refers to quantitative features in order to clearly describe it. This “translation” permits to compare different entities in a stable and repetitive way by referring to the newly identified concrete aspects, and to discover similarities or differences that were unrevealed until then.