-
Essay / Interaction metadata from multimodal interactions
With the advent of digital cameras, people are taking more photos than ever [1], as a result, our photo collections have exploded in size. Retrieving the right photography from these enormous collections appears to be an obvious problem. However, it turned out that people lack the motivation to accomplish the arduous task of tagging and indexing these huge photo collections [2, 3, 4]. As Fleck M [5] points out, people do not see the point in annotating and indexing photographs when adding new images to the collection. The real need only surfaces when they already have a huge collection of indexed photos and the task of annotating the entire collection is no longer attractive enough. Much work has been done to make it easier for the user to mark photographs. Some systems make this easier by allowing the user to drag and drop names from a list. Work is underway on using voice input rather than typing content information. Apart from this, a lot of work is being done to make this content scraping work automatic. There are some computer vision algorithms that can attempt to find the content of images and attempt to infer the occasion [6, TBD]. For example, the presence of a bride in a photo could suggest that it is a photo of a wedding. These methods are still unreliable and are very expensive problems to solve. GPS data has also been used to extrapolate context [7, 8], for example a photograph taken next to a tourist landmark could help label the photograph. If the four main values of the structure of a photo are supposed to be what, who, place and emotion [Unpublished thesis!], it can be argued that these methods can best give us what, who and place; they are incapa...... middle of article......7[14] Paulo Barthelmess, Edward Kaiser, David R. McGee, Towards content-aware multimodal tagging of personal photo collections, Proceedings of the 9th International Conference on Multimodal Interfaces, November 12-15, 2007, Nagoya, Aichi, Japan[15] Rodden, Kerry and Kenneth Wood (2003) "How People Manage Their Digital Photographs", CHI 2003, pp. 409-416.[16] Schiano, Dirme .J., Coreen P. Chen, Ellen Isaacs (2002) “How Adolescents Take, View, Share, and Store Photos,” CSCW 2002.[17] Worthington P (2004) Kiosks and printing services for consumer digital photography. Future analysis of the image market[18] Y. Qian and LMG Feijs. Exploring the potentials of combining photo annotation tasks with the fun of instant messaging. In MUM '04: Proceedings of the 3rd International Conference on Mobile and Ubiquitous Multimedia, pages 11–17, New York, NY, USA, 2004. ACM Press.[19]