Otherwise known as the Redial final project (and possibly the Mobile Apps final project)....

For a while, I have been interested in creating something where a mobile phone could control visual information in the real world. I worked on a project last semester using Max/MSP that sort of addressed these interests using the mysql object and live user-submitted photographs (although the actual performance of the project was compromised thanks to Tonic's unreliable internet connection- but that’s a different story). Upon taking the Redial class, I obviously became very interested in how the human voice could become a control mechanism and renewed my previous interest in speech recognition.

So far my final project in Redial I came up with an idea to combine all of this into one. I will be working on the project with Nanna Halinen and a few others (no firm commitments yet), and a special thanks has to go out to Jadie and Ben, who recently made me rethink the complexity of the idea (it’s always great to know people smarter then yourself..hehe). I was initially inspired by the concept that as we have become more inclined to communicate through digital means (email, text messaging, myspace and the like, etc.) we have perhaps become less likely to communicate through traditional means, such as face to face or through voice on the phone. Not that I believe that email or text messaging is a bad way to communicate, simply that it is interesting to see how information is translated differently in that format, specifically information that deals with memory and emotions. I touched on similar concepts in the first semester of ITP when I worked on the SMS Case project with Nanna and Alex Bisceglie. However, our goals were never fully realized due to a fried Bluetooth module (no comments on this please) and the perceived limitations of J2ME. I also think in retrospect that we were making the concept to complex. So here is my new thought:

There has been a lot of work and research done surrounding speech recognition and analysis in an attempt to glean some kind of emotional context automatically through analyzing pitch, tone, intonation and the actual spoken words. I have also seen a number of projects that allow bloggers to use voice to make comments on photographs they have taken (which is particularly interesting because, especially if you are in a rush or not the best text messager in the world, this seems to be a much easier way to describe an event you have witnessed or contextualize a memory and have it immediately posted). For this project, I want to create a interactive screen display of user-generated photographs that are location based, perhaps visualized on an actual map, perhaps not. Viewers would be able to submit photographs to the screen via an MMS message. Then they could call in and leave a comment about the photograph (either a description of the event they attended, a memory of that location, the reason they took the photograph..etc,). Each photograph will have a unique ID displayed on the screen. Other viewers can then call in, find out where the photo was taken, hear the recorded voice comment and leave a comment/memory of their own. Each comment will be put through some kind of speech analysis to determine its emotional context. That emotion will then be tagged to that location in the database.

Viewers can then download an application (from the web or from a Bluetooth location, i.e. my computer) that will assign a specific ring tone and visual graphic to a set of common emotional responses on their mobile phone. (the ring tones and graphics will initially be pre-determined but can be changed at any time through the application itself or on the website) Then when the person is out in Manhattan, they can send a text message with their location and receive a phone call that will alert them to the emotional context of the location they are in by playing a ring tone and displaying a graphic on their phone. If they choose to answer the call, they can listen to the comments left about the location, leave their own comments and choose to have the most recent photographs viewable on their phone. In addition, if they are in an 'unhappy' location, for instance, they can choose to 'find' a 'happier' location, and the application will alert them as to which direction to travel to find a specific emotion.

Much of this idea still needs to flushed out, but this is the general concept. More info will be available as we work on it.