Video Sensing Final Project
If you refer to past posts, my video sensing final project was to create a midlet for mobile phones in J2ME that allows users to upload video to a website without sending an mms message. See my presentation page
If you refer to past posts, my video sensing final project was to create a midlet for mobile phones in J2ME that allows users to upload video to a website without sending an mms message. See my presentation page
In the spirit of my research on cell phone video applications and the current project I am working on for dynamic web and producing participatory media, I plan to create in J2ME an app that allows video and photos to be automatically uploaded with privacy settings, tags, comments and forum tags to the website that I am working on free4md.com. I wll post image and code examples soon
For today's presentation, I did some research on some of the available social applications for cell phones that attempt to contexualize cell phone content and allow you to share it with others and publish it immediately online. My research is also directly related to a project I am currently working on for DWD/PPM that invloves creating a java application for your cell phone that automatically uploads content and creating a website that stores the content and allows you to create video forums to discuss and view topics of interest. The forums will also be available through rss.
I plan to relate this to this class by attempting to expand the java application to automatically record video when the phone senses sound over a certain threshold. My research:
Continue reading "Video Sensing Tech Demo/Presentation (DWD/PPM Final Project)" »
After having some issues and with some help, I finally managed to get my java application to send captured images to a perl script that automatically uploads them to my mobile video blog. The next step is to get the application to capture images when it finds skin and then to capture short segments of video. Once I get this working, my hope is to get it working on a cell phone so that the cell phone, in effect, becomes the web camera. I plan to research cell phone applications for the student tech demos and maybe get something simple working on the phone itself.
I am very interested in the idea of using low quality video creation tools, such as cell phones and web cameras, to produce online media content. I am also interested in playing around with the idea of surveliance and photography. Last semester my final project was an interactive photograph. When you touched the photographic projection in a certain places the image would change and in one particular place the image would become a snapshot of the viewer standing in front of the web camera. The snapshots, when created, were saved to the hardrive of my computer, but I had ultimately wanted them to be displayed online in some form so that the display itself became an online camera. Unfortuately, I did not have the knowledge then to make this work. For my midterm project, I would like to expand upon this idea, looking ultimately to recording small segments of video from the web camera rather then just jpeg images. I am going to modify the BestPixelTracker class so that it recognizes a specific color automatically, probably something easy like red or orange, When a color like this is in front of the camera a snapshot of the viewer will be taken and automatically uloaded to my server using the code from webCam. Then the picture will be emailed to a specific email address using a perl script where another perl script that is set to automatically check the email every minute will post the image to a blog.
Thanks to Shawn, I already have part of this working. He gave me a script that checks an email address and posts video, sound or photographs to a blog using xmlrpc. I created an email address for my dreamhost account and modified the script to work with this account and my blog This is actually supposed to be used to send video from a camera phone. It works pretty well right now. You can view some tests at http://www.catmindeye.com/video or post to the blog by sending a picture or video to blogpost@catmindeye.com. The part that I am having trouble with is the script that takes the image from the server and emails it to this account. I have looked into the perl sendmail stuff but have not found anything yet that I can understand which sends mail with an attachment. I am thinking that I may modify the script I already have so that it checks a folder on the server instead of an email address but I'm not sure if that is going to work yet.
Here is a link to the script Download file
My first step to complete this week's assignment was to modify the BestPixelTracker class so that when the user pressed the mouse click to select a particular pixel color to track, a jpg image would be created and FTPed to the server. Since I am currently learning perl, I figured that I could create a perl script that would receive the image and print it to a webpage. This seemed to be easier for proof of concept then trying to figure out how to email the image to flickr and have it post directly to the blog. Unfortunately, I ran into some problems getting the image to post to the server. This was the error I received:
IOException: Server returned HTTP response code: 500 for URL: http://itp.nyu.edu/~clc219/cgi-bin/UpLoad.php
Here is the BestPixelTracker modified:
Yesterday, with some help, I managed to connect my Flickr account to my blog. The picture below was sent via email to a specific email address and then posted automatically by flickr to the blog. It has been suggested to me that if I look at the java email api I can find a way to grab images from my web camera and email them to a specific email address. Unfortunately, in conjunction with trying to figure this out I am having extensive problems with WinVdig. It was working perfectly until today. Dan has suggested several remedies, among the first being to use an older version of vdig (vdig 99- available on the VXP wiki), which I am going to try first in order to avoid using JMF and setting up preferences. My next attempted remedy will be to use http://abstractplane.com.au we'll see what works. Ultimately, I am much more interested in being able to transmit some kind of video to my blog. I am currently doing some research into how this would be possible.
This week I began thinking about an idea that I would like to attempt, not only for this week's assignment, but for the midterm as well. I am very interested in using web cameras and cell phones with video capture capability to transmit video and still images via MMS and email to my blog. For this week's assignment, I would like to attempt to show a proof of concept by altering the RectangleTracker class so that when the camera finds a particular shape it transmits a still image via FTP to my server. Then, I would like to transmit the image to my blog via email through an automatic email-to- blog-post option on Flickr. I am currently working with others on figuring out the email-to-blog aspect of this, as well as altering the RectangleTracker example shown in class.
I am still unsure as to exactly what will be the ultimate result of this project. But I am very interested in the idea of being able to have video and images automatically published by anyone at anytime by using their cell phone. I am also interested in the idea of surveliance footage being published in the same way when the camera finds particular things of interest. See future blog posts for more details.
My first problem with this assignment was I didn't realize how difficult it was going to be to figure out how to play a quicktime movie. After some suggestions from others, I checked out the QTJava api and did a google search to find sample code. I found some stuff:
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.io.File;
import quicktime.*;
import quicktime.app.*;
import quicktime.app.players.*;
import quicktime.app.display.*;
import quicktime.io.*;
import quicktime.std.*;
import quicktime.std.movies.*;
public class Window extends Frame{
Movie movie;
public Window (String title) {
super (title);
try {
QTSession.open();
FileDialog fd = new FileDialog (this,"comlab.mov",FileDialog.LOAD);
fd.show();
if (fd.getFile() == null)
return;
// get movie from file
File f = new File (fd.getDirectory(), fd.getFile());
OpenMovieFile omFile =
OpenMovieFile.asRead (new QTFile (f));
movie = Movie.fromFile (omFile);
MoviePlayer player = new MoviePlayer (movie);
QTCanvas canvas = new QTCanvas();
canvas.setClient (player, true);
add (canvas);
addWindowListener (new WindowAdapter() {
public void windowClosing (WindowEvent e) {
QTSession.close();
System.exit(0);
}
});
} catch (Exception e) {
e.printStackTrace();
}
}
public static void main (String[] args) {
Window frame = new Window ("Cat's Window");
frame.pack();
frame.setVisible(true);
try {
frame.movie.start();
} catch (Exception e) {
e.printStackTrace();
}
}
and it worked, after a little while, much to my surprise.
Then I created a new class within my project to utilize "FindSkin" because I realized that I could not extend two classes.
import java.awt.event.KeyEvent;
import java.awt.event.MouseEvent;
public class WindowSkin extends PixelOperator {
int[] backgroundPixelArray;
float redLower = .35f;
float redUpper = .55f;
float greenLower = .26f;
float greenUpper = .35f;
WindowSkin() {
super();
setTitle("FindSkin");
}
public static void main(String[] args) {
WindowSkin fs = new WindowSkin();
ws.startVideo();
}
public void performPixelOp(int _x, int _y) {
int[] rgb = ps.getPixel(_x, _y);
float total = rgb[1] + rgb[2] + rgb[3];
float normalizedRed = rgb[1] / total;
float normalizedGreen = rgb[2] / total;
if (normalizedRed < redUpper && normalizedRed > redLower && normalizedGreen < greenUpper && normalizedGreen > greenLower) {
ps.setPixel(_x, _y, 255, 0, 0, 0);
}
}
public void keyPressed(KeyEvent keyEvent) {
String keyString = KeyEvent.getKeyText(keyEvent.getKeyCode()); // change
if (keyString.equals("Up")) { // use arrow keys to adjust a
redUpper = redUpper + .01f;
redLower = redLower + .01f;
} else if (keyString.equals("Down")) {
redUpper = redUpper - .01f;
redLower = redLower - .01f;
} else if (keyString.equals("Right")) { // use arrow keys to adjust a
greenUpper = greenUpper + .01f;
greenLower = greenLower + .01f; // variable
} else if (keyString.equals("Left")) {
greenUpper = greenUpper - .01f;
greenLower = greenLower - .01f;
} else {
super.keyPressed(keyEvent);
}
}
}
The problem came when I tried to combine functionality together. I will post the code I was working with in my next post.
import java.awt.Graphics;
public class Cattimelapse extends WebCam {
long lasttimetaken = 0;
public static void main(String[] args) {
Cattimelapse mytimelapse = new Cattimelapse ();
}
public void newFrame (){
long now = System.currentTimeMillis();
if (now- lasttimetaken >10*1000){
put("grabit");
grabPicture(freezeFrame);
makeJPegFile(freezeFrame, "C:\\",getTimeString()+ "test.jpg", 1.0f);
//postFileToITP("C:\\", "test.jpg", "/home/clc219/public_html/VideoSensing/", "WC" + getTimeString() + ".jpg" );
lasttimetaken = now;
}
super.newFrame();
}
public void paint(Graphics g) {
g.drawImage(liveImage ,0,0,null);
g.drawImage(freezeFrame, kWidth, 0, null);
}
}
For the second video sensing assignment I want to recreate, with more success I hope, my ICM midterm in which I modified a class example that finds skin. The skin pixels remained the actual pixels of the captured video image while all the pixels that were not skin became the pixels of a quicktime movie. I looked all over the java api trying to find the documentaion for playing a quicktime movie with little success. I am finding it very difficult to navigate through the documentation to find what I want, which makes me miss processing very much. But after a google search, I think I have found something that I might be able to use. This will hopefully be the first step in using video as a sensor to change imagery on the screen. I'm not sure how far I will get this week but I am going to try to make something interesting.
After reading Mirror Neurons by V.S. Ramachandran i have a few things to point out about his conclusions. While I feel that he is correct in stating that the discovery of mirror neurons in the frontal lobes of monkeys is a very relevant and interesting discovery and that it does have ramifications in terms of thinking about human brain evolution, I feel that in his rush to establish their importance in explaining the "big bang" of the human cultural revolution he has ignored several important points. Firstly, I think he is correct in stating that environmental factors played a huge part in the explosion of technological change that brought about the beginning of language, tool use, art and music. However, I feel that, while he does state that mirror neurons and the ability to learn, absorb and imitate things more quickly cannot fully explain the beginning of this phenomenon, he is remiss in stating that cultural aspects of communication such as art, music or dance do not have evolutionary benefits to the species. It is quite a well accepted fact in athropological circles that the human capacity to express and share ideas through artistic means is beneficial to the ultimate survial of any given group of people. If you study the earliest forms of cave art in African tribal peoples, they are mostly about hunting, gathering and collecting food and the general physical survival of the tribe. Music and dance can also be seen as ways to communicate apects of emotional survival, as in their earliest forms they were used as a way to connect and have trancendental experiences with the dead. It is therefor likely that these forms of communication were as important if not more so to earlier cultures then language itself as they provided a means to communicate ideas with large groups of people.
I understand that this was not really his main point and that he was mostly stating that the rise of such cultural advances were transmitted so quickly because of these mirror neurons. I'm sure that this is partly true. However, in discussing this point he completely ignores the topic of how any particular advancement actually began and why other animals such as Chimpanzees who obviously have the same brain capacity do not exhibit the same cultural advances in the same explosive way as humans.
He goes further to state that the industrial revolution of the 19th century in comparison to the development of uniquely human traits 40,000 years ago was much larger and would appear if studied side by side to be of much greater evolutionary importance. On a certain level I agree, but I also think that while the advancements made in recent history are truly amazing, they cannot possibly be equated to the beginning of human cultural intelligence simply because one would not have occured without the major changes in human society of the other. I agree that the industrial revolution proves that certain environmental factors precipitate such an explosion but I also think it proves that there are evolutionary benefits and immediate needs for such advancement to occur and be transmitted throughout a population rather then it simply occuring because of a particular ability of the brain to immitate seen behavior.
In conclusion, I think that his discussion of the actual outcomes of the studies of this particular neuron behavior are the most interesting fact stated in this essay rather then his attempted correlation between such a discovery and the evolution of human culture. I definitely think that it plays a part but I am not convinced that its as relevant in that aspect as he tries to make it seem.
So there were many problems with my idea. I had no trouble downloading the vxp.jar or understanding the basic functions of the WebCam application. With Dan's help, I created a new class called Cattimelapse that would extend WebCam and change the method newFrame so that the application, instead of capturing an image with a key press, iwould capture an image based on the current time, At first it seemed to be working but then something happened. The first problem was that I stopped getting live video in the left side of the window. I solved this, after some help, by removing the following statements from my if statement:
ps.grabFrame();
liveImage = ps.getImage();
repaint();
Unfortunately, this did not solve the problem that the application was only capturing a JPEG freezeframe when the application first began and then not doing so ever again. I thought that the problem lay in the variable lasttimetaken
public void newFrame (){
long now = System.currentTimeMillis();
if (now- lasttimetaken >1000*60*60){
lasttimetaken = now;
grabPicture(freezeFrame);
makeJPegFile(freezeFrame, "C:\\",getTimeString()+ "test.jpg", 1.0f);
//postFileToITP("C:\\", "test.jpg", "/home/clc219/public_html/VideoSensing/", "WC" + getTimeString() + ".jpg" );
}
ps.grabFrame();
liveImage = ps.getImage();
repaint();
}
It seems that with this method lasttimetaken would always be equal to the current time. It was then suggested to me that I needed a listener that would inform me when the system current time had changed. I tried to implement this but I couldn't figure out how to create the method. I am now hopelessly stuck and confused as to how it ever worked in the first place.
When initially thinking about constructing a web camera that records an architectural space, I began thinking about issues surrounding surveillance. One of the things that has struck me as being very odd about the concept of the public being watched all the time through video cameras, other then the obvious violation of privacy, is who actually sits there and watches these things and how boring a job it must be in most circumstances. However, while I cannot imagine anything more frightening then the knowledge that someone in a position of power is watchng and recording my every move, I have recently realized that there are circumstances when survelliance is not only necesssary but actually somewhat interesting. The school where my daughter goes, she's three so in all seriousness I'm speaking of day care, has just announced that they are planning on installing what is effectively web cameras in all of their classrooms so parents can watch their children online every minute they are in school. I have heard of this being done in other schools as well. Now I am by no means the type of parent who would monitor my child's every move online durring the day, such a task would be tedious and obsessive quite frankly. But it did get me thinking as to what kind of surveillance might be important or interesting to me and how it could be implemented so it is more then simply a continous feed of video.
In thinking about these concepts, my initial idea was to create a 24 hour web camera for surveliance of my daugher's bedroom. I have a tendency not to hear my child wake up these days, and she has a tendency to surprise me either by waking up at 3am to watch Dora the Explorer on DVD or by performing an "organizing" exercise of all her worldly possessions at 6am while I am about an hour and a half away from being awake. I thought it would be interesting to capture the daily and nightly state of her room that would not only provide me with a clue of what is actually going on in there but would also be visually interesting.
In order to accomplish this the camera would run continuosly with live video feed into the left window of the java application. Then the application would freeze frame an image every five minutes or every half hour into the right window of the application. Ideally the images would be uploaded to either my or the stage server and be displayed sequencially on a web page. Unfortunately, I am only currently learning perl and database stuff and have no knowledge of PHP so I planned only to save the images to my C drive and then organize them in a kind of slide show that would represent a day and night in the life of my daughter's room.