Video Analysis / Analytics for Home Automation

[naething]

As the scope of my home automation expands I'm slowly looking for new things to tackle. One thing I've been thinking about for some time is applying computer vision to the webcams I have in and around my house. I'd like to do this for occupancy sensing in particular rooms, detection of mine and visitors cars, and also to identify individuals for personal lighting / music preferences.

Currently I have quite a few webcams running on pogoplugs through 'motion', a linux webcam application. I also have a few commercial ip cams. It is pretty easy to grap jpegs from any of these, and right now I am just planning on feeding these to an OpenCV program on a set period of time (say, every 15 seconds) and then doing something interesting with the results.

There are some trivial things I can tackle (facial / human detection for occupancy sensing) but I am interested in pushing this a bit further. I would like to have it detect the specific person rather than just 'a person', but I haven't found a good open source library that performs this task. I could always write my own Eigenface code (and I think I actually had to once for a class project), but I figure this has to exist somewhere.

I actually would prefer to just tie in to my Picasa install because it already has all the faces I need programmed in and does a great job. Too bad it doesn't seem to have an API that provides this feature!

Another idea I have is detection of license plates. This fortanetly is available in open source:

http://javaanpr.sourceforge.net/

Any ideas? Also, does anyone have any thoughts on the legality of doing this? I don't plan on recording images, just processing them (unless my house is alarmed in which case I'll be transfering images offsite in real time). I haven't run this all by my wife yet so hopefully she doesn't mind too much.... :unsure:

 

[tmbrown97]

Sounds awesome! I wish I had the time and the skillset to do it... selfishly speaking, think you could make this a plugin for Elve? ^_^

 

[BraveSirRobbin]

You have a cameras with a high enough resoultion to perform facial recognition? I would be first worried about that.

 

[naething]

@BraveSirRobin, that is a good point. I have a mix of mostly 640x480 along with a few 720p and 1080p webcams. I'm pretty sure facial detection will work without a problem but having enough resolution for facial recognition might be a problem. I do have a couple of 10MP+ Canon cameras that I run CHDK on. I suppose in combination with an Eye-Fi memory card I could script the capture of some higher resolution images. Quite honestly facial detection (for occupancy) is probably more useful anyway.

@Work2Play, I have no experiance with Eve, sorry! However, if I get anything exciting working I'll probably post the code, and maybe you could tie it into Eve?

 

[Rupp]

Sounds like a boat load of work for what? What are you looking to use this information for?

 

[naething]

@Rupp, actual facial detection (as apposed to recognition) is very little work. All the heavy lifting has been done by OpenCV. In fact, it is about a dozen lines of python. I already have these cameras running (great for checking up on the house when I'm on travel), so it seems like an easy way to improve my occupancy detection. I think (but I'm not sure!) that while full facial recognition might be tough I can at least get the system to differentiate between my wife and I.

I'm hoping this information will allow me to make my house lighting more automatic. As I walk around I can have the lighting adjust without any manual control. Knowing what room I'm in will allow me to have one command that is executed in different ways. For instance, "turn on music" could just turn on music in the room that I'm in. Differentiating between my wife and myself would also allow certain commands to be smarter. For instance, if I turn on the tv it could tune to a different channel by default than if she does.