The past few days, I have been playing around with a good motion segmentation scheme so that I can break down user motions into discrete, atomic "actions". The plan for these actions is that they will form the "alphabet" of my motion gesture language. Combination of actions or individual actions in the correct context and mode (remember from my previous posts?) will lead to interactions with our onscreen interface.
I used the concept of zero crossing to segment motion, thus motion is broken into actions by a combination of velocity and acceleration. The basic principle is that a abrupt change in velocity (high acceleration) signals the end of a previous action and the start of a new action.
Thus I used a finite state machine to represent the breakdown of these motions.
Then after each motion has been completed, I save the many attributes of that specific motion and put in my queue of previous motions that will then be interpreted.
Here is what I store (so far).
Finally, here is an trial of my motion segmentation. On the bottom left I keep track of the various states that the motions go through, and everytime a motion is completed it is pushed onto the queue and the total number of motions increase. As you see, its fairly accurate but fails to pick up very rapid minute motions, but in the grander scheme of things I think it is good enough for our purposes.
Hi Joe, The cool thing is that motion segmentation and motion recognition are entirely separate. So when I segment motions, I just keep track of everything of that motion (avg vel, max vel, avg accel, max accel, direciton, duraction....etc) Thus we will be able to compare motions easily. Furthermore, I made it so that we can have many motion recognizer finite state machines running at the same time, and each one can have different parameters for acceleration/velocity sensitivity and duration, thus we can filter the different motions this way.
Probably an ongoing task will be to figure out the best constants/thresholds to use in the segmentation engine.
I like how a finite state machine can be used to applied to process these motions.
I'm also curious to see how well the Kinect is at distinguishing and detecting small accelerations. I'd assume better than the Wii since it has absolute, not relational, data.
Also, I want to point out that somehow you're stuck tweaking constants again :)
David this looks nice so far.
ReplyDeleteHow are you comparing if two motions are similar?
I guess my real question is the problem - can you detect subtle acceleration changes or classifing them?
ReplyDeleteHi Joe,
ReplyDeleteThe cool thing is that motion segmentation and motion recognition are entirely separate. So when I segment motions, I just keep track of everything of that motion (avg vel, max vel, avg accel, max accel, direciton, duraction....etc) Thus we will be able to compare motions easily. Furthermore, I made it so that we can have many motion recognizer finite state machines running at the same time, and each one can have different parameters for acceleration/velocity sensitivity and duration, thus we can filter the different motions this way.
Probably an ongoing task will be to figure out the best constants/thresholds to use in the segmentation engine.
-David
I like how a finite state machine can be used to applied to process these motions.
ReplyDeleteI'm also curious to see how well the Kinect is at distinguishing and detecting small accelerations. I'd assume better than the Wii since it has absolute, not relational, data.
Also, I want to point out that somehow you're stuck tweaking constants again :)