TangoChen

A mouse control application for Kinect v2, including a couple of options for various uses.

I’m so glad that I’ve just become a Microsoft Kinect MVP since this July, so I think I need to do something more.

One request asked most since the original Kinect came out is, “can I use my hand to control the mouse cursor?” There you can find a few applications for doing this with original Kinect on the web.

These days, some people still asked me about this kind of application for Kinect v2. Then I strangely found there’s no such downloadable applications on the web. I gotta release a Kinect v2 version app. And here’s it, with source code!

I initially develop this for Francis Kim(He’s a Magento Developer). Thanks Francis for letting me release this freely!

Options:

• Mouse Sensitivity
• Pause-To-Click Time Required

The time you hold your hand for as a click

• Pause Movement Thresold

How large the circle range you hold your hand inside for a little while would be a click

• Cursor Smoothing

The large it is, the smoother the cursor would move, more slowly as well.

• Grip Gesture
  Grip to drag/click
• Pause To Click
  Hold your hand and don’t move for a little while to click.
• No clicks, move cursor only

Github: KinectV2MouseControl
EXE files: [download id=”24″]

One day I felt that using one Kinect on one PC isn’t enough. There could be more interesting things to do with multiple screens.
So I consider sharing the Kinect data with multiple devices. You may know that I created an app called Kv2 Viewer that can access Kinect data on the phone. With the similar technology, I successfully send Kinect data to multiple devices.
Firstly I write a server program to make a Win 8.1 PC as a server. Then all other clients that want the Kinect data would just need to connect to this PC.

Here are the simple demos I made in the video:

Body views

They all get the joint positions, joint states first and generate the body views on screen, not reciving body view images from the server.

Girls staring at target

The girls from different directions would stares at you (According to your head position). And if you act like pointing at something, they would stare at that (According to your hand position.)

The virtual shoe fitting store from Goertz in 2012 is quite impressing. They used 3 original Kinects. I decided to make my virtual shoe fitting application with a Kinect v2 last month. It’s still something rare, though we’ve seen many virtual dressing room demos.

I placed the Kinect low on a box, so it can capture my feet only. In this case, we can see more details of the shoe. But, I’ll need to write my own feet tracking algorithm as the original body tracking won’t work. This is one of the big challenges.

Another challenge is that I need to hide some parts of the shoes that covered by my feet. If you’ve ever tried implementing the virtual dressing room, you’ll also face this problem. Using single-sided materials is one of the solutions. But it won’t work in some cases and it’s flawed. For some reasons, I’d keep my solution private. 😉

Overall, the application doesn’t work very well due to the CPU capability and the current algorithm. But I enjoy the process of making it!

Inspired by the movie Transformers 4, I decided to make a transformers-alike game. Imagine
that I can control the transformer and switch between robot and car, hit the enemies. That would be a lot of fun! So I do this.
I bought the 3D models on the web. It’s not one of the official Transformers though. (I will not be able to buy and use it if it is…)
It’s not so good-looking when looking from the back. Here’s the front looking.

[Click Play button on bottom-left to play the video]

By using Unity 4, all the work is done in 4 days.
To do it, what I needed to solve are:

• Controlling the 3D model
• Walk/Run/Turn left/Turn right detecting
• Two-hands-above-head gesture to switch to car/robot
• Animations of switching to car/robot
• Driving gesture detecting
• Spawn enemies(Helicopter/Car) and make them follow the target(Transformer robot)
• Enemies exploding when hit. “Hit Targets” +1

If you like my works, you can subscribe to my Youtube channel.

This is my first Kinect 2.0 project.

I downloaded a free 3D model on the web.
It’s all done in 2 days by using Unity3D.

Neither the program nor the source code is available for download for now.

This is based on preliminary software and/or hardware, subject to change.