Virtual Shoe Fitting Experiment with Kinect v2

The virtual shoe fitting store from Goertz in 2012 is quite impressing. They used 3 original Kinects. I decided to make my virtual shoe fitting application with a Kinect v2 last month. It’s still something rare, though we’ve seen many virtual dressing room demos.

I placed the Kinect low on a box, so it can capture my feet only. In this case, we can see more details of the shoe. But, I’ll need to write my own feet tracking algorithm as the original body tracking won’t work. This is one of the big challenges.

Another challenge is that I need to hide some parts of the shoes that covered by my feet. If you’ve ever tried implementing the virtual dressing room, you’ll also face this problem. Using single-sided materials is one of the solutions. But it won’t work in some cases and it’s flawed. For some reasons, I’d keep my solution private. ;)

Overall, the application doesn’t work very well due to the CPU capability and the current algorithm. But I enjoy the process of making it!

Posted in Kinect, Tech | 4 Comments

Kv2 Tongue Tracking – Tip of Tongue Tracking with Kinect v2

I had been so busy recently. Finally got some time and done this work, tracking the tip of tongue with Kinect v2.

Here’s the steps to track tip of tongue with Kinect sensor.

  • Get the mouth area in the depth image by using face tracking.
  • Get the smallest depth(then it is closest to the Kinect sensor) inside the mouth area. That’s the depth part of the tongue’s tip!
  • Get the relative position according to the mouth area to know the tongue’s direction.
  • Show the tracking results when the mouth is open.

* P.S. I gotta say the researchers from Japan made it 2 years ago, too. They beat me to it. So it’s not me that showed you the tip-of-tongue tracking ability first. :)

Posted in Kinect, Tech | Tagged | 1 Comment

Kinect v2 Transformers Game Project

Kinect v2 Transformers[10-00-30]

Kinect v2 Transformers[10-14-02]

Kinect v2 Transformers[10-16-09]

Inspired by the movie Transformers 4, I decided to make a transformers-alike game. Imagine
that I can control the transformer and switch between robot and car, hit the enemies. That would be a lot of fun! So I do this.
I bought the 3D models on the web. It’s not one of the official Transformers though. (I will not be able to buy and use it if it is…)
It’s not so good-looking when looking from the back. Here’s the front looking.

[Click Play button on bottom-left to play the video]

By using Unity 4, all the work is done in 4 days.
To do it, what I needed to solve are:

  • Controlling the 3D model
  • Walk/Run/Turn left/Turn right detecting
  • Two-hands-above-head gesture to switch to car/robot
  • Animations of switching to car/robot
  • Driving gesture detecting
  • Spawn enemies(Helicopter/Car) and make them follow the target(Transformer robot)
  • Enemies exploding when hit. “Hit Targets” +1

If you like my works, you can subscribe to my Youtube channel.

Posted in Kinect, Tech | 4 Comments

Kinxct Ray – Turn the New Kinect v2 Sensor Into an X-Ray Machine

This is my first Kinect 2.0 project.

I downloaded a free 3D model on the web.
It’s all done in 2 days by using Unity3D.



Neither the program nor the source code is available for download for now.

This is based on preliminary software and/or hardware, subject to change.

Posted in Kinect, Tech | 3 Comments

The Idea of Playing Blocks Game w/ Leap Motion Is Done!

I post an idea here on Nov 10th last year. Now I finally did it!

The laptop get data from Leap Motion controller and send it to the Android pad via Wifi network.
The pad gets the direction to the virtual blocks via gyroscope.

Posted in Life | 2 Comments







P.S. Chrome还是我最常用最喜欢的浏览器…

Posted in Life | Leave a comment

MotionSensing.TV for sale
It’s obviously for Motion Sensing TV…

[Nothing more for reading]

Posted in Life | Leave a comment

Share An Idea of [Leap Motion + Pad] App

What can we do with an Android Pad connected with Leap Motion?
Currently the Leap Motion is not able to connect to tablet..
So I connect it to the PC, then send the Leap Motion data to the Android Pad.. Hope it work…

We can do a lot of things with them..
Here I just got an idea…
Due to lack of resources I can’t make it immediately…

not that clearly…

Posted in Leap Motion | 3 Comments

Why The Flat Design In My Opinion



再后来..聊聊iOS 7…
iOS 7与之前的iOS系统一些区别是出现同时有至少两个面的操作界面多了..原本也一直有呈现多个面的界面..这样就更有需要了…




Posted in Life | Leave a comment

Thoughts of Designing Leap Motion App Gestures

I want to share my [Thoughts of Designing Leap Motion App Gestures] with you.. :)

  • Stable
    Let the user act stably..
    Let the device detect it stably..
    Make the interaction stable..

  • Intuitive
    Easier to remember as well…

  • Don’t be too dumb.. Or make it funny
    Leap Motion is a cool thing…

  • Won’t let the user get too tired easily
    Sometimes, How long the user wanted to use the app for – How long the user used the app for ≈ How Tiring it is
    Tiring may make it less interesting.. but not always a problem.. It depends on the what users mainly use the app for…
    I may use an app for a little while, but I’ll use it frequently… It should be funny/addictive/cool enough.. Maybe for fun..
    I may use an app for a longer while for work.. Shouldn’t be too tiring…

    Commands can be done with one hand should better be done with one hand, especially the frequent ones..

  • Detecting Quickly
    It’s annoying if the user has to wait a little bit after a whole gesture is done every time then see the result..
    The movement patterns(Circle, Swipe, Key Tap, Screen Tap) recognized by Leap itself delay a little bit..
    We can make the movement patterns(not only official ones, like sign languages recognition) feel like real-time interacting by showing the movements in real-time like painting app..

  • Easy to Learn
    It’s confusing if the user has to practice a gesture many times.. (Maybe it’s not stable…)

  • Be clear with other gestures
    Not very good if you design these 3 gestures:
    1. Extending one finger only
    2. Extending two fingers
    3. Extending three fingers

    It may be tough(unstable) cause:

    • Sometimes Leap device detects one more/less finger than it is by mistake..
    • Switching from one-finger gesture to three-finger gesture may cause two-finger gesture
  • Gestures can be something more
    Don’t limit it with only finger controlling…

Bad Experiences:
Stable / Intuitive / Detected Quickly / Easy to Learn / Clear with other gestures:

  • “Did I do the gesture correctly?..” *waiting for the result*
  • Sometimes it works, sometimes not…
  • “Sometimes I was going to … but it’s detected as …”
Posted in Leap Motion, Tech | Tagged | 1 Comment