Virtual Shoe Fitting Experiment with Kinect v2


The virtual shoe fitting store from Goertz in 2012 is quite impressing. They used 3 original Kinects. I decided to make my virtual shoe fitting application with a Kinect v2 last month. It’s still something rare, though we’ve seen many virtual dressing room demos.

I placed the Kinect low on a box, so it can capture my feet only. In this case, we can see more details of the shoe. But, I’ll need to write my own feet tracking algorithm as the original body tracking won’t work. This is one of the big challenges.

Another challenge is that I need to hide some parts of the shoes that covered by my feet. If you’ve ever tried implementing the virtual dressing room, you’ll also face this problem. Using single-sided materials is one of the solutions. But it won’t work in some cases and it’s flawed. For some reasons, I’d keep my solution private. ;)

Overall, the application doesn’t work very well due to the CPU capability and the current algorithm. But I enjoy the process of making it!

Posted in Kinect, Tech | 4 Comments

Kv2 Tongue Tracking – Tip of Tongue Tracking with Kinect v2

I had been so busy recently. Finally got some time and done this work, tracking the tip of tongue with Kinect v2.
Video:
Github: https://github.com/TangoChen/Kv2TongueTracking

Here’s the steps to track tip of tongue with Kinect sensor.

  • Get the mouth area in the depth image by using face tracking.
  • Get the smallest depth(then it is closest to the Kinect sensor) inside the mouth area. That’s the depth part of the tongue’s tip!
  • Get the relative position according to the mouth area to know the tongue’s direction.
  • Show the tracking results when the mouth is open.

* P.S. I gotta say the researchers from Japan made it 2 years ago, too. They beat me to it. So it’s not me that showed you the tip-of-tongue tracking ability first. :)

Posted in Kinect, Tech | Tagged | 1 Comment

Kinect v2 Transformers Game Project

Kinect v2 Transformers[10-00-30]

Kinect v2 Transformers[10-14-02]

Kinect v2 Transformers[10-16-09]

Inspired by the movie Transformers 4, I decided to make a transformers-alike game. Imagine
that I can control the transformer and switch between robot and car, hit the enemies. That would be a lot of fun! So I do this.
I bought the 3D models on the web. It’s not one of the official Transformers though. (I will not be able to buy and use it if it is…)
It’s not so good-looking when looking from the back. Here’s the front looking.

[Click Play button on bottom-left to play the video]

By using Unity 4, all the work is done in 4 days.
To do it, what I needed to solve are:

  • Controlling the 3D model
  • Walk/Run/Turn left/Turn right detecting
  • Two-hands-above-head gesture to switch to car/robot
  • Animations of switching to car/robot
  • Driving gesture detecting
  • Spawn enemies(Helicopter/Car) and make them follow the target(Transformer robot)
  • Enemies exploding when hit. “Hit Targets” +1

If you like my works, you can subscribe to my Youtube channel.

Posted in Kinect, Tech | 4 Comments

Kinxct Ray – Turn the New Kinect v2 Sensor Into an X-Ray Machine

This is my first Kinect 2.0 project.

I downloaded a free 3D model on the web.
It’s all done in 2 days by using Unity3D.

KinxctRay_Screenshot_0

KinxctRay_Screenshot_1

KinxctRay_Screenshot_2
Neither the program nor the source code is available for download for now.

This is based on preliminary software and/or hardware, subject to change.

Posted in Kinect, Tech | 3 Comments

The Idea of Playing Blocks Game w/ Leap Motion Is Done!

I post an idea here on Nov 10th last year. Now I finally did it!
Video:

The laptop get data from Leap Motion controller and send it to the Android pad via Wifi network.
The pad gets the direction to the virtual blocks via gyroscope.

Posted in Life | 2 Comments

我的Chrome/Google.com用户体验建议

一些习惯..也不多..可能暂时忘了一些..

1.
Chrome的默认首页改为8个选项卡上一搜索引擎文本框后..用起来总感觉有点不妥…
文本框在窗口中央..用户视线先聚焦在此..一输入文本就跳到上方的地址栏…
这样做的目的大概是为了说明地址栏已经有搜索框的功能…
但一瞬间的Focus转移在体验上感觉有点欠缺…
我要这样做的话..可能会做一个矩形…一般截图功能可能会用到的半透明闪动的框…
从窗口中央的文本框在短时间内逐渐移动缩放到上方的搜索框…
告诉用户这个时候焦点做了转移…
Chrome

2.
地址栏有搜索引擎的功能..直接敲击要搜索的内容就能进入搜索..
但我还是很少用..甚至避免..
通常打开搜索引擎网页..或者打开隐身模式再使用(隐身模式下用地址栏的搜索功能)…
因为希望地址栏的提示显示有效常用的列表…不希望一次相比下无关紧要的搜索内容列在列表内…
(其他记录都挺有用..不想这样一次搜索把他们挤开或占一个位置..)
有时抓狂的是打错的内容..一条输错的搜索内容列在列表上很不养眼…
为此我会趁这是刚输错的搜索去历史记录里删了这条搜索记录…
如果搜索记录列表后有个X可以删掉这条记录就好了…
Chrome_1

3.
我不太习惯Google页面上的常用应用集合小窗口(暂称为”窗口”)设计…
上下滚动后窗口大小改变..这个功能区域大小非固定..界面体验是先进的..但对我可能觉得添了一点凌乱的感觉…
页面共两页..滚动不是固定上下两个状态..(要么在第一页,要么在第二页)可以是介于中间的状态…
之前那个滚动条还是Windows内置的..不美观..现在改好了…
Chrome_3
Chrome_4
Chrome_5

4.搜索引擎按了搜索框右边的语音搜索..返回时焦点没有恢复到搜索文本框…

P.S. Chrome还是我最常用最喜欢的浏览器…

Posted in Life | Leave a comment

MotionSensing.TV for sale

http://MotionSensing.tv
It’s obviously for Motion Sensing TV…

[Nothing more for reading]

Posted in Life | Leave a comment

Share An Idea of [Leap Motion + Pad] App

What can we do with an Android Pad connected with Leap Motion?
Currently the Leap Motion is not able to connect to tablet..
So I connect it to the PC, then send the Leap Motion data to the Android Pad.. Hope it work…

We can do a lot of things with them..
Here I just got an idea…
Due to lack of resources I can’t make it immediately…

LeapPadIdea
not that clearly…

Posted in Leap Motion | 3 Comments

Why The Flat Design In My Opinion

之前一段时期很多东西都趋向于3D(自智能手机图形界面发展水平开始越来越高的时候,不是阿凡达那个时候的3D)..
图标什么的都做成凸起的3D..
(那时候)一个屏幕呈现的3D视觉效果很”Xuan”…

逐渐地..有点泛滥…
普遍的凸起圆角图标像小孩的积木一样显得有点幼稚…
只是有点..还是可以的…
iPhone机身的设计就有这种变化…

再后来..聊聊iOS 7…
我没怎么用iOS系统..但我有些印象..希望不会有错…
iOS 7与之前的iOS系统一些区别是出现同时有至少两个面的操作界面多了..原本也一直有呈现多个面的界面..这样就更有需要了…
比如原本的..背后是滚动界面..前面是虚拟键盘…
然后是控制中心..没有布满整个屏幕..模糊&半透明..

这样扁平化的好处就是让多个面更清晰..能看清后面是一个整体的平面..前面也是…
而不会出现前后两个面..后面也有凸起的图标..而且程度不同..前面也是…
看起来会很凌乱..产生视觉疲劳…
就算只有一个界面..越来越繁多的操作控件..立体效果的形状,大小,位置各不同…
也会很凌乱…

另外是很多东西多趋向于科技感..但变得不那么友好…
之前的界面对小孩来说是不错的…
但人们对此的审美标准会越来越习惯&认同…

不管这些原因..直接看到一个扁平的图标也觉得是一种新的美的视觉体验…(大部分都)没有那么花哨..清晰简洁了…

Posted in Life | Leave a comment

Thoughts of Designing Leap Motion App Gestures

I want to share my [Thoughts of Designing Leap Motion App Gestures] with you.. :)

  • Stable
    Let the user act stably..
    Let the device detect it stably..
    Make the interaction stable..

  • Intuitive
    Easier to remember as well…

  • Don’t be too dumb.. Or make it funny
    Leap Motion is a cool thing…

  • Won’t let the user get too tired easily
    Sometimes, How long the user wanted to use the app for – How long the user used the app for ≈ How Tiring it is
    Tiring may make it less interesting.. but not always a problem.. It depends on the what users mainly use the app for…
    I may use an app for a little while, but I’ll use it frequently… It should be funny/addictive/cool enough.. Maybe for fun..
    I may use an app for a longer while for work.. Shouldn’t be too tiring…

    Commands can be done with one hand should better be done with one hand, especially the frequent ones..

  • Detecting Quickly
    It’s annoying if the user has to wait a little bit after a whole gesture is done every time then see the result..
    The movement patterns(Circle, Swipe, Key Tap, Screen Tap) recognized by Leap itself delay a little bit..
    We can make the movement patterns(not only official ones, like sign languages recognition) feel like real-time interacting by showing the movements in real-time like painting app..

  • Easy to Learn
    It’s confusing if the user has to practice a gesture many times.. (Maybe it’s not stable…)

  • Be clear with other gestures
    Not very good if you design these 3 gestures:
    1. Extending one finger only
    2. Extending two fingers
    3. Extending three fingers

    It may be tough(unstable) cause:

    • Sometimes Leap device detects one more/less finger than it is by mistake..
    • Switching from one-finger gesture to three-finger gesture may cause two-finger gesture
  • Gestures can be something more
    Don’t limit it with only finger controlling…


Bad Experiences:
Stable / Intuitive / Detected Quickly / Easy to Learn / Clear with other gestures:

  • “Did I do the gesture correctly?..” *waiting for the result*
  • Sometimes it works, sometimes not…
  • “Sometimes I was going to … but it’s detected as …”
Posted in Leap Motion, Tech | Tagged | 1 Comment