MIDI Reality – Mixed Reality with Kinect and MIDI Keyboard


Tech & music are my favorites. I’d love to do them together.
So I had this idea to try covering a song with a MIDI keyboard, and a Kinect v2 sensor for mixed-reality floating caption effects.
That was when the song (Life Is Worth Living by Justin Bieber) was still new. However, the recording process’s been so procrastinated.
Since it had taken for a very long time, I decided to not making a better cover, but this somewhat bad, weird and incomplete one. And, explaining how that was done more focusedly.

You can consider it as how most mixed reality / augmented reality effects are done behind nowadays. Literally overlaying parts of(or full) virtuality on a reality layer. I can not be 100% sure though, cause I haven’t been looking into any references for this project.

Furtherly, using regular AR marker technology to track position of MIDI keyboard, Kinect can capture around for a movable camera view(Requires a stored 3D background data also). Or just use HoloLens!

About TangoChen

Original name: Jingzhou Chen. VR Prototyper at Tencent, Microsoft MVP on Emerging Experiences(More Personal Computing), dropout.
This entry was posted in Augmented Reality, Kinect, Mixed Reality, Tech. Bookmark the permalink.

2 Responses to MIDI Reality – Mixed Reality with Kinect and MIDI Keyboard

  1. Pingback: Mixed Reality with Keyboard and Kinect – Be Analytics

  2. Pingback: Combined Actuality with Keyboard and Kinect | Coding4Fun Kinect Tasks - New Geekers

Leave a Reply

Your email address will not be published. Required fields are marked *

Time limit is exhausted. Please reload CAPTCHA.