Mike Hodnick's Blog

Kinect Sensor in an MVVM App

Posted: 20 July 2011

Source code: https://github.com/kindohm/kinect-mvvm

Now that the official Kinect SDK is out I've been looking for good patterns when incorporating it into Windows 7 applications. The Model-View-ViewModel (e.g. MVVM) pattern is my pattern of choice for WPF applications and I wanted to demonstrate how I incorporate the Kinect runtime within the MVVM pattern.

I've created a screencast walk-through of how you can incorporate the Kinect runtime into MVVM (or read on below for a brief description of the approach).

The main concept of incorporating the Kinect runtime into MVVM is to abstract all of its behaviors out into its own service interface:

public interface IKinectService { }

In my approach, a simple event is defined in the interface that is used to signal when some Kinect runtime data has become available, such as new skeleton information:

public interface IKinectService
{
	event EventHandler<SkeletonEventArgs> SkeletonUpdated;
}

Basically, the Kinect service just fires events whenever it wants (probably hundreds of times per second).

At this point in the Kinect SDK's lifetime, failing to "uninitialize" it causes problems and can sometimes require a reboot. Thus, putting some methods on the interface to allow us to deal with starting up and shutting down the runtime would be helpful:

public interface IKinectService
{
	event EventHandler<SkeletonEventArgs> SkeletonUpdated;
	void Initialize();
	void Uninitialize();
}

From there we can implement a basic KinectService:

public class ConcreteKinectService : IKinectService
{
	Runtime runtime;

	public void Initialize()
	{
		runtime = new Runtime();
		runtime.SkeletonFrameReady += 
			new EventHandler<SkeletonFrameReadyEventArgs>(runtime_SkeletonFrameReady);
		runtime.Initialize(RuntimeOptions.UseSkeletalTracking);
	}

	void runtime_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e)
	{
		var skeleton =
		   e.SkeletonFrame.Skeletons
		   .Where(
				s => s.TrackingState == SkeletonTrackingState.Tracked)
		   .FirstOrDefault();

		if (skeleton == null) return;

		var position = skeleton.Joints[JointID.HandRight].Position;
		if (this.SkeletonUpdated != null)
		{
			this.SkeletonUpdated(this, 
				new SkeletonEventArgs() 
				{ 
					RightHandPosition = position 
				});
		}
	}

	public void Cleanup()
	{
		if (runtime != null)
		{
			runtime.Uninitialize();
		}
	}

	public event EventHandler<SkeletonEventArgs> SkeletonUpdated;
}

This IKinectService can now be injected into a ViewModel whenever it is created. Using the "Locator" pattern from the MVVM Light toolkit, this is done when the locator constructs its ViewModels:

public class ViewModelLoader
{
	static ViewModel viewModelStatic;
	static IKinectService kinectService;

	public ViewModelLoader()
	{
		kinectService = new ConcreteKinectService();
		var isInDesignMode = ...; // detect if running in VS designer
		if (!isInDesignMode) kinectService.Initialize();
	}

	public static ViewModel ViewModelStatic
	{
		get
		{
			if (viewModelStatic == null)
			{
				viewModelStatic = 
					new ViewModel(kinectService);
			}
			return viewModelStatic;
		}
	}

	public ViewModel ViewModel
	{
		get
		{
			return ViewModelStatic;
		}
	}

	public static void Cleanup()
	{
		if (viewModelStatic != null)
		{
			viewModelStatic.Cleanup();
		}
		kinectService.Cleanup();
	}
}

A static copy of the ViewModel is used so that it can be "cleaned up" easily from any point in the application. This cleanup eventually unhooks events and uninitializes the Kinect runtime further down the call stack.

There are a variety of benefits to this approach:

1. The Kinect runtime logic is de-coupled from the ViewModel and View/UI
2. Multiple types of Kinect services could be implemented. Internally, a concrete IKinectService could use either event-based or polling-based skeleton tracking. With an IKinectService you could swap out one for the other and the application won't care.
3. The Kinect service can be mocked during unit testing.

Hopefully this approach can help make your MVVM-based Kinect apps more fun to code! Enjoy!