Project Natal

Eurogamer have a pretty interesting Natal interview up.  We talk about it a little on this week’s podcast, but I have a few extra observations.

First, in 2006 I met with a company who were in final talks to sell a “two-camera, full-body mapping” technology to Microsoft.  We met because we wanted to see if Determinance should work with this tech – it never came to anything, but given what I was told about their product I would be extremely surprised if it wasn’t the first part of Natal.  I don’t offer any conclusions, but it is interesting to consider how long it took to get to this stage, and the way in which Natal is being marketed.  It’s also mildly interesting that they don’t mention the 3rd party company, but kind of imply that it was a fully internal product.  Not at all surprising mind you – but this is the kind of datapoint which makes for more informed analysis of first party decisions.

Second, here’s an exerpt from the interview:

Say I’m tracking a wrist, which is what I do for Burnout. I can look at that on a single frame and I can see what direction, acceleration and confidence I have for that joint. Why is that interesting? Because it allows me to not only know where you are, but to know where you’re going to be. This is how we do the directing and the predictive behaviour.

If you think about swinging a baseball bat, by the time you’re halfway done with the swing, I know not only where you’re going to end but when you’re going to end. There are very precise and predictable ways so you can have that immediate payoff of my baseball bat hitting the baseball.

…What?  Surely you… will actually be hitting the baseball though?  Why do you need to predict anything?

This says to me that no matter how advanced Natal is, Microsoft are still very much thinking in terms of gestures.  You wouldn’t need to predict anything if you were actually mirroring the player’s actions and responding realistically.  This shows that it’s a big consideration for them that the game knows before you do it what action you’re performing… which really is a very wiimote way of doing things.

I’m also not making a value-judgement on this – my work on Determinance and Guile has shown me that players often really don’t want full analogue control over everything.  Game design is partially about making the set of choices the player has the correct size – too big is just as bad as too small.  I think we can probably all agree that Wii Tennis is too limited in this regard, but I’m also sure you don’t want people to actually have to be good at tennis to play a mainstream game.

This is a tangent now… but how far is this going to go in Golf games?  It seems to me that Natal or MotionPlus could both get pretty close to simulating real swing mechanics… in which case, will the average player start having a handicap in the tripple-figures?

Next up:

Or you could have a hardcore gamer like me playing a game with a controller, while a non-hardcore person sitting next to me enjoys the experience by playing with Natal. I could be having my Halo experience with the controller and the friend next to me, who’s not a hardcore gamer, could be throwing grenades or driving the Warthog or doing any number of things with Natal.

Wow.  Natal as the noob control option for games?  That’s an insane, amazing idea.  That is facing the problem of accuracy vs ease in a controller head on.  I love this idea.

So we have a custom chip that we put in the sensor itself. The chip we designed with Microsoft will be doing the majority of the processing for you, so as a game designer you can think about the sensor as a normal input device – something that’s relatively free for you as a game designer.

Sounds like the signal processing is pretty hard work – I can see no reason for not doing it on the 360 itself unless it took considerable resources.

Share this gibberish!

This entry was posted on Friday, June 5th, 2009 at 11:45 am and is filed under News. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.

5 Responses to “Project Natal”

1. Shivoa:

   § June 5th, 2009 at 2:26 pm

   I really wouldn’t process the images on the 360 passing the image data over the USB link. The 3 core (HT) base is already loaded on the 360 with a decent game design (360 OS takes one thread, XNA takes another if you’re using that) and having dedicated silicon to calculate the information to pass to the framework fits in with the MS approach.

   The tech seems to be what I was expecting when I first heard the rumours, using IR with an emitter and dual cameras to get 3D positional info (which takes out any low light conditions and even has the natural advantage of humans being IR sources) and then a stock camera with a few MP to get basic image data. This is all edge detection processed into a skeletal model of the player(s) with size and position on the floor and also some data for other moving objects (allowing you to capture the area of the plain image where a held object exists). I think the only thing that came as a surprise is the other extra data, Natal checks out the surroundings to the player to check for sofas and walls and returns this as an available space to the game. In hindsight this is something you should obviously do but hasn’t been in the previous implementations of camera mapping.

2. Ian:

   § June 6th, 2009 at 10:47 am

   Hey Geoffrey. Bandwidth is a good point I hadn’t thought of, but I really don’t imagine they are high res enough to cause USB 2.0 a problem.

   Yeah, I like that sofa thing. This interview has me more excited for Natal.

3. Ian:

   § June 6th, 2009 at 10:49 am

   Also, interesting that XNA uses a whole core to itself.

4. Shivoa:

   § June 6th, 2009 at 2:30 pm

   Thread, not core. It takes over half a core, which kind of makes sense as it is Managed DirectX.

   The problem with USB (over networking with a proper network card or firewire) is it is basically a software solution nowadays so once data rates go high you’re going to be dealing with a CPU load (even high end PC CPUs can see measurable load when you max out USB 2.0 bandwidth). I don’t think it is a massive issue compared to the CPU load of doing the image processing on the box but it’ll all add together to cripple the available CPU resources in a game if they don’t use a processor on the Natal hardware.

5. Ian:

   § June 9th, 2009 at 3:01 pm

   Aparently, Natal is not based on 3DV’s work…

   http://www.eurogamer.net/articles/e3-natal-not-derived-from-3dv

Leave a Reply