Paul Coulton's Blog

Here is the latest update for our markerless augmented reality project in which we are now able to  explore a full 3D model of Lancaster University. My PhD student Klen Copic Pucihar has taken the system originally developed by Dan Hutchinson and applied some sensor fusion to improve the pose estimation and tracking so we have the results shown in the following video using an N900.

We have made some tweaks to the rendering but need to improve it a bit more to make it even more impressive . 

In my last post I showed a video that Dan and I produced realted to our work on markerless augmented reality on N900. There were some issues with colours realted to capturing straight out of the device. We managed to get round those issues so here is a short video of an AR truck on my desk. The video capture still doesnt really get the redering of the model properly but its much improved. Hopefully next we can show some improvemnets to pose estimation and scaling.

Last time I introduced a project from my student Dan Hutchison related to enabling markeless augmented reality in maemo with the aim of making mobile AR more contextually aware of the environment. Here is where we are so far with a 3D object that can be viewed from different angles by moving the mobile phone around the virtual object object whose plane is referenced from objects in the environment rather than a fiducial marker. This short video show one example of using a poster in my office with the phone moving the second use my head as reference with the phone held still. The object in this case is a crude box but you get the idea and coming soon we will show it coupled to a much more interesting model so stay tuned!

Here is a little teaser of a project my student Dan Hutchinson is working on that will use the Parallel Tracking and Mapping (PTAM) algorithm for marker-less augmented reality developed on Maemo and the N900. This video represents the first stage of the project the implementation of the Fast Detection Algorithm indicated by the white dots (not normally shown) that will ultimately produce the reference plane for the 3D AR components of the scene.

We are finding the Maemo platform incredibly powerful and the N900 a great device for this project and I will keep you updated on this blog as things develop further.

As we start a new decade lets leave the 'my touch screen is better than your touch screen' rhetoric  in the old one and sets our sights on the Holy Grail of human computer interaction the brain computer interface. While I was at GDC last year I came across the NeuroSky and their Mindset headset which allows you to pick up signal from users brain activity. Very kindly they were able to provide me headset and we have been working on a number of ideas. Brain Maze is our first game which uses tilt and brain interaction. In particular it requires users to adopt the appropriate mental state (either medidtative or attentive) at certain points in the game to allow them unlock brain controlled gates within the game.

Although this interaction is relatively simple at the moment it highlights the enormous potential brain computer interaction could provide and at some point in the future when we look back at touch screens and they will probably seem very primitive indeed.

Having spent my summer monster hunting I have been neglecting my blogging duties so here is an interesting little project based on an idea that’s been festering in the back of my mind for a while. Not content with inflicting 3D glasses on you for anaglyph photography I thought it would be interesting to explore using them in a mobile game and finally managed to persuade a student, Ahmed Alansari, it would be a good idea. Basically it’s a touch based game similar to the old Asteroids game except with 3D anaglyph blocks flying out at you which you destroy by tapping with your finger. Here is the current prototype which still needs some work on the explosions but even with this poor quality video you get the idea. I am of course assuming you all still have those 3D glasses ;).

Im pretty sure it the first game of this type and quite probably the last ;)

In terms of my games Free Londons Monsters is the mobile augmented reality game I have previously discussed but the following video is the Mk2 version of the Magic Monstervision Machine in action.

MashReality is a playful experience in the form of a community driven interactive narrative played out as a visualisation on a large public screen with player interacting using text Messages. In many respects its and evolution of our original TxtBk project but designed to be more accessible to all. 

In any one is down in London that weekend I would encourage you to check out the programme of great games available (in addition to mine of course ;) ) and come down and play. If you are there feel free to say hi

We had good news from the Flash Developer Challenge yesterday as our new location based game Big Game Huntr took the innovation prize out of 150 entries worldwide. I am particularly pleased to get the innovation prize as I believe the game represents a new generation of mobile gaming designed specifically for the medium and harnessing player creativity in an exciting new direction. I would also acknowledge the hard work of the two students involved Mark Lochrie and Kate Lund in getting it to this point. We are hoping to enter Open Beta testing soon and I look forward to offering it to the community here as first testers.

I have been working on the Mobile Magic Monstervision Machine to improve the functionality and introduce better game play and the following images are screenshots captured during the first full scale trial. I added the holding trap (in a homage to Ghostbusters) to indicate your progress during the game.  When a particular monster has been captured its position is indicated on the map and the time of capture indicated in the holding trap inventory. To aid exploration I introduced a hot, warm, cold metaphor indicated on the Monstervision Machine by snowflake, sun, and flame icons which are triggered by measuring your distance from a  monster (in this trial warm was 30m and hot 20m) which seems to work really well with the players thus far. As an added touch when the Monstervision Machine is hot the phone vibrates and shouts MONSTER which is fun. Finally rather than have a GPS bar I have indicated this as ‘Captoplasm’  as it indicates the importance of having a signal but doesn’t detract from the game concept.

Next week we intend to test the game in London with more REAL people and new user generated monsters .

I have been working on an interesting take on a location based game recently which incorporates user generated content in the form of sketches of monsters. It’s in collaboration with my friend and artist Andrew Wilson who ran a workshop at Institute of Contemporary Arts in London where people could sketch there idea of what the London’s monsters might look like.  I met up with Andrew a few of weeks ago at Futuresonic and we hatched a plan to turn the sketches into a mobile location based game called Free London's Monsters to be played around the actual locations where the monsters were envisaged. I had a small team of students (Frank, Laura, Richard, Rebecca, and Tabona) put together a first prototype using some bits of code I had lying around which uses the phone camera as the Magic Monstervision Machine (that’s an S60 phone to the uninitiated) to capture monsters when the players enter their lair. It’s worth noting this was done in less than two weeks with no previous experience of PyS60. Here’s a short video by the students of the prototype in action.

Iam am currently extending the application to include a map, game-time, and capture lists and a few other fun features but the basic functionality will be as shown. We are planning to test the first prototype at the Sandpit event at the end of June (ehich are free and if you in London around RCA pop along) before we hopefully present the more elaborate version at this year’s Hide and Seek festival which introduces social games and playful experiences from 31st July to 2nd August at the Southbank Centre, London.

I think the idea of incorporating the user generated content within the game is very interesting and is likely to greatly enhance the user experience. This concept also forms the basis of another location based game I am working on as I think UGC can be used to great effect in location based games

It’s a sad day for me as the Widsets platform is officially closed which means that the many innovative widgets we produced will suffer the same sad demise. In particular the loss of Bombus and Boom! which finish with 1.5 million users.  I would like to say both thanks to everyone who played the game but also pay a large tribute to the team who developed Widsets which I think was a wonderful piece of work.

We did consider changing the widgets to J2ME an put them on OVI but the requirement to have J2ME applications Java verified (not sure of logic behind this)makes it to costly for us to consider so Bombus and Boom! RIP.

Having warmed myself up with a small game rant (sorry Andreas) I am off to a hopefully sunny San Francisco (hence the quote from Oscar Wilde) for the great gathering of the game nerds at GDC. I am speaking at Mobile GDC on How do we socialise mobile games? and instead of often technology inspired talks I am getting all ‘touchy and feely’ around the premise of games as social information appliances or social objects. If anyone from here is there feel free to say hi and I will be the one NOT talking about the IPhone.

It seems a long time ago since I blogged about the operation of the accelerometer sensors and made a plea for their inclusion as a standard feature (I am of course assuming the positive outcome was because people actually listened to me rather than me being just prophetic ). Anyway I have already discussed in previous blogs about Streetview and other aspects of Point ‘N’ Seek utilising the magnetometer sensor (generally described as the digital compass) so here are some practical details of how it works in practice (I always feel a visualisation helps). As with the accelerometer in its raw form you have three outputs x,y,z which means that you can find directional information ,were your phone is pointing, whatever its orientation. For example the Figure below shows the outputs on the three axis for the phone held with the screen parallel to the users face (i.e. upright) and in the normal portrait orientation and when the phone is held so that the screen is parallel with the ground as you would use a normal compass. The plots start with the phone facing North then rotated through East, South and West and pausing at each alignment for around 5 seconds and you can clearly see the four compass points.

The illustrate the actual rotational change the following figure illustrates the three outputs for a continuous clockwise rotation with the phone held upright going clockwise from North.

The second graph of the figure is after applying some simple signal processing whereby we can combine these outputs along phone orientation from the accelerometer to produce an output that could easily be turned into an angle from North. Indeed the sensor API does provide a range of options one being for an angle from North as shown. 

The problem but we have found this appears to be slightly sluggish (as illustrated) and my feeling is that a low pass filter has been applied to give a smooth scrolling of Maps when used for navigation. However, I have other plans for the sensor so will likely create my own algorithms using the raw outputs previously illustrated. Hopefully people will find this blog as useful as the one on accelerometers but judging by the time lag from last time you might want to return to this in about 12 months .

BTW should have said the data is from a S60 application running on the Nokia 6210 Navigator

As Arto has recently done some interesting blogs relating to bio-signal and pain monitoring and Nokia launching a phone Heart rate Monitor combo I thought its useful to discuss the extension of the work  Carlos and I have done using an Heart Rate Monitor  to provide mobile exergaming. In this research we experimented with four different user input controls to the game to try and find a balance between benefits to the heart and game play as shown in the following video.

The following figure shows Carlos heart rate for the four schemes presented in the video. Interesting although the third scheme (all actions triggered by a small physical jump) provides the greatest exertion its probably the hardest to play and the final scheme ( all physical movement control ) provides the best balance between exertion and game play.