Just to make life easier to anyone who wants to do AR on N900 i have a few suggestions.
1.) Use openCV library for any tracking and image manipulation. It is an amazing library that is available in the repo.
$ apt-get install libcv4 libcvaux4 libhighgui4
$ apt-get install libcv4 libcv-dev opencv-doc libcvaux-dev libcvaux4 python-opencv libhighgui-dev libhighgui4
2.) For capturing video use openCV library calls or go through the hassle of GStreamer (appsink element) which can be integrated with QT using QGLWidget object. I attached source code of a project GstOpenGL that demonstrates how this can be done. Make sure you have qt 4.6 on your N900 or scratchbox. The GstOoenGL does not use openCV.
The code has not been cleaned and optimized but should save you time.