Getting access to the frames in video stream provided by QCamera

[aapiskukko]

I have a problem for which I cannot find a solution. I’m using QCamera class (Qt Mobility 1.1) and I need to get access to the video frames(e.g. QVideoFrame) provided by the camera in N900. I need to do some image processing to the frames before they are shown in for example QVideoWidget. How can I do that?

I have tried the method illustrated in http://developer.qt.nokia.com/forums/viewthread/370 with no luck.
If i don’t define “camera->setViewfinder(viewfinder);”, all I get is some Camerabin “internal data flow error” after a while.

I have also considered subclassing QVideoWidget and reimplementing paintEvent, but don’t see how can I actually get access to the frames before they are drawn to the widget.

I do not want to process snapshot images that are already saved in file.

Is the any good solutions/examples how to solve this problem?

Thank you.

[chriadam]

Hi,

I think you can set the viewfinder without necessarily showing the viewfinder widget, but nonetheless it sounds like a bug if you _need_ to set the viewfinder widget before you can grab data from the camera. As to your actual question, I'd suggest joining the #qt-mobility channel on the Freenode IRC network and asking in there. The mobility multimedia developers are in GMT+10 timezone, so if you're in Finland there should be a couple of hours of overlap in your morning.

Cheers,
Chris.

[aapiskukko]

Hi,

Thanks for the reply. I'll try asking from the IRC channel.

[DianaM]

I think it would be much easier to use GStreamer to get raw frames rather than QCamera.

[aapiskukko]

Hi,

I managed to solve this issue. My main problem was really that I didn't realize that N900 camera outputs YUVY pixel format. Here are the rough steps to get access to the camera video frames:

1. Make an instance of the QCamera class.

2. Make own implementation of the QAbstractVideoSurface by subclassing it. present() and supportedPixelFormats() pure virtual functions must be implemented at least. present() function is where you can get access to the QVideoFrames. supportedPixelFormats() function must be set to support YUVY pixel format. See VideoWidget example in mobility API examples for details. Pay attention that QVideoFrame::imageFormatFromPixelFormat() does not support UYVY to RGB conversion.

3. Use this kind of code to be able to receive video frames to your own video surface:

 QVideoRendererControl *control = qobject_cast<QVideoRendererControl *>(
                                     camera->service()->requestControl("com.nokia.Qt.QVideoRendererControl/1.0"));

    if (control)
    {
        control->setSurface(yourOwnVideoSurface);
    }

4. To be able to show camera frames in widget, YUVY to RGB conversion must be made. I was adviced to look at the qgraphicsvideoitem_maemo5.cpp file from mobility api multimedia source codes. There you can find neon accelerated converter suitable for at least the N900 phone. Check multimedia.pro file in mobility api source codes to see how neon acceleration is enabled in maemo5. If you use the code, pay attention for licensing. I made own converter class which utilizes that code and converted frames in YourOwnVideoSurface::present() class.

Hope this was helpful for somebody. Happy christmas!

[dfreee]

Hi,
please help, as I am looking for exactly that solution.
I don't know how to make the connection between the qcamera and the widget.
I tried to integrate the 'videowidgetsurface' files into my project. Then I used

    VideoWidgetSurface * videoSurface = new VideoWidgetSurface(ui->uiWidget);
    QVideoRendererControl *control = camera->service()->requestControl<QVideoRendererControl *>();
    if (control) control->setSurface(videoSurface);

in my widget's class.
Then what next?
The widget stays black.
In the videowidget example there is the paint() fuction and the present() function, but I don't know how to integrate them.
Could you explain what I should do or better - could you send me your (ripped off) solution, so I can see what to do.

Thank's

[dfreee]

Ok. I guess this is a problem that people resolve by themselves.
The 'missing link' was actually the paintEvent() function.
The ingredients to make it work.
1. Take the camera files from the camera example.
2. Take the videowidgetsurface files from the videowidget exmaple.
3. In QT Designer, replace the viefinder widget from the camera example with a plain widget. Name it for instance uiWidget.
3. in the constructor of camera add

videoSurface = new VideoWidgetSurface(ui->uiWidget);

4. in setCamera() comment out the setViefinder line:

 //camera->setViewfinder(ui->viewfinder);

5. Add instead:

    QVideoRendererControl *control = camera->service()->requestControl<QVideoRendererControl *>();
    if (control) control->setSurface(videoSurface);

6. Add the paintEvent() function as is to the camera files.
7. in camera.cpp

#include <QVideoRendererControl.h>

8. in camera.h:

#include "videowidgetsurface.h"

That should be it more or less.
Good luck.

[Carlo.Rossini]

Ok. I guess this is a problem that people resolve by themselves.
The 'missing link' was actually the paintEvent() function [...]

Sorry, I used your code "as is" but I can't achieve your goal.
It returns to me the error: "Viewfinder format not supported" and the widget remains black.

Is this error known? I can't find any post about it.
I tried almost everything...

But your post said you had success.. so I'm missing something...
Could you please suggest me where I'm wrong?

Thanks,
Carlo.