application stucks at gst_app_sink_pull_buffer call

[zr_yao]

Hi: I went through all steps Klen has done and stuck in the end, help is needed...

Basically I try to do the similar thing as Klen, port maemo Camera example to qt4-maemo5-gles1 version example Hellogl_es (I choose to use gles1 version for my later project purpose)type of GLWidget, to draw video overy the GLWidget, I create the gstreamer pipe line in a CameraN900.cc in initialize_pipe()function (which is called in CameraN900:CameraN900()) as:

Camera_src ! csp_filter ! app_sink

and I only require "video/x-raw-rgb" caps when connect camera_src and csp_filter, unlike Klen case, it seems request raw-rgb work in my case (this is in fact the maemo example_camera.c did)
unlike the camera.cpp in qtcartonizer I set:
gst_element_set_state(m_pipeline, GST_STATE_PLAYING); //the qtcartonizer set it to GST_STATE_NULL
///////////////The CameraN900:initialize_pipe() code//////////////////////////////////////

Code:
---------
bool CameraN900::initialize_pipeline()
{
GstElement *camera_src, *app_sink;
GstElement *image_queue, *image_filter;
GstElement *csp_filter;
GstCaps *caps;
GstBus *bus;

gst_init( NULL, NULL);
m_pipeline = gst_pipeline_new("test-camera");

bus = gst_pipeline_get_bus(GST_PIPELINE(m_pipeline));
//gst_bus_add_watch(bus, (GstBusFunc)bus_callback, &m_appData);
gst_object_unref(GST_OBJECT(bus));
camera_src = gst_element_factory_make(VIDEO_SRC, "camera_src");
csp_filter = gst_element_factory_make("ffmpegcolorspace", "csp_filter");

app_sink = gst_element_factory_make("appsink", "app_sink");
if(!(m_pipeline && camera_src && csp_filter && app_sink))
{
qDebug() << "Couldn't create pipeline elements";
return FALSE;
}

m_appsink = GST_APP_SINK(app_sink);
g_object_set(G_OBJECT(app_sink),
"signal-handoffs", TRUE, NULL);
g_object_set (G_OBJECT (app_sink), "emit-signals", TRUE, NULL);
g_signal_connect (app_sink, "new-buffer", G_CALLBACK (new_buffer_callback), NULL);

gst_bin_add_many(GST_BIN(m_pipeline), camera_src, csp_filter, app_sink, NULL);
caps = gst_caps_new_simple("video/x-raw-rgb",
"width", G_TYPE_INT, 320,
"height", G_TYPE_INT, 240,
NULL);
if(!gst_element_link_filtered(camera_src, csp_filter, caps))
{
qDebug () << "capability not supported for link camera->csp-filter";
return FALSE;
}
gst_caps_unref(caps);
if(!gst_element_link_many(csp_filter,app_sink, NULL))
{
qDebug () << "gst pipeline init fail";
return FALSE;
}

gst_element_set_state(m_pipeline, GST_STATE_PLAYING); //GST_STATE_NULL);
qDebug () << "gst init N900 ok";
return TRUE;
}
---------
//////////////////////////////////////////////////////////
hoping the camera will play from the very start. I then connect the CameraN900 with the original glwidget through GLWidget::initializeGL() by adding the following line in the end (class GLWidget has a member CameraN900 *camn900; ):

The initializeGL function for GLWidget
GLWidget::initializeGL{
...
camn900 = new CameraN900(); // start to play at the end, in play mode
}

The main drawing function paintGL in glwidget.cc looks

Code:
---------
void GLWidget :: paintGL()
{
QPainter painter;
painter.begin(this);
glEnable(GL_TEXTURE_2D);
camn900->refresh_buffer(); //manually call appsink to fatch for data in the buffer
unsigned char* data=(unsigned char *) GST_BUFFER_DATA (camn900->buffer);
glBindTexture(GL_TEXTURE_2D, m_uiTexture);
glTexImage2D( GL_TEXTURE_2D, 0, GL_RGB, 320,240, 0, GL_RGB, GL_UNSIGNED_BYTE, data );

glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR );
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR );
paintTextured();//paint the video as background
painter.end();
swapBuffers();
}

---------

in CameraN900.cc (adapted from qtcartonize's camera.cpp) I have


Code:
---------
static GstBuffer* gstbuffer;

void CameraN900::refresh_buffer(){
if(m_appsink!=NULL){
qDebug () << " in refresh_buffer(), do gst_app_sink_pull_buffer()...";
gstbuffer = gst_app_sink_pull_buffer(m_appsink); //buffer us a static GstBuffer *
buffer = (unsigned char *) GST_BUFFER_DATA (gstbuffer);
qDebug () << "refresh buffer done";

// buffer is a class member of CameraN900
}
}

---------
I have done things like in mainwindow.cc
Code:
---------
...
glwidget->setFixedWidth(320);
glwidget->setFixedHeight(240);
...
---------
to make sure glwidget has same size as video frame.
Now the application stucks at gst_app_sink_pull_buffer call without moving, seems pull buffer is waiting for a never ready buffer to come, I'm lost here what to do, could anyone please shed me some light? I feel I'm very close to a happy end...

BR/ zr_yao

[divanov]

Does pipeline go to GST_STATE_PLAYING? You can know this by attaching a watch to to a bus like in this GStreamer example written by Felipe Contreras:
http://felipec.wordpress.com/2008/01...r-hello-world/
then check for messages with type GST_MESSAGE_STATE_CHANGED like here:
http://www.gstreamer.net/data/doc/gs...-state-changed

[zr_yao]

Daniil: Thank you very much!

You are right, the pipeline never manage go to PLAYING mode, I use the tips you suggest and figure out more issue, at the moment I'm stuck again.

It seems the pipeline move from NULL to READY and then PAUSED then simply abort, I try to run a modified example_camera (from maemo example) using exact the same pipeline as mine in the GLwidget, and for testing purpose I also change my pipeline to only use image_sink instead of app_sink to see what's wrong, I remove the screen_sink branch and tee from the original pipeline (the upper branch and also the tee element), the pipe line looks like

camera_src | csp_filter | image_queue | image_filter | image_sink

I expect the drawing should be simple blank window. It runs well in the Gtkwidget (the modified maemo example) but not mine, I could see the difference lies in that:

In example_camera case, the piple line bus get "new clock " msg and then "async done" msg, but it never show up in my GLWidget case before it abort at typicall place "capsfilter0 change from READY to PAUSED" ... I could see stream status has go through type 0 and 1 as the example_camera case.

according to some web page:
"...
When the Gst::Pipeline performs the PAUSED to PLAYING state change it will select a clock for the elements. The clock selection algorithm will by default select a clock provided by an element that is most upstream (closest to the source). For live pipelines (ones that return Gst::STATE_CHANGE_NO_PREROLL from the Gst::Element::set_state() call) this will select the clock provided by the live source. For normal pipelines this will select a clock provided by the sinks (most likely the audio sink). If no element provides a clock, a default Gst::SystemClock is used.
..."

in my test, both case has camera_src at front end and image_sink at end, I have no idea now how to break this. the bus message break in middle with "Abort", any tip to get more hint from the broken ones?

I however notice in both case they dont print my debug message in the new-buffer call back that I connect to image_sink, that's wired to me, I have the code of callback as

static gboolean new_buffer (GstElement *app_sink, GstBuffer *buffer, GstPad *pad, AppData *appdata)
{
g_print ("................in new_buffer callbback.............. \n");
}

and I did the signal and connect in the initialize_pipeline(), is this normal?

g_object_set(G_OBJECT(image_sink),
"signal-handoffs", TRUE, NULL);

g_object_set (G_OBJECT (image_sink), "emit-signals", TRUE, NULL);
g_signal_connect (image_sink, "new-buffer", G_CALLBACK (new_buffer), NULL);

Any help would be greatly appreciated!

zr_yao

[divanov]

You are right, the pipeline never manage go to PLAYING mode.

It could be actually because of media policy, suppressing your application.

the pipe line looks like camera_src | csp_filter | image_queue | image_filter | image_sink

This pipeline is a way too complicated, for example this one works without any problems:
$ gst-launch-0.10 -v -t v4l2camsrc num-buffers=1000 driver-name=omap3cam ! xvimagesink

and I did the signal and connect in the initialize_pipeline(), is this normal?

Yes.

[zr_yao]

Many thanks!

Klen kindly post one working example, everyone can follow it and escape the gstreamer pipeline headache...

http://discussion.forum.nokia.com/fo...d.php?t=202336

For sure I'm not complaining I see the value of flexibility of gstreamer there.

cheers!

[stone.chao]

Hi!
I have the same problem. Klen's example is very good.
But after I added an additional timer to work with Gst::Pipeline, it cause the blocking problem.
Don't understand why was that.
Does the additional timer event interrupt the process of changing pipeline mode?

Thanks for any comments