MeeGo OpenCV performance issues.

[Klen]

Hello everyone,

I am porting an opneCV application from Maemo to MeeGo. Managed to compile the application, using OpenCV 2.3.1, but got stuck with performance issues.

The application runs on N9 at 10 FPS, which is surprising when compared to 23 FPS on N900.

When I did some code benchmarking it seams like the performance of the image processing done by openCV library dropped segnificantly.

Question:
Any idea why? Do you also share the same experience?

BTW. I am using an older version of openCV (2.0) on N900.

Thanks,
Klen

[ohookins]

I have a feeling the Qt libraries doing the display are doing something funny, it wouldn't surprise me if the Qt support in OpenCV is not as good as its GTK support.

When you say image processing, which calls are you making to OpenCV and have you isolated it to any particular call taking the most time?

[Klen]

I know about the QT issues. Initially had the application written in QT but moved to SDL (Simple Direct Medlia Library) beacuse of perforamnce issues. The computationlly intensive operations from OpenCV that I use are:
cvCalcOpticalFlowPyrLK();
cvFindHomography(); //using RANSAC as filtering method
CvLevMarq solver //to refine camera pose decomposition by minimizing reprojection error.

Just realised that I did not recalibrate the camera! If no improvement in performance, I plan benchmarket these three operations on N900 and N9 and post results here.

Thanks for your reply,
Klen

NOTE: Recalibrating the camera did not affect the performance of the system.

[Klen]

Results of performance benchmark (average execution times of 15 frames). From results bellow it is obvious that the Feature tracking phase on N9 is performing much worse than in case of N900. The Feature Tracking code is representing only one function call :

NOTE: flags=CV_LKFLOW_PYR_A_READY // checked the API and did not find this flag as an option in openCV 2.3 cvCalcOpticalFlowPyrLK function call. Nonetheless, If I use flags=0 on N9 and N900 the performance difference does not get affected.

cvCalcOpticalFlowPyrLK(
       prev_grey, 							/* Previous Image */
       grey,								/* Current Image or candidate image */
       prev_pyramid[0], 						/* Image pyramid for previous image */
       pyramid[0],  							/* Image pyramid for current image */
       points[0],							/* Points to be tracked */
       points[1],							/* Their new position */
       featuresIn1,							/* Number of features */
       cvSize(SEARCH_WINDOW_SIZE, SEARCH_WINDOW_SIZE),	/* Window size */
	2,
        status,								/* Array of positive feature matches */
	errors,								/* Difference measure between observed and initial */
	cvTermCriteria(CV_TERMCRIT_ITER | CV_TERMCRIT_EPS , 5, 0.03),/* Iteration criteria for each pyramid level */
	flags);

Legend:
Feature C - Number of features tracked
Feature T - Feature tracking using Luca- Kanade Optical Flow (OpenCV stuff)
Pose E - Camera pose estimation : Homography decomposition (OpenCV stuff)
Rendering - Rendering of the openGL scene with the texture of a video stream.
Pool F - Pooling a frame from the camera.

N9

Feature C.: 20 | Feature T.: 54 | Pose E.: 27 | Homography E.: 20 | Pose R.: 5 | Init ST.: 221 |Init Map.: 222 | Sub. A.: 0 Un-P: 0| | Pool F.: 8.31 | Rendering: 5.31
Feature C.: 20 | Feature T.: 57 | Pose E.: 28 | Homography E.: 18 | Pose R.: 9 | Init ST.: 221 |Init Map.: 222 | Sub. A.: 0 Un-P: 0| | Pool F.: 7.75 | Rendering: 5.50
Feature C.: 20 | Feature T.: 58 | Pose E.: 28 | Homography E.: 22 | Pose R.: 5 | Init ST.: 221 |Init Map.: 222 | Sub. A.: 0 Un-P: 0| | Pool F.: 7.75 | Rendering: 5.56
Feature C.: 20 | Feature T.: 54 | Pose E.: 31 | Homography E.: 25 | Pose R.: 5 | Init ST.: 221 |Init Map.: 222 | Sub. A.: 0 Un-P: 0| | Pool F.: 9.31 | Rendering: 5.25

N900 600 MhZ
Feature C.: 20 | Feature T.: 18 | Pose E.: 17 | Homography E.: 12 | Pose R.: 4 | Init ST.: 221 |Init Map.: 222 | Sub. A.: 0 Un-P: 0| | Pool F.: 16.56| Rendering: 10.12
Feature C.: 20 | Feature T.: 12 | Pose E.: 25 | Homography E.: 20 | Pose R.: 2 | Init ST.: 221 |Init Map.: 222 | Sub. A.: 0 Un-P: 0| | Pool F.: 5.31| Rendering: 10.94
Feature C.: 20 | Feature T.: 19 | Pose E.: 17 | Homography E.: 13 | Pose R.: 3 | Init ST.: 221 |Init Map.: 222 | Sub. A.: 0 Un-P: 0| | Pool F.: 5.69| Rendering: 10.12
Feature C.: 20 | Feature T.: 18 | Pose E.: 18 | Homography E.: 15 | Pose R.: 2 | Init ST.: 221 |Init Map.: 222 | Sub. A.: 0 Un-P: 0| | Pool F.: 5.31| Rendering: 11.81
Feature C.: 20 | Feature T.: 19 | Pose E.: 26 | Homography E.: 18 | Pose R.: 8 | Init ST.: 221 |Init Map.: 222 | Sub. A.: 0 Un-P: 0| | Pool F.: 5.50| Rendering: 11.31

N900 clocked to 1000 MhZ
Feature C.: 20 | Feature T.: 8 | Pose E.: 14 | Homography E.: 9 | Pose R.: 4 | Init ST.: 149 |Init Map.: 150 | Sub. A.: 0 Un-P: 0| | Pool F.: 4.31| Rendering: 10.62
Feature C.: 20 | Feature T.: 8 | Pose E.: 16 | Homography E.: 9 | Pose R.: 5 | Init ST.: 149 |Init Map.: 150 | Sub. A.: 0 Un-P: 0| | Pool F.: 4.38| Rendering: 10.88
Feature C.: 20 | Feature T.: 9 | Pose E.: 19 | Homography E.: 17 | Pose R.: 1 | Init ST.: 149 |Init Map.: 150 | Sub. A.: 0 Un-P: 0| | Pool F.: 3.81| Rendering: 10.75
Feature C.: 19 | Feature T.: 14 | Pose E.: 17 | Homography E.: 13 | Pose R.: 3 | Init ST.: 149

Thanks for your help.

Klen

[ohookins]

Sorry, I wasn't receiving replies to this thread and didn't see your response. I'll look at your results and see if I can figure out what is going on.

[ohookins]

One thing I can also mention, unfortunately I compiled the current version of the lapack libraries on my laptop in the Scratchbox Harmattan environment so they may not be optimised for the actual N9/N950 processor. As far as I can tell, the sbrsh compile option is no longer available on Harmattan so I was limited in this choice.

I'll investigate further though as it may be possible this library is not that efficient, and the slow operations you are mentioning are math-based.