QPainter performance in Harmattan

[lostdev]

Hi all,

I wrote QML-based image editor that can run both on S^3 and Harmattan and noticed that QPainter performance in Harmattan (tested on my N950) is noticeably lower than in S^3 (tested on my N8), even with latest firmware (1.2011.34-2 - Beta 2). For example, the following operation:

painter->drawImage(option->exposedRect, image, option->exposedRect);

performs at ~15 fps on Harmattan, however on N8 it performs at least 60 fps. I tried the following quirks:

1. painter->drawImage(QPoint&, QImage&) to avoid clipping. There was no effect.
2. Conversion from QImage to QPixmap and then painter->drawPixmap(QRect&, QPixmap&, QRect&) to avoid non-optimized (comparing to QPixmap) QImage rendering. No effect as well.
3. Conversion from QImage to QPixmap and then painter->drawPixmap(QPoint&, QPixmap&) to avoid clipping and non-optimized QImage rendering. Still no effect.
4. Call of setViewport(new QGLWidget()) for QmlApplicationViewer to make QPainter operations using OpenGL. It does work in general (QGLWidget object was valid, rendering was ok, no artifacts), but performance was the same.
5. QApplication::setGraphicsSystem("raster"). It was really fast in rendering of my own object, but all QML objects on screen was broken, and application output was full of messages regarding unsupported graphics system.
6. QApplication::setGraphicsSystem("opengl"). QML objects on screen was broken as well, and "unsupported graphics system" errors was in place.

Should I try something else to achieve the same performance of QPainter as on N8? In Release Notes for latest N950 firmware I have read that "Performance of all use cases has not been optimized". Is my problem related to this, and in final version performance will be improved?

Thanks in advance.

[gnuton]

Did you try to use "meego" graphics system [1]?
That's the one you should use on MeeGo and which is actually used by default by MeeGo apps.

[1] https://qt.gitorious.org/~mdk/qt/mdk...ssystems/meego

[lostdev]

Yes, I tried it. When I call QApplication::setGraphicsSystem("meego"), performance is the same as without this call (i.e. low), but there is a bunch of messages like this in application output:

file:///usr/lib/qt4/imports/Qt/labs/components/native/Button.qml:77:5: QML BorderImage: Failed to get image from provider: image://theme/meegotouch-button-background-horizontal-center
Valid eglHandle received but not running with meego compatible graphicssystem.

I'm a little confused: isn't a "meego" graphics system a default system on N950? At least I see the following message in application output on startup when running without QApplication::setGraphicsSystem() at all:

Successfully resolved MeeGo graphics system: /usr/lib/qt4/plugins/graphicssystems/libqmeegographicssystem.so

[lostdev]

It seems that I found a bugreport that is similar to my case:

http://bugreports.qt.nokia.com/browse/QTBUG-20734

I don't know what is the max texture size on N9, however, when I call painter->drawImage() for QImage that is significantly bigger than the N9 screen size (for example, four times bigger), I notice a very slow performance. However, when I call painter->drawImage() for QImage that fits in screen, there is no performance penalty. I should note also that exposedRect() in all cases does not exceed screen size, so region to draw is the same in both cases. Can I promote this bug somehow to have a chance to test fix for it (if it is really a bug)?

[gnuton]

Looking at the MeeGo graphics system plugin [1] I can see those lines:
QEglProperties *properties = new QEglProperties();
properties->setValue(EGL_FIXED_WIDTH_NOK, width);
properties->setValue(EGL_FIXED_HEIGHT_NOK, height);

Where those are defined here [2] as:
#define EGL_FIXED_WIDTH_NOK 0x30DB
#define EGL_FIXED_HEIGHT_NOK 0x30DC

So, maybe 12000x12000 is maximum "allowed" size to not fall into performance penalty.

[1] https://qt.gitorious.org/+grym/qt/gr...hicssystem.cpp

[2] http://svn.netlabs.org/repos/qt4/tru...goextensions.h

[lostdev]

Looking at the MeeGo graphics system plugin [1] I can see those lines:
QEglProperties *properties = new QEglProperties();
properties->setValue(EGL_FIXED_WIDTH_NOK, width);
properties->setValue(EGL_FIXED_HEIGHT_NOK, height);

Where those are defined here [2] as:
#define EGL_FIXED_WIDTH_NOK 0x30DB
#define EGL_FIXED_HEIGHT_NOK 0x30DC

So, maybe 12000x12000 is maximum "allowed" size to not fall into performance penalty.

[1] https://qt.gitorious.org/+grym/qt/gr...hicssystem.cpp

[2] http://svn.netlabs.org/repos/qt4/tru...goextensions.h

OK, may be I'm wrong with that... However performance penalty is really exists. I wrapped call to drawImage() this way:

qint64 msec = QDateTime::currentMSecsSinceEpoch();
painter->drawImage(option->exposedRect, ScaledCurrentImage, option->exposedRect);
qDebug(QString("Image: %1 %2, exposedRect: %3 %4 %5 %6, drawImage time: %7").arg(ScaledCurrentImage.width()).arg(ScaledCurrentImage.height()).arg(option->exposedRect.x()).arg(option->exposedRect.y()).arg(option->exposedRect.width()).arg(option->exposedRect.height()).arg(QDateTime::currentMSecsSinceEpoch() - msec).toAscii());

On small QImage the following results are observed:

Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 10
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 10
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 18
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 12
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 14
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 14

But on scaled (bigger) version of the same QImage we see the following:

Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 36
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 45
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 42
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 43
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 42
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 41
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 40

We have about 3x performance penalty when drawing almost the same exposedRect! It seems, however, that problem lies in some preparations that Qt makes before painting an image, because I change QImage just before every draw. If I comment out this in my code, the results will be as follows:

Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 13
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 0
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 0
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 0
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 0
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 0
Image: 674 592, exposedRect: 0 0 480 592, drawImage time: 0

Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 45
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 0
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 0
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 1
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 0
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 1
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 0
Image: 1348 1184, exposedRect: 0 0 480 803, drawImage time: 0

As you can see, first call takes the same time, but consequent calls are take almost zero time. Can I somehow "help" Qt to prepare images for drawing? I tried to alter QImage format, but there was no result at all.

[lostdev]

It seems that the following quirk:

painter->drawImage(option->exposedRect, ScaledCurrentImage.copy(option->exposedRect.toRect()));

helps - in this case performance almost doesn't depend on size of original image, and depends only on exposedRect size. Just curious, is this a normal behaviour? It seems that on N8 QPainter::drawImage() implemented by some other way, because it doesn't require such quirks to work fast.

[gnuton]

Hi,
QImage::copy crops the image. To crop an image is way faster than scaling... but it's not the same thing.

In your previous post you said that painting the same image lowers the drawing time to 0.
Well, if you got that value painting the SAME image, I think you result is not valid
because the internal buffer is not updated. That explains the 0. Which is REALLY low!

If you want to evaluate better and having some basis to understand and improve the performance of drawing on harmattan
you should analyze it using Valngrid.

[lostdev]

I do not replace scaling by cropping. I say that call to

painter->drawImage(option->exposedRect, ScaledCurrentImage.copy(option->exposedRect.toRect()));

is SIGNIFICANTLY faster than call to

painter->drawImage(option->exposedRect, ScaledCurrentImage, option->exposedRect);

with the same image, if the image is new at every call. There is no scaling at all. It seems that drawImage() have no advantage of exposedRect in the second case, i.e the whole image is processed, and not only the part covered by exposedRect. I can give you a test case, but I could't attach files here.

P.S. I guess that Qt internally draws QImages on Harmattan as gl textures, which, of course, requires some preparation. If this is really the case, I can understand ignoring of exposedRect in my case - there is no reason to crop a texture, if there is a probability that other parts of the same texture can be drawn later. If I'm right, it would be good to explain this in the documentation, because such unexpected performance penalties can look really strange.

[bokovoy]

Some observations. Yes, GL textures are used to draw pictures (or parts of them). That means there is upload to happen which is slightly slower than simple memory copy due to synchronization reasons between kernel driver and SGX hardware. Actually, if you are not playing special tricks like working with physically contiguous memory provided by framebuffer driver (100% not your case), drawing each new image after you have changed QImage will incur at least one copy from QImage::bits() to OpenGL's internal buffer that is then locked until the image is transferred to the SGX side. The transfer is actually by doing a copy into RAM-locked segment of a memory allocated by the SGX driver. Afterwards and before the texture is drawn, SGX does some transformation of the texture for faster transformation processing (that means there will be yet another copy most likely).

As long as original QImage not changed, once it is drawn to the screen and application stays in foreground, the corresponding texture is cached and no additional transfer is required. This is the fastest way. If application goes to a switcher, its GL context is destroyed and all associated texture caches go away as well.

Once you start changing original QImage, associated textures (if any) are evicted from the texture cache that Qt maintains. End result is that you go through texture uploading routine all the way each time you change QImage's content.

More to that, images greater than the texture size SGX is able to support (2048x2048) are uploaded in chunks. It means they'll be first processed by Qt and later uploaded as separate textures. In worst case it may mean additional memory allocations if referencing partial sub-region is not possible -- like with scaling that is not multiple of 8, for example (though my memory might serve a fail here on alignment).

In short, you need to use QPixmap if you do have content that you are absolutely sure to not change during lifetime of your scenes. If you have something changeable, you need to play tricks -- either using QImage way or following special APIs for "live textures" that Harmattan gives as Qt extension. Your application will not be portable then but you'll be able to exploit most of device performance then.

[bokovoy]

Also, from a perspective of Harmattan's image editor that is part of the Gallery application. When we do edits, we maintain a smaller, less than max texture size image that is used for preview, and apply editing operations to both original (potentially large) image, in background, and to this preview image, in foreground. This significantly improves usability and also gives you good compromise on what user will be able to see anyway and what will be available. You may look at Quill code to see how it is managed, albeit it only has "backend" code there and not the UI parts. https://maemo.gitorious.org/meego-image-editor -- there is one simple Qt GUI as well, that shows how the backend parts are integrated in real application. Not QML, but real Qt widgets, so runs well on desktop without any Harmattan APIs.

[lostdev]

Some observations. Yes, GL textures are used to draw pictures (or parts of them). That means there is upload to happen which is slightly slower than simple memory copy due to synchronization reasons between kernel driver and SGX hardware. Actually, if you are not playing special tricks like working with physically contiguous memory provided by framebuffer driver (100% not your case), drawing each new image after you have changed QImage will incur at least one copy from QImage::bits() to OpenGL's internal buffer that is then locked until the image is transferred to the SGX side. The transfer is actually by doing a copy into RAM-locked segment of a memory allocated by the SGX driver. Afterwards and before the texture is drawn, SGX does some transformation of the texture for faster transformation processing (that means there will be yet another copy most likely).

As long as original QImage not changed, once it is drawn to the screen and application stays in foreground, the corresponding texture is cached and no additional transfer is required. This is the fastest way. If application goes to a switcher, its GL context is destroyed and all associated texture caches go away as well.

Once you start changing original QImage, associated textures (if any) are evicted from the texture cache that Qt maintains. End result is that you go through texture uploading routine all the way each time you change QImage's content.

More to that, images greater than the texture size SGX is able to support (2048x2048) are uploaded in chunks. It means they'll be first processed by Qt and later uploaded as separate textures. In worst case it may mean additional memory allocations if referencing partial sub-region is not possible -- like with scaling that is not multiple of 8, for example (though my memory might serve a fail here on alignment).

In short, you need to use QPixmap if you do have content that you are absolutely sure to not change during lifetime of your scenes. If you have something changeable, you need to play tricks -- either using QImage way or following special APIs for "live textures" that Harmattan gives as Qt extension. Your application will not be portable then but you'll be able to exploit most of device performance then.

It's pretty much what I suspected. Thank you for explanations.

Also, from a perspective of Harmattan's image editor that is part of the Gallery application. When we do edits, we maintain a smaller, less than max texture size image that is used for preview, and apply editing operations to both original (potentially large) image, in background, and to this preview image, in foreground. This significantly improves usability and also gives you good compromise on what user will be able to see anyway and what will be available. You may look at Quill code to see how it is managed, albeit it only has "backend" code there and not the UI parts. https://maemo.gitorious.org/meego-image-editor -- there is one simple Qt GUI as well, that shows how the backend parts are integrated in real application. Not QML, but real Qt widgets, so runs well on desktop without any Harmattan APIs.

I understand this way for speed up of image editing process, but this will not work in my case, because I need to give user an ability to edit small parts of the original image in realtime, so I need to allow user to edit zoomed (with up to 4x zoom) image in realtime. For now, I use image cropping to reduce image size that is redrawn on each image change - to avoid load of large textures to OpenGL.

[bokovoy]

I understand this way for speed up of image editing process, but this will not work in my case, because I need to give user an ability to edit small parts of the original image in realtime, so I need to allow user to edit zoomed (with up to 4x zoom) image in realtime. For now, I use image cropping to reduce image size that is redrawn on each image change - to avoid load of large textures to OpenGL.

If you are able to limit the area that is affected by edits, copy out that part and do fast edits on the copy. Later, merge it onto the original image. You may need to maintain fast mapping of (sub-)fragments so that the image is never used as a whole during edits/viewing but gathered together from edited fragments when saved in background. This is not an easy job but that's exactly what you'll need to do for limitless resolution editing anyway. On mobile all the tricks are worth to use.

[lostdev]

If you are able to limit the area that is affected by edits, copy out that part and do fast edits on the copy. Later, merge it onto the original image. You may need to maintain fast mapping of (sub-)fragments so that the image is never used as a whole during edits/viewing but gathered together from edited fragments when saved in background. This is not an easy job but that's exactly what you'll need to do for limitless resolution editing anyway. On mobile all the tricks are worth to use.

I use somewhat like this now