Getting bigger Camera Viewfinder Images / Getting smaller Camera captured Images

[JohannesS]

Hi all,

I'm developing a Symbian app that uses the camera of the phone to capture an image and uses image processing to decide if the quality of the image is good enough to match certain criteria. The aim was to do this without making the user press a "capture image" key every time.
In order to get the object to be photographed to be sharp, only a small rectangle in the center of the view is taken to be processed, the rest of the image data (>= 50% of the image data) can be ignored.
As basic application, I took the Camera Example v3.0, my test device is a Nokia N79 (S60 3rd, FP2).

So much for the project, now the problem:
The unsolved problem is the resolution of the image that is to be processed. I tried two different approaches and both failed:

1. Using the viewfinder data
   I could read in the forum that the viewfinder only produces images that have at max the same resolution as the display:
   http://discussion.forum.nokia.com/fo...d.php?t=115481
   It seems to me that when a bigger viewfinder resolution is assigned, for example
   

   iViewFinderSize = TSize(1280,960);
   //...
   TRAPD(err2,iCameraWrapper->StartViewFinderL(iViewFinderSize));

   the viewfinder image is interpolated from i.e. 240x320 to the desired size. As the image processing in my app needs a much bigger image than 240x320 pixels the viewfinder can't be used for this...
   Am I wrong? Is there a way to get viewfinder images in higher resolutions that are not interpolated?
   
   
2. Capturing images
   To get bigger images I tried capturing an image with my desired resolution. In order not to get to much data my first thought was to use the the format CCamera::EFormatMonochrome, 8bit grayscale. But CCameraEngine::PrepareL leaves with error -5, meaning that this mode is not supported by the phone.
   Anybody have any ideas why the mode is not supported?
   I had no choice so I used one of the CFbsBitmap modes, CCamera::EFormatFbsBitmapColor4K.
   However now, no matter how small the image size is set with for example
   

   iCaptureSize = TSize(640,480);
   //...
   TRAPD(err,iCameraWrapper->PrepareL(iCaptureSize, CCamera::EFormatFbsBitmapColor4K));

   it always takes like 5s until the callback Method MceoCapturedBitmapReady() is called.
   Is there a way to find out how the camera sensor works? Does the camera API get the full 5 Megapixels image as a jpeg, decodes it and scales it down to the desired resolution? Could you explain to me why the bitmap capturing process takes so long?
   Is there a way to configure the camera's sensor to get take smaller resolution pictures? Similar to a digital camera with 10 megapixels that can be configured to take "Email-size" pictures in resolution 640x480.

As you can see both approaches failed:
- viewfinder: image resolution is too small
- capturing: time to process capturing takes too long

Sorry for writing a small novel to describe the problem but it seems kind of complex to me and I don't have a clue how to go on.
Any help is greatly appreciated. Hope someone can help me
Thanks in advance!

JS

[wizard_hu_]

The viewfinder stuff is intentional, providing screen-sized images is its purpose.
For capturing, TCameraInfo::iImageFormatsSupported actually tells you what is supported and what is not, so you do not have to rely on trial-and-error.
Though that 5s sounds a bit long, it is true that CaptureImage is not that fast. Depending on your use case, you can try capturing video instead - the good news is that it will be fast (you can expect 10-15 or even more FPS with VGA resolution), the bad news is that it might not be that sharp (focusing may happen to be amongst the factors slowing down CaptureImage), and the frames will be in some kind of YUV format (TCameraInfo::iVideoFrameFormatsSupported can tell you that). However if you do not need the colours (as your question suggests), you can simply ignore the UV part, and use Y as 8-bit grayscale.

[JohannesS]

Thanks for your fast answer, wizard_hu_!
The idea of using video is a really good one.
The problem is that the resolution of the video is 640x480 and in case of my N79 can't be higher. Or can it?

In my app, however, the image resolution needs to be higher. The image processing algorithms used are optimized for 640x480 images. The object to be photographed can only be focused by the camera if there is a certain distance between the camera and the object. So the actual resolution of the image needs to be higher than 640x480. My description sounds a bit odd, I hope you understand what I mean.
So my approach was to draw a rectangle on the screen and make the user "aim" the camera in a way that the object to be photographed is within this rectangle. Then the object is focused and the image processing is applied to the small rectangle only (which prefarably is in resolution 640x480) and not to the whole picture, to get more efficient processing.
So depending on the focal range of the camera the image needs to be for example (2*640)x(2*480) pixels.

Do you think that this is too much data to process in <= 1s?
Is there any other way than capturing the image and throwing away half of the images data?

I also tried your hint with TCameraInfo::iImageFormatsSupported and it tells me of course that only the CFbsBitmap and JPEG modes are supported. Does that depend on the camera or the camera API? It would have been great to get a 8bit grayscale picture, because it would have been nearly 70% less data to process.

[wizard_hu_]

Unfortunately I have no better suggestion. Video capturing is probably not supported for higher resolutions.
I do not really understand what you exactly mean on having an algorithm optimized to 640x480, but if it is a restriction arising from the mathematical background of your code, you could try simply doubling the pixels from that target rectangle.

[janza]

Some answers:

Generally view finder is not available in sizes bigger than screen. In some devices it might be, but there is no way you can know which devices and what size. So if you need bigger images than screen size - don't use view finder

I am not familiar with the CameraWrapper class you are using. But what comes to ECam (CCamera class): as wizard_hu_ mentioned there are methods to interrogate which image sizes and formats are supported. Generally the device HW is optimized to produce JPEG format images. All the image sizes might not be supported in bitmap format and the capture can involve decoding the actual JPEG format image that was captured. Once again this is HW dependant so there is no single correct answer.

the image processing is applied to the small rectangle only (which prefarably is in resolution 640x480) and not to the whole picture, to get more efficient processing.

In case you are only going to use the center part of the image have you considered applying digital zoom? That should do the trick.

I also tried your hint with TCameraInfo::iImageFormatsSupported and it tells me of course that only the CFbsBitmap and JPEG modes are supported. Does that depend on the camera or the camera API?

I am not quite sure what you refer to with "camera"? Basically the image data can be converted into what ever format, but the camera SW in the device will only provide the two aforementioned formats. In case you want something else you need to do the conversion by yourself.

It would have been great to get a 8bit grayscale picture, because it would have been nearly 70% less data to process.

As wizard_hu_ mentioned, you can use the YUV format video frames as grayscale image by just discarding the U and V components.

[JohannesS]

First of all I want to thank all of you for your answers!

We solved our problem, using a combination of your suggestions. Our image processing is now done in a kind of a two-phase processing:

1. First a kind of preselection is done on a 320x240 viewfinder picture. This is possible because we discovered that the first part of the image processing algorithm is stable enough to be performed on small res pictures. The small resolution in combination with digital zoom reduces the amount of data to be processed in a way that the first phase of the image processing can be performed with 15 frames per second.
2. The second phase is only performed when the image succeeds the test in phase one. To get higher resolution pictures in phase two, an image of 640x480 pixels is captured and the more complicated and more demanding part of the image processing algorithm is performed on this image.

The second phase takes about 1s, but that's no problem because the preselection in phase one discards most of the unusable frames and it works with 15fps, so the application as a whole matches certain "real time" criteria and the user's patience isn't taxed to much.

Thanks again,

JS