CMMFCoded: hardware or software?

[Rocco77]

Hi, I'm interested in knowing whether the codecs we usually adopt for PCM-AMR conversion or the equivalent for GSM, are software only or hardware accelerated codec.
I mean when we use:
CMMFCodec* codecpcm2AMR = CMMFCodec::NewL(TUid::Uid( 0x101FAF68 ));

From Nokia's document "Symbian_OS_Multimedia_Framework_And_Other_Multimedia_APIs_v1_0_en.pdf":

There are two types of codecs: MMF software codecs and hardware-accelerated (DSP-based) codecs.
Software codecs are part of the MMF controller framework and implemented as standard ECOM plug-ins. Use of DSP codecs has the advantage of increased processing speed while lowering the power consumption of a device. DSP-based codecs are commonly used for A/V encoding and decoding on OMAP processors.
Software codecs are derived from the CMMFCodec class and operate in their own threads. They convert source data in a certain FourCC coding type to a destination buffer of another FourCC coding type usually via CMMFDataPath or CMMFDataPathProxy. [...]

Support of hardware-accelerated codecs in the MMF relies upon the CMMFHWDevice interface. Loading and managing of hardware codecs is beyond the controller’s competency, and therefore it is left out to the lower layers, such as CMMFDevSound and CMMFHwDevice, to control them. Unlike software codecs, hardware-accelerated codecs provide an asynchronous interface for data transformation. Typically, they would be equipped with direct hooks to the hardware offering better performance and more flexibility to the third-party developers.

How do I understand if my device (for exemple a Nokia 6630) uses hardware acceleration for my encoding process?
Are there any possible queries?

Thank you for any information
best regards

Rocco

[groys]

----------------------------------------------------------------------------
Hi, I'm interested in knowing whether the codecs we usually adopt for PCM-AMR conversion or the equivalent for GSM, are software only or hardware accelerated codec.
I mean when we use:
CMMFCodec* codecpcm2AMR = CMMFCodec::NewL(TUid::Uid( 0x101FAF68 ));


These codecs are software codecs, they are very simple and dont need H/W acceleration. These codecs are created by the controller framework


----------------------------------------------------------------------------
Support of hardware-accelerated codecs in the MMF relies upon the CMMFHWDevice interface. Loading and managing of hardware codecs is beyond the controller’s competency, and therefore it is left out to the lower layers, such as CMMFDevSound and CMMFHwDevice, to control them.

All said above is true ...these codecs are also Ecom plugins but created by Devsound.

----------------------------------------------------------------------------
How do I understand if my device (for exemple a Nokia 6630) uses hardware acceleration for my encoding process?

Donno this myself....i dont work on nokia phones... plus i dont know where nokia gives details of its internal architecture.
If you find out nething do tell me

hope this helps
groys

[Rocco77]

Hi groys, thank you for your reply.
I agree that AMR coding is not an heavy task for current ARM processors but I also think that devices have dedicated DSP for audio processing in the telephony uplink/downlink communication.
For S60 (symbian) WCDMA phones where AMR is THE mandatory codec (that in one of its variants can generate EFR GSM too), I hoped we could access the same hardware for our audio processing.
At page 32 and 34 of this presentation
http://www.symbian.com/news/events/d...sm04public.pdf
or at page 8 of this one:
http://ncsp.forum.nokia.com/download/?asset_id=11731
there are schemes where we can clearly see a: CMMFAMRHWDevice MDF (media device framework) plugin that should be used.
But doubts remain about whether with our actual encoding via
CMMFCodec* codecpcm2AMR = CMMFCodec::NewL(TUid::Uid( 0x101FAF68 ));
we actually use and access it...


Rocco

[groys]

Hi,
the two documents you quoted were very interesting, i wasn't aware of them,
alas for the bits n pieces documentation of mmf.

I work on the techview platform with the devkit and not on series 60. I am sorry i mistook amr codec to be very simple, in the reference implementation in the devkit there is no software amr codec given.

1. Hence i think when u instantiate a codec as u have specified
CMMFCodec* codecpcm2AMR = CMMFCodec::NewL(TUid::Uid( 0x101FAF68 ));
(is this how u actually do it?)
the hardware accelerated codec must be getting used.

2. "I hoped we could access the same hardware for our audio processing."
why are u worried about this? Are there multiple ways in which you can do your audio processing or do you always have to use the codec with above uid? If there is only this way i think u can write your application and plugin and let the framework do the work.

do reply the conversation is getting interesting

[Rocco77]

Hi,
the two documents you quoted were very interesting, i wasn't aware of them,
alas for the bits n pieces documentation of mmf.

I'm pleased to hear this :-)

1. Hence i think when u instantiate a codec as u have specified
CMMFCodec* codecpcm2AMR = CMMFCodec::NewL(TUid::Uid( 0x101FAF68 ));
(is this how u actually do it?)
the hardware accelerated codec must be getting used.

This is the way I do, of course.
I doubt the HW codec is used because in another document there are hints about a way to get "direct" AMR input/output streaming (as you can see I tryed to get as much documentation as a non Platinum and non Nokia PRO member can).
http://www3.symbian.com/faq.nsf/0/89...6?OpenDocument
This way relies on the presence of "Media Device Plugins" exactly as the CMMFAMRHWDevice is.
I tried this on my 6630. It works but not as expected. I don't get "KErrNotSupported" errors but the MaiscBufferCopied callbacks in this case behave weirdly and I have problems to send the generated frames to sockets.
The same method used for GSM encoding, on my 6630 doesn't work at all: "not supported" error!
This gives some clues. Surely the "direct" way that acquires in AMR straight from DevSound uses the HW encoder but I'm not able to manage it to work correctly.The "old" working way that works via the CMMFCodec probably IS still software.I'd bet on it.
I'd like to find someone pointing me to some explicit document but also to the right method to use the "direct" way.

Are there multiple ways in which you can do your audio processing or do you always have to use the codec with above uid? If there is only this way i think u can write your application and plugin and let the framework do the work.

I actually use the CMMFCodec way.
A HW one would be optimal for processor load (obviously) and I wonder if I'm already using it :-)

do reply the conversation is getting interesting

Here it is ;-)

Rocco

[groys]

Hi,

here are a few paragraphs from the devkit documentation,
1. MMF Codec Plugin How-To Document

Within MMF there are two ways in which a codec can be used:
1. Codecs implemented within the source and sink itself. The source and sink is a software abstraction of real hardware that can source or sink multimedia data (see ref R3). As such the hardware itself may support more than one data type which can be set via the MDataSink/MDataSource API (ref R3). This would typically be implemented by a codec contained within a CMMFHwDevice (ref R4). Typically the codecs will be available within the source and sink and the source and sink will determine which codec to load up.
2. Codecs implemented as a CMMFCodec plugin. These are codecs that are used when it is not possible for the source and sink data to send data to each other using the same data type. A CMMFCodec is instantiated in order to code/decode the source data into a data type that is acceptable to the sink. It is this type of codec that is described by this document.

2. MMF HwDevice Plugin using Codec Wrapper How-To Document

An MMF hardware device is an abstraction of a hardware accelerated audio codec as described in ref[R3]. An MMF hardware device provides the interface used by DevSound [R4] to the audio hardware.

From this there is a hint that if u were using an h/w codec u would be using cmmfhwdevice instead of cmmfcodec, but this is only an hint, and knowing mmf this is not much to go by. Nokia might be doing nething internally.

I tried this on my 6630. It works but not as expected. I don't get "KErrNotSupported" errors but the MaiscBufferCopied callbacks in this case behave weirdly and I have problems to send the generated frames to sockets.
The same method used for GSM encoding, on my 6630 doesn't work at all: "not supported" error!
This gives some clues. Surely the "direct" way that acquires in AMR straight from DevSound uses the HW encoder but I'm not able to manage it to work correctly.The "old" working way that works via the CMMFCodec probably IS still software.I'd bet on it.

I have stopped betting on nething related to mmf cant comment on the phone really because i am not working on it.

(as you can see I tryed to get as much documentation as a non Platinum and non Nokia PRO member can).

yup u r doing great.

I donno what is the way to find, what exactly nokia is doing inside its code. MMF in itself is a very difficult thing to document due to all its complexities. By the way do u know whether video playing on nokia phones is also via mmf?

regards
groys

[Rocco77]

Thank you for the excerpts groys!

For your question about video, I really don't know.
I'm working only with audio features and that is really enough difficoult...

I don't know how is the nokia implementation and if it is on top of DevVideo...
From "Datasheet of Series 60 Developer Platform SDK's for Symbian OS, For C++"
http://sw.nokia.com/id/cc78018c-b9ba...2-180ac61a0ae6
it seems that only starting from s60 3rd edition several video API were introduced:
MDF DevVideoRecord API
MDF DevVideoRecord Proxy API
MDF Video Recorder Media Device API
MDF Video Encoder Media Device API
MDF Video Pre-Processor Media Device API
MDF Clock Source API
MDF DevVideo General API

I hope this helps...

Rocco

[raghu_an]

Hi,
For using PCM to AMR Hw accelerated Codec you need to provide some configuration parameter which are codec(AMR) specific (Such as DTX on/off, bit-rate, optional header).
and also you need to provide characteristic of input PCM stream(frequency and channels). If you provide these then there should be no problem(theoretically) for using DevSound/CMMFAMRHwDevice. Can you check you are providing these info???
I am not aware about the API's present in S60 for providing theses parameters!!!!but for sure there will be some API....

BR,
RAGHA

[RyanSantos]

To create or initialize, where "this" is derived from MDevSoundObserver and you have included mmf\common\mmffourcc.h if you get compile errors or if non of your system header files have included it yet

TInt initError;

iMMFDevSound = CMMFDevSound::NewL();

#ifdef PCMAUDIO
TFourCC fourCC(KMMFFourCCCodePCM16);
#else //PCMAUDIO
TFourCC fourCC(KMMFFourCCCodeAMRNB);
#endif //PCMAUDIO
TRAP(initError, iMMFDevSound->InitializeL(*this, fourCC, EMMFStateRecording));
return initError;

I defined PCMAUDIO somewhere to switch between PCM16 and AMR.

It seems that if you use KMMFFourCCCodeAMRNB, the data/sound gotten from the phone mic which comes in after you do a iMMFDevSound->RecordInitL() and MDevSoundObserver::BufferToBeEmptied is called, the stream received is automatically converted to AMRNB. I tried using FourCC codes to create an output stream. I have not been successful as of the moment. I was able to do

Input ===> PCM16 ===> AMR ===> PCM16 ====> Output speaker

and

Input ===> PCM16 ====> Output speaker

I am now experimenting other FourCC codes like PCM8 but am not successful as of the moment.