Terrible performance of the J2ME sound subsystem

[Utumno]

Fact: a call to Manager.playTone() consumes about 1/3 of CPU on a N97 mini. Playing a ToneSequence, WAV or - God forbid - MP3 takes even more (although surprisingly by not that much)

Don't take my word for it, try this simple MIDlet:

1. Class SoundMIDlet

import javax.microedition.midlet.*;
import javax.microedition.lcdui.*;

public class SoundMIDlet extends MIDlet
  {
  private SoundCanvas mCanvas = null;

  public void startApp()
    {
    mCanvas = new SoundCanvas();
    Display.getDisplay(this).setCurrent(mCanvas);
    }

  public void pauseApp()  { }
  public void destroyApp(boolean uncond)  { }
  }

2. Class SoundCanvas:

import javax.microedition.lcdui.*;
import javax.microedition.media.Manager;

public class SoundCanvas extends Canvas implements Runnable
  {
  private Thread mSoundThrd= null;
  private boolean playSound = false;

  protected  void pointerPressed (int x, int y)
    {
    if( mSoundThrd==null )
      {
      mSoundThrd = new Thread(this);
      mSoundThrd.start();
      }

    playSound= !playSound;
    }

  public void paint( Graphics g)  { }

  public void run()
    {
    while(true)
      {
      if( playSound )
        {
        try
          {
          Manager.playTone(67, 500, 20);
          }
        catch( Exception ex1 ) {}
        }

      try
        {
        Thread.sleep(200);
        }
      catch(InterruptedException ex) {}
      }
    }
}

As you can see, you can toggle the sound by touching the screen. Now run the EcmtAgent in your phone and connect a Diagnostic Screen to it. You'll notice that when playSound==true, the MIDlet uses about 30-35% of N97 mini CPU ( a 434 MHz ARM11 ), and when playSound==false, the MIDlet takes exactly 0%.

So it really looks like the innocent-looking call to playTone() consumes 434/3 = 140MHz !! Anybody can explain that?

[Utumno]

I've got some more findings. Consider this MIDlet:

1. class SoundCanvas ( SoundMIDlet is the same like in the previous example )

import javax.microedition.lcdui.*;
import javax.microedition.media.Manager;

public class SoundCanvas extends Canvas implements Runnable
  {
  private Thread mSoundThrd= null;

  private int duration = 100;
  private int sleeping = 100;
  private int loudness = 20;

  private int height,width;
  private long time=0;

  public SoundCanvas()
    {
    this.setFullScreenMode(true);

    mSoundThrd = new Thread(this);
    mSoundThrd.start();

    height = getHeight();
    width  = getWidth();
    }

  protected  void pointerPressed (int x, int y)
    {
    if( x<width/3 )
      {
      if( y<height/2 ) duration+=10;
      else if( duration>0 ) duration-=10;
      }
    else if( x<2*width/3 )
      {
      if( y<height/2 ) { if(loudness<=90 ) loudness+=10; }
      else if( loudness>0 ) loudness-=10;
      }
    else
      {
      if( y<height/2 ) sleeping+=100;
      else if( sleeping>0 ) sleeping-=100;
      }

    repaint();
    serviceRepaints();
    }

  public void paint(Graphics g)  
    {
    g.setColor(0,0,0);
    g.fillRect(0, 0, width, height );
    g.setColor(255,255,255);
    g.drawString( "Duration", 1*width/6, height/2, Graphics.TOP|Graphics.HCENTER );
    g.drawString( "Loudness", 3*width/6, height/2, Graphics.TOP|Graphics.HCENTER );
    g.drawString( "Sleeping", 5*width/6, height/2, Graphics.TOP|Graphics.HCENTER );

    g.drawString( ""+duration, 1*width/6, height/2+60, Graphics.TOP|Graphics.HCENTER );
    g.drawString( ""+loudness, 3*width/6, height/2+60, Graphics.TOP|Graphics.HCENTER );
    g.drawString( ""+sleeping, 5*width/6, height/2+60, Graphics.TOP|Graphics.HCENTER );

    g.drawString(""+time, width/2, height/2 +120, Graphics.TOP|Graphics.HCENTER );
    }

  public void run()
    {
    long tmptime;

    while(true)
      {
      tmptime = System.currentTimeMillis();

      if( duration>0 )
        {
        try { Manager.playTone( 67, duration, loudness); }
        catch( Exception ex1 ) {}
        }
      if( sleeping>0 )
        {
        try { Thread.sleep( sleeping ); }
        catch(InterruptedException ex) {}
        }

      time = System.currentTimeMillis() - tmptime;
      }
    }
  }

Using the above, we can ( by touching appropriate sections of the screen ) adjust how long the sound should be, how loud it should be, and how long we should sleep in between calls to playTone.

Findings:

1) CPU usage does NOT depend on:
- how long the sound is supposed to play
- how loud it is ( even when loudness==0, we still incur the 30-35% penalty )
- if the phone is in 'silent' mode ( even in this mode it still takes 30-35% of CPU )
- surprisingly, it also does not depend if we call a 'playTone' or we are playing a tone sequence, or we play a WAV file. In all those 3 cases ( I did not check MP3 or MIDI ) CPU usage is very similar.
2) CPU usage DOES depend on:
- how often we are calling playTone() , i.e. on the value of 'sleeping' variable above.

and so, if for example duration==100 and sleeping==200 then CPU==~33%
if duration==500 and sleeping==1000 then CPU==~20%
if duration==1000 and sleeping==2000 then CPU jumps between 3 and 10%
if duration==2000 and sleeping==4000 then CPU jumps between 0 and 10%, more precisely it stays at roughly 2.5 seconds at 10%, then for the remaining part of the cycle it goes down to 0%

Surprisingly, if duration=10 and sleeping==4000 then still CPU== 0-10%

So, it is clear that calling a 'playTone' ( or 'player.start() for that matter ) kinda makes the system very busy for about 2.5 seconds, regardless of what is the type of the sound, how long it is supposed to play, how loud, etc.
Also, another important point: in the Diagnostic Screen ->Task Manager->Applications tab, please take a look at individual processes running on your phone when you're playing the sound. You will notice that it is actually not the MIDlet itself that consumes the 30-35% of CPU - it consumes only about 10%, the rest are a lot of other processes suddenly waking up and consuming a few percent each.

I would love if a Nokia engineer would enlighten me on what is going on here! This behavior is making it very hard to develop any faster game. N97 mini w/ 12.0.110 here.

[spectra]

I think its because of the whole system. I mean PlayTone and other sound stuff works not directly with device. it goes from virtual environment.
The process seems something like this(in a easy view):
Checking sound type like midi or stream (wav, mp3) or tone (which works the same like midi and I think its bad for game\software dev).
If you want to play midi, the system loads a sound bank(its about 100 little wav files) to memory and then plays your tune using it.
if you want to play tone, the system must use something closer to hardware (like this "beep" which was in these old DOS days, or something like game boy sound system) BUT it uses the same midi sound bank and picks the one which was set to play tones and then plays your tone commands. Thats why my project will use stream types. As I remember working with stream type is better than midi.

ps: sorry for my terrible english

[Utumno]

Hmm... Are you saying that MIDI, playTone and Tone Sequences are in reality still WAVes ? And to add insult to injury, those WAVes are loaded and unloaded every time I call playTone?

[spectra]

This is my opinion for now. It will be good if someone who knows about this subject better than we do, will tell us the truth of this realization. I dont see other options for now. I have experience in developing for windows, dos and game boy, so I think this is the problem for tone sounds (playTone uses the same "midi engine" to play tone but using only one instrument from midi sound bank).

By the way windows has this "midi" sound bank too (GM bank).

[Utumno]

You will notice that it is actually not the MIDlet itself that consumes the 30-35% of CPU - it consumes only about 10%, the rest are a lot of other processes suddenly waking up and consuming a few percent each.

I am stubborn with this and by now I am pretty sure that the real problem lies in the system, i.e. it really seems like playing a sound makes the system go berserk and wake up a few random processes ( like Calendar ) that have absolutely nothing to do with sounds. Is Nokia interested?

[spectra]

eh, my game got 20 mb of sound files. ambient sound, effects and music... the music is a chiptunes rendered to mp3... ah if only there was a special hardware chip to play these bleep blops... it was perfect for 50% of games