OutOfMemoryException

[thechameleon]

hello, i need to do some crypto ops on files.

however, trying to do it on a file as large as 700MB causes this Exception (at least on the emulator). i know, it's a really ridiculous size for a phone.

how much is the maximum Java/S60 can handle?

so, wat is the limit on each byte Array, and wat is the total limit?

thanks.

[thechameleon]

Happy day Thechameleon ! Files of crypto opts subjects give full idea of size Java S60. Thanks. Yours Deep

uh... wat do u mean?

right now, i havent even gotten to the crypto part. the emulator cant create a byte Array of 700MB. im asking the max size the S60 emulator / phones can handle.

[grahamhughes]

The ultimate limit for a byte array is 2Gb (because the index is an int). One or two older devices have small limits (Siemens S55/SL55 (16k), Panasonic X60/X400 (64k)), but these devices are so old that you can forget them.

Of course, you cannot create an array that exceeds the size of the heap. Even on a Series 60, this is going to be in the region of 10Mb (depending on model). I'm not aware of any device with any operating system that has 700Mb of memory.

Graham.

[thechameleon]

The ultimate limit for a byte array is 2Gb (because the index is an int). One or two older devices have small limits (Siemens S55/SL55 (16k), Panasonic X60/X400 (64k)), but these devices are so old that you can forget them.

Of course, you cannot create an array that exceeds the size of the heap. Even on a Series 60, this is going to be in the region of 10Mb (depending on model). I'm not aware of any device with any operating system that has 700Mb of memory.

Graham.

is there a way that i can find the heap size?

the N95 8GB has an internal memory of 128MB. but how much is actually acting like the memory and not the storage?

[grahamhughes]

Experimentally... something like this...

public static int howMuchHeapIsThere() {
    final int GRANULARITY = 1024 * 100;
    int heapsize = GRANULARITY;
    try {
        while (true) {
            byte[] test = new byte[heapsize];
            test = null;
            heapsize += GRANULARITY;
        }
    } catch (OutOfMemoryError oom) {
        heapsize -= GRANULARITY;
    }
    return heapsize;
} 

The bigger you make GRANULARITY (100k in the example), the faster it will run. The smaller, the slower, but also the more accurate. On my N96, this tells me that the heap can expand to about 16Mb (at least, I can create a byte[] that size).

Graham.

[thechameleon]

Experimentally... something like this...

public static int howMuchHeapIsThere() {
    final int GRANULARITY = 1024 * 100;
    int heapsize = GRANULARITY;
    try {
        while (true) {
            byte[] test = new byte[heapsize];
            test = null;
            heapsize += GRANULARITY;
        }
    } catch (OutOfMemoryError oom) {
        heapsize -= GRANULARITY;
    }
    return heapsize;
} 

The bigger you make GRANULARITY (100k in the example), the faster it will run. The smaller, the slower, but also the more accurate. On my N96, this tells me that the heap can expand to about 16Mb (at least, I can create a byte[] that size).

Graham.

thanks... i put the code into my J2ME code and ran it on the emulator. it can go to the maximum int limit.

however, when i actually try to read a 700MB file, the error occurs as an Unhandled Exception (java.lang.OutOfMemoryError), which is caused by the line:

System.out.println("before");
byte [] fileBytes = new byte [((int) fileConnection.fileSize())];
System.out.println("after");

gets me:

before
Callback threw: java.lang.OutOfMemoryError

so... im quite confused.

[grahamhughes]

i put the code into my J2ME code and ran it on the emulator. it can go to the maximum int limit.

Exactly what figure does it go to?

[thechameleon]

Exactly what figure does it go to?

it actually overran the int limit and became a negative value. i did a try catch, but it caught nothing. i thought it's quite odd too...

anyway, the code has been modified since to take 1024 * 2000000 bytes at max.

try{
			while(true){
System.out.println("\n\tTesting heap size: " + heapSize);
				byte [] test = new byte [heapSize];
System.out.println("\tTest OK");
				test = null;
				//up to 10MB
				if(heapSize < (1024 * 10000) && heapSize >= 0){
					heapSize += granularity;
				}
				//at 10MB
				else if(heapSize == (1024 * 10000)){
					//set granularity to 1MB
					granularity = 1024 * 1000;
					heapSize += granularity;
				}
				//from 10MB to 100MB
				else if(heapSize > (1024 * 10000) && heapSize < (1024 * 100000)){
					heapSize += granularity;
				}
				//at 100MB
				else if(heapSize == (1024 * 100000)){
					//set granularity to 10MB
					granularity = 1024 * 10000;
					heapSize += granularity;
				}
				//from 100MB to 1GB
				else if(heapSize > (1024 * 100000) && heapSize < (1024 * 1000000)){
					heapSize += granularity;
				}
				//at 1GB
				else if(heapSize == (1024 * 1000000)){
					//set granularity to 100MB
					granularity = 1024 * 100000;
					heapSize += granularity;
				}
				//upwards of 1GB
				else if(heapSize > (1024 * 1000000) && heapSize < (1024 * 2000000)){
					heapSize += granularity;
				}
				else if(heapSize == (1024 * 2000000)){
					break;
				}
			}//while
		}
		catch(OutOfMemoryError e){
			//1 granularity less
			heapSize -= (granularity);
System.out.println("\tMaximum allowed file size: " + heapSize);
		}
		catch(Exception err){
System.out.println("\tException err: " + err.toString());
		}
System.out.println("FileManager.findMaxHeapSize(): " + heapSize);

last print out:

Testing heap size: 2048000000
Test OK
FileManager.findMaxHeapSize(): 2048000000

[grahamhughes]

anyway, the code has been modified since to take 1024 * 2000000 bytes at max.

This is far, far too big. You're trying to measure the heapsize to the nearest 2Gb, which is not going to give you a useful value. Put it back to (1024 * 100).

Having tried this on an N96 and a 5800XM, the heap expands to 16Mb on both (making the maximum available to the application slightly small than that).

Graham.