Reading inputStream using threads...

[kurteknikk]

Hi guys,

I'm in desperate need of some help please.

I'm developing a dynamic application for multiple phones, currently working on: Nokia 6120 Classic & Sonim Quest

Previously i was told that i shouldn't use the inputStream.available() and that i should read data in another thread which can block and doesn't effect my main thread.

So, this is what i have done, i've got a thread filling a ByteArrayOutputStream (every 1sec) and my main thread is reading that same variable (every 1 sec).

First of all, i'm using getBaos() to get the ByteArrayOutputStream which is syncronized, so that there won't be problems of thread trying to access the same variable.

Secondly this is the code i'm talking about, which creates a new thread and while there is data, it writes the data into the ByteArrayOutputStream.

    public void readData(boolean rd) {
        read = rd;
        try {
            baos = new ByteArrayOutputStream();
            buffer = new byte[512]; // 512/1024 is a good size...
            readTimer = new Thread() {

                public void run() {
                    try {
                        while (read) {
                            try {
                                // retrieving the data from Web server
                                while ((bufferUsed = inputStream.read(buffer)) > 0) { 
                                    getBaos().write(buffer, 0, bufferUsed); 
                                }
                            } catch (Exception ex) {
                                //Do nothing just keep reading when it's available
                            }
                            Thread.sleep(1000); //TODO no need to sleep because it tries to read and it blocks...
                        }
                        readTimer.interrupt();
                    } catch (Exception ex) {
                        ex.printStackTrace();
                    }
                }
            };
            readTimer.start();
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }

I've tried to use multiple try catch so that a single problem won't hang my whole application.

The thing thats getting me mad is that on Sonim & on Nokia's nfc emulator it's working 100% correct without any problems.
And on the actual Nokia 6120 classic, i'm sending the HELLO~ message, the OK~ is being sent back from the server (i know from the server log) but it's not being received by the phone for some reason :s

Any help friends ? I do really need help here please because the deadline is in a week time... :/

[suryakantb]

Hi,
Check this out.
Basically, we are not directly using string in Java switch case because Java only permit to use Char and Integer variable directly in switch cases. So, in following example I have explained about how to handle String easily in Java switch case.

First of all possible string values are declared within a enum then in the application using that enum we can handle the string values in the switch case:

Example
public class Test{

public enum Day
{
SUNDAY, MONDAY, TUESDAY, WEDNESDAY,
THURSDAY, FRIDAY, SATURDAY;
}
public static void main(String args[]) {

String str="SUNDAY";

switch (Day.valueOf(str))
{
case MONDAY:
System.out.println("Day is Monday");
break;

case TUESDAY:
System.out.println("Day is Tuesday");
break;

case WEDNESDAY:
System.out.println("Day is Wednesday");
break;

case THURSDAY:
System.out.println("Day is Thursday");
break;

case FRIDAY:
System.out.println("Day is Friday");
break;

case SATURDAY:
System.out.println("Day is Saturday");
break;

case SUNDAY:
System.out.println("Day is Sunday");
break;
}
}
}
Hope this was helpful.
Thanks
Suryakant

[kurteknikk]

Hope this was helpful.
Thanks

Guess it was posted my mistake to the wrong thread right ?

[grahamhughes]

i'm using getBaos() to get the ByteArrayOutputStream which is syncronized, so that there won't be problems of thread trying to access the same variable.

This is a bad plan. You're only synchronizing access to the variable, not to the object. When you're writing to the ByteArrayOutputStream, you're no longer in a synchronized{} block, so there is nothing to stop your other thread from reading at the same time. Basically, you have unsynchronized access to the object.

                            Thread.sleep(1000); //TODO no need to sleep because it tries to read and it blocks...

You're still trying to poll the connection, which is exactly what traud and I suggested you don't do.

                        readTimer.interrupt();

I don't know why you're doing this. I'm guessing you're trying to "wake up" another thread. Use wait() and notifyAll() for this - that's what they're designed for.

I've tried to use multiple try catch so that a single problem won't hang my whole application.

Many small try..catches are likely to hide problems from you, especially when you do nothing in the catch{}. All this does is make the diagnostic information about the problem disappear, and make the program carry on running even when it is unable to. I suggest you don't catch Exception: catch the exception class that you actually want to handle. And, catch only exceptions that you intend to handle. "Handling" means, doing something useful to resolve or report the problem. printStackTrace() is not "handling".

What is involved in processing the information that comes from the server? You might be better to get this working in the same thread as you're reading, rather than trying a producer-consumer model. You don't seem familiar with multi-threaded programming, so avoid these synchronization issues.

Structure like this:

boolean exit = false;
while (!exit) {
    byte[] data = new byte[BUFFER_SIZE];
    int amountOfData = in.read(data);
    if (amountOfData < 0) {
        exit = true;
    } else {
        processData(data, amountOfData);
    }
}

No sleep(). You don't need to sleep(). sleep() causes the Thread to block for a specific time. read() will cause the Thread to block until more data arrives, which is much more useful.

For now, do the processing in the same thread. You can always separate it out later if it proves necessary.

Graham.

[kurteknikk]

Hi Graham,

First of all thanks a lot for your help. Yes, you're true i'm still very new to multithreading infact that's why i posted the code so that you will point out all my mistakes, and infact i really appreciated your reply.

So i'm following all your guidelines and i've got a couple of more questions if its not a problem.

This is a bad plan. You're only synchronizing access to the variable, not to the object. When you're writing to the ByteArrayOutputStream, you're no longer in a synchronized{} block, so there is nothing to stop your other thread from reading at the same time. Basically, you have unsynchronized access to the object.

Yes you are 100% right, the thing is that i'm accessing the object in two different ways, reading & writing and i wasn't sure how to syncronize because i can't have a common method to write to the object and syncronize on it because i'm accessing the object in two different ways.

I don't know why you're doing this. I'm guessing you're trying to "wake up" another thread. Use wait() and notifyAll() for this - that's what they're designed for.

I was actually using interrupt to stop the thread when read == false... but i think that i should interrupt the thread from the other thread right not make a while loop and when it exists, the loop will be interrupted.

What is involved in processing the information that comes from the server?

Actually its very simple, i will just create a string with the array of bytes read from the inputstream, with a specific encoding (loaded by default by the application) and then that specific string/msg/data is processed according to its contents.

For now, do the processing in the same thread. You can always separate it out later if it proves necessary.

Actually i will need to do the processing in another thread because since i'm having the inputStream from the socketConnection it will block until more data arrives (since there's no end of stream) so as you previously suggested it will be better to use different threads, and i will have more experience and learn more about multithreading while working on this part of my project.

[grahamhughes]

Actually i will need to do the processing in another thread because since i'm having the inputStream from the socketConnection it will block until more data arrives (since there's no end of stream) so as you previously suggested it will be better to use different threads, and i will have more experience and learn more about multithreading while working on this part of my project.

This won't stop the processing from happening. It won't block when there is data, so you read some data, process some data. It will only block when there is no data to read (and, therefore, no data to process).

The important thing Thread-wise, is that you don't perform the input in the UI thread. Blocking the UI's thread is very bad. (The UI thread is the one that's executing when methods like startApp() and commandAction() get called.)

If your deadline is looming, focus on getting something that works. You don't have time to become a multi-threading guru. You can do that later. Better to have something that works than something that is clever and broken. Once it works, if you have time, you can try a multi-theaded, producer-consumer model if you want. You'll have a working version to fall back on.

Graham.

[kurteknikk]

Are you sure mate? Because the code you just gave me:

boolean exit = false;
while (!exit) {
    byte[] data = new byte[BUFFER_SIZE];
    int amountOfData = in.read(data);
    if (amountOfData < 0) {
        exit = true;
    } else {
        processData(data, amountOfData);
    }
}

That is blocking @ int amountOfData = in.read(data);

The reason is that:
1st loop there is data, ammounOfData > 0 therefore processData
2nd loop there is no data to read, so in.read(data) blocks just right there.

So there's no reason to do if(ammountOfData < 0) exit = true; because that condition is only true when the stream is closed therefore connection is closed.

You're getting my point mate ? That was my whole problem from the begining that i need to either now beforehand if there's data in the inputstream, which is impossible to do without using inputStream.available() which still is not a stable function.

Or either, use multithreading, and read data in one thread, and retrieve it from the other.

The thing that's worrying me is that the same code works on Sonim & Emulator but on Nokia 6120 classic isn't working.

Btw this is the new code that i'm talking about: (Maybe you'll notice something which i'm doing wrong)

    public void readingData(boolean rd) throws Exception {
        read = rd;
        buffer = new byte[512]; // 512/1024 is a good size...
        readTimer = new Thread() {

            public void run() {
                while (read) {
                        if (inputStream != null) {
                            // retrieving the data from Web server
                            while ((bufferUsed = inputStream.read(buffer)) > 0) { 
                                writing = true; //i'm using this to make sure that the other thread doesn't read the baos while this thread is writing
                                baos.write(buffer, 0, bufferUsed); //TODO int offset should be always zero?
                                writing = false;
                            }
                        } else {
                            read = false;
                        }                   
                }
            }
        };
        readTimer.start();
    }

Thanks a lot for your help mate, i do really appreciate it...

[grahamhughes]

That is blocking @ int amountOfData = in.read(data);

The reason is that:
1st loop there is data, ammounOfData > 0 therefore processData
2nd loop there is no data to read, so in.read(data) blocks just right there.

Yes. Why is that a problem? That's the way it is supposed to work.

So there's no reason to do if(ammountOfData < 0) exit = true; because that condition is only true when the stream is closed therefore connection is closed.

You need a way to exit the loop, or the thread never dies. So, this is absolutely necessary.

The thing that's worrying me is that the same code works on Sonim & Emulator but on Nokia 6120 classic isn't working.

This is precisely why I warn you about depending on a specific implementation.

To understand why your code isn't working, I'd need to see the other thread's code. But my guess would be that there is a problem synchronizing the two, since don't have any synchronization.

Certainly, don't use a non-local boolean as a semaphore to control access. Use synchronized{} and wait()/notifyAll(). If you're not sure how to use these then, seriously, avoid multi-threading, because you have absolutely no chance of getting it to work reliably without proper synchronization.

The reason why get different results is probably because of different Thread scheduling algorithms. On implementations where it works, you are probably lucky, and Thread switches are occuring at times that happen to suit you. This is common with multi-threaded code that is not adequately synchronized.

Graham.

[kurteknikk]

Hi, sorry for a late reply i was really busy...

Yes.  Why is that a problem?  That's the way it is supposed to work.

The problem is that if i'm processing that in the same thread, its going to block my application until more data is received and i don't want that to happen, i want to continue doing other things and i will check every second if there's data available without blocking, thats why i'm using multithreading.

You need a way to exit the loop, or the thread never dies.  So, this is absolutely necessary.

Yes, i need the way to exit the loop but when the user presses the exit button from the UI. It will set read = false and so it will exit the thread. BUT still that doesn't work because the thread will be still in the inputstrem.read() so i need something else to just terminate the thread immediately.

My guess would be that there is a problem synchronizing the two, since don't have any synchronization.

The only syncronization that i currently have is that i've got two methods:

public byte[] getRxdData() throws IOException { //TODO IMPORTANT settle receiving data !! (test with netbeans)
byte[] data;
synchronized (baos) {
data = baos.toByteArray();
baos.reset();
}
return data;
}

public int dataAvailable() throws IOException{
try {
synchronized (baos) {
if (!read) {
return -1;
} else if (baos != null) {
if (baos.size() > 0) {
return 1;
} else {
return 0;
}
} else {
return 0;
}
}
} catch (Exception ex) {
ex.printStackTrace();
return 0; //When disconnected we're not alerted for the first time
}
}

I'm not sure if the syncronization is correct because last time you noted something about the syncronization that i should change and i wasn't sure about it, i just knew that how i tried to syncronize didn't make much sense.

But since i'm a bit new to syncronization i'm not sure if its synronizing well on the object or not, because i read an article i should syncronize the object not the method for my particular use.

[grahamhughes]

The problem is that if i'm processing that in the same thread, its going to block my application until more data is received and i don't want that to happen, i want to continue doing other things and i will check every second if there's data available without blocking, thats why i'm using multithreading.

No, it won't block your appliction. It will block only that thread. It will prevent that thread from processing any more data, but only when there isn't any data to process, so that's OK.

You're suggesting using sleep()... well, sleep() blocks the thread too. sleep() will block the thread whether there is any data to process or not, and so will stop your application from processing data as soon as it becomes available.

Yes, i need the way to exit the loop but when the user presses the exit button from the UI. It will set read = false and so it will exit the thread. BUT still that doesn't work because the thread will be still in the inputstrem.read() so i need something else to just terminate the thread immediately.

You need to make sure that the loop terminates when the socket closes. Then you can terminate the loop by closing the socket.

The only syncronization that i currently have is that i've got two methods:

So, is your other thread also looping and sleeping and checking some variable to see if data is available?

Graham.

[kurteknikk]

No, it won't block your appliction. It will block only that thread. It will prevent that thread from processing any more data, but only when there isn't any data to process, so that's OK.

Ok sry i thought that you were talking about (if i won't use multi threading) yes that is obviously true, and thats why i'm using multi threading so that it will never block my application heh

You're suggesting using sleep()... well, sleep() blocks the thread too. sleep() will block the thread whether there is any data to process or not, and so will stop your application from processing data as soon as it becomes available.

Wait... i'm not using the sleep anymore when reading data from the inputstream (since you told me so, and obviously it will still block when there's no data in the inputstream, so there's no point to sleep the thread)

You need to make sure that the loop terminates when the socket closes. Then you can terminate the loop by closing the socket.

I think that its impossible to terminate it EXACTLY when the socket closes because the thread might be blocked @ inputstream.read() so at that particular time it won't be checking if read == true/false (therefore the need to exit the loop)

I can prove this because i read somewhere else, that by closing the connection, the inpuStream.read() will throw an exception and therefore the loop will exit. BUT the thread is processing inputStream.read() (so it's blocked waiting for data) the exception isn't throwed, and the thread which closes the connection will wait for the thread trying to read and before it reads it won't throw the exception. (I don't know if you're getting what i'm saying i can explain further more if you want)

So, is your other thread also looping and sleeping and checking some variable to see if data is available?

Yes my other thread is looping every second, checking if the ByteArrayOutputStream.size() > 0
If true - there's no data so don't process anything
If false - there's data, so read the ByteArrayOutputStream and process the data...

[grahamhughes]

Ok sry i thought that you were talking about (if i won't use multi threading) yes that is obviously true, and thats why i'm using multi threading so that it will never block my application heh

I'm talking about using one Thread, in which to do both the reading and processing, and not about using two threads, reading in one, processing in the other.

I can prove this because i read somewhere else, that by closing the connection, the inpuStream.read() will throw an exception and therefore the loop will exit. BUT the thread is processing inputStream.read() (so it's blocked waiting for data) the exception isn't throwed, and the thread which closes the connection will wait for the thread trying to read and before it reads it won't throw the exception. (I don't know if you're getting what i'm saying i can explain further more if you want)

Did you try this? When read() is blocking, it is waiting for something to happen that will enable it to complete. "Complete" is to return data, to return -1, or to throw an exception. You think that if the server closes the connection that read() will block forever?

Yes my other thread is looping every second, checking if the ByteArrayOutputStream.size() > 0

You don't appear to have any way to synchronize writing with reading, to prevent both happening at the same time.

The point of multi-threading is that you don't poll like this. If you were to use separate threads for reading and processing, you should use wait() and notify(), as I've previously described, so that the thread wakes up at the right time. However, I strongly recommend you don't do this, but instead put both reading and processing in the same thread.

Synchronizing the operation of two threads is complicated, and requires a thorough understanding of threads and thread monitors, and will result in very hard to find bugs if you get it wrong. Avoid multi-threading like the plague.

Graham.