FAQ(?) Howto Support HTTP/GPRS properly for EVERYONE..

[Niall]

Hi,

I have found that none (or every app i tried) of the available Series 60 applications will work properly on my 7650 - where a HTTP connection is required (using GPRS/WAP).

In order to work out why I have written some J2ME applications which finally allow this sort of functionality for me in my own software - and thought I'd share my experience. If this has been covered already - my apologies(!)

My WAP/GPRS browsing was working correctly on the real device however - and the setttings for my J2ME applications matched. Even though the example J2ME applications worked fine on the Emulator the moment I tried to run them on the real device I would just get errors.

I tried Nokia Tech Support in Australia as well as my Mobile Phone/GPRS providor - and they couldn't help me at all. They seemed also very new to these technologies and didn't really understand what I was talking about. I basically got nowhere.

I remembered however (from some previous work done with WAP) - that my Mobile Phone Company (Singtel/Optus in Australia) used GPRS Gateway products made by CMG. These products are designed for WAP content however and are defaulted to actually parse the WML content that passes through and only allow certain mime types (such as WBMP's and WMLS). Basically unless your content complies with these standards it will actually replace the content with an error message (even though it was able to source the request).

I have a feeling this is also a limitation for many people around the world. Whilst Gateways are configured to support the new .jad/.jar file types - many operators dont understand that people also want it for customized use where only specific raw data is downloaded (in applications like PocketTV for example) - and restrict all other content.

As a solution I would ask people writing apps to consider this issue for many of us, and implement the following workaround :

Bring your Raw data through packaged into a compliant WML,WMLS or WBMP file - and then parse for it within your application. (Remember that this is not just the extension required but the format OF DATA required).

Your Web server should also be configured to support the correct MIME-TYPES required for WAP content (such as text/vnd.wap.xml etc etc)..

For example your application could request the following WML page. This page would not be for browsing with your WAP browser - but rather disguised as valid content for the purpose of it getting through the GPRS gateway.

<WML etc.etc.>
<deck>
<card>
<p id="start">#### INSERT YOUR RAW TEXT DATA IN HERE #####</p>
<p id="moredata">#### INSERT OTHER RAW TEXT DATA IN HERE #####</p>
</card>
</deck>

If you setup the correct encoding attributes in your WML file you can also bring through binary encoded data in this manner. Additionally - I believe you may be able to get away with shoving data inside a WBMP file (by having a correct header and using the pixel data area).

For downloading images however - GIF is normally supported and can be directly sent through most gateways. So for example if you were writing a Weather Report application, you could directly download a .GIF file - however the associated weather forecast data would need to be embedded into a WML file and parsed on your end.

Doing this has it's advantages - as the WML is XML - you can easily parse it to find the start/end tokens for your data. Alternatetively - simply searching for all text between <p id="start"> and </p> may be sufficient (just using the string object).

So - If you are writing applications please remember consider these workarounds - and maybe even make it an option inside your application - so those without these limitations can still download the raw data without the additional chars for the WML tags! (Nokia - also please consider maybe addressing this issue with your Application Style Guides)

Thanks - any comments welcome/appreciated!

Niall G
Melbourne, Australia

[alex70]

I think that an easer work around is to place 0.0.0.0 in the gateway.

[sweekeat]

Yup, I've tried the 0.0.0.0 trick before as well and it didn't work for me. No idea what my carrier's setuo is like...

One question regarding Niall's original suggestion: if we're gonna dump data into a 'WBMP', does it matter whether the data sent actually conforms to the WBMP format, or is it enough simply for the server to indicate the WBMP MIME-type? Basically, do WAP gateways that do such content-type restrictions actually check incoming WBMPs if they are the right format? If there's no such checking, then it would be pretty simple to just send all 'raw' data using WBMP MIME-type and we won't need to worry abt making it 'look real'.

[copperdragon]

I have tried quite long to get a correct HTTP request while the language settings of the 7650 is something else than english (both "Deutsch" and "Fracais" did not work).
It did also not work to turn off the wap gateway (set it to 0.0.0.0).
I guessed that the 7650 has a bug with _both_ protocolls (WAP and HTTP) - at least with my current firmware version 3.16.

Luckily there is a workaround using direct TCP sockets:
Using "socket://www.hostname.com:80" I succeeded in opening a connection to the webserver and echange data with it directly.
Of course this totally works around any wap gateway that may be configured to be used for connections - but as long as your Mobile Phone Compony does not block data that is not targeted to the gateway it should work.

I had to be very carefull with this connection though, because its implementation is also quite buggy and (as far as I have read) does not allow simultaneous read and write. But for HTTP this is not necessary.

Well, I wrote this code and it worked for me:
(call it in an extra thread, because using blocking operations
in the GUI thread is a very bad thing!)
I hope, this helps!

private void doSocketRequest() {
// need to extract hostname and document name from request
if (!request.startsWith("http://&quot) return;
String uri = request.substring(7);
String hostname = null;
String docname = null;
int slashpos = uri.indexOf('/');
if (slashpos<0) {
hostname = uri;
docname = "/";
} else {
hostname = uri.substring(0,slashpos);
docname = uri.substring(slashpos);
}

// open the socket and send the get request
try {
StreamConnection c = (StreamConnection)
Connector.open("socket://"+hostname+":80"
try {
OutputStream os = c.openOutputStream();

os.write( (
"GET "+docname+" HTTP/1.0\015\012"
+ "Host: "+hostname+"\015\012"
+ "Cache-Control: no-cache\015\012"
+ "Pragma: no-cache\015\012"
+ "\015\012&quot .getBytes());
os.flush();

InputStream is = c.openInputStream();

// read the HTTP response header and try to determine
// the length of the answer in advance (if given by
server)
String headerlow = new String(readAll(is,-1,true));
String header = headerlow.toUpperCase();
int contentlength=-1;
int tagidx = header.indexOf("CONTENT-LENGTH: "
if (tagidx>=0) {
int startidx = tagidx + 16;
int endidx = header.indexOf("\015\012",startidx);
if (endidx>=0) {
try {
contentlength = Integer.parseInt
(header.substring(startidx, endidx));
} catch (NumberFormatException e) {}
}
}
// now read the answer itself
answer = readAll(is, contentlength, false);
// clean up after operation
is.close();
os.close();
} catch (IOException e) {}
c.close();
} catch (IOException e) {}
}

[peterd.p]

Many, many thanks.

This is also a solution to talk to a local webserver (localhost/127.0.0.1) running on the mobile device without the neccessarity of a access point for the localhost, while beeing able to keep a Internet connection open.

Thanx!

[walkal]

I use the same carrier as Niall - Optus in Australia - and I have been able to use networked applications, such as Backgammon from j2x, on my series 40 phone (6610).

Was I just lucky, and hit on apps that were using the sorts of work-arounds discussed? Or is there some difference between series 40 and 60 phones in this area?

Please excuse me if my question is stupid. I don't know anything about midlets using the WAP network, but I would like to try it out some time in my own games, so I am curious to know how tricky it is.

Alan Walker
www.lexigame.com

[lmtang]

I'm having a problem with HTTP connections on a Nokia 6630. I'm posting data to a .php page, and retrieving a response in XML format.

It works fine in the emulator (well, doesn't everything!), but I cannot get it to work on the actual phone. Get a HTTP 500 error code.

I tried using copperdragon's technique by making use of socket connections. However, I come across this error "Target port denied to untrusted applications ".

Also tried using internet access point as opposed to WAP access point, and reset gateway ip address to 0.0.0.0. No luck either.

my content length is also returned, so i doubt that is the problem.

Does anyone have any ideas? I'm seriously at a dead end. No idea what else to try.

Thank you in advance!

[mreaves]

I have done quite a bit of work with is and it can be a minefield, different operators in different countries all have different problems (I am based in t he UK). I have even had a two different 6600 phones using the same operator act slightly differently (obviously a slightly different firmware).

In my experience making sure the phone’s settings are set up to use HTTP and the access point is pointing to a valid Internet Access Point usually does the trick, I also never change the gateway, I just point to our PHP server to request data and use a binary content type.

I think there are still one or two problems and the big problem I wanted to solve was one whereby the phone could obviously connect via GPRS (i.e. it could download JAD/JAR files via the WAP browser) but didn’t seem to want to connect in a Java app via HTTP, I really wanted to connect to my server when the phone was only set up with a WAP connection but there just didn’t seem to be anyway to do it. I am willing to stand corrected though.

[suterma]

Hi copperdragon,

I have a question for you. I am trying to do wap on 7610 and 6630, with no success in dropping a request which is properly understood by some commercial servers. I stubled over your line

>I have tried quite long to get a correct HTTP request while the language settings of the 7650 is something else than english (both "Deutsch" and "Fracais" did not work).

Do you have any suspicions, why the language settings could affect the request. Could this be an issue for me?

Best Regards, Marcel

[holtsch]

Originally posted by walkal
[B]I use the same carrier as Niall - Optus in Australia - and I have been able to use networked applications, such as Backgammon from j2x, on my series 40 phone (6610).

Was I just lucky, and hit on apps that were using the sorts of work-arounds discussed? Or is there some difference between series 40 and 60 phones in this area?

There is a MAJOR difference between Series 40 and Series 60 in this area. I've been having trouble with internet connections since the Nokia 7650... on that device, I have things working now (back then, it was just a matter of the correct settings, Gateway = 0.0.0.0 etc.).

Now, with the 6630 I have the same trouble AGAIN (possibly, because I've moved from a Java-server to ASP.NET for a new application, and I've also noticed that the Gateway-Setting has been removed on the 6630, so I have no way of changing this to 0.0.0.0)...

I must say I find this a very annoying issue - and this has been trouble with Nokia Series 60 devices for a couple of YEARS now...

I've never had any trouble with Series 40 devices on this. Just connect and everything is fine (never even had to change any settings or anything)...

Does the "mimick WBMP"-solution work smoothly? That sounds like the simplest solution to me... if it works...

kind regards,
david

[walkal]

Originally posted by holtsch
I've never had any trouble with Series 40 devices on this.

Is Nokia completely crazy? They had something that was working fine, and now it's broken. And surely the ability to communicate is one of the great potential strengths of the mobile phone as an application platform. Why make it so hard?

[shmoove]

It's not just Nokia's fault. The telco's back end plays a very big part in the equation, and every telco will have a slightly different configuration.

shmoove

[walkal]

Which makes it even crazier, because it's the telcos who would benefit from data traffic generated by more Java applications communicating with servers.

[shmoove]

Yes, but it's not like they are doing this on purpose.

shmoove

[holtsch]

Originally posted by walkal
Is Nokia completely crazy? They had something that was working fine, and now it's broken. And surely the ability to communicate is one of the great potential strengths of the mobile phone as an application platform. Why make it so hard?

I think that is a misunderstanding. Series 60 is not following Series 40 - it's simply a different platform with different specs. I guess the major difference is that Series 60 means Symbian OS while I guess Series 40 is a proprietary Nokia OS.

I wouldn't be surprised if the problem is simply interaction between mobile device, Symbian OS and J2ME...