Java Verified Experiences

[grahamhughes]

Oh.. the other interrupt issue to watch out for is with games that allow you to hold a key down (setting a flag on keyPressed() and resetting on keyReleased()). On some devices, you may lose the keyReleased() event if the interrupt occurs while the key is held down. If you're using hideNotify() and showNotify(), make sure you reset any key flags in the showNotify(). You're probably already doing this...

Graham.

[pjohnsen]

Testing is now ongoing .. so keeping my fingers crossed

[grahamhughes]

Who did you choose for testing it?

Are you paying the same for all devices, or do you pay a reduced price for the other devices after the first? I was reading the new "testing on multiple devices" stuff, which I eventually decided was not as complicated as it looks, but if I understand it, you can get away with quite a lot of reduced testing.

Graham.

[pjohnsen]

I went with Babel. They gave me a discount on additional languages, but pricing was the same for each device group.

The other quotes I got was priced per language per device group.

Initially I'm only doing 2 languages, but it's still good to get discount

As I understand the UTC 3.0, testing of additional language will only go through a subset of the test cases. But the reply I got from the other test provider was that if it was a separate JAD/JAR, then they would go through a full test. As the game contains quite a lot of text, it is not really feasible to package multiple languages in the same JAD/JAR, so I went with Babel.

[pjohnsen]

So we got the response back from the first test.

It failed 2 test-cases

UI2 (UI consistency):
- We were using 'Quit' both from the main menu to exit the app. and from the game menu to exit the game and go back to the main menu.
- Some minor inconsistency in dialogs, some had 'Continue' softkey, some had 'Back'.
- Some inconsistent 'Back' behavior, in some cases it would return to gameplay instead of going one step back in the menu.

FN14 (Data Deletion):
- When choosing 'New Game' when a save game is present, the save game is deleted without informing the user.

These issues have now been fixed and the updated version of the game submitted for re-testing.

The learning point is to make sure to study the UTC document in all details and check every corner of your app's UI.

[grahamhughes]

That's not a bad result for a first time. No nasty device-related issues to address.

Yes, you need to run through the application as if you were testing it. Interpret all the requirements literally (as the testers will). It gets easier once you are familiar with the requirements. You won't put those issues in in the first place, and you'll spot issues quickly.

Good luck with the re-test!

Graham.

[21n9]

So we got the response back from the first test.

It failed 2 test-cases

UI2 (UI consistency):
- We were using 'Quit' both from the main menu to exit the app. and from the game menu to exit the game and go back to the main menu.
- Some minor inconsistency in dialogs, some had 'Continue' softkey, some had 'Back'.
- Some inconsistent 'Back' behavior, in some cases it would return to gameplay instead of going one step back in the menu.

so what do they expect your quit button to be called on the pause menu ??????

What do they mean by inconsistency in dialogs, does that mean you can’t use the right softkey for anything else other than BACK.

Ive got on some screens, a tick icon left_softkey to make a selection and a cross right_softkey to go back to the previous menu, but then on some screens such as my pre game controls screen Ive got a down arrow left_softkey to go to the next screen and a tick right_softkey to finnish and PROCEED to the game. Would that be an EXCUSE for them to fail it and squeeze me for more money.

[grahamhughes]

There would be a definite fail if:

* on some screens "cross" takes you back to the previous screen, but on other screens "cross" skips the current screen onto the next one (without making a selection);

or

* on some screens "cross" is the "back" action, and is on the left softkey, while on other screens "cross" is the "back" action, and is on the right softkey.

The requirement pjohnsen's game failed was:

Two different actions must not be named with a same title (for example exit must be used to exit the application to the devices and not to exit the application to the Main Menu, back could be used instead).

In your case, there is some confusion in that some of your screens seem to have to "proceed" options (one to proceed to the next screen, another to proceed to the game). So, on some screens the RSK is "backwards", and on some it's "forwards".

I'd recommend you review that, and perhaps use left and right to move between screens in the pre-game section, with a softkey to skip, or use the softkeys for next and back.

Graham.

[pjohnsen]

Thanks for the encouragement Graham

Truth be told, I kind of expected to fail, this being the first time going through JV. But of course couldn't help being a bit disappointed and feeling a bit stupid about some of the problems, when the result came back.

After realizing that the issues were all quite easily fixable, and looking at all the tests that passed, I do feel pretty good about the game passing the re-test.

Regarding softkey usage, my recommendation is to try and follow the native UI, i.e. always have the 'negative' option (Exit, Back, etc.) on RSK and 'positive' on LSK.

For the 'Quit' option we now changed it to say 'Exit' in the main menu and 'Quit game' in the in-game menu.

[grahamhughes]

Oh, I think you should be very happy with the performance of your game.

Each time I suggest on this forum that making people pass JV would not be such a bad thing, I get a torrent of replies saying that JV is no guarantee of quality, and that it's just a barrier to independent developers.

True, passing JV will not certify that your game is a good game. But it will certify that it passes some standards of stability, performance, usability and so on.

A barrier? You passed about 95% of the requirements without breaking into a sweat, and 100% with a couple of text changes.

I think it's too expensive. Orange have a discount deal with NSTL for testing in their Orange Partner program, and I see no reason why Nokia couldn't make some similar arrangement. Given the quantity of content that will go through Ovi, all the test houses would be eager to get that business. Nokia could also subsidise it, rather than paying for their own testers (as they seem to be doing now). At least you'd know what the testing process is, and you'd have certification you could take to another sales channel.

In my previous job, I was involved in the production of thousands of builds that went through JV, so the requirements are all familiar to me.

I'd be interested to know your opinion, after going through this for the first time. Are the testing requirements sensible, well documented? Was the feedback from the testing house useful? What about the price? Was it painful from a paperwork point of view?

I fully expect your next post to tell us that you passed!

Cheers,
Graham.

[pjohnsen]

Got response back on the re-test and the English version of the game now passed for the N95 device group, however with the comment that I forgot to update the help text to say Exit instead of Quit ... DOH.

But surely good news

Now I am a bit confused about how this works with the different device groups. I have the signed app. which is certified for N95. I tried installing on N85 (which is from the N78 group) and it installs just fine without any cert. errors.

The tester is telling me that I need to submit separate builds to JV for each device group, even if it uses the exact same JAD/JAR. This seems rather cumbersome to me as I now have to enter all the information (flowchart, connection usage etc.) multiple times. Also the JV site lets me choose several device groups when I submit a build ... confusing.

I'm also not sure if Ovi will let me submit the same version multiple times (for multiple devices).

But now on to submitting to Ovi. I wonder if they will now put the app. through the same tests that it just passed :?

[srdjandakic]

Pjohnsen, thank you for the great thread!

It would be great to know how much money at the end you spent on publishing tasks, before your application went live in Ovi store.

[21n9]

Hi thanks Graham and Pjohnson, as much as I hate the whole JV thing, especially them telling me how to design my UI, I think Im going to have to go that route. Even though Nokia have ditched it as a requirement, the other option they now give, to just use the Verisign or Thawte signing, would actually work out more expensive at the moment $2100 for the 7 year duration of the JV certificate and $3500 for 7 years of the verisign, although I can use that on as many products as I like.

But like Pjohnson Im going for the same 3 devise groups you chose which works out at $525, Im quoting dollars not sure how it will equate to pounds.

Its interesting that your signed N95 midlet runs on the N85, does that run without any warning messages on install? I wonder if that means the certificate doesn’t recognize which handset its running on if you know what I mean.

I had set my game up according to UTC 2 test criteria, back always on the right when it was used ect, but they keep moving the goal post, and far too subtly for my liking!

Your post has been really interesting, much appreciated. Keep us updated with your progress oh and congrats on passing mate

[pjohnsen]

Yes, it installed without security warnings on N85 and when I look at the installed application it is also listed as belonging to trusted 3rd party domain. I tried on a couple of other phones, and some lists the app. as untrusted (E71, SonyEricsson W715). As far as I can see, all the phones have the same GeoTrust root cert. So I am a bit in the dark on how the device group matching happens.

Anyway I submitted the game to Ovi yesterday evening, it is now in Pending QA state. I have set distribution to only allow S60 3rd FP1 240x320 (which should correspond to the N95 JV device group) and written a comment stating the app is JV'ed for N95. I hope this will make the QA process faster.

[21n9]

Hmmm the E71 is landscape ie 320x240 not 240x320 like the N95, and obviously the w715 is Sony Ericsson not Nokia.

Its obviously checking some kind of criteria, from the above information I would surmise its checking the screen size and manufacturer . additionally what JSR’s are you using, maybe this is a criteria too.

The only difference I can see between the devise groups is that some have JSR’s that the others don’t for example the N78 and the N95 devise groups have all the same JSR’s except the N78 group has an additional JSR, (JSR 248 Mobile service architecture), but if you don’t use this, then there isn’t a problem with it running on your N85.

Assuming this is the case I wonder if Nokia would except you submitting your JV signed N95 build for the N78 group, one of the main reasons Nokia originally sited for requiring apps to be JV signed was that no warning messages are displayed by the app. If this is the case then that’s great news, It would certainly resolve the whole JV debate as far as I’m concerned, it would resolve the issue of costs and ensure quality which is something Graham has been asking for, I have to say Ive never been against testing full stop, it’s purely an issue of cost for me.

After your N95 build gets approved (which I’m sure it will) you could submit it for the N78 and see how that goes