Ovi Store Java content publishing and Java Verified testing requirement confusions

[Aleksi]

Hi,

I've read the discussions here and elsewhere and it seems there's a big confusion around Ovi Store and Java Verified testing. I'm working for Nokia S60 Java Runtime so I do not directly work with Ovi Store but the talk about Java Verified in relation to Ovi Store all over the Internet made me to check what are really the requirements.

So I contacted Ovi Store guys and then posted information regarding Java apps on Ovi Store also on our blog in S60 blogs:
http://blogs.s60.com/2009/05/ovi-sto...-java-verified

It's not thus necessary but optional to use Java Verified when publishing content to Ovi Store. The bare minimum requirement seems to be that content is signed using content signing (i.e. VeriSign or Thawte).

And btw. at the same time my colleague Görkem also from S60 Java had done the exactly same thing, here's his post:
http://www.gorkem-ercan.com/2009/05/...-verified.html

-Aleksi Uotila, from S60 Java Runtime

[grahamhughes]

That's a shame, I was hoping it would be a requirement.

Perhaps you can find out what the testing requirements are? Applications, apparently, do go through some kind of testing...

Graham.

[21n9]

According to Nokia

Application signing: remember you have to have your applications signed prior to submittal.

- For Symbian SIS files you can use Symbian Signed or Symbian Express Signed.

- For Java files you can use Java Verified or 3rd party Java signing such as Verisign or Thawte.


It seems they have eased up a bit on the java verified issue but Verisign costs $500 a year, so unless you keep up the payments your customers apps will stop working correctly when the certificate expires at the end of that year (

[grahamhughes]

The issue here is testing, not signing.

Graham.

[21n9]

The necessity of testing/signing is the issue which has been posted

[grahamhughes]

No, that was the issue of signing.

It is clear that applications do not have to pass any testing prior to submission, since a Java Verified signature is not a requirement.

However, my understanding is that Nokia perform some kind of testing on submitted applications. My question relates to the test-cases that a submitted application must pass.

Graham.

[21n9]

Hi Graham sorry for the confusion I wasn’t answering your specific question, merely adding to the original post.

Not sure what the Ovi specific testing criteria is, would be interesting to find out, I’m sure it’s not as strict as Java Verified, I would suspect its concentrates on basic stability, what it does with sensitive information and issues regarding excessive violence and sexual content.

[grahamhughes]

would be interesting to find out

Sure would. I'm not taking a run at the bar until I know how high I'm going to have to jump...

Actually, my concern is not that it will be too high, but that it will be too low. If it's so low that any idiot can jump it, then any idiot will jump it. And I don't see much point in joining them. I don't want to publish my software through a channel that very quickly gets a reputation for being filled with junk.

[21n9]

Although Nokia hasn’t said specifically that they are vetting content for overall quality, I think it is a consideration when you register for Ovi they require a url to your content ie a promotional page or something like that, failing that a url to your home page, most likely so they can get a feel for your company and products, before they will accept you.

I don’t think that java verified would ensure the quality of your application in that respect, its merely a barrier for start ups and indie developers, Take a look at some of the movie tie ins churned out by the big publishers to see what I mean.

Big publishers are your biggest competition, because they can muscle their way to the top of the listings, unless that is Nokia adopts apples system where by they feature apps they feel are good, regardless of who developed it.

Games like stick man bashes stickman only serve to highlight the quality of your app

[mobileperry]

There's some more info here: http://www.eweek.com/c/a/Security/No...y-Apps-348402/

[grahamhughes]

Of course, JV is not intended to determine if your product is a good product. However, if I part with money for a mobile phone game/app, then I expect at least that:

* it doesn't crash
* it's performance is not so poor as to render it unusable
* if I get a call in the middle of a game, the game doesn't play music in the middle of the call
* it doesn't crash when I get a text message
* the general UI makes some kind of sense
* there is some kind of help text
* it doesn't delete information without warning me

And so on. These are only basic requirements, and I dispute that they represent "a barrier to start ups and indie developers". At least, not a significant barrier. Take a look at pjohnsen's account of JV, an indie's experience of getting through JV. It hasn't presented him with a major barrier. But I think he's demonstrated that he's serious about wanting to deliver a professional quality product.

We could discuss whether JV is or is not a suitable vehicle for assessing these basic requirements. But, they are currently the only industry standards, and Nokia are co-conspiritors in their creation. If they're wrong, it's better to change them than to dispense with the idea of QA entirely.

(My understanding is that Nokia do have some such criteria, and my original question was just: what are they???)

As a former development manager for a major publisher, I have been involved in delivering a number of movie tie-ins, so I'm curious as to which you mean! If it came through my hands, then, good game or not, it will have met the requirements I listed above. I strongly suspect that some of the very large publishers are able to circumvent JV requirements for operator listing, so I wouldn't assume that every product you see from them has passed JV.

Hmmm... maybe Nokia do vet on product quality (as in "is it a good product"), maybe they don't. If you're suggesting that they should, I would have to agree with you. But it's not clear to me that they are.

My concern is not simply about competition, but that a free-for-all, with no functional standards, will lead to a large number of apps that users will find extremely dissatisfactory, and that users will look elsewhere for their content.

And I'm not convinced that big publishers are the biggest competition. Freeware is the biggest competition. People who have to meet no standards can always undercut you. See if you can find out what percentage of iphone downloads are paid for...

Before I am accused (again, having just read a reply to my post on another thread!) of believing that all indie or freeware developers are incompetent morons, let me make it clear that I am in no way suggesting that your product, or that of any other specific developer, is substandard or undeserving of publication! I, quite sincerely, wish you the very best of luck with your product(s)!

Graham.

[grahamhughes]

There's some more info here: http://www.eweek.com/c/a/Security/No...y-Apps-348402/

Well, Bill, that's clearly garbage.

Java applications must be certified by Java Verified, Thawte or VeriSign.

Thawte and Verisign run certification processes for mobile Java??

Here's a thought: why don't you tell us what your QA test cases are, rather than using a news agency as a half-witted intermediatary?

Cheers,
Graham.

[21n9]

Jesus where did that come from, I was merely expressing an opinion. You seem to think that because Im not keen on Java Verified that I don’t want my products to pass any kind of scrutiny. Did I say in any part of this post, or any other post on this forum for that matter, that I was against testing full stop?????

A while ago I put a game through boot test on numerous handset, with a testing house TO ENSURE QAULITY. These boot tests cost £20 per device set, less than that for devices located at their test house in India. Each test had a half hour duration, and the tests were identical to JV test criteria except they didn’t tell me how to design my user interface.

What I have a problem with is having to fork out £135 for a 1 hour test, when a half hour test which looks at the things which really matter costs £20. 1 hour = £20 x 2, so where does the other £95 go, do you think?

These are only basic requirements, and I dispute that they represent "a barrier to start ups and indie developers". At least, not a significant barrier.

Many Independent developers have gone to the wall in the past 5 years or so, precisely because of the porting and testing costs, and those who didn’t jumped ship to other more stable platforms such as Xbox live, wii ware, ps3 and more recently iphone, and most have been making good money since, at least the ones I know. I don’t know how many builds or languages you have but you might want to get your calculator out and have another look at your business plan if you think JV isn’t a disincentive.

As a former development manager for a major publisher, I have been involved in delivering a number of movie tie-ins, so I'm curious as to which you mean!

As for Movie tie ins being 9 times out of 10 complete %&$# its a widely held opinion amongst reviewers the world over, and that’s on consoles. Being a development manager you most likely wouldn’t have been responsible for overall costs, but the majority of the budget goes on the IP and porting and testing costs, leaving very little for the developer, which doesn’t exactly give them much incentive to develop the world’s best game, coupled this with tight deadlines to make sure the games launch coincides with the launch of the film.

Hmmm... maybe Nokia do vet on product quality (as in "is it a good product"), maybe they don't. If you're suggesting that they should, I would have to agree with you. But it's not clear to me that they are.

My concern is not simply about competition, but that a free-for-all, with no functional standards, will lead to a large number of apps that users will find extremely dissatisfactory, and that users will look elsewhere for their content.

You say that you don’t have an issue with how polished an app is and your only concern is that customers don’t download a buggy app, and then you turn round and say on top of the JV certificate Nokia should vet an apps quality!!!! It seems to me your primary concern is centred on the issue of competition, not customer satisfaction

And I'm not convinced that big publishers are the biggest competition. Freeware is the biggest competition. People who have to meet no standards can always undercut you. See if you can find out what percentage of iphone downloads are paid for...

Free ware is your biggest competitor! thats like saying a television program is competition for the adverts in between, freeware is what keeps customers returning to Apples app store everyday , and then when they’re there buying stuff like Peggle Trism, and ifighter because they are clearly better quality. If the apps store was full of £4.99 apps I wouldn’t go on it every day more like once a month if that. I personally have bought more apps on the appstore in 6 months than I did in 5 years of owning java enabled handsets, by a considerable margin, and its not simply because of ease of use. I’m compelled to go on the app store everyday, because of the wealth of apps and I can download something for free, and then when I spot something of real quality like Peggle or ifighter I buy it simple as that!!!!!

[Aleksi]

In my opinion it's good that Java Verified is optional as it doesn't force developers to such service but still they can use it if needed. Each developer may themselves identify if the service provides value back. As discussed Java Verified is no guarantee for quality, as it only tests apps based on a fixed set of criteria. And thus with or without Java Verified there will be of course applications and games that are in the "less value" range but only good value and good quality applications survive in stores like Ovi Store. That's because the system recommends to normal users only those apps that are e.g. popular. So when people and recommending apps to the their friends applications are gaining popularity and when they gain a certain level of popularity also people start to see them directly from the system home page without need to dig deep. What I'm saying the store itself is part of the market and bad quality apps won't get air time, and even if they are available, they are not that visible at all with basic browsing. That of course happens when there's sufficient amount of content available.

[grahamhughes]

I didn't wish to suggest that you personally were against quality. I'll save that judgement for when I see your product! I apologize unreservedly if I gave that impression.

Hmmm... to some extend, TV programmes are competition for the adverts. How many people switch channel when the adverts come on? On Ovi, there is an option to list freeware only - akin to having a button on your TV remote to show no adverts.

Yes, I know exactly how much is spent on what. I can assure you, for the projects I've been invovled in, we spent as much or more money on a TV or movie tie-in than on a non-IP game.

The major cost is porting, not certification. A big problem has been the domination of operators like Vodafone, who often impose "drop lists"... sometimes more that a 100 devices that must be supported in order to get a chance of listing. A platform like Ovi, without this requirement, is a major step forward and, even if it were to impose JV, would still represent a massive lowering of barriers to entry.

Ovi was touted as a way for independent developers of make money from their efforts. My opinion (right or wrong) is that independent developers will get squeezed between big developers with brand names, and a huge mass of freeware. If I'm right, this saddens me immensely.

Creating a new set of testing procedures also worries me. First, I'm staggered that Nokia are not publishing their testing procedure.

Second: JV was set up because, at the time, every potential sales channel, every operator, every manufacturer, had its own set of testing requirements. A different set of hoops for developers to jump through every way they turned. JV is, at least, only one hoop. (It has not made the problem go away, mainly because the operators in the US have not embraced it as much as the European operators.)

I doubt Ovi will be the last such store to appear. How long until Sony Ericsson, Motorola, Samsung, et al, join suit? For a Java developer, all of these are potential target markets. Nokia are really the only manufacturer with the clout to set a different benchmark than Apple's. I hate the idea of seeing a dozen different manufacturer-specific app stores, 18 months from now, with a dozen different test requirements. Indie developers would have a nightmare following this, and would at best have to work to the tightest requirement from each set. Having had to target JV as well as testing processes from Three, T-Mobile, Sprint, etc., I know how much of a pain that is, even with a huge team. I have a distinct feeling that I will see developers posting here 18 months from now, complaining about fragmentation in certification requirements.

I agree wholeheartedly with you that JV is too expensive. Again, no one has more power to change that than Nokia. Orange arrange a discount deal for JV, for developers on their Partner programme. With the potential scale of Ovi, Nokia are in a fantastic position to negotiate a better price (and subsidise it, rather than spend money on setting up and running their own testing process).

Aleksi: I think there are two flaws in your argument. One: users never get to see a game or app before they download it. I can tell (from having seen plenty of sales figures from my previous employer), there is very little correlation between the quality of the product and it's download rate. Two: Ovi doesn't appear to give developers much chance to indicate what quality standards their product meets, so customers cannot make an informed choice.

Ultimately, I'm guessing. I haven't seen any Apple figures about how many of the downloads are actually paid, how many free trials get converted into purchases, how much money an average iphone app makes. I certainly know developers who have not even covered the cost of the Mac they had to buy...

Well... Ovi will be what Ovi will be. For me, I won't spend any more time looking at it, I don't see it as a serious business opportunity.

To those of you with different opinions, I look forward to you changing my mind with your success stories!!

Graham.