The reason why C# is required comes from the constraint that in WP7, your application is confined run on top of the CLR (Common Language Runtime). The CLR concept is very similar to a Java virtual machine :  the code is converted to an intermediary language that the virtual machine will execute. The CLR environment is commonly called the “managed” mode. The managed mode comes with many great advantages, most notably garbage collection and reflection.  C# is a managed language.

That changed in Windows Phone 8! In addition to the managed framework, developers can write in “native” code. Native code, in comparison with managed code, does not run on top of a virtual machine, but is compiled directly into machine code. As you might expect, native code typically execute slightly faster than managed code. On WP8, native code is written in C or C++.

It is possible to glue pieces of native code with pieces of managed code, thanks to the Windows Phone Runtime framework. The framework takes care of the magic required to convert objects across managed and native runtimes. This Nokia developer wiki gives a quick tutorial how to get started with the Windows Phone Runtime, while this article C++ support from Windows Phone 8 goes into deeper details.

There are some limitations how one passes objects back and forth between managed and native runtimes, and while the “Windows Phone Runtime Component”  code template shipping with Visual Studio provides all the necessarily plumbing to call a Native object from a Managed object, it may not be that obvious how to call a managed object from a Native object. For example, when a developer needs to call from C++ the functionality only provided by  .NET, like modifying live tiles, send a SMS, … The same wiki article provides a solution for such a scenario.

Of course, it’s not because you can that you should!  Stick with C#, unless you really need that little extra speed improvement. Or if you have a C++ library that does a great job and just do not want to convert it to C#: that would be such a waste of your time.

You might have heard from the tech news outlets about the Leap Motion. If you haven’t yet, you will soon. It’s a black box about half the size of an average mouse (computer, not actual) that sits on your desk and tracks the 3D movement of your hand (actual, not computer). Essentially, it’s like an XBox Kinect but instead of tracking limb movements, it tracks fingers. These devices are being released mid-July but the here.com team were lucky enough to get into the developer preview program so that we could integrate the Leap with here.com.

(A quick aside: it’s pretty cool that we get to play with cool toys like this as our actual job. You should totally think about joining our team.)

3D and the Web

All the finger tracking and so on would be pretty impressive on its own but the developers of the Leap have made it extremely versatile by providing a JavaScript library to allow you to integrate it into a web app. So we did exactly that. The Leap driver sits in the background on your computer relaying the data from the device to a WebSocket. Web sites and applications can check for the existence of this WebSocket and, if they find it open and active, listen to a stream of JSON objects coming out of it. In essence, you move your hand above the Leap and your web app gets told there’s a hand moving around, how many fingers it has, where the fingers are, what direction they’re pointing and so on.

Enough Talk, show me!

If you’ve got a Leap, read this far and just want the demo, go to here.com/leap. If you haven’t got one yet, this video should explain what we did:

Note: WebGL isn’t supported across all browsers yet. If you don’t have a WebGL-enabled browser, there’s still the rest of here.com to explore.

Challenges

Leap-enabled applications are very CPU-intensive. The device collects a big amount of sensor data that is streamed through the USB port to a Leap Motion Controller application that does some computation and sends the parsed data to applications via WebSocket. All of this is a lot of work for the CPU, which means that soon enough, even on a high-end laptop or desktop computer fans start spinning quite fast. The Leap sensor itself becomes pretty hot after using it for a while.

We added WebGL to the mix, so our users can use the Leap to control a paper airplane and fly around the buildings of the coolest cities in the world. This is an additional element that keeps the CPU busy, but the experience is still nice and smooth with 3D maps rendered at 60FPS.

Gestures are hard

There are numerous native libraries available to help process gestures but translating this to an entirely web-based application is not easy. Gesture detection can be computationally heavy and, as mentioned above, we’re already pushing the computer quite hard. We prototyped with the $1 gesture library but in the end, removed gesture recognition from our demo.

No established patterns

The biggest thing we discovered that might not be obvious at first was not strictly a technical one. There are very few examples of 3D interactions. This is a new paradigm of computer interfaces and we struggled to get something that was familiar enough to be intuitive but accurate enough to not be unforgiving. Most of our early attempts involved introducing an arbitrary constraint to the ‘gesture space’. The most successful was probably the ‘virtual screen’ which registers movements as being ‘active’ if they are within a small z-distance from the device. In essence, it reduces the space in which user interactions are detected to an invisible screen parallel to the leap. By doing this, you get to fallback to typical touch gestures – pan, pinch-zoom – with the benefit of also having a hover state when the user’s fingers are detectable but not ‘touching’ the virtual screen.

The most successful interaction is the one we eventually ended up going with – direct control. To take most advantage of the hand-tracking, we decided to simply map the movements of the user’s hand to the position and angle of the airplane. This not only simpified things but also had the effect of being intuitive. The user can quickly learn that what they do is represented on screen.

On a technical level, this was much easier to accomplish than gesture-based control as we now take the information about the hand position coming from the Leap and pass it directly through to the position of the 3D camera in the WebGL world.

What do you think? Is this the future of computer interaction or just a great way to explore HERE 3D?

One major API change that affects the UI and UX has been done for the CategoryBar API; it now supports two different modes. You can use it as a tab bar (like it has been used for Full touch), and in addition as a toolbar for the most frequently used actions. At the same time the CategoryBar extension has been disabled for Nokia Asha UI. There is a fixed number of four items you can place in the Category bar, both in portrait and in landscape.

You probably remember my blog post from a while back, where I discussed this same issue but saying that actions are not allowed in the CategoryBar? Well, that still holds for the Full touch; the API change doesn’t apply there. So, when you are bringing your Full touch MIDlet to Nokia Asha and you have “played by the rules” (i.e. not having actions in your CategoryBar), you only need to consider the amount of tabs you have (only 4 possible with Nokia Asha UI, compared to 15 with Full touch).

But if you decide to create a new MIDlet for Asha and use actions in the CategoryBar, you will need to have another solution for Full touch, due to the facts described in my old post.

It’s also important to keep in mind that tabs and actions cannot be mixed in the same CategoryBar. This also comes back to the visualization of the component. So either use tabs (one item is always selected, used for changing the view) or actions (none of the items is in selected mode, used to trigger an action).

See the Nokia Asha design guidelines if you haven’t already!

Happy developing,
Sanna

The Nokia Asha design guidelines provide a comprehensive set of UI component descriptions and UI patterns. For app icons, templates and guidelines are of course included. And to make things as simple as they can get, there’s also a UX checklist.

Stay tuned for more updates on the Asha UI in this blog. And don’t forget to check the Event calendar for Asha UI webinars!

Happy developing,
Sanna

Asha web apps are a lightweight application development framework for creating apps for low end Nokia devices (Asha series and earlier). While JME is the most popular technology for creating apps for Asha, web apps are a reasonably quick and convenient way to expose content on the web – basically, your garden variety front ends for restful services, spiced by some simple navigation.

Since Asha devices don’t have local javascript execution, javascript is executed on proxy server. “Normal” web pages execute all javascript on the proxy, but Asha web apps provide some “shortcuts” for doing simple DOM manipulation (add/remove css classes, set values in input fields) without the proxy roundtrip. Check out “MWL” library for details.

Standard workflow

Normally, you use Eclipse to develop and test Asha web apps. Nokia provides the “Nokia Web Tools” IDE for this; it’s a rather mainstream Eclipse based IDE, spiced with webkit based “local” simulator (that runs javascript locally), and the “cloud” simulator that runs Javacript on the real Nokia proxy servers (but still renders the DOM on the webkit based simulator).

While this is fine for most web developers, there are lots of us relentlessly looking to optimize our workflows, to make it little more efficient, little more fun (or more tolerable, depending how bad an aversion you have for Javascript and HTML) and little more readable and maintainable.

For this kind of developer, there are tools like CoffeeScript, Brunch, Jade, SASS, LESS, HAML. In this exposition we’ll use CoffeeScript, Brunch and Jade.

CoffeeScript is a language that compiles to idiomatic and readable/debuggable JavaScript. It makes producing JavaScript more tolerable for people used to modern languages without semicolons, `function` keywords
and a selection of other nastiness that plagues JavaScript.

JADE is a language that compiles to HTML.

This Jade:

        doctype 5
	html(lang="en")
	  head
	    title= pageTitle
	  body
	    h1 Jade - node template engine
	    #container
	      if youAreUsingJade
	        p You are amazing
	      else
	        p Get on it!

Compiles to this html:

<h1>Jade - node template engine</h1>
<div id="container">
 
You are amazing
 
</div>

As you can see, this makes it easier to “see forest from the trees”, as the eye is not misdirected by stream of redundant characters and endless <div>’s.

Brunch is an “assembler” for HTML5 applications. It picks up all the pieces you have (CoffeeScript files, JADE templates, etc.) and produces a runnable web application out of that. It can watch the files on the background, compiling CoffeeScript files to JavaScript the second you save them, running unit tests immediately, refreshing the browser in real time etc.

We use Brunch as the “skeleton” of our Asha web application.

Getting started

Prerequisites: This flow has been tested on Linux and Windows. Have to install Node.js, git, the usual stuff every self respecting web developer has set up on day one. Anecdotal evidence suggests it’s a good idea to compile Node.js from source instead of using the packaged versions, if you are lucky enough to be on Linux.

We clone the basic scaffolding from github and install the dependencies:

$ git clone git@github.com:vivainio/awa-brunch.git awa-helloworld
$ cd awa-helloworld/
$ npm install
npm http GET https://registry.npmjs.org/javascript-brunch
npm http GET https://registry.npmjs.org/css-brunch
npm http GET https://registry.npmjs.org/uglify-js-brunch
npm http GET https://registry.npmjs.org/clean-css-brunc
.. etc ..
This install Brunch, CoffeeScript compile and lots of other packages with npm.

Let’s take a look at the project structure:


ville@ville-tp:~/p/awa-helloworld$ tree app
app
├── assets
│   ├── config.xml
│   ├── icon.png
│   ├── index.html
│   └── s40-theme
│   ├── css
│   │   ├── s40-theme.css
│   │   ├── single_landscape.css
│   │   └── single_portrait.css
│   ├── images
│   │   ├── arrow-close.png
│   │   ├── arrow-open.png
│   │   ├── back_40x40.png
│   │   ├── button-bg_40x40.png
│   │   ├── close_40x40.png
│   │   ├── grid_40x40.png
│   │   ├── option-menu_titlebar.png
│   │   ├── refresh_40x40.png
│   │   ├── search_40x40.png
│   │   ├── tab_bg.png
│   │   └── title-bar.png
│   └── js
│   ├── s40-theme.js
│   └── screensize.js
├── index.jade
├── README.md
├── templates
│   └── mainlist.jade
├── testCs.coffee
└── testJs.js

Now, we start “brunch watch” that compiles the application in real time as we edit the files, and for added convenience launches a webserver serving on :3333:


ville@ville-tp:~/p/awa-helloworld$ brunch watch --server
26 Apr 22:03:48 - info: application started on http://localhost:3333/
26 Apr 22:03:48 - info: compiled in 115ms
26 Apr 22:03:49 - info: compiled in 206ms

The resulting application that we can deploy, or preview in browser, gets generated in awa-helloworld/public directory:

ville@ville-tp:~/p/awa-helloworld$ tree public
public
├── config.xml
├── icon.png
├── index.html
├── javascripts
│   ├── app.js
│   ├── templates.js
│   └── vendor.js
├── js
│   └── templates.js
└── s40-theme
├── css
│   ├── s40-theme.css
│   ├── single_landscape.css
│   └── single_portrait.css
├── images
│   ├── arrow-close.png
│   ├── arrow-open.png
│   ├── back_40x40.png
│   ├── button-bg_40x40.png
│   ├── close_40x40.png
│   ├── grid_40x40.png
│   ├── option-menu_titlebar.png
│   ├── refresh_40x40.png
│   ├── search_40x40.png
│   ├── tab_bg.png
│   └── title-bar.png
└── js
├── s40-theme.js
└── screensize.js

“Sources” under app/ directory were processed to produce these files; .coffee files were compiled and concatenated to “app.js”, index.jade was converted to index.html, and so on.

There is one extra directory we didn’t show yet, the “vendor” directory:


ville@ville-tp:~/p/awa-helloworld$ tree vendor/
vendor/
├── jade_runtime.js
└── README.md


If you are using JavaScript libraries like jQuery, you would just copy it under vendor/ and all of them get automatically concatenated to vendor.js. This makes the app fast to load as it reduces the amount of separate files to download. For production, Brunch can also minify the Javascript files to further reduce page weight. jade_runtime.js is here to support *precompiled Jade templates*

Shut up and show it running

In the interest of making things as “familiar” and frictionless as possible, we will preview the application in standard Chrome browser instead of Nokia Web Tools. So we launch Chromium and disable web security to prevent CORS from cramping our style and killing our Ajax requests:

	$ chromium-browser --disable-web-security http://localhost:3333

This doesn’t look at all what you would expect on the phone screen, because we haven’t specified a viewport size:

Chromium inspector has a handy feature of overriding the viewport size (Settings -> Overrides -> Device Metrics), so we use it to set the resolution to match Asha Full Touch devices (240 x 400):

Now, we have a low-fidelity representation of the phone experience! While this certainly wouldn’t satisfy an UI designer, it’s sufficient for a busy software developer that mostly needs to focus on the “logic” side of the web app.

Launching in NWT

Now is the time to get this running with “real” IDE, i.e. Nokia Web Tools. Launch NWT, do File -> New -> Import web app, select public/config.xml and launch the application in local preview:

Is this it?

There is more to show: how to utilize precompiled templates, how to inject the MWL library, and how Jade can help you create MWL UI “components” with reusable, encapsulated logic. These, and other things to smoothen the development flow will be covered in another blog post, as this already got way too long.

The Category bar uses permanent highlight for the currently selected category—one of the category icon buttons is always in the pressed-down state (Pro tip: See the Series 40 UI component demos to see how it’s done on the code side!). Let’s assume that we have placed an action there; if your user wants to trigger the same action twice in a row, how would that be possible when the button is pressed down after the first go? The answer is: there would be no way.

So even though it’s technically possible to place actions in the Category bar, you really shouldn’t do that because your app will become more or less unusable at that point.

Now that we agree that actions do not belong to the Category bar and that it is a navigation element (used for changing the view), let’s go through another burning issue: Category bar should remain level contextual.

Parallel view navigation basically means that all the views are on the same hierarchy level; you don’t drill down when you change the category. Since Back is following hierarchy (not history), this also means that with simple tab navigation the Back will exit the MIDlet (and not change to the previously visited category).

If you combine drill down functionality to either of your category views, you shouldn’t show the Category bar anymore when drilling down. If you did, what would happen when the user taps “Back”? Would they exit the app or move to the main level of the drilled down view? Would you change the backstepping paradigm to make it historical? The user wouldn’t know what to expect anymore.

Because of this conflict you never should show the main Category bar on any drill-down view.

To summarise the main things for the Category bar, here’s a recap from the Full touch UX checklist:

• Categories are used to provide visibility and navigation between different sub-apps, functionalities, or modes within an app.
• The functionality of the Category bar can be compared to tabs.
• The Category bar is level contextual, i.e. you should not show it when user drills down in hierarchy.
• The Category bar is not used for representing actions.
• The currently active category is highlighted.
• If there are more categories than what fits to the Category bar, the categories are accessible via the Category bar extension.
• The Category bar extension icon (three dots) is exclusively used for opening the extension view. Do not use the icon for any other category (or action in Action button 1).
• Back is part of the Category bar, positioned at the rightmost location.

Happy developing,
Sanna

The answer is simple: We are now getting to a saturation point—actually we have been there for a while. In this context, saturation means that relevant UI components will not benefit from higher resolution.

Take the case of fonts. There is a minimum font size which people can read conveniently without glasses at a certain age (see Guy Kawasaki’s formula for minimum font size in a presentation). The issue isn’t limited to fonts, however.

It’s important to be aware of a big trap in developing for mobile. You are not developing for people using a 30 inch display, but for people using a 3 inch display. As a rule of thumb, today the pixels on your computer screen are roughly double the size they are on your mobile phone screen. The font you are seeing from the emulator has twice the size you see on the actual phone. Maybe you have enlarged the output of your emulator, say double. So you see the font four times larger than what your user would see on-device.

Pro tip: Test your app on a real phone!

Let’s continue the thought from a different point of view. You are sitting in a comfortable chair. Your light is not flickering. Your computer is not moving anywhere or shaking because it is not installed in a bus which is driving on a bumpy road. If there is direct sunlight on your computer screen you can block the sun out by either closing the blinds or turning the screen. You do not have to do initial readings from a new screen all the time because you sit in front of the screen for a longer period and you know the screen.

In addition, people grow older and in many cases their eyesight gets worse. Double the age, double the font size!

Don’t forget to consider eye-finger coordination of people using a touch-enabled phone. At the computer you use a mouse on a target 16 times larger than on a phone, with a pointer device of 10th of a millimeter (or better) precision. Your user in the wild, commuting with a phone and your application has only one hand free (the other hand is used for securing herself), and she can probably only use her thumb with a precision of a bit better than a centimeter. Even worse, the actual press down point is a bit lower than the perceived center of the finger tip due to parallax.

Put it this way; while developing, you have plenty of unfair advantages over your user and you must be aware of that.

Now that you are aware of your advantages, what should you do? For full touch phones, keep in mind the following guidelines:

• The minimum font size should be not less than 14 pt
• No side of a touch area should be less than 7 mm (43 pixels in full touch) for finger use
• For thumb use (in the case of one handed use) no side should be less than 9 mm (55 pixels in full touch)
• Include margins of 1 mm (6 pixels in full touch) between all touch areas
• Use icons large enough for its purpose.
• Further information can be found in the UX checklist.

This all means that you cannot fit endless amounts of content on a phone screen and especially on phones with smaller screens (3 inches or smaller). This will be a real challenge for you, but not impossible.

A final piece of advice: Test—on a real phone and with a real user. Take your mom on the shakiest bus ride you have in town and make her operate your app.

With these few considerations, you can help ensure that your app will be accessible for most of your users and even better, they will thank you (at least subconsciously) for paying attention to their needs.

Happy developing!

Sanna

From a developer point of view, detecting if NFC is enabled or not is usually the first step  before using the proximity API to perform NFC-enabled features. There are two steps to check in order to sort out all the scenarios:

1. Check if there are any proximity devices installed on the device.
2. Check if “Tap+Send” setting is switched on or off. The UI for this setting is illustrated below.

In order to check if there are any proximity devices installed, the developers has two options:

• Option 1:  Add ID_REQ_NFC as a HW requirement for the application.
• Option 2:  Use the following code to figure it out:

var proximityDevice = Windows.Networking.Proximity.ProximityDevice.GetDefault();
if (proximityDevice != null) {
   // at least one proximity device installed
}
else {
   // no proximity device installed
}

 
Using the ID_REQ_NFC will hide the application from the app store for the customer if his or her device does not have a NFC chip. However, sometimes NFC is not necessarily the key feature of the application. Therefore, option 2 might be more practical to determine whether to provide NFC related features in the application or not.

Currently (Apr 2013) there is no public API on WP8 for third-party developers to check if “Tap+Send” is switched on or off. If the setting is switched off, Proximity API will behave as if it is working properly without exceptions, but both publish and subscribe requests will be ignored without notification. This could cause your customers confusion.

As currently there is no public SDK to check “Tap+Send” setting, developers have to remind customers to switch on “Tap+Send” setting no matter whether it is currently switched on or off. Also unlike other settings e.g. wifi or bluetooth, “tap+send” does not have yet a public URI schemes from MSDN.

There are few extremely good reasons to follow a platform style guide::

• People will immediately know how to use the basic functionality and they feel kind of “home” right away.
• They will make fewer mistakes.
• The app will look and feel like an integral part of the phone, something that belongs there.
• People will perceive that you have taken care about your customers, i.e. your users.

(Psst… You can find the style guides for Series 40 full touch, Series 40 touch and type, Series 40 non-touch, and Windows Phone 8 online!)

You can also look at this the other way around. Say you do not follow the style guide, but just do a direct port e.g. from an application optimised for a non-Series 40 phone to a Series 40 phone. It is very likely that:

• People won’t know at first how to navigate your app since the main UI elements are not where they would expect them to be. They will be confused even before they started exploring your app! This is a not a good start for trying out something new.
• People will make a lot of mistakes. A simple example: For a Series 40 full-touch application, say you place the Options menu bottom right and the back button top left. Since the application is used by Series 40 users, they will open the Options menu every time they want to step back, since all the other applications in the phone work this way. Even if the rest of your app works flawlessly, people might easily give you bad ratings, just because of this one flaw.
• It really doesn’t help if your app is getting nice reviews for the “non-Series 40 app”; in this example, the users of full-touch phones are not used to the UI style of that “non-Series 40 platform”.
• The application might not render correctly and might look broken.
• People might get the impression that you just want to make money and don’t care about your customers!

Especially for mobile applications (no matter for which platform), it is very important that your application is re-designed and optimised according to the style guide of the platform or even according to the edition of the platform. This has been proven in several large studies. (For some examples, see Jacob Nielsen’s site.)

It also means:

• There is no universal style guide to cover all mobile phone platforms.
• There is no universal answer about where to place common UI components, e.g. “Do I put YES or NO on the left button of a query”. This depends entirely on the platform you are targeting.

If your app is getting bad ratings:

• Read the users’ comments and try to fix what is wrong.
• Have a look into our UX checklist. Following it often helps fix the most common flaws.
• Check the UX webinars online; we have listed some in the Full touch library.
• Check the Event calendar for our UI Clinic webinars and submit your app; we can then figure out together how to improve your app.

For more information on our design resources:

• Designing for Series 40
• Designing for Windows Phone

Oh, and one final thought: We are talking about a style guide, not a style law. Don’t forget to have fun.

Happy developing,
Sanna

The Proximity API in WP8 provides two main classes for NFC related features: ProximityDevice and PeerFinder. ProximityDevice enables functionality like publish/subscribe for NFC messages to and from tags or another devices, while PeerFinder focuses on pairing two devices with a socket connection using NFC tapping or browsing without NFC involvement.

Launching an application from another device is one of the most important features that NFC provides. In most cases, launching the same application from another device is desired to enable further interactions on the same app context between two devices, such as multi-player games, messaging apps, media sharing, etc. Both ProximityDevice and PeerFinder can perform this task.

By using ProximityDevice, developers need to make a URI association for the app and then publish the custom URI to another device. This approach enables the app to interact with other applications as well. However, sometimes app developers do not want to disclose the interface to others due to confidentiality. In such cases, the URI association might not be the best option.

On the other hand, many developers have encountered issues where if both apps are running in the foreground on separate devices and either Fast App Resume (FAR) or BackgroundExecution is defined in the manifest, then passing the URI through NFC will not trigger the MapUri function to launch the desired page in the application. In such cases, the PeerFinder approach to launch the same app on another device becomes more reliable option for developers.

So, how do you launch the same app on another device using PeerFinder? The solution is pretty simple. Developers just need to include the following code in their application. As it requires NFC to make tapping gesture work, the application needs to have ID_CAP_PROXIMITY and ID_REQ_NFC defined in the manifest.

...
PeerFinder.TriggeredConnectionStateChanged += PeerFinder_TriggeredConnectionStateChanged;
PeerFinder.Start();
...
void PeerFinder_TriggeredConnectionStateChanged(object sender, TriggeredConnectionStateChangedEventArgs args)
{
//to-do
}

After PeerFinder is started on device A, then when another device (“B”) enters into proximity range, device B will behave according to the following rules:

• Case 1: If device A has the app but device B doesn’t, device B will get a prompt to search for the app in the store as in the first illustration.
• Case 2: If both device A and B have the app installed and the app is not yet launched on B, then B will get a prompt to launch the app as in the second illustration.
• Case 3: If both device A and B have the app installed and both have launched the same app, then TriggeredConnectionStateChanged will get notified and further actions can be performed.

If the app needs a socket connection for further action between two devices, then it can check the TriggeredConnectState property in the  TriggeredConnectionStateChangedEventArgs passed in with the  TriggeredConnectionStateChanged event. Once the state is changed to TriggeredConnectState.completed, then the Socket property under TriggeredConnectionStateChangedEventArgs will be ready to use.

Some developers want to distinguish between normal application launching and launching by PeerFinder. This can be achieved by using the URI mapper and checking the URI passed. In the PeerFinder case, the URI passed to the MapUri function will look like this:

/MainPage.xaml?ms_nfp_launchargs=Windows.Networking.Proximity.PeerFinder:StreamSocket


Launching the same app on another device using PeerFinder does not require URI association, which implies that no other application can pretend to be the application by associating the same URI theme. This ensures that only the expected app is launched. It also resolves the issue mentioned above in the FAR or BackgroundExecution case. However, if developers want to share their application experiences with other applications, then using ProximityDevice with URI association is the proper option to choose.