Behind the Scenes: Headset Camera app for the N9

The logical step after the “Volume+ as Camera Button” app (Nokia Store link) for the N9 is another app that allows you to take photos while not touching your N9 at all. While time-triggered photos are fun, remote-triggered photos are.. erm.. “funner”? So what kind of remote “buttons” can we easily get on the N9? The remote control button on the headset is both “remote” and a “button”. Also, as seen in Panucci and gPodder versions since the N900, Bluetooth headset buttons can also be queried by applications. So what do we get by combining remote control and photo taking? The Headset Camera app (Nokia Store link) for the N9! Or – for the visual reader – this:

If you want to integrate such features into your own app, the code for querying the headset buttons is readily available in the gPodder source tree (src/gpodder/qmlui/helper.py):

import dbus
 
class MediaButtonsHandler(QtCore.QObject):
    def __init__(self):
        QtCore.QObject.__init__(self)
        headset_path = '/org/freedesktop/Hal/devices/computer_logicaldev_input_0'
        headset_path2 = '/org/freedesktop/Hal/devices/computer_logicaldev_input'
 
        system_bus = dbus.SystemBus()
        system_bus.add_signal_receiver(self.handle_button, 'Condition',
                'org.freedesktop.Hal.Device', None, headset_path)
        system_bus.add_signal_receiver(self.handle_button, 'Condition',
                'org.freedesktop.Hal.Device', None, headset_path2)
 
    def handle_button(self, signal, button):
        if signal == 'ButtonPressed':
            if button in ('play-cd', 'phone'):
                self.playPressed.emit()
            elif button == 'pause-cd':
                self.pausePressed.emit()
            elif button == 'previous-song':
                self.previousPressed.emit()
            elif button == 'next-song':
                self.nextPressed.emit()
 
    playPressed = QtCore.Signal()
    pausePressed = QtCore.Signal()
    previousPressed = QtCore.Signal()
    nextPressed = QtCore.Signal()

MediaButtonsHandler is already a QObject subclass, so you can easily expose an instance of this class to your QDeclarativeView rootContext() and connect to the signals in QML (such a “headset button handler” might actually be a good candidate for inclusion into nemo-qml-plugins in Sailfish OS and Nemo Mobile?). As it’s really just using the Python D-Bus bindings to get property changes from Hal devices, the code above should be easy (read: trivial) to port from Python to Qt/C++. Be aware that you need to connect to both …/computer_logicaldev_input_0 and …/computer_logicaldev_input, which can both exist if you have a cable headset and a Bluetooth headset connected at the same time.

You can get the Headset Camera App for the N9 in Nokia Store now, there is also a video on YouTube showing the app. Or start integrating headset button features into your own app or scripts by adapting the code above. One use case that comes to mind is using the previous/next buttons on a Bluetooth headset to control a photo slideshow on the N9 connected to TV-Out. Enjoy :)

HTML5 Web Apps on Mobile Devices

Get out your Buzzword Bingo cards, we’re talking HTML5. And Canvas2D. And WebGL. See? Check them off and then continue reading. So, while writing “native” apps using JavaScript is definitely possible and works great with QML, some games are just simple enough (or want to have a broad enough audience) to warrant writing everything in HTML5.

This might also be a good time to check off XmlHttpRequest on your BB card, even if none of the following games use it – you might want to use it in your applications or game for things such as server-side stored high scores.

As far as Maemo and MeeGo is concerned, you might want to try out some of these games (especially the WebGL variant of One Whale Trip) in Fennec (aka Mobile Firefox – get it for: N900, N950/N9, Nemo Mobile).

Color Lines: This one simply uses Canvas2D, and works nicely on all mobile browsers that I tested – Maemo, MeeGo, Android, WP7.5, BB Playbook, iOS. Comes in at about 650 lines of rather well-documented JavaScript, and could easily be ported to use QML as a renderer if need be (it would be good to have a QML Plug-In that provides a JavaScript context + (a subset of) the Canvas2D API – without using WebKit (cross that off, too), that is). Also, the N900′s stock browser has performance problems rendering this, while on the same device in Fennec it’s quite playable.

Circle1D: This is a straight Python-to-JavaScript port of a lame 2D “Physics” Engine. It’s kept very (read: very, very) simple, and collision detection could be done in a nicer way, but the inefficiency of it provides a nice benchmark for comparing JavaScript engine performance (I’m sure you can find “engine performance” on your bingo card as well) on mobile devices. The N900′s default browser can’t handle it at all, but Fennec can at least render/simulate it, albeit slowly.

One Whale Trip: This game actually started out as a Python game for PyWeek last September, which was also ported to the N950/N9, but as a test for trying out WebGL, I decided to port it from Python/PyGame to JavaScript/Canvas2D and then to WebGL (the Python version also contains two renderers – a “blitting” one using PyGame surfaces, and an OpenGL one using OpenGL [ES 2 on mobile]). The Canvas2D version works again in all modern mobile browsers (same as above), the WebGL only works on browsers supporting WebGL, for example Fennec/Firefox on both the N900 (even though very slowly) and not in any of the stock browsers (even not the one on the N9). As WebGL is “roughly” the same as OpenGL ES 2.x, one could imagine sharing at least shader programs for a possible C++-or-JavaScript cross-platform application.

So yeah, for smaller applications and/or games, HTML5 is definitely an option. In Firefox OS, your HTML5 web app will – also with WebGL – work and integrate nicely as “native” app. If you also want to create “native” applications (maybe after finishing the HTML5 version), consider encapsulating your JavaScript code so that you can re-use it in QML code, or (in case of WebGL apps), at least design the rendering part of your application in such a way that the code/architecture and shader programs can be shared with a C++ port of your existing HTML5 app.

Another option that’s worth considering: Writing a compatiblity application layer that can load (specially-crafted) WebGL subset applications and display them on a fullscreen SDL-(or Qt)-provided window. Applications written in this WebGL subset could then be deployed on the web as HTML5 application or “natively” running on top of a JavaScript engine only. I’d call that “webglenv“, and no, I haven’t written it yet.

Petals for Harmattan – A pure Qt4/Qt5 JS/QML puzzle game

Next up in my list of things I did in the last weeks/months and never blogged about is Petals (Nokia Store link), a “beautiful, brain-teasing puzzle game for 1-4 players” if the game’s website is to be believed (I would like to think it is…). As always, there’s some technical details about the porting and creation of this game. While another recent game (Tetrepetete) has been done on a low level (C++ using no frameworks, and interfacing with multiple front-ends directly, including an OpenGL ES frontend, a console-based ncurses frontend(!) as well as a server-sent events/XHR/Canvas2D-based HTML5 frontend(!!)), this one is approaching things from a very high level: JavaScript.

Petals: A puzzle game written in pure JavaScript and QML

The gameplay logic of the game is implemented in pure JavaScript (without any QML dependencies), so it could easily be ported to, say, HTML5, but for integration reasons, QML seemed like the better choice for a release on the N9/Harmattan. Also, writing things in JavaScript wouldn’t preclude a console-based frontend using nodejs and node-ncurses from happening should the need arise (making the flowers look good in ASCII art would be the challenge there – or cheating by using libcaca). Ok, ok – stop cursing, I’ll stop talking about curses (cue laugh track).

Writing pure QML applications has the advantage of easing porting to Qt 5. While QtQuick 1.1 still exists on Qt 5 (and is the only QML option at the moment if you are also targetting iOS), QtQuick 2.0 is usually the better choice for performance reasons.

In my case, the changes necessary to port from QtQuick 1.1 to QtQuick 2.0 were:

  • Change “import QtQuick 1.1” to “import QtQuick 2.0” (sed(1) helps here)
  • Instead of assigning a JavaScript function to a property to create a dynamic property binding (item.someprop = function() { return otheritem.otherprop * 3.0; }), this function has to be wrapped in a call to Qt.binding() in Qt 5 (see “Creating Property Bindings from JavaScript” in the Qt 5 docs)
  • Instead of using SQL Local Storage directly as in QtQuick 1.1, use QtQuick.LocalStorage 2.0, which you can still do in your .js files – use “.import” as described in this blog post
  • In your C++ launcher (in case you need one), QApplication becomes QGuiApplication, and QDeclarativeView becomes QQuickView
  • Use “QT += quick qml” instead of “QT += declarative” in your qmake project file

And that’s basically it. Of course, as this is a full-screen game with custom UI, no platform-specific components (such as Harmattan Components or Sailfish Silica) are used, so porting is a bit easier there (no need to “wait” for specific components to be compatible with QtQuick 2.0, which might realistically not happen at all for Harmattan Components). More screenshots of Petals and download links for multiple platforms can be found on the Petals Website.

Upcoming: Billboard 1.0.9 for Nokia N9

Turns out I haven’t posted here for two months, so here we go again: Billboard, your favorite low-power mode standby screen will soon receive a new update – version 1.0.9 has been uploaded to Nokia Store QA two days ago, and should hopefully pass QA and be available as an update in the next few days. This release brings a few major under-the-hood improvements and small bugfixes:

  • Fixed MeeCast icon (in 1.0.8, you can already use <<{meecast-icon-src}>>)
  • New formatter that allows you to nest {} expressions used for adding dynamic content
  • Optional image dithering (using # after the filename) for better colors in low power mode

With the new formatter, you can now output {} expressions in your scripts so that they get replaced, and similarly pass {} expressions as parameters to your scripts (for example to modify them in some way before displaying). This should allow for even more customization, some examples of what users have been doing on their N9 standby screen can be seen in the Billboard Standby Screen support thread on talk.maemo.org.

If you are looking for additional ways to tweak and enhance your Billboard-on-N9 experience, have a look at billboard-scripts, a growing collection of Shell and Python scripts that provide even more ways of customizing your standby screen.

If you haven’t purchased Billboard from Nokia Store yet, you can get the current version now for your N9, and get the upgrade to 1.0.9 as soon as it’s available. If you are already a happy user, watch your application updates in the next few days, and get the new version.

Tetrepetete, SMS Backup and apkenv updates

Whew, January is pretty much over already, and there haven’t been any updates here. So this means here’s a short cumulative update of what’s happened over the last few weeks:

Tetrepetete
If you are not afraid of falling blocks, and don’t mind the lack of colors, give Tetrepetete a try. Not to be confused with a game of a totally different name, this game brings falling blocks of 4 to your N900 and N950/N9. While the full color version is still not available as such, the free greyscale version is up for grabs as .deb on the website. Including a cameo appearance by That Rabbit from That Rabbit Game for no good reason. Play or discuss. Or be productive. Or something.

SMS Backup GUI
Instead of rolling my own solution from scratch, I found the wonderful MeegoSMSBackupRestore project by Tony Wang, which is a command-line tool for Harmattan devices to backup and restore SMS messages (as if that wasn’t clear from the application name..). It was missing a GUI, so I’ve added one. The announcement and download, as well as the Debian source package (.dsc, .tar.gz) are available from an Internet near you. Feel like Git? We have you covered as well.

apkenv updates
If you haven’t been watching the apkenv Git repo closely (I know you haven’t), you might have missed the Pandora port by crowriot, which – while not being directly useful to Maemo/MeeGo users as such – also brought improvements to the Cut the Rope module, which somehow works now. To be discussed at talk.maemo.org, and patches (as always) welcome.

In other (totally unrelated) news, the thesis is done (yay!).

gPodder 3.4.0 for MeeGo 1.2 Harmattan

As promised, following the 2.20.3 release for Maemo 4 and Maemo 5, a new version of gPodder 3 has been released today. This release brings some new improvements to the QML UI that should make scrolling in the episode list faster, as well as align a bit more with the Harmattan UX Guidelines.

gPodder 3.4.0 running on MeeGo 1.2 Harmattan (N9)

As always, updated packages will be available via Nokia Store once the QA process has finished. As there have been problems with apps.formeego.org recently (3.3.0 isn’t even out of the staging area yet, after 3 months), we now provide a package of gPodder for MeeGo 1.2 Harmattan on gpodder.org/downloads for those who want to avoid or can’t use Nokia Store (you have to enable packages from unknown sources, and you might have to uninstall the previous version if it came from Nokia Store or Apps For MeeGo due to the Aegis security policy). The source and packaging is available in the “harmattan” branch in Git, as has always been the case.

I hope you like the new UI changes. If you are missing the feed update button in the lower left corner, try “pull to refresh” on the podcast list (this avoids accidental updates when pressing the back button twice). I’d like to create a nicer “pull to refresh” implementation, but the limitations of Qt 4′s QML Flickable prevent me from creating a better one (if you have a better implementation than gPodder’s PullDownHandle, I’d love to hear from you). Scrolling the episode list should now also be faster (than before), and some unnecessary animations were removed in the process.

apkenv 42.1.0 source code release

The compatibility layer for applications packed as .apk (that are running natively and use OpenGL ES) has seen its source code released yesterday. Supports Maemo 5 (Fremantle) and MeeGo 1.2 (Harmattan), which means your N900, N950 and N9 are covered. Documentation is provided in the source, and the wrapper generator scripts are also released. Looking forward to contributions and new modules from the community. Details can be found on the apkenv website.

The quick way to USB tethering via SSH on your N9

Here’s something obvious (and I’m sure it has been discussed before, I just can’t find the link right now), but it might be helpful if you haven’t played around with SSH much yet, and your Wi-Fi Hotspot isn’t working (the case for me right now). The goal is to get an internet connection over your N9/N950 from a PC (tested with Linux, expected to work with OS X and probably also works on Windows – on Windows you might need something like PuTTY).

  1. Enable developer mode on your device
  2. Connect your device to your computer via USB
  3. Select “SDK mode” when asked for the USB connection type
  4. Use the SDK connection utility, and select USB connection
  5. Note the password displayed in the “Connectivity Details” screen
  6. On your computer, use “ssh -D 9898 developer@192.168.2.15″
  7. Accept the host key question, and enter the password from step 5
  8. You should be greeted by a Busybox prompt “/home/developer $” – leave that open in the terminal window in the background
  9. At this point, a SOCKS proxy server is running on port 9898, and you can use it in any applications supporting a SOCKS proxy (there are even utilities like socksify(1) (Debian package: dante-client) that make generic network applications work through a SOCKS proxy)
  10. To use it in Firefox, go to Edit – Preferences – Advanced – Network – Settings…, then choose “Manual proxy configuration” and set “SOCKS Host:” to localhost and port to 9898 (be sure to disable the proxy again when you want to browse via a normal Wi-Fi/Ethernet connection)

Instead of doing steps 4 and 5, you can set up a permanent password for your “user” account on the N9, and even set up a SSH key on the N9 to avoid having to enter the password. You can make step 6 simpler by adding an entry to your ~/.ssh/config file (on your host) – see ssh_config(5) for details:

Host n9proxy
    HostName 192.168.2.15
    User developer
    DynamicForward 9898

After that, a “ssh n9proxy” (possibly followed by the developer password) is all you need to set up the proxy. This method is arguably easier (and definitely safer) than using the Wi-Fi hotspot, and instead of using up battery on your N9, it gets charged via the USB port while you are using it.

By the way: You will have to manually connect your N9 to your mobile internet connection, this won’t happen automatically.

Dalvik "Hello World" on Harmattan (in a chroot)

Here’s a quick and boring tutorial on how to get a Java “Hello World” application running on the Dalvik VM of Nitdroid within Harmattan. This requires the Nitdroid Open Mode Kernel (might or might not work with Inception), all Nitdroid files and a bit of patience. This might brick your device or worse, and require a reflash at best. As you can see from the screenshot below, it’s not that exciting ;)

  1. If you want to avoid the Open Mode Warning on each boot, remove the disclaimer from the cal area (this can only be done while in closed mode, as the cal area becomes read-only when flashing open mode until closed mode is re-flashed. This is also the reason why you can’t set a device lock code in open mode – the lock code is saved in the cal area) – this is an optional step, and just a cosmetic fix.
  2. Flash the Nitdroid Open Mode Kernel (be sure to read the “Known limitations” on that post) – this works both on the N950 and N9.
  3. Boot into Harmattan open mode and replace the preinit script to allow booting into Nitdroid (again, read and understand the warnings on that post)
  4. Then, download a Nitdroid tarball (the latest is alpha 5, but I had better luck with alpha 4) and extract it to /home/nitdroid/ – the post explains the steps in great detail, including the kernel flashing and preinit replacement
  5. Reboot into Nitdroid (this might be optional, but do it just to check if your Nitdroid installation is working)
  6. Reboot into Harmattan again – the following commands are all carried out as root user inside Harmattan (using “devel-su“)
  7. Bind mount /dev/ and /sys/ into Nitdroid:
    mount –bind /dev/ /home/nitdroid/dev/
    mount –bind /sys/ /home/nitdroid/sys/
  8. chroot into the Nitdroid hierarchy:
    /usr/sbin/chroot /home/nitdroid/
  9. Set some environment variables (not all of them might be needed, you can find them in “setup the global environment” of init.rc — save these commands into “setupenv.sh” or something in /home/nitdroid/ and save yourself some typing):
    export PATH=/system/bin:/system/xbin:/usr/sbin:/usr/bin:/sbin:/bin
    export LD_LIBRARY_PATH=/system/lib
    export ANDROID_BOOTLOGO=1
    export ANDROID_ROOT=/system
    export ANDROID_ASSETS=/system/app
    export ANDROID_DATA=/data
    export EXTERNAL_STORAGE=/mnt/sdcard
    export ASEC_MOUNTPOINT=/mnt/asec
    export LOOP_MOUNTPOINT=/mnt/obb
    export BOOTCLASSPATH=/system/framework/core.jar:/system/framework/core-junit.jar:/system/framework/bouncycastle.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/android.policy.jar:/system/framework/services.jar:/system/framework/apache-xml.jar:/sys
  10. Start the Dalvik VM:
    / # /system/bin/dalvikvm
    Dalvik VM requires a class name

Ok, great – 10 steps to start the Dalvik VM, but I promised a Hello World. For that, you need the Android SDK (and the corresponding Java JDK) and follow the “Basic Dalvik VM Invocation” instructions (steps 1 to 6 on your host computer, from step 6 on your Harmattan device):

  1. With your favorite editor (mine is vim), create a file “Foo.java” with the following content:
    public class Foo {
        public static void main(String [] args) {
            System.out.println(“Hello, world”);
        }
    }
  2. Compile it:
    javac Foo.java
  3. Use the “dx” utility (from the Android SDK) to package the Foo.class (generated by javac) into a .jar
    dx –dex –output=foo.jar Foo.class
  4. Bonus exercise: “less” the foo.jar file to see its contents:
    % less foo.jar
    Archive:  foo.jar
     Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
    ——–  ——  ——- —- ———- —– ——–  —-
          72  Defl:N       70   3% 2012-10-23 15:51 1294a38f  META-INF/MANIFEST.MF
         732  Defl:N      404  45% 2012-10-23 15:51 b6e3654e  classes.dex
    ——–          ——-  —                            ——-
         804              474  41%                            2 files
  5. Copy this over to your Harmattan device (assuming USB Networking + Developer Mode + “user” access, “developer” should also work):
    scp foo.jar user@192.168.2.15:
  6. SSH into your Harmattan device, become root (devel-su) and copy the .jar file into the Nitdroid hierarchy:
    cp /home/user/foo.jar /home/nitdroid/
  7. chroot into Nitdroid again and set up the environment (see above)
  8. Finally, run Dalvik VM, telling it to run the class “Foo” and using a classpath of “foo.jar”:
    / # /system/bin/dalvikvm -cp foo.jar Foo