Revision as of 17:30, 15 November 2012

This tutorial explains how to combine the Windows Phone 8 voice services with Nokia Music to make a voice-controlled Windows Phone 8 music information app.

Note: This is a community entry in the Windows Phone 8 Wiki Competition 2012Q4

Introduction

With Windows Phone 8, apps can register to handle specific commands. For instance, a calculator app could handle the "add," "subtract," "multiply," and "divide" commands to provide instant, easy access from any place in the OS. Windows Phone 8 also provides text to speech capabilities, and Nokia has provided Nokia Music API for Windows Phone which can be used to query a web service for music information.

This article explains how we use these three features to create a voice-controlled music information app. The app will respond verbally to a voice command along the lines of "Music, what are the top songs by Adele?"

For clarity, the rest of this tutorial has been split up into three parts, Text to Speech, Nokia Music, and Voice Commands.

Before you Begin

Before you begin, we'll need to add a few references, capabilities, and using statements to our project.

Create a new Windows Phone 8 C#/XAML application and name it "Music Info."

References

To use the Nokia Music API in our app, we'll need to reference it. Right-click on References in the solution explorer, then choose Manage NuGet Packages. Use the search bar in the top-right of the NuGet window to search for nokiamusic, and then press the Install button to automatically reference the Nokia Music API and its dependencies in your application.

Capabilities

In the Visual Studio solution explorer of your new project, expand "Properties" and double-click WMAppManifest.xml Choose the Capabilities tab and check both ID_CAP_MICROPHONE and ID_CAP_SPEECH_RECOGNITION.

Using Statements

Open MainPage.xaml.cs and add the following using statements to the top of the page:

using Nokia.Music.Phone;
using Nokia.Music.Phone.Types;
using Windows.Phone.Speech.Synthesis;
using Windows.Phone.Speech.VoiceCommands;

Text to Speech

Our music app will need to be able to respond to the user verbally, and new features in Windows Phone 8 make that extremely easy.

With Windows Phone 8, Microsoft made speech recognition and synthesization simple with the new Windows.Phone.Speech namespace. Our code for the SpeakText function is fairly straightforward:

public async void SpeakText(string TextToSpeak)
{
  SpeechSynthesizer synth = new SpeechSynthesizer();
  synth.SetVoice(InstalledVoices.Default); //you can change the voice to use here
  await synth.SpeakTextAsync(TextToSpeak);
}

Basically, we're going to create a new SpeechSynthesizer, set it's voice to the default installed voice (if you prefer another voice you can change it with synth.SetVoice()), and then await while it speaks whatever text is passed to the function.

Nokia Music

API Key

Before we begin, you'll need to have signed up for an API key at nokia.ly/apisignup.

Creating the music client

We'll start off by creating a blank function that will allow us to respond to an artist's name with that artist's top songs. This function will be called from our voice command handler. Place the below in your MainPage.xaml.cs file:

public void RespondWithTopSongs(string ArtistName)
{
}

Next, we'll add the code necessary for the Nokia Music API. Add the following line to the RespondWithTopSongs function we just created, filling in the App ID and App Code accordingly:

MusicClient nmClient = new MusicClient("App ID", "App Code"); //use the App ID and App Code values that you got from signing up at nokia.ly/apisignup

We now have a Nokia MusicClient that will allow us to query Nokia Music for information.

Searching for the Information

With the Nokia Music API, you find information by searching for artists, songs, albums, and more.

MusicClient has a SearchArtists() function that we will use to search for the user-specified artist. Here's the code we'll use to search for the artist:

nmClient.SearchArtists(
  (ListResponse<Artist> artistResponse) =>
  {
    Artist artist = artistResponse.First<Artist>(); //let's assume that the first returned artist is the one we're looking for
  }, ArtistName);

The code uses nmClient.SearchArtists to search for all artists that contain the ArtistName. The Nokia Music API will by default return the artist who best matches our search first, so that's the artist we assign to the "artist" variable.

Next, we need to use the SearchProducts() function to search for all the products this artist has created, and then narrow that search down to only Tracks and Singles.

The following code will go after we set the artist variable:

nmClient.GetArtistProducts(
  (ListResponse<Product> productResponse) =>
  {
    var songs = productResponse.Where((x) => //productResponse will include all products, not just songs. this will only return tracks and singles
      {
        if (x.Category == Category.Track || x.Category == Category.Single)
        {
          return true;
        }
        else
        {
          return false;
        }
      }
    );
    List<Product> productList = songs.ToList<Product>();
    if (productList.Count >= 3)
      {
      toSpeak = "The top songs by " + ArtistName + " are " + productList[0].Name + ", " + productList[1].Name + ", and " + productList[2].Name;
      }
    else
      {
      toSpeak = "The top song by " + ArtistName + " is " + productList[0].Name;
      }
    SpeakText(toSpeak);
  }, artist);

The code will assign the top three returned songs to a string which we will now need to read out to the user. We can use the SpeakText() function we created in the previous section to accomplish this:

SpeakText(toSpeak);

After you do the above, your RespondWithTopSongs() function should look like this:

public void RespondWithTopSongs(string ArtistName)
{
  MusicClient nmClient = new MusicClient("App ID", "App Code");
  nmClient.SearchArtists(
  (ListResponse<Artist> artistResponse) =>
    {
    Artist artist = artistResponse.First<Artist>(); //let's assume that the first returned artist is the one we're looking for
    nmClient.GetArtistProducts(
    (ListResponse<Product> productResponse) =>
      {
      var songs = productResponse.Where((x) => //productResponse will include all products, not just songs. this will only return tracks and singles
        {
        if (x.Category == Category.Track || x.Category == Category.Single)
          {
          return true;
          }
        else
          {
          return false;
          }
        }
        );
        List<Product> productList = songs.ToList<Product>();
        string toSpeak = "";
       if (productList.Count >= 3)
         {
          toSpeak = "The top songs by " + ArtistName + " are " + productList[0].Name + ", " + productList[1].Name + ", and " + productList[2].Name;
         }
      else
        {
        toSpeak = "The top song by " + ArtistName + " is " + productList[0].Name;
        }
      SpeakText(toSpeak);
      }, artist);
    }, ArtistName);
}

Voice Commands

Anatomy of a Voice Command

Voice commands consist of three parts: the app name, command, and phrase:

• App Name: The app name is simply the name of your app. This part of your command is only necessary if the user is not using your app - if they're already in your app, there's no need for them to say the app name.
• Command: This is the portion of the voice command which actually tells you what the user wants to do. For example, in our app, the command would be "what are the top songs by"
• Phrase: The phrase provides more details about what exactly the user wants to do. In our app the phrase would be "Adele."

Registering a Voice Command

Voice Command Definition

Right-click the project name in Visual Studio and then go Add->New. Select "Voice Command Definition" and name it something like "VCD.xml"

Once the file opens, copy and paste the following:

<?xml version="1.0" encoding="utf-8"?>
 
<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.0">
  <CommandSet xml:lang="en-US">
    <CommandPrefix>Music Info</CommandPrefix>
    <Example> What are the top songs by Adele? </Example>
 
    <Command Name="MusicInfo">
      <Example> What are the top songs by Adele? </Example>
      <ListenFor> what are the top songs by {Artist} </ListenFor>
      <Feedback>Thinking...</Feedback>
      <Navigate />
    </Command>
  <PhraseList Label="Artist">
    <Item>Adele</Item>
    <Item>Johnny Cash</Item>
    <Item>Katy Perry</Item>
    <Item>Garth Brooks</Item>
  </PhraseList>
  </CommandSet>
</VoiceCommands>

This definition file will look for someone saying "What are the top songs by" and then a string. After it receives a command, it will show "Thinking..." until our app launches, at which point we'll need to override the default OnNavigatedTo() handler and replace it with code to start looking for top songs.

Now that we have the VCD file, we have to add the commandset it contains to the system. We want to do this when the app is launched, so we'll override the default OnNavigatedTo() handler. Add the following to your MainPage.xaml.cs file:

protected async override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
  base.OnNavigatedTo(e);
  await VoiceCommandService.InstallCommandSetsFromFileAsync(new Uri("ms-appx:///VCD.xml", UriKind.RelativeOrAbsolute));
}

Note: You may have noticed in the VCD file above that we specified four specific artists instead of using a wildcard to allow the user to specify any artist of their choice. With voice commands in Windows Phone 8, there is no effective wildcard. You can use {*} to detect that the user has said something, but you can't figure out what the user said. There are a few workarounds to this problem, including asking the user for the artist's name once the app is started. For more information, look here and here.

Handling the Command

Once we get a voice command, we'll need to add to our newly overridden OnNavigatedTo() handler to detect when it's being launched as a voice command. Copy and paste the below code as the OnNavigatedTo() function you just made:

protected async override void OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
{
  base.OnNavigatedTo(e);
  await VoiceCommandService.InstallCommandSetsFromFileAsync(new Uri("ms-appx:///VCD.xml", UriKind.RelativeOrAbsolute));
  if (NavigationContext.QueryString.Any())
    {
    RespondWithTopSongs(NavigationContext.QueryString["Artist"]);
    }
}

This code will check to see if there is a QueryString, and if so it will call the function we made in the last section with the artist name passed by the phone.

Congratulations!

Congrats - you now have a working app that incorporates a few of the awesome new features in Windows Phone 8. You can learn more about these features with the links below.

Bibliography/See Also

• What's new in Windows Phone 8 (Nokia Developer Wiki)
• How Many Lines of Code Does it Take to Build a Complete WP8 Speech App (Rob Chambers Blog)
• How to create a PhraseList in a VCD file that includes every word (wildcard)? (MSDN Forums)
• The Nokia Music WP API Client Test App (Github)
• Voice Commands in Windows Phone 8 (wiki)