Universal Windows App Development with Cortana and the Speech SDK 1 and 2 @Channel9

Course Website: http://channel9.msdn.com/Series/Universal-Windows-App-Development-with-Cortana-and-the-Speech-SDK

1. Introducing Cortana & Getting Started with Speech

Speech is Personal, Experiential.

Bing powers Cortana. Cloud recognizer runs algorithms
Core Functions: Communicate. Remember. Find.

Fun Cortana. Can tell jokes. Can sing a song.

2. Using Speech Synthesis in Mobile Apps

2.1 Basic Speech Synthesis Integration

2.1.1 Speech Synthesis Options

Speak with default speech settings. -Windows.Media.SpeechSynthesis
namespace
Speak with any installed speech language
Customize Text-to-Speech(TTS) voice with speech synthesis Markup Language (SSML)
TTS support introduced in WIndows Phone 8 SDK and Windows Store 8.1 SDK

2.1.2 In Application Speech Synthesis (Using default voice)

//windows phone store app

Speech Thesis uses a MediaElement control to speak text

// Synthesis
<MediaElement Name="audioPlayer" AutoPlay="True".../>

C# code behind: //Speak text
//Fundtion to speak a text string

private async void SpeakText(MediaElement audioPlayer, string textToSpeak){

SpeechSynthesizer synthesizer=new SpeechSynthesizer();

SpeechSynthesisStream ttsStream= await synthesizer.SynthesizeTextToStreamAsync(textToSpeak);

audioPlayer.SetSource(ttsStream,"");
// This starts the player because AutoPlay="True"
}

<strong>Demo to use default voice: </strong>

private async void ButtonLookup_Click(object sender, RoutedEventArgs e)
{
string location = txtLocation.Text.Trim();

var wr = await owms.GetWeather(location);
if (wr != null)
{
var weatherText = "The current temperature in {0} is {1}°F, with a high today of {2}° and a low of {3}°.";
string weatherMessage = string.Format(weatherText, wr.Name, (int)wr.MainWeather.Temp, (int)wr.MainWeather.MaximumTemp, (int)wr.MainWeather.MinimumTemp);
lblMessage.Text = weatherMessage;
lblTemp.Text = string.Format("{0}°", (int)wr.MainWeather.Temp);
ReadText(weatherMessage);
}
}

private async void ReadText(string mytext)
{
//Reminder: You need to enable the Microphone capabilitiy in Windows Phone projects
//Reminder: Add this namespace in your using statements
//using Windows.Media.SpeechSynthesis;

// The media object for controlling and playing audio.
MediaElement mediaplayer = new MediaElement();

// The object for controlling the speech synthesis engine (voice).
using (var speech = new SpeechSynthesizer())
{
//Retrieve the first female voice
speech.Voice = SpeechSynthesizer.AllVoices
.First(i => (i.Gender == VoiceGender.Female && i.Description.Contains("United States")));
// Generate the audio stream from plain text.
SpeechSynthesisStream stream = await speech.SynthesizeTextToStreamAsync(mytext);

// Send the stream to the media object.
mediaplayer.SetSource(stream, stream.ContentType);
mediaplayer.Play();
}
}

2.1.3 Required Capability

Microphone Capability in package.appxmanifest

-Required for TTS in Windows Phone apps

- Optional for TTS in Windows Store apps

<Capabilities>
<Capability Name="internetClientServer"/>
<DeviceCapability Name="microphone"/>
</Capabilities>

2.2 Speech Synthesis Voice Settings

VoiceInformation currentVoice=(VoiceInformation) lstVoice.SelectedItem;

speech.voice=currentVoice;

SpeechSynthesisStream stream=await speech.SynthesizeTextToStreamAsync(myText);

mediaplayer.SetSource(stream,stream.ContentType);

mediaplayer.Play();

2.3 Speech Synthesis Markup Language

<speak
version='1.0'
xmlns='http://www.w3.org/2001/10/synthesis'
xml:lang='en-us'>
<voice name='Microsoft Zira Mobile'>
<prosody pitch='low' rate='0.9'>
This is the text what will be read by the speech synthesizer.
<prosody>
<voice>
<speak>

private async void ReadSsmlText(string mytext)
{
VoiceInformation currentVoice=(VoiceInformation)lstVoices.SelectedItem;

string Ssml=@"<speak version='1.0' " +
"xmlns='http://www.w3.org/2001/10/synthesis' xml:lang='" + currentVoice.Language+"'>"+
"<voice name='"+ currentVoice.DisplayName+"'>"+
"<prosody pitch='"+lstPitch.SelectedItem.ToString() +"' rate=' "+sldRtae.Value.ToString()+"' </voice></speak>";

SpeechSynthesisStream stream=await speech.SynthesizeSsmlToStreamAsync(Ssml);

mediaplayer.SetSource(stream,stream.ContentType);

mediaplayer.Play();

}

2.4 Advanced Text-to-Speech Topics

2.4.1 Save Speech Audio Streams to a File

//open the output stream
Widnows.Storage.Streams.Buffer buffer =new Windows.Storage.Streams.Buffer(4096);
IRandomAccessStream writeStream=(IRandomAccessStream) await file.OpenAsync(FileAccessMode.ReadWrite);
IOutputStream outputStream = writeStream.GetOutputStreamAt(0);
DataWriter dataWriter =new DataWriter(outputStream);

//copy the stream data into the file
while(synthesisStream.Position<synthesisStream.Size)
{
await synthesisStream.ReadAsync(buffer,4096,InputStreamOptions.None);
dataWriter.WriterBuffer(buffer);
}

2.4.2 Play Speech Audio in the Background

MediaElement can play in the background
 
AudioCategory="BackgroundCapableMedia"    for Windows Store 8.1 only

Windows phone 8.1 requires a background audio agent

SystemMediaControls