.NET [Source] How to make a voice controlled program!

[.NET] [Source] How to make a voice controlled program! - Printable Version

+- Hack Forums (https://hackforums.net)
+-- Forum: Tech (https://hackforums.net/forumdisplay.php?fid=444)
+--- Forum: Coding (https://hackforums.net/forumdisplay.php?fid=151)
+---- Forum: Software Development (https://hackforums.net/forumdisplay.php?fid=118)
+---- Thread: [.NET] [Source] How to make a voice controlled program! (/showthread.php?tid=4359083)

Pages: Pages 1 2

[Source] How to make a voice controlled program! - Dubito - Jul 25, 2014

Welcome to my tutorial thread on how to code a Voice Controlled "Personal Assistant" with the addition on how to gather weather information from your local area!

Lets Begin!

What will you need?

Visual Studio (I'm using Visual Studio Ultimate 2013

"Some" knowledge in coding in VB.net

.Net 4.5 framework (Needed for speech imports)

Creating the Personal Assistant base

First we need to grab the imports needed, in the namespace (Above the line that says "Public Class Form1" you need to add the following:

PHP Code 


<?php 
Imports System.Speech.Synthesis
Imports System.Speech.Recognition
Imports System.Speech.AudioFormat
Imports System.Speech
Imports System.Net
Imports System.IO
Imports System.Linq
Imports SpeechLib
Imports System.Xml 

Now, this WILL return an error! To fix this, simply add a System Reference! Spoiler (Click to View) Go to Assemblies/Framework and look for the reference "System.Speech"
Spoiler (Click to View)
Click ok and the errors will be fixed!

If errors are not fixed, you may have forgotten to click the checkbox in the reference menu. Try to add the reference again but look for the checkbox!

Now add the following code in Form1's class:

PHP Code 


<?php 
Public synth As New Synthesis.SpeechSynthesizer
    Public WithEvents recognizer As New Recognition.SpeechRecognitionEngine
    Dim gram As New System.Speech.Recognition.DictationGrammar()
    Dim WithEvents reco As New Recognition.SpeechRecognitionEngine 

This sets the program up ready for speech recognition and equally important, Speech synthesis! (The computer's ability to talk back)

Next you will need to go into the private sub generated for when Form1 Loads, paste the following code:

PHP Code 

<?php 
reco.SetInputToDefaultAudioDevice()

      Dim gram As New Recognition.SrgsGrammar.SrgsDocument

      Dim wordRule As New Recognition.SrgsGrammar.SrgsRule("word")

      Dim wordList As New Recognition.SrgsGrammar.SrgsOneOf()

      wordRule.Add(wordList)

      gram.Rules.Add(wordRule)

      gram.Root = wordRule

      reco.LoadGrammar(New Recognition.Grammar(gram))

      reco.RecognizeAsync()

The next step is important!
Reco.recognizeAsync will only recognize SINGULAR words!
Delete the closing bracket ")" and press the down arrow key once, this will change it to Multiple recognition mode. Then replace the closing bracket ")" and your done.
Spoiler (Click to View)

Next, add a second private sub preparing to take over should the multiple Async fail:

PHP Code 

<?php 
Private Sub reco_RecognizeCompleted(ByVal sender As Object, ByVal e As System.Speech.Recognition.RecognizeCompletedEventArgs) Handles reco.RecognizeCompleted

      reco.RecognizeAsync()

    End Sub

Now we need to make the code listen in-CASE the user says a keyword, to do this we need to setup the "case" sub:

PHP Code 


<?php 
Private Sub reco_SpeechRecognized(ByVal sender As Object, ByVal e As System.Speech.Recognition.RecognitionEventArgs) Handles reco.SpeechRecognized

      Select Case e.Result.Text
'------

'------
      End Select 

End Sub
Your finished! You now have the base to a personal assistant! All you need to do is add keywords, and their case's!
Currently your program won't build, this is because there are now terms for it to listen to! Lets start by adding a weather function!

Firstly, goto the line which says

PHP Code 


<?php 
Dim wordList As New Recognition.SrgsGrammar.SrgsOneOf() 

Your going to want to add the string "weather to it", it should end up looking like this:

PHP Code 


<?php 
Dim wordList As New Recognition.SrgsGrammar.SrgsOneOf("weather") 

CAUTION! Only use lowercase in the terms and cases, otherwise said keyterm/case WILL NOT WORK

Now your program is constantly looking for the user to say the word "weather". However, once weather is said, it doesn't know what to do! To fix this, we need to add a case. A case works in the same way as it does in the english language. for example: Take an umbrella inCASE it rains.
In the context of this program, we need to tell the code to do a certain thing inCASE the user says a certain thing.

To do this, find the two big lines that look like this:

PHP Code 


<?php 
Select Case e.Result.Text
      '------

      '------
      End Select 

Inside the two lines, you will need to add your case!
Paste the following inside the two lines:

PHP Code 


<?php 
Case "weather"
      Dim mean
      Dim ad As New SpVoice
      Dim URLString As String = "http://api.openweathermap.org/data/2.5/weather?q=*****&mode=xml"
      Try 

This sets up the start of our weather program. Replace '******' with your city, state or country. Now we need to try to parse the XML and read certain nodes. To do this, paste the following code:

PHP Code 


<?php 
Dim m_xmld As XmlDocument
      Dim m_nodelist As XmlNodeList
      Dim m_node As XmlNode
      Dim m_node2 As XmlNode
      'Create the XML Document
      m_xmld = New XmlDocument()
      m_xmld.Load(URLString)
      m_nodelist = m_xmld.SelectNodes("current/weather")
      For Each m_node2 In m_nodelist
      Dim weather = m_node2.Attributes.GetNamedItem("value").Value
      mean = weather
      Next 

Lets break this code down. m_xmld is the XML file we downloaded containing the information we need. m_nodelist is a list of all the nodes in the file. m_node and m_node2 are singular nodes in the file.
Futher in the code, we state that "m_nodelist = m_xmld.SelectNodes("current/weather")
Since current is the title of the XML and weather is the branch we need, this line tells the program that we want a specific detail from this branch.
Now paste this code:

PHP Code 


<?php 
m_nodelist = m_xmld.SelectNodes("/current/temperature")
      'Loop through the nodes
      For Each m_node In m_nodelist
      Dim temp = m_node.Attributes.GetNamedItem("value").Value
      temp = temp - 272.15
      Dim temp2 As Decimal = temp
      Convert.ToInt32(temp2)
      ad.Speak("Current temperature in ***** is " + temp2.ToString)
      ad.Speak("Degrees. The " + mean.ToString)
      Next
Catch errorVariable As Exception
      'Error trapping
      Console.Write(errorVariable.ToString())
      End Try 

This code finds a second node, containing temperature and gets the temperature vaule. Since the temperature in the XML file is in Kelvin, we need to take away 272.15 (To convert into degrees).
Again, replace *** with your city, state or country.

Now plug in your microphone, then build or run your program.
Say the word 'weather' and your computer should talk back to you!

Full source (In case you can't follow the steps)
Spoiler (Click to View)

PHP Code 


<?php 
Imports System.Speech.Synthesis
Imports System.Speech.Recognition
Imports System.Speech.AudioFormat
Imports System.Speech
Imports System.Net
Imports System.IO
Imports System.Linq
Imports System.Xml

Public Class Form1
    Public synth As New Synthesis.SpeechSynthesizer
    Public WithEvents recognizer As New Recognition.SpeechRecognitionEngine
    Dim gram As New System.Speech.Recognition.DictationGrammar()
    Dim WithEvents reco As New Recognition.SpeechRecognitionEngine

    Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
      reco.SetInputToDefaultAudioDevice()

      Dim gram As New Recognition.SrgsGrammar.SrgsDocument

      Dim wordRule As New Recognition.SrgsGrammar.SrgsRule("word")

      Dim wordList As New Recognition.SrgsGrammar.SrgsOneOf("weather")

      wordRule.Add(wordList)

      gram.Rules.Add(wordRule)

      gram.Root = wordRule

      reco.LoadGrammar(New Recognition.Grammar(gram))

      reco.RecognizeAsync()
    End Sub

    Private Sub reco_RecognizeCompleted(ByVal sender As Object, ByVal e As System.Speech.Recognition.RecognizeCompletedEventArgs) Handles reco.RecognizeCompleted

      reco.RecognizeAsync()

    End Sub

    Private Sub reco_SpeechRecognized(ByVal sender As Object, ByVal e As System.Speech.Recognition.RecognitionEventArgs) Handles reco.SpeechRecognized

      Select Case e.Result.Text
      '------

      Case "weather"
      Dim mean
      Dim ad
      ad = CreateObject("sapi.spvoice")
      Dim URLString As String = "http://api.openweathermap.org/data/2.5/weather?q=Bristol&mode=xml"
      Try
      Dim m_xmld As XmlDocument
      Dim m_nodelist As XmlNodeList
      Dim m_node As XmlNode
      Dim m_node2 As XmlNode
      'Create the XML Document
      m_xmld = New XmlDocument()
      m_xmld.Load(URLString)
      m_nodelist = m_xmld.SelectNodes("current/weather")
      For Each m_node2 In m_nodelist
      Dim weather = m_node2.Attributes.GetNamedItem("value").Value
      mean = weather
      Next
      m_nodelist = m_xmld.SelectNodes("/current/temperature")
      'Loop through the nodes
      For Each m_node In m_nodelist
      Dim temp = m_node.Attributes.GetNamedItem("value").Value
      temp = temp - 272.15
      Dim temp2 As Decimal = temp
      Convert.ToInt32(temp2)
      ad.Speak("Current temperature in Bristol is " + temp2.ToString)
      ad.Speak("Degrees. The " + mean.ToString)
      Next
      Catch errorVariable As Exception
      'Error trapping
      Console.Write(errorVariable.ToString())
      End Try

      '------
      End Select
    End Sub
End Class 

Plus a C# Version of the code for those whom want to see the difference:
Spoiler (Click to View)

PHP Code 


<?php 
using Microsoft.VisualBasic;
using System;
using System.Collections;
using System.Collections.Generic;
using System.Data;
using System.Diagnostics;
using System.Speech.Synthesis;
using System.Speech.Recognition;
using System.Speech.AudioFormat;
using System.Speech;
using System.Net;
using System.IO;
using System.Linq;
using System.Xml;

public class Form1
{
    public Synthesis.SpeechSynthesizer synth = new Synthesis.SpeechSynthesizer();
    public Recognition.SpeechRecognitionEngine recognizer = new Recognition.SpeechRecognitionEngine();
    System.Speech.Recognition.DictationGrammar gram = new System.Speech.Recognition.DictationGrammar();
    private Recognition.SpeechRecognitionEngine withEventsField_reco = new Recognition.SpeechRecognitionEngine();
    public Recognition.SpeechRecognitionEngine reco {
        get { return withEventsField_reco; }
        set {
            if (withEventsField_reco != null) {
                withEventsField_reco.RecognizeCompleted -= reco_RecognizeCompleted;
                withEventsField_reco.SpeechRecognized -= reco_SpeechRecognized;
            }
            withEventsField_reco = value;
            if (withEventsField_reco != null) {
                withEventsField_reco.RecognizeCompleted += reco_RecognizeCompleted;
                withEventsField_reco.SpeechRecognized += reco_SpeechRecognized;
            }
        }

    }
    private void Form1_Load(object sender, EventArgs e)
    {
        reco.SetInputToDefaultAudioDevice();

        Recognition.SrgsGrammar.SrgsDocument gram = new Recognition.SrgsGrammar.SrgsDocument();

        Recognition.SrgsGrammar.SrgsRule wordRule = new Recognition.SrgsGrammar.SrgsRule("word");

        Recognition.SrgsGrammar.SrgsOneOf wordList = new Recognition.SrgsGrammar.SrgsOneOf("weather");

        wordRule.Add(wordList);

        gram.Rules.Add(wordRule);

        gram.Root = wordRule;

        reco.LoadGrammar(new Recognition.Grammar(gram));

        reco.RecognizeAsync();
    }

    private void reco_RecognizeCompleted(object sender, System.Speech.Recognition.RecognizeCompletedEventArgs e)
    {
        reco.RecognizeAsync();

    }

    private void reco_SpeechRecognized(object sender, System.Speech.Recognition.RecognitionEventArgs e)
    {
        switch (e.Result.Text) {
            //------

            case "weather":
                dynamic mean = null;
                dynamic ad = null;
                ad = Interaction.CreateObject("sapi.spvoice");
                string URLString = "http://api.openweathermap.org/data/2.5/weather?q=Bristol&mode=xml";
                try {
                    XmlDocument m_xmld = default(XmlDocument);
                    XmlNodeList m_nodelist = default(XmlNodeList);
                    XmlNode m_node = default(XmlNode);
                    XmlNode m_node2 = default(XmlNode);
                    //Create the XML Document
                    m_xmld = new XmlDocument();
                    m_xmld.Load(URLString);
                    m_nodelist = m_xmld.SelectNodes("current/weather");
                    foreach ( m_node2 in m_nodelist) {
                        dynamic weather = m_node2.Attributes.GetNamedItem("value").Value;
                        mean = weather;
                    }
                    m_nodelist = m_xmld.SelectNodes("/current/temperature");
                    //Loop through the nodes
                    foreach ( m_node in m_nodelist) {
                        dynamic temp = m_node.Attributes.GetNamedItem("value").Value;
                        temp = temp - 272.15;
                        decimal temp2 = temp;
                        Convert.ToInt32(temp2);
                        ad.Speak("Current temperature in Bristol is " + temp2.ToString);
                        ad.Speak("Degrees. The " + mean.ToString);
                    }
                } catch (Exception errorVariable) {
                    //Error trapping
                    Console.Write(errorVariable.ToString());
                }

                break;
            //------
        }
    }
    public Form1()
    {
        Load += Form1_Load;
    }
} 

Thank you for reading my tutorial and I hope you enjoy my program!
This has endless possibilities!

Some ideas:
Youtube Video Player: Spoiler (Click to View)

(Adding more ideas soon)

Many thanks as always.
Yours,
Dubito 🤝

What did YOU use this source for? Let me know in the replies.

* - ArtMachine - Jul 25, 2014

Really good and detailed one!
I am going to try this soon.

Thanks for your contribution to the community

* - Increment - Jul 25, 2014

Even if I say chubaka it recognizes it as the word weather and reply me..

* - +Haze - Jul 25, 2014

I'm wondering If I should attempt to make a personal assistant like SHIELD. ( The one Iron Man uses ).
Edit: Jarvis

* - SaketSaket - Jul 25, 2014

Nice tutorial... very useful for new learners like me.. ;)

* - SIM-1456245 - Jul 25, 2014

Yeah yeah yeah, It's nice. I've done a tutorial aswell: https://hackforums.net/showthread.php?tid=3275428. Not that anyone knows how to use this properly, but still.

* - SIM-1847363 - Jul 25, 2014

Its a good tutorial, nice work. i only read bits and pieces but it seems like a really nice tutorial. Good job

* - Dubito - Jul 26, 2014

Thanks everyone! I hope you enjoyed it!

* - zeroexp - Jul 26, 2014

Nice detalied tutorial. Thank you for your contribution. :D

* - AnonyPwn - Jul 26, 2014

Made one of these apps a while back but my mic stopped working so i gave up lol... Great tut tho

* - VirinomTR - Jul 27, 2014

Is system.speech.recognation namespace only for english or can we use in any installed language pack ?

* - Malcolm˞˞˞˞˞˞˞˞˞˞˞˞˞˞˞˞˞˞ - Jul 27, 2014

So genius, i made a program asking for the words: loren i want gay
I cracked man..

* - Yui Asuna - Jul 27, 2014

I found this tutorial kinda hard to follow, especially because the code snippets aren't organized at all.
Also, I noticed some Public declaration at the beginning, when they should have been Private.

Overall, nice tutorial & thanks for the share.

* - Dubito - Jul 27, 2014

(Jul 27, 2014 05:55 AM)Yui Asuna Wrote: I found this tutorial kinda hard to follow, especially because the code snippets aren't organized at all.
Also, I noticed some Public declaration at the beginning, when they should have been Private.

Overall, nice tutorial & thanks for the share.

Apologies if you found it hard, would you like me to assist you?
Add me on Skype and i'll take you through step by step 🤝

* - Yui Asuna - Jul 27, 2014

(Jul 27, 2014 06:35 AM)Dubito Wrote: Apologies if you found it hard, would you like me to assist you?
Add me on Skype and i'll take you through step by step 🤝

Huh, I meant it was kinda unclear, but nah, I'm not having troubles, it's quite easy, lol