Control multiple devices with one voice command using natural language

Learn how you can make a command like "turn off the tv and turn on the bath room light" work!

  1. joaomgcd
    At the end of this tutorial you'll be able to do something like this:

    You will be able to say a voice command to turn on and off multiple devices all in one go.
    For example Turn on the heater and turn off the living room light and the kitchen light will make it all happen all at once.

    STEP 1 - CREATE LIGHT AND STATE TYPES


    • In AutoVoice go to Natural Language -> Types
    • Create a new type called light
    • Add some of your devices in it like living room or bathroom and set some synonyms you may use for each. For example, you may also want to call the living room main room
    • Add another type called state
    • Add an on and an off state
    (i) Read more about natural language types here.


    STEP 2 - CREATE LIGHT-STATE COMPOSITE TYPE


    • Create another type and call it light-state
    (i) This is going to be a type that's composed of a combination of both the light and state types above.
    • Add a value with the text @state:state the @light:light
    (i) This means that you want this type to be recognized with you say a state, followed by the word "the" followed by the name of a light. For example it matches on the living room as in "turn on the living room light"
    • Go back, touch the new light-state type and choose Toggle Reference Mode so that this type can reference other types.


    STEP 3 - CREATE VOICE COMMAND TO CONTROL DEVICES


    • Go the Commands section of the Natural Language menu and create a new command
    • As the commands enter turn off the living room light and turn on the bathroom light,turn off the living room light
    • As the response enter turning off the living room light
    • Select off the living room and create a variable with that expression called $lightstate. Make sure to replace all other instances of similar expressions (like on the bathroom) with the variable
    • Set the type of the $lightstate variable to the previously created light-state type
    • Call the command control lights and give it the controllights action


    STEP 4 - SET PROMPT AND MAKE LIGHTSTATE A LIST


    • Tap the newly created command and set the parameter prompt to something more friendly like how do you want to control your devices?
    • Tap the command again and set the $lightstate parameter to be a list
    (i) By allowing the $lightstate parameter to be a list you will be able to say multiple device names and statuses in the same command and all of them will be sent to Tasker as a list. If you didn't set this you would only ever receive one device and one state


    STEP 5 - CREATE TASKER PROFILE TO CONTROL LIGHTS


    • Create a new profile in Tasker with the AutoVoice Natural Language event condition
    • Select the control lights command and accept
    • In the task add a Flash action with the text Controlling lights: %lightstatus()


    STEP 6 - TESTING SO FAR


    Back out of Tasker to save and test out the command by saying
    turn the bathroom light on and turn the living room light off
    You should see your newly created tasker Flash showing you the contents of the %lightstatus() variable. As you can see it contains a list of JSON objects. Learn more about reading JSON in Tasker here.
    We're now going to read each of these JSON objects one by one and toggle the devices accordingly.


    STEP 7 - GET EACH DEVICE AND STATUS


    • Add For and End For actions in the Task
    • In the For action set Variable to %index and Items to 1:%lightstate(#)
    (i) This creates a loop that sets the %index variable to 1,2,3,etc... for as many elements there are in %lightstate(). In normal circumstances you would be able to use the For Loop directly on the %lightstate() array, but because this array contains JSON which contains commas you can't do that directly or else the loop would break up the JSON objects in parts. This would happen because the For loop thinks that the items are separated by commas
    • Inside the For loop, add a Varible Set action where you set %item to %lightstate(%index)
    (i) Since %index will contain 1,2,3,etc... , one at a time, using %lightstate(%index) will get each of the items in the array one at a time. It would be the same as doing %lightstate(1),%lightstate(2),%lightstate(3),etc...


    STEP 8 - READ LIGHT AND STATUS VALUES FOR EACH ITEM


    • Inside the For add an AutoTools JSON Read action
    • Set the Json field to %item
    • Set the Values field to light,state
    • Add a Flash action with the text %light,%state
    (i) This reads the light and state field from the JSON object in %item and shows them in a toast


    STEP 9 - TURN EACH DEVICE ON OR OFF


    • Inside the For add an If-Else-End If block and make the condition %state matches on
    • In the If part turn on the device with the name %name. I'm using AutoVera here but you can use any home automation action that allows you turn on or off a device with a given name
    • In the Else part do the same but turn off the device instead


    STEP 10 - TEST COMMANDS


    If you now say Turn on the living room light you'll see that it'll show the appropriate toast with the text living room, on
    If you say Turn off the bathroom light and turn on the living room light you'll see a toast with bathroom,on and another one with living room, on


    The way it's setup now, if you want to turn on multiple devices you'll have to say turn on the living room light and turn on the bathroom light for example.
    It would be great if you could simply say turn on the living room and the bathroom light wouldn't it?
    Let's make that small change now :cool:

    STEP 11 - ADD POSSIBILITY TO NOT SAY STATE


    • Go into AutoVoice -> Natural Langugage -> Types and edit the light-status type
    • Add another entry with the text @light:light
    (i) This enables the natural language processor to be able to recognize the command even if you don't say the state part of it.


    STEP 12 - FINAL TEST


    If you now say something like turn on the living room and the bathroom light the For loop will go through the JSON items. When it can't find a state part in the item it'll retain the previous value.
    In this example the JSON objects would be something like

    Code (Text):
    {"light":"living room","state":"on"}
    and
    Code (Text):
    {"light":"bathroom"}
    It woud read the first item and set
    • %light to living room
    • %state to on
    then it would read the second item and set
    • %light to bathroom
    It would not set the %state variable because the value wouldn't exist in the second JSON and so %state would be kept as on

    Because of this you can now say what you see in the video for example. Something like:

    turn on the heater and turn off the corner and the other light :cool: