Reply to Messaging App Messages With Natural Language

Learn how you can reply to whatsapp messages from your Google Assistant like on Google Home

  1. joaomgcd
    In this project you're going to set your latest unreplied WhatsApp messages in context so that the Google Home (or the AutoVoice Natural Voice action) assistant will know that you have a notification and then be able to reply to it.

    Note: this will work with most other messaging apps but I'll use WhatsApp as an example.

    STEP 0 - SETUP YOUR API.AI ACCOUNT


    Make sure that you have setup your account and have configured AutoVoice to work with API.AI


    STEP 1 - ENABLE NATURAL LANGUAGE FOR GOOGLE ASSISTANT


    • Open AutoVoice
    • Click Google Assistant
    • Click Commands
    • Enable Use Natural Language at the bottom


    STEP 1 - CREATE PROFILE TO INTERCEPT WHATSAPP MESSAGES


    • Create a new profile in Tasker with the AutoNotification Intercept condition
    • Set it to intercept WhatsApp messages
    • Create a new task


    STEP 2 - CHECK CURRENT NOTIFICATIONS


    • Add an AutoNotification Query action and set it to get WhatsApp notifications
    • Make sure to only get notifications With Reply Actions
    • Add a Variable Set action where you set %text to You have a WhatsApp message from %antitle: %antext
    • Add a Flash action with %text


    STEP 3 - DELETE CONTEXT IF NO MESSAGE


    • Add an If-Else-End If chain where the condition is %anreplyaction Is Not Set
    /!\ I used the wrong condition the first time around, so follow this video if you have trouble setting the right condition.
    • In the If part use the AutoVoice Natural Language action and set it to delete the context named notifications


    STEP 4 - SET CONTEXT IF NOTIFICATION IS PRESENT


    • In the Else section add an AutoVoice Natural Language action
    • Set the Context Name to notifications again
    • Set the Field Names to text,replyaction,name
    • Set the Field Values to %text,%anreplyaction,%antitle
    (i) This will create a context with 3 fields: text, replyaction and name. They will have the corresponding values set so that you can then use them in the intent on API.AI
    /!\ If %text contains a comma this will break, so use a Variable Search and Replace Tasker action beforehand and replace , with nothing so that all the commas are removed from the text before using it in AutoVocice
    • accept everything and back out of Tasker to save


    For more information on contexts in natural voice commands check this tutorial.

    STEP 5 - CREATE INTENT FOR NOTIFICATIONS


    • Open AutoVoice -> Natural Language -> Commands
    • Add a new command
    • Set the commands that you'd like to use to ask if there are notifications. Something like Check my notifications or What are my notifications
    • Set the reply to #notifications.text which will contain whatever was in the %text variable in the task you created before in Tasker
    • Accept
    • Set the input context to notifications
    (i) By setting the input context to notifications you make sure that this command will only trigger if the notifications context exists


    STEP 6 - CREATE INTENT FOR NO NOTIFICATIONS PRESENT


    • Create another intent just like the one you created but set the response to No notifications present and don't set a context
    (i) By not setting a context you assure that this command will trigger if there's no notifications context present


    STEP 7 - TEST SO FAR


    If you now get a WhatsApp notification and use the Natural Language home screen shortcut or Google Home and ask AutoVoice to check your notifications you'll see that it says your notification out loud.
    If you dismiss the notification it'll reply that you don't have notifications


    STEP 8 - CREATE INTENT TO REPLY


    • Create a new intent in API.AI
    • Set the context to notifications
    • Set the User Says part to some commands like reply with text hello, reply with hello, reply hello or simply reply
    • Mark all instances of hello as a variable with entity type @sys.any so that part of the command is a variable
    • Name that variable reply
    • Add a replyaction variable with the type @sys.any and with the value #notifications.replyaction
    • Add a name variable with the type @sys.any and with the value #notifications.name
    (i) This will create 2 variables for this command called replyaction and name whose values come from the notifications context. You set these context values every time you get a new WhatsApp message as seen in step 4
    • Set the prompt for the reply variable to What do you want to reply to #notifications.name?
    (i) This will allow AutoVoice to prompt you for the text if you just say reply for example
    • Set the response to Ok, replying $reply to #notifications.name
    (i) $reply will contain whatever is the reply variable that you created above
    • Set the Action to replynotification
    • Save


    STEP 9 - REPLY IN TASKER


    • Create a new Tasker profile with the AutoVoice Natural Language event condition
    • Select the reply intent from the list
    • Accept. As you can see the %reply, %replyaction and %name variables are automatically created from the API.AI intent
    • In the task add a Flash action with the text Replying %reply to %name
    • Add an AutoNotification Reply action and set Reply ID to %replyaction and Reply Text to %reply


    STEP 10 - FINAL TEST


    Now if you receive a WhatsApp message and ask to reply, it'll reply directly! :cool: