Actions on Google deep dive

As you expand your bot's capabilities onto voice platforms, you'll need to think through some of the UX consequences of having no way to visually display information to the user, or to receive input from the user via button clicks and text input.

Below is a summary of details you'll want to consider as you develop your voice bot, or customize an existing bot for voice.

G Suite vs Gmail

If you are using a corporate G Suite account with Actions on Google, you will see some unexpected behaviour in the simulator.

1. Saying the main invocation "Talk to <MY_BOT>" will return "Sorry, this action is not available".
2. In a multi-turn conversation (e.g. a multi-step recipe), the second turn will return random text.

The solution to both of these issues is for your G Suite administrator to enable web history for the entire organization.

If you are using a personal Gmail account with Actions on Google, there are no settings to adjust; the simulator should work as expected.

Draft mode

Other users and developers you invite to your Actions on Google project, will need to refresh the simulator with the latest version of the project before using the simulator.

To do this, they can click Change Version near the top of the simulator page, and choose an option from the drop-down menu, then click Done. This refreshes the simulator after having made changes to the settings.

Time delays

Actions on Google does not support time delays (i.e. delay: relative: 5) between components. Instead, it will terminate the conversation.

Engine turn timeout

Actions on Google expects the bot to respond within 4-5 seconds. Long-running custom Python components may result in the engine turn timeout being reached after which the conversation will be terminated.

Push notifications

Actions on Google does not currently support push notifications (i.e. broadcasts, delayed/scheduled notifications, etc.), although it appears Google does intend to support this in the future.

Entity extraction

Meya does not currently support entity extraction.

For example, this interaction...
User: Okay Google, tell MyApp to set the temperature to 72 degrees.
Bot: Okay, setting the temperature to 72 degrees.

...would be expressed like this:
User: Okay Google, tell MyApp to change the temperature.
Bot: Okay, what temperature would you prefer?
User: 72 degrees.
Bot: Okay, setting the temperature to 72 degrees.

Invocation modes

There are two ways of invoking, or activating, your bot.

1. A main invocation

Example: Okay Google, talk to MyBot
In this example, no specific action was provided by the user. This is the voice equivalent of the start_chat trigger for the Meya Web integration. You could use this invocation to launch a welcome flow, an "about me" flow, or to send the user right into the main flow.

You can specify which flow should be launched in response to the main invocation by setting its trigger type to start_voice_chat.

2. An invocation with an action phrase

Example: "Okay Google, ask MyBot to book an oil change"
In this case, an action was provided: book an oil change. If a flow exists with a trigger that matches book an oil change, that flow would be launched instead of the main invocation flow defined by start_voice_chat.

Publishing a bot

Before becoming accessible to the public, you’ll need to publish your cognitive app. This will make it accessible in the Actions on Google directory and allow users to start interacting with your bot.

Publishing your bot will provide you with access to analytics including usage, health and how your bot is doing within the directory. More information about publishing your cognitive app can be found in the Actions on Google documentation.

Custom components

You can make your custom Python components voice integration-friendly by setting the speech property, like this:

from meya import Component


class HelloWorld(Component):
    def start(self):
        text = "Hello, World!"
        speech = "What a beautiful world!"
        message = self.create_message(text=text, speech=speech)
        return self.respond(message=message, action="next")