Categories
Programming

Control Philips Hue with Google Assistant SDK on Ubuntu

In this article, I’m going to teach how I manage to integrate Philips Hue bridge with Google Assistant SDK to control my smart lights.

Introduction

I’ve a Philips Hue bridge and a set of bulbs. On the other hand, Google has an SDK for their Google Assistant. By combining these two, I was able to create a simple Linux application that identifies your custom voice command and map those commands to control the bridge.

In this guide, I will go through step by step on creating this project so that you can also control your Philips Hue bridge from your Linux desktop.  As a matter of fact, one you go through the steps, you will also learn how to define custom voice commands and execute custom code blocks.

However please note that I’ve wrote this after prototyping and getting things working. So some of the approaches I have used might not be conventional.

Things you need

  • A gmail account
  • Philips Hue Bridge and a philips hue light
  • A computer with:
    • Ubuntu(better to be a newer version)
    • A microphone
    • A Speaker
    • Stable internet connection

Part 1 – Architecture Setup

Create a Google developer project

First you need to create a Google Developer project. We will use this project to test “Actions on Google”. Actions on Google is a program that runs via Google Assistant. It enables us to create custom commands(actions) and execute them on Google Assistant API.

Go to https://console.cloud.google.com/projectcreate

For project Name give ‘test’ (assuming you already don’t have a project called test) and click create. Then you will be navigated to the Dashboard page for the project. Bookmark this page as we might come back to this.

Enable Google Assistant API

Then we need to enable Google Assistant API under the created project. This is an API that enables voice commands that is captured from a voice input device to be recognized.

To enable this go to ‘API and Services -> Dashboard’ From the left menu. Then click enable APIs and services Button. From the search Google assistant API. Then enable the service.

We also need to enable OAuth grant access from the device to the API. For this, go to credentials from the left menu and select OAuth consent screen. Then under select your email address for the Email Address drop down and click save.

Register the PC as a device To connect with Google assistant and the PC we need to register the PC as a device in Google actions.

for that go to: https://console.actions.google.com/

Then click add/ import Project.

For project name search the previous project name and select from the drop down. Then click import project.

Now you should be able to see getting started Project page. In this page, click skip button. Once done, you should see a page similar to below.

In this page click “Device registration” and click “register model”. Now you should see the register model form.

For this, type a product name (e.g. My PC) and something similar for the Manufacturer name. Then register the model. After that, download the credentials file. We will need this to be located in our PC. For Traits options, do not select any and click ‘Skip’.

Part 2 – System setup

Now we can dive into the coding part. We are going to use python3 that comes with Ubuntu package manager. We will install our dependencies in a Python virtual environment.

First lets ensure we have the correct packages.

sudo apt-get update
sudo apt-get install python3-dev python3-venv portaudio19-dev libffi-dev libssl-dev libmpg123-dev

Then create a directory for the project and a virtual python environment called ‘env’ for the project. The directory we create here will be used to store all the files related to the project.

mkdir hue-test
cd hue-test
python3 -m venv env

Now you should source to use the virtual Python environment.

source env/bin/activate

Then install pip packages

python -m pip install --upgrade pip setuptools wheel google-assistant-library google-auth-oauthlib[tool] phue

 

Generate Credentials

The next step is to generate credentials so that we can run our code and use the Assistant SDK. For client-secret option, pass the location of the previously downloaded credential file.

google-oauthlib-tool --scope https://www.googleapis.com/auth/assistant-sdk-prototype \
      --scope https://www.googleapis.com/auth/gcm \
      --save --headless --client-secrets /path/to/client_secret_client-id.json

Once you run that, you will see a link in the terminal. Follow that link and grant permission. Then copy the verification code and paste in the terminal when it asks.

If authorization was successful, you will see a response similar to the following:

credentials saved: /path/to/.config/google-oauthlib-tool/credentials.json

Part 3 – Coding

First we need to define our custom action and send it to Google Actions. Lets create a json file . touch actions.json

In this tutorial the intention is to turn off/on a bulb in Philips Hue system. So we are going to create an action that handles a voice command like “Turn off table lamp” where “off” is a state and “table lamp” is the bulb name in our network.

So let’s create a file called actions.json and put the following.

{
  "actions": [
    {
      "name": "foo.bar",
      "availability": {
        "deviceClasses": [
          {
            "assistantSdkDevice": {}
          }
        ]
      },
      "intent": {
        "name": "foo.bar",
        "parameters": [
          {
            "name": "state",
            "type": "LightState"
          },
          {
            "name": "light",
            "type": "LightName"
          }
        ],
        "trigger": {
          "queryPatterns": [
            "turn $LightState:state (my)? $LightName:light"
          ]
        }
      },
      "fulfillment": {
        "staticFulfillment": {
          "templatedResponse": {
            "items": [
              {
                "simpleResponse": {
                  "textToSpeech": "As you wish."
                }
              },
              {
                "deviceExecution": {
                  "command": "foo.bar",
                  "params": {
                    "state": "$state",
                    "lightName": "$light"
                  }
                }
              }
            ]
          }
        }
      }
    }
  ],
  "types": [
    {
      "name": "$LightState",
      "entities": [
        {
          "key": "ON",
          "synonyms": [
            "on"
          ]
        },
        {
          "key": "OFF",
          "synonyms": [
            "off"
          ]
        }
      ]
    },
    {
      "name": "$LightName",
      "entities": [
        {
          "key": "table lamp",
          "synonyms": [
            "table lamp"
          ]
        }
      ]
    }
  ]
}

Here is the basic structure of the actions.json

actions.intent.name : This is the identifier for our custom action.

actions.intent.parameters : Here we define a list of parameters that are populated by the voice command

`actions.intent.trigger.queryPatterns` : Here we define the custom voice command. As you can also see we also used the defined parameters so that SDK will populate the users given value.

types : Here we define the values for custom parameter types. Actions also support some Schema.org types by default.

Let’s see how this all map. Suppose you tell “Hey google, turn ‘off’ my ‘table lamp'”. Assistant SDK will send the voice command to Actions. Actions will map ‘off’ value to lightState and ‘table lamp’ value to lightName. We will get those mapping back from the API.

To learn more about Actions read the docs at https://developers.google.com/actions/extending-the-assistant.

Now you need to upload the fiule. For this you need to download “gactions” tool. Please downland it from https://developers.google.com/actions/tools/gactions-cli and put in the current “hue-test” directory. Then make it executable. After that you can upload it to your project.

./gactions update --action_package actions.json --project project_id

When you are using it for the first time for the project, you have to authenticate. Click the generated URL in the terminal and grant permissions. Then paste the verification back on the terminal.

Then we need to enable test mode for the actions.

./gactions test --action_package actions.json --project project_id

Next lets create a file called user_config.json to store the configuration such as Google project ID and device ID.

{
    "project_id": "id",
    "device_id": "id"
}

 

Now we need to create the python script to use these voice command and turn off/on the bulb. So lets create a main.py file in the current directory (hue-test). Notice that we also use Phue python library to give commands to the Philips bridge.

from __future__ import print_function

import json
import os.path
import google.oauth2.credentials
from google.assistant.library import Assistant
from google.assistant.library.event import EventType
from google.assistant.library.device_helpers import register_device
from phue import Bridge

try:
    FileNotFoundError
except NameError:
    FileNotFoundError = IOError

def process_event(event):
    if event.type == EventType.ON_CONVERSATION_TURN_STARTED:
        print()

    print(event)

    if (event.type == EventType.ON_CONVERSATION_TURN_FINISHED and
            event.args and not event.args['with_follow_on_turn']):
        print()
    if event.type == EventType.ON_DEVICE_ACTION:
        for command, params in event.actions:
            print('Do command', command, 'with params', str(params))
            if command == "foo.bar":
                # ip address of your bridge
                hue = Bridge('192.168.0.100')  
                # press the button on the bridge and call connect() for the first time
                hue.connect()
                lights = hue.lights
                lights = list(filter(lambda light: light.name == params['lightName'], lights))
                lights[0].on = True if params['state'] == 'ON' else False

def main():
    with open("user_config.json", 'r') as f:
        user_config = json.load(f)

    with open(os.path.join(os.path.expanduser('~/.config'),'google-oauthlib-tool','credentials.json'), 'r') as f:
        credentials = google.oauth2.credentials.Credentials(token=None, **json.load(f))

    try:
        with open('device_config_library.json') as f:
            device_config = json.load(f)
        should_register = False
    except FileNotFoundError:
        should_register = True

    project_id = user_config['project_id']
    device_model_id = user_config['device_id']

    with Assistant(credentials, device_model_id) as assistant:
        events = assistant.start()

        device_id = assistant.device_id
        print('device_model_id:', device_model_id)
        print('device_id:', device_id + '\n')

        if should_register:
            register_device(project_id, credentials,
                                device_model_id, device_id)
            with open('device_config_library.json', 'w') as f:
                    json.dump({
                        'last_device_id': device_id,
                        'model_id': device_model_id,
                    }, f)
        for event in events:
            process_event(event)


if __name__ == '__main__':
    main()

Now you can simply run python main.py and say “Hey Google, turn off my table lamp”.  The command should be identified by Google and send with the correct identified command name with parameters.  If everything works correctly, the command that connect to bridge and turn on/off the lights should be executed.

This guide is simply a prototype for connecting Google SDK and Philips Hue as a smart device. Hopefully, now you should be able to go with your requirements and build something fancy. 🙂

I hope my guide helped you at least for some extent.

Troubleshooting

If it complains about activity permission error, make sure you have granted corrected Activity controls. See “Set activity controls for your account” on

https://developers.google.com/assistant/sdk/guides/library/python/embed/config-dev-project-and-account

 

0 0 vote
Article Rating

By Sinaru Gunawardena

Sinaru is a software developer at work and a painter in free time. He enjoys writing tech notes on his personal website. He currently lives in Dublin, Ireland.

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments