reading

● This is an Elektor Publication. Elektor is the media brand of Elektor International Media B.V.78 York StreetLondon W1H 1DP, UKPhone: (+44) (0)20 7692 8344© Elektor International Media BV 2020First published in the United Kingdom 2020

● All rights reserved. No part of this book may be reproduced in any material form, including photocopying, or storing in any medium by electronic means and whether or not transiently or incidentally to some other use of this publication, without the written permission of the copyright holder except in accordance with the provisions of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London, England W1P 9HE. Applications for the copyright holder's written permission to reproduce any part of this publication should be addressed to the publishers. The publishers have used their best efforts in ensuring the correctness of the information contained in this book. They do not assume, and hereby disclaim, any liability to any party for any loss or damage caused by errors or omissions in this book, whether such errors or omissions result from negligence, accident or any other cause.

● British Library Cataloguing in Publication DataCatalogue record for this book is available from the British Library

● ISBN 978-1-907920-94-3

● EISBN 978-3-89576-383-0

● EPUB 978-3-89576-382-3

Prepress production: DMC ¦ daverid.comPrinted in the Netherlands by Wilco

Images and logos used in this book are courtesy of Material Design Icons, Cisco, and the Raspberry Pi Foundation

Table of Contents

• Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Chapter 1 • Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.1 • What is home automation?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

1.2 • Why use a Raspberry Pi as a home automation gateway? . . . . . . . . . . . . . . . . 15

1.3 • The properties of a good home automation system. . . . . . . . . . . . . . . . . . . . . 16

1.3.1 • Secure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.3.2 • Modular . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.3.3 • Open-Source . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.3.4 • Self-sufficient. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.4 • How to use this book. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

1.5 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Chapter 2 • The Raspberry Pi as a home automation gateway . . . . . . . . . . . . . . . . . 27

2.1 • Which Raspberry Pi models are suitable for home automation?. . . . . . . . . . . . . 27

2.2 • Requirements for a reliable home automation gateway. . . . . . . . . . . . . . . . . . 30

2.3 • Installing Raspberry Pi OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.4 • Setting up network connectivity with Ethernet or Wi-Fi. . . . . . . . . . . . . . . . . . 35

2.4.1 • Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.4.2 • Wi-Fi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.4.3 • Setting a fixed IP address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.5 • Remote access using SSH . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2.5.1 • Enabling the SSH server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2.5.2 • Connecting with the SSH client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

2.6 • Basic setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.7 • The tmux terminal multiplexer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.7.1 • The basics of tmux: windows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

2.7.2 • Working with tmux sessions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.7.3 • Seeing more at the same time with panes. . . . . . . . . . . . . . . . . . . . . . . 42

2.7.4 • Copying and pasting text. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.8 • Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.8.1 • Virtual environments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.8.2 • Package requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

2.9 • Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.9.1 • Installing Docker . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

2.9.2 • Installing Docker Compose. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

Control Your Home with Raspberry Pi

2.9.3 • Creating a Docker Compose YAML file. . . . . . . . . . . . . . . . . . . . . . . . . . 49

2.10 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

Chapter 3 • Secure your home automation system . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.1 • Some general computer security principles. . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.2 • Isolate your home automation devices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.2.1 • Physical isolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

3.2.2 • VLANs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

3.2.3 • Firewalls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.3 • User management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.3.1 • Permissions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.3.2 • Passwords. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

3.3.3 • Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.4 • Encryption. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.4.1 • Your threat model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.4.2 • TLS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

3.4.3 • Setting up your own CA with mkcert . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.4.4 • Creating a CA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.4.5 • Creating and signing a certificate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

3.4.6 • Keeping your root CA key secure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.5 • Keeping your software up-to-date. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

3.5.1 • Update apt packages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

3.5.2 • Update Docker images. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

3.5.3 • Update pip packages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.5.4 • Update manually installed packages. . . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.5.5 • Update your home automation devices. . . . . . . . . . . . . . . . . . . . . . . . . 78

3.6 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

Chapter 4 • MQTT (Message Queuing Telemetry Transport). . . . . . . . . . . . . . . . . . . 80

4.1 • What is MQTT?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

4.1.1 • Central intermediary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

4.1.2 • Hierarchical names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

4.1.3 • Using wildcards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

4.2 • Installing and configuring the Mosquitto MQTT broker. . . . . . . . . . . . . . . . . . . 83

4.2.1 • A basic Mosquitto setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

4.2.2 • Testing your setup with the Mosquitto clients. . . . . . . . . . . . . . . . . . . . . 85

4.2.3 • A secure Mosquitto setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

4.2.4 • Testing your secure setup with the Mosquitto clients. . . . . . . . . . . . . . . . 90

4.2.5 • Default options for Mosquitto clients . . . . . . . . . . . . . . . . . . . . . . . . . . 92

4.3 • Using graphical MQTT clients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

4.3.1 • MQTT.fx . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

4.3.2 • MQTT Explorer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

4.4 • Using MQTT in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

4.5 • Direct communication between other containers and Mosquitto . . . . . . . . . . . 100

4.6 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Chapter 5 • TCP/IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

5.1 • Wake other network devices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

5.2 • Remote control with SSH. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

5.2.1 Run commands on other devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

5.2.2 • Secure passwordless logins using SSH keys. . . . . . . . . . . . . . . . . . . . . 109

5.3 • Collecting information from devices using SNMP . . . . . . . . . . . . . . . . . . . . . 111

5.3.1 • Walking through the MIB tree. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

5.3.2 • Collecting your router's version using SNMP . . . . . . . . . . . . . . . . . . . . 113

5.3.3 • Collecting your printer's ink levels. . . . . . . . . . . . . . . . . . . . . . . . . . . 114

5.4 • Using devices with a HTTP/REST API . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

5.4.1 • Setting up a Shelly device for secure remote control. . . . . . . . . . . . . . . 117

5.4.2 • Using Shelly's HTTP API with curl. . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

5.4.3 • Using the HTTP API in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

5.5 • Creating a video surveillance system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

5.5.1 • Turn your Raspberry Pi into an IP camera. . . . . . . . . . . . . . . . . . . . . . 123

5.5.2 • Turn your Raspberry Pi into a camera controller. . . . . . . . . . . . . . . . . . 125

5.5.3 • Viewing your remote cameras. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

5.5.4 • Motion detection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

5.5.5 • Notifications on motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

5.6 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Chapter 6 • Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

6.1 • An introduction to Bluetooth Low Energy . . . . . . . . . . . . . . . . . . . . . . . . . . 134

6.1.1 • Broadcasting data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

6.1.2 • Connecting to services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

6.2 • Enabling Bluetooth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

6.3 • Investigating Bluetooth Low Energy devices . . . . . . . . . . . . . . . . . . . . . . . . 138

6.3.1 • Scanning for Bluetooth Low Energy devices. . . . . . . . . . . . . . . . . . . . . 139

Control Your Home with Raspberry Pi

6.3.2 • Dumping raw Bluetooth broadcast data . . . . . . . . . . . . . . . . . . . . . . . 140

6.3.3 • Discovering device characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

6.3.4 • Reading device characteristics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

6.4 • Reading BLE sensor values in Python. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

6.4.1 • RuuviTag Sensor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143

6.4.2 • Miflora . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

6.5 • Relaying Bluetooth sensor values with bt-mqtt-gateway . . . . . . . . . . . . . . . . 148

6.5.1 • Configuring bt-mqtt-gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148

6.5.2 • Running bt-mqtt-gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

6.6 • Presence detection with Bluetooth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

6.6.1 • Presence detection with monitor.sh . . . . . . . . . . . . . . . . . . . . . . . . . . 152

6.6.2 • Configuring and running monitor.sh. . . . . . . . . . . . . . . . . . . . . . . . . . 153

6.6.3 • Trigger arrival and departure scans in monitor.sh. . . . . . . . . . . . . . . . . 155

6.7 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

Chapter 7 • 433.92 MHz. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

7.1 • 433.92 MHz protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157

7.2 • Hardware requirements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

7.2.1 • Receiver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158

7.2.2 • Antenna . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

7.3 • Receiving sensor values with rtl_433 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

7.3.1 • Installing rtl_433toMQTT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

7.3.2 • Configuring rtl_433. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

7.4 • Publishing 433.92 MHz sensor values to MQTT. . . . . . . . . . . . . . . . . . . . . . . 165

7.5 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Chapter 8 • Z-Wave. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

8.1 • An introduction to Z-Wave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

8.1.1 • The specification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

8.1.2 • How does Z-Wave work? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

8.2 • Choosing a Z-Wave transceiver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170

8.2.1 • Transceiver on the GPIO header: RaZberry . . . . . . . . . . . . . . . . . . . . . 171

8.2.2 • USB Transceiver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

8.3 • OpenZWave and Zwave2Mqtt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

8.3.1 • Installing Zwave2Mqtt. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

8.3.2 • Configuring Zwave2Mqtt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

8.3.3 • Using the Zwave2Mqtt Control Panel . . . . . . . . . . . . . . . . . . . . . . . . . 179

8.4 • Using your Z-Wave devices with MQTT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

8.4.1 • Reading sensor values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184

8.4.2 • Controlling switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

8.5 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

Chapter 9 • Zigbee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

9.1 • An introduction to Zigbee. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

9.1.1 • The specification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188

9.1.2 • How does Zigbee work?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

9.2 • Creating a Zigbee transceiver. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

9.2.1 • Connect the downloader cable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

9.2.2 • Install the flashing software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

9.2.3 • Flash the firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

9.3 • Zigbee2mqtt and Zigbee2MqttAssistant . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

9.3.1 • Connecting the CC2531. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

9.3.2 • Installing Zigbee2mqtt and Zigbee2MqttAssistant. . . . . . . . . . . . . . . . . 195

9.3.3 • Configuring Zigbee2mqtt and Zigbee2MqttAssistant . . . . . . . . . . . . . . . 196

9.3.4 • Using Zigbee2MqttAssistant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198

9.4 • Using our Zigbee devices with MQTT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200

9.4.1 • Reading sensor values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

9.4.2 • Controlling switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201

9.5 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202

Chapter 10 • Automating your home. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203

10.1 • Node-RED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

10.1.1 • Installing Node-RED . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

10.1.2 • Adding authentication to Node-RED . . . . . . . . . . . . . . . . . . . . . . . . . 205

10.1.3 • Using Node-RED over HTTPS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207

10.1.4 • Creating Node-RED flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

10.1.5 • Installing extra nodes in Node-RED . . . . . . . . . . . . . . . . . . . . . . . . . 213

10.1.6 • Creating a dashboard in Node-RED . . . . . . . . . . . . . . . . . . . . . . . . . 215

10.2 • Home Assistant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

10.2.1 • Installing Home Assistant. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

10.2.2 • Integrating MQTT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221

10.2.3 • Creating automation rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

10.3 • AppDaemon. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

Control Your Home with Raspberry Pi

10.3.1 • Installing AppDaemon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

10.3.2 • Creating an AppDaemon app with MQTT: the time . . . . . . . . . . . . . . . 229

10.3.3 • Creating an AppDaemon app with MQTT: garage door alert . . . . . . . . . 231

10.4 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

Chapter 11 • Notifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

11.1 • Forwarding local email. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

11.1.1 • Installing Nullmailer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

11.1.2 • Testing Nullmailer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

11.1.3 • Using Nullmailer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

11.2 • Forwarding emails from Docker containers . . . . . . . . . . . . . . . . . . . . . . . . 237

11.2.1 • Installing docker-postfix. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237

11.2.2 • Sending emails to docker-postfix. . . . . . . . . . . . . . . . . . . . . . . . . . . 239

11.3 • Push notifications with Gotify . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

11.3.1 • Installing the Gotify server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242

11.3.2 • Adding applications to Gotify. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243

11.3.3 • Using Gotify applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244

11.3.4 • Using Gotify clients. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247

11.4 • Notifications on receiving MQTT messages. . . . . . . . . . . . . . . . . . . . . . . . . 248

11.4.1 • Installing mqttwarn. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248

11.4.2 • Sending emails with mqttwarn . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

11.4.3 • Transforming and filtering payloads . . . . . . . . . . . . . . . . . . . . . . . . . 252

11.5 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255

Chapter 12 • Voice control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

12.1 • A basic Rhasspy setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256

12.1.1 • Hardware requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

12.1.2 • Configure audio hardware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257

12.1.3 • Installing Rhasspy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260

12.1.4 • Rhasspy's settings. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261

12.1.5 • Configuring audio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

12.1.6 • Configuring the wake word. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

12.1.7 • Configuring text to speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263

12.1.8 • Configuring speech to text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264

12.1.9 • Configuring intent recognition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

12.1.10 • Configuring dialogue management. . . . . . . . . . . . . . . . . . . . . . . . . 266

12.1.11 • Testing your Rhasspy setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

12.2 • A Rhasspy base with satellites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

12.2.1 • Hardware requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268

12.2.2 • Setting up the satellites. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

12.2.3 • Setting up the base. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270

12.2.4 • Testing your base and satellites. . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

12.2.5 • Enable UDP audio streaming. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273

12.3 • Train your sentences. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

12.3.1 • Rhasspy's template language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

12.3.2 • Slots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

12.4 • Intent handling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

12.4.1 • Intent handling with MQTT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

12.4.2 • Intent handling with AppDaemon and MQTT . . . . . . . . . . . . . . . . . . . 280

12.4.3 • Intent handling with WebSocket in Node-RED. . . . . . . . . . . . . . . . . . . 282

12.5 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287

Chapter 13 • Remote access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

13.1 • Three ways for remote access. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

13.1.1 • Port forwarding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289

13.1.2 • A localhost tunneling solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

13.1.3 • A virtual private network (VPN). . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

13.2 • Updating your dynamic DNS with ddclient. . . . . . . . . . . . . . . . . . . . . . . . . 297

13.3 • Running WireGuard on your Raspberry Pi . . . . . . . . . . . . . . . . . . . . . . . . . 298

13.3.1 • Installing PiVPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299

13.3.2 • Adding a VPN client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300

13.3.3 • Connecting with a VPN client. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302

13.3.4 • Managing your VPN clients . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304

13.4 • Summary and further exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305

Chapter 14 • Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

14.1 • A dashboard for all your services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306

14.2 • More about home automation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310

• Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312

15.1 • Getting the name and ID of a serial device . . . . . . . . . . . . . . . . . . . . . . . . 312

15.2 • Switching USB ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

15.3 • Disabling the onboard radio chips . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313

15.3.1 • Disabling onboard Bluetooth. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

Control Your Home with Raspberry Pi

15.3.2 • Disabling onboard Wi-Fi. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

15.4 • Disabling the on-board LEDs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314

15.4.1 • Raspberry Pi Zero (W). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

15.4.2 • The big Raspberry Pi models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

15.4.3 • Ethernet models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315

15.4.4 • Raspberry Pi Camera Module. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316

15.5 • Securing insecure web services with a reverse proxy . . . . . . . . . . . . . . . . . 317

15.5.1 • Using nginx as a reverse proxy with HTTPS. . . . . . . . . . . . . . . . . . . . 317

15.5.2 • Adding basic authentication to nginx . . . . . . . . . . . . . . . . . . . . . . . . 321

15.6 • Bridging two MQTT brokers securely. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323

• Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327

• Preface

Ever since the Raspberry Pi was introduced, the popular single-board computer has been used by enthusiasts to automate their home. That's not a coincidence: the Raspberry Pi is a powerful computer in a small package, with lots of interfacing options to control various devices.

In this book, I'll show you how you can automate your home with a Raspberry Pi. You can do this in many ways and with various software and hardware choices. I'll show you one way, which is a bit different from what you'll read in many other books, but my approach has its merits, and I'll explain why.

You'll learn how to use various wireless protocols for home automation, such as Bluetooth, 433.92 MHz radio waves, Z-Wave, and Zigbee. Soon you'll automate your home with Py-thon, Node-RED, and Home Assistant, and you'll even be able to speak to your home auto-mation system. All of this in a secure way, with a modular system, completely open-source and without relying on third-party services.

At the end of the book, you can install and configure your Raspberry Pi as a highly flexible home automation gateway for your protocols of choice and link various services with MQTT to make it a system of your own. This DIY (do it yourself) approach is a bit more laborious than just installing an off-the-shelf home automation system, but in the process, you learn a lot, and in the end, know exactly what's running your house and how to tweak it. And that's why you were interested in the Raspberry Pi in the first place, right?

Koen Vervloesem, May 2020

Control Your Home with Raspberry Pi

Chapter 1 • Introduction

In this introductory chapter, I give a short overview of what home automation is and why you would use a Raspberry Pi as a home automation gateway. Then I describe what I consider properties of a 'good' home automation system:

secure

modular

open-source

self-sufficient

In the rest of this book, I'll explain step by step how to create such a good home automation system with a Raspberry Pi.

1.1 • What is home automation?

Home automation is the process or result of automating systems that are running at home: lighting, HVAC (heating, ventilation, and air conditioning), appliances such as washing machines, blinds, and roller shutters, and so on. A home automation system is also able to use information from environmental sensors (temperature, humidity, pressure, CO2, …), smart meters, movement sensors, presence sensors, cameras, and so on.

A home automation system typically consists of:

a central gateway (also called controller or hub), which controls devices and reads sensor measurements

controllable devices

sensors

The controllable devices and sensors are regularly called "smart devices", although almost none of them are really smart. Another name you'll see for them is IoT (Internet of Things) devices because they can be (directly or indirectly) linked to and controlled over the internet.

Figure 1.1 A home automation system consists of a central gateway and various controllable devices and sensors.

Chapter 1 ● Introduction

A home automation gateway always has a user interface. Of course, the purpose of home automation is to automate as much as possible, so the idea is that the user shouldn't have to use this user interface that much. But a user interface is still essential to:

configure the home automation gateway: for instance if the sun goes down, close the blinds;

manually control your devices: this should still be possible because you can't automate everything;

show you a nice dashboard of sensor measurements: for instance to see the inside and outside temperature.

This user interface can come in many forms:

Most home automation gateways have a web server running, which supplies a web interface as the user interface. You can access this web interface on any computer or mobile device.

Some systems have a mobile app for Android or iOS, which is generally better adapted to the specific requirements of mobile devices, such as a smaller screen.

It's also possible to use a dedicated touch screen, for instance hanging on the wall, to control your home automation system, and to show you some nice graphs.1

Last but not least, in recent years home automation systems have added a new kind of user interface: speech. With a so-called voice assistant or smart assistant (again, they are not that smart), you can give speech commands to your home automation system and it replies with spoken messages.

This book is not focused on any of these user interfaces; it's more about the backend services and how to link and automate them. However, I cover two home automation projects with a web interface in Chapter 10 (Home Assistant and Node-RED), and create a voice assistant for your home automation system with Rhasspy (Chapter 12). You should consult the documentation of these projects if you're more interested in the user interface side of home automation.

1.2 • Why use a Raspberry Pi as a home automation gateway?

If you buy an off-the-shelf home automation system, the gateway is a small box that looks somewhat like a router or a Wi-Fi access point. In this book, we'll show you how you can create your own home automation gateway with a Raspberry Pi.

But why would you do that? Because you can, of course! More seriously, the Raspberry Pi is what makes the approach in this book possible. We'll go into the advantages of this approach in the next section, but the number one reason to use a Raspberry Pi as your home automation gateway is: you are in control.

An off-the-shelf home automation gateway isn't flexible: you can only do with it what the

1 In many cases this touch screen is just running a fullscreen web browser visiting the home automation gateway's embedded web server.

Control Your Home with Raspberry Pi

manufacturer allows, you rely on the manufacturer's goodwill to receive updates and new functionality, and most of the time you can't "hack" on it yourself.2

Contrast this with the Raspberry Pi. You can choose your operating system (which we'll do in the next chapter), you can choose which communication protocols you'll want to support (which we'll cover in various chapters in this book), you can choose your user interface, and so on. You can even choose the case to protect your Raspberry Pi and which expansion boards you connect, as there's a whole ecosystem of hardware for the Raspberry Pi.

Of course, you can also run home automation software on a more powerful system, such as a NAS (network-attached storage) or a home server. But the Raspberry Pi has several advantages to these systems:

It's very low-power, so it doesn't cost you much to keep it running 24/7.

For most home automation tasks you don't need the processing power that these other systems offer.

The ecosystem of software and hardware for the Raspberry Pi is immense, as well as the number of resources where you find more information about it.3 This is also a reason to choose the Raspberry Pi over similar single-board computers from other manufacturers.

1.3 • The properties of a good home automation system

A good home automation system should:

be secure, so you don't risk someone else controlling your house or spying on you at home;

be modular, to make it easy to plug in other protocols or applications;

only use open-source software;

be self-sufficient, not relying on cloud systems from Google, Amazon, or other parties.

This is my highly opinionated vision, and it's this vision that I build upon in this book.

If you consider these properties for a moment, you'll see that this vision is almost diametrically opposed to most off-the-shelf systems you'll find. I'll give some examples in the next subsections.

It's possible that you don't agree with some of these properties, or that you don't have such strong feelings about them as I do. That's OK: while I explain an approach in this book

2 With hacking I don't mean gaining unauthorized access to a computer (which is the connotation the word unfortunately has received). The Jargon File describes a hacker as "a person who enjoys exploring the details of programmable systems and how to stretch their capabilities, as opposed to most users, who prefer to learn only the minimum necessary." (The Jargon File, http://www.catb.org/jargon/html/H/hacker.html)

3 For instance, the Raspberry Pi Foundation publishes its official magazine about the Raspberry Pi, MagPi (https://magpi.raspberrypi.org), and Elektor (the publisher of this book) publishes Dutch (https://www.magpi.nl) and French (https://www.magpi.fr) editions of the magazine.

Chapter 1 ● Introduction

that implements this vision, thanks to its modularity you can certainly plug in proprietary software or cloud systems if you prefer these. Heck, you can even add monolithic and insecure software, but I don't tell you how.

In the next subsections, I'll go over these four properties in more detail, and I hope that at the end of this chapter you'll agree with me that this approach to home automation is the right one.

1.3.1 • Secure

Of course, a home automation system should be secure. No one can be against it, can't they? A home automation system controls your home, so whoever can break into it can make your life very miserable.

Unfortunately, even if a manufacturer tells you that his system is secure, chances are that it isn't. Security is very difficult to attain, and most manufacturers don't want to spend the resources needed to secure their system.4

Home automation and IoT devices are notoriously insecure. At the Usenix Security Conference 2019, the Czech security software company Avast and Stanford University presented their research of household IoT devices. Avast scanned 83 million IoT devices in 16 million homes around the world of people who agreed to share these data. The results of the study published in "All Things Considered: An Analysis of IoT Devices on Home Networks" (https://press.avast.com/hubfs/stanford_avast_state_of_iot.pdf) were staggering:

7% percent of all IoT devices support an obsolete, insecure, and completely unencrypted protocol such as Telnet or FTP.

Of these, 17% exhibit weak FTP passwords, and 2% have weak Telnet passwords.

Surveillance cameras have the weakest Telnet profile, with more than 10% of them that support Telnet with weak credentials.

3% percent of the homes are externally visible on the internet and more than half of those have a known vulnerability or a weak password.

This is not an isolated study. Not a week goes by without some news items about insecure devices, most of the time because basic security measures such as strong passwords are not enforced by the manufacturer or basic programming errors have been made. To give you an idea about what can happen: in 2018 nude videos of the Dutch women's handball team appeared on a popular porn website because the surveillance cameras of the dressing room of a sauna were broken into. Imagine if someone can access your baby monitor with a camera or your security camera in your living room or bedroom…

So what can you do to secure your home automation system? If you choose an off-the-shelf system: not much. You fully rely on the manufacturer's ability to create a secure

4 Most consumers probably wouldn't want to pay more for a secure home automation system anyway.

Control Your Home with Raspberry Pi

system and the goodwill to keep supplying patches that solve security issues that have been discovered. And the home automation and IoT industries have clearly shown they are not up to the task. This is one of the reasons why I prefer open-source software. Not because it is always secure, but because the transparency of the open-source development process forces developers to create more secure software.

Security is such an important property of a home automation system that I dedicate an entire chapter in this book about it. It's such a vast topic that entire books are written about it, and I encourage you to read much more about computer security than I can tell you here. In Chapter 3 I'll cover the most important tools you need to secure your home automation system, so you don't need to be paranoid and continuously think about the possibility that someone is currently spying on you.

1.3.2 • Modular

There are many competing standards and communication protocols for home automation, such as Z-Wave, Zigbee, and KNX. Other protocols aren't specific to home automation but are very usable in this domain too, such as Wi-Fi, Bluetooth, or Near Field Communication (NFC).

Unfortunately, many off-the-shelf home automation gateways support only a small subset of these protocols or even use a proprietary protocol that locks you into using devices of the same manufacturer. That severely limits your choice of products.

You can't know which protocols will become popular in a few years, and maybe you like one product that uses Z-Wave and another product that uses Zigbee. It should be easy to interconnect these devices, even when they use different protocols.

This is why a good home automation system should be modular, which makes it possible to plug in new components when you want to support a new protocol, add a new user interface or extend its functionality in another way.

Many of the wireless communication protocols for home automation need a dedicated transceiver because they work on a specific radio frequency. That's where the Raspberry Pi shines: you can easily connect Z-Wave, Zigbee, or 433.92 MHz transceivers using the USB ports or the GPIO header. So you can start with a basic Raspberry Pi setup supporting only IoT devices that are communicating over Wi-Fi and Bluetooth, add an RTL-SDR USB dongle to read the measurements of your 433.92 MHz weather sensors, later add a Z-Wave HAT on the board when you start adding Z-Wave sensors to your house and then add a Zigbee USB transceiver when you want to control some Zigbee lights.

Figure 1.2 A good home automation system is modular enough to support many home automation protocols.

Chapter 1 ● Introduction

Modularity is also important for software. There's a lot of user-friendly software to make your Raspberry Pi a home automation gateway.5 So you just install this software on your Raspberry Pi and that's it: you have a gateway that supports several devices. Some of these systems are very modular and extensible, others aren't. Many of them support MQTT (Message Queuing Telemetry Transport), a common language to exchange messages.

MQTT has become the standard for interoperability between various home automation devices. For instance, if your home automation gateway of choice doesn't support Zigbee but it does support MQTT, then you only have to run the Zigbee2mqtt software (see Chapter 9), which translates the Zigbee protocol to MQTT messages. Your gateway can then talk to your Zigbee devices using MQTT.

Modularity also means that you don't have to have one gateway. You can perfectly have your main gateway in your basement, but install a second gateway with your 433.92 MHz receiver for your environmental sensors in your living room because that gives you better coverage to receive data from these wireless sensors. If you're using MQTT, that's very simple to implement: you just relay the sensor readings that your gateway in the living room receives to your MQTT broker, after which your main gateway receives the readings in the MQTT format.

In short: a good modular home automation system means that you can mix and match the devices that you like, irrespective of their protocol, and you can use the software and hardware components of your choice, in various locations in your house.

1.3.3 • Open-Source

Source code is code written in a human-readable programming language, that specifies the actions a computer has to perform. The source code of a program is then compiled to machine code that the computer can execute, or it's interpreted on the fly to machine code and thus immediately executed by the computer.

Most software is being distributed as machine code, so it's not readable for us. If you buy an off-the-shelf home automation gateway, you generally don't get access to its source code, so you cannot peek into it to see what it does or to assess its quality. You just have to believe the manufacturer on his word. Is that enough for you if it's about software that will get to know you intimately because it processes sensor readings and even camera images about you in your home? Not for me.

But there's a type of software where you do get access to the source code: free and open-source software (sometimes abbreviated as FOSS or even FLOSS).6 If you really want to be precise, there's free software and open-source software, but for the end-user, the differences are minimal and mostly philosophical. When I talk about "open-source

5 An example is Home Assistant, which I will introduce in Chapter 10.

6 The L in FLOSS stands for "libre", which is kind of a synonym to "free" but making it clearer that it's about maintaining the user's civil liberties: "free" as in "free speech", not as in "free beer", as they say.