IoT Based Smart Home Using Multiple Language Voice Essay

Anu prabha RS, Athul Chandran BS and Vivek Raj

Department of Computer Science and Applications

Amrita School of Engineering,Amritapuri

Amrita Vishwa Vidyapeetham,India

Abstract— The rapid increase in the number of users of the

internet over the past decade has made the Internet a part of

the day to day life, and IoT is the latest and emerging internet

technology. IoT includes a variety of objects like smartphone,

sensors, tablets. Once all these devices are connected, they

enable smarter processes and services that support our basic

needs. With the development of technology and usage of

Smartphone people start to follow a smart way of living.Hence

this paper presents a proposal for home automation using

Google Assistant in multiple languages like English and

Malayalam to control various home appliances and also a

smart door unlocking system.

Keywords- Homeautomation; ESP32; Relayboard;

Arduinocam; Blynk application; IFTTT application;

I. INTRODUCTION

The world has become a global village due to the revolu-

tion in the technology; in this revolution,Information Tech-

nology played an important role.

Similarly, the development

of IT makes people dream come true to have an automated

home. In this fast-moving world, people are more attached

to smart technologies. People need all to be automated and

easily usable. Because of this, homes of this century become

more and more automated due to the ease it provides. Home

automation is one of the fast-growing industry that can

change the way people live due to their ease of use and

wide operations capabilities. Even though people are lucky

to have new smart devices, Most of the home automation

system is a bit too expensive. However, not everyone is

wealthy enough. Hence, the need to find an inexpensive

smart assistant for simple users keeps growing. Some of these

home automation systems target those seeking luxury. The

elder and differently abled person find it hard to get used

to these changing technologies, this helps them to control

devices using their local languages.This paper proposes such

an inexpensive system that reduces the effort put in by the

user to control a device manually. The wireless network

based smart home systems have become very popular as

they provide comfort, security, and safety for all types of

users. The aim of this project to design and develop a

home automation system which is wireless. This system uses

Google Assistant SDK, to communicate with the ESP32 and

the users. The system that can respond to voice commands

and control the on/off status of electrical devices, such as

lamps, fans and smart door unlock . Integrating Google

Assistant technology to home automation systems make the

system more user-friendly and easy to operate.

II. RELATED WORKS

Several methods have been proposing for home

automation. Most of the system uses single language

voice commands and too expensive also not suitable

for elderly persons. Some of them notify with SMS.

To solve this issue many new technological approaches

are introduced. But above all these technologies, home

automation using multiple language voice commands is

reliable and efficient.

This paper aims to set Home automation, which

is operated using a smartphone. This system uses a

Bluetooth module used to control the home appliances

that are connected using an FPGA board which

enables communication between them using serial

communication[1].

The objective of this paper is to design and implement

a cost-effective and secure home automation system. It is

a smart home system controlled based on SMS and voice

recognition and also have fault identification system which

ensures home appliances goes On or Off[2] .

The proposed system is a wireless home automation

system. This system work based on voice commands. The

system consists of an RF Zigbee wireless communication

module, which helps to control all lights and electrical

appliances in a home or office[3].

This paper presents a different home automation system

which can be idle for paralyzed and elder ones. The main

focus of the system is to control the electrical appliances

like switching the light On or Off, turning ac on by voice

commands. But there is no voice automation for bed

elevation in this system[4].

The prime objective of this project is to build a smart

wireless home security system which will send an alert

message to the owner by using the internet. The advantage

of preferring this system is that the user gets an alert

message to his phone from any distance irrespective to

mobile phone connected to the internet[5].III. SYSTEM ANALYSIS

A. Problem statement

Homes of the future have been a recurring theme for

more than 50 years in popular culture and the technology

industry. When Hanna-Barbera created The Jetsons cartoon

in 1962, for example, they had some interesting ideas about

what the world would look like in 2062. Fifty-four years

later and some of those ideas don’t look so out of place;

mobile phones, flat screen televisions and video calls are

now all firmly established features of everyday life. The main

challenges of the home automation system are reliability,

security, digital transformation and integration, and liability.

The aim of this paper was to propose a home automation

system controlling general appliances found in ones home

using Google Assistant. This system is highly reliable and

efficient for aged people because we are using English and

Malayalam voice commands to control the system.

receives it like media message and the user can unlock the

door using voice commands.

Then do the hardware assembly for the system and power

up the ESP32 and Relay board using the 5V mobile charger,

and a 12V power supply to the camera.

B. Proposed System Feature

The proposed system is a home automation system that

controls home appliances using Google assistant with mul-

tiple languages like English and Malayalam, and it contains

new innovative modules like smart door unlocking. The

system uses the Ardu cam mini(OV 2640). The camera is for

live face detection and sending the image captured to the user

as a media message. Then the user can give commands using

google assistant for unlocking the door. ESP32 is used as the

working microcontroller for the device. The other important

components are Electromagnetic lock and a relay board.

IV. METHODOLOGY

The software part of the proposed system consists of

mainly three applications Blynk, IFTTT, and Slack. This

system uses ESP32 as the microcontroller.Relay board,

Electromagnetic Lock, camera and calling bell switch

interfaced to ESP32.To Configure the ESP32 and relay

board. Install the Blynk application on a smartphone. Then

log in to it and create a new project. Give a name to

the project and select the microcontroller ESP32 also set

connection type as Wi-Fi and hit create button. Blynk

will send an authentication token to the registered email

id. The authentication token can be used later to link the

application with NodeMCU. Since using a four channel

relay, add four buttons to the project and click on the button

to add a name. Download Arduino IDE and configure Blynk

libraries and do the necessary settings and upload the code

to NodeMCU to connect Blynk app.Then setup IFTTT

application to understand the commands for triggering the

proper pins on ESP32 with the help of Blynk.To control the

appliances using multiple language voice commands. Create

a triggering applet for google assistant in IFTTT. Specify

the language and triggering command while creating it.

For smart door unlocking when the calling bell switch

press, the camera activates and captures the image. Then

ESP32 sends this image to the slack application using a

web Post method through a web server and the user

Fig. 1.

Architecture diagram

V. HARDWARE AND SOFTWARE COMPONENT

A. ESP32

ESP32 is a wifi SOC (system on a chip) produced by

Espressif Systems.It is based on the ESP32-12E WiFi

module.It is a highly integrated chip designed to provide

full internet connectivity in a small package.It can be

programmed directly through the USB port using LUA

programming or Arduino IDE.By simple programming we

can establish a WiFi connection and define input/output pins

according to your needs exactly like Arduino,turning into a

web server and a lot more.ESP32 is the WiFi equivalent of

ethernet module.It combines the features of the WiFi accesspoint and station + microcontroller.These features make it

an extremely powerful tool for WiFi networking. It can be

used as an access point or connect to the internet to fetch

or upload data.

designed for the Internet of Things. It can control hard-

ware remotely, display sensor data, store data and visualize

it.Blynk App setup is required; we set it up as per the

need. We begin by creating a new project in the Blynk app

and selecting the microcontroller we are using. After which

we can drag and drop the needed toggle button for each

relay associated withthe digital pins of the microcontroller.

Once this is complete, Blynk sends a unique identifier

Authentication token for our hardware to our registered email

id which is needed to connect to the device. This token

should be noted and saved for its use while programming

the NodeMCU and setting up the IFTTT application.

D. Ardu cam mini ov2640

Fig. 2.

NodeMCU

B. Relay Board

A relay is an electromagnetic switch operated by a rela-

tively small electric current that can turn on or off a much

larger electric current. The heart of a relay is an electro-

magnet.You can think of a relay as a kind of electric lever:

switch it on with a tiny current and it switches on another

appliance using a much bigger current. As the name suggests,

many sensors are incredibly sensitive pieces of electronic

equipment and produce only small electric currents. But

often we need them to drive bigger pieces of apparatus that

use bigger currents. Relays bridge the gap, making it possible

for small currents to activate larger ones. That means relays

can work either as switches or as amplifiers

Ardu cam mini ov2640 is a high definition spy cam. It

integrates 2MP or 5MP CMOS image sensor and provides

miniature size as well as easy to use a hardware interface.

Ardu cam mini can be used in platforms like Arduino,

Raspberry Pi, Maple, Chipkit, Beaglebone black and also

offers capabilities to add a camera interface which doesnt

have in some low-cost microcontrollers, but also can add

multiple cameras to a single microcontroller.

Fig. 4.

Ardu cam mini ov 2640

E. IFTTT Application

Fig. 3.

Relayboard

IFTTT (If This, Then That) is a simple service that lets you

create powerful connections between Automatic and many

other devices and digital services across the web. It is both a

website and a mobile app and has the slogan ”Put the Internet

to work for you”.Here, IFTTT application is used to bridge

the gap between the Google Assistant commands and the

Blynk app. To configure IFTTT go to IFTTT’s website and

sign in using your Google account.After which we need to

create an applet and then This,i.e. the trigger, here we select

Google Assistant and then we will type in the commands

to which the Google Assistant should respond and to this

command it should control the relay associated with it.Also,

we can give the required response from the Google assistant.

VI. EXPERIMENT AND RESULT

C. Blynk Application

Blynk is an application which helps to control Arduino,

Raspberry Pi, NodeMCU etc over the Internet in which it

supports Platform with iOS and Android apps. It’s a digital

dashboard where we can build a graphic interface of our

project by simply dragging and dropping widgets.Blynk was

In our experiment, we create a prototype of the system.

The main objective of this system is to propose a home

automation system which can be control home appliances

using multiple languages voice commands like English and

Malayalam. The Blynk and IFTTT application help to con-

nect Google assistant to the ESP32 directly. The followingfigure show how to configure the Relay board using Blynk

app.

Fig. 5.

Relay board configuration

Select the pin as the digital pin for the relay and do the

hardware assembly. To work ESP32 with voice commands

configure IFTTT application as below by creating on/off

triggers for each relay. Then invoke Google assistant by

Fig. 7.

Proposed system

VII. CONCLUSIONS AND FUTURE WORK

This paper proposes a cost-efficient and effective manner

of multiple language voice commands based home automa-

tion system. The proposed system implemented successfully

and observed that controlling the home appliances smartly.

This system is highly useful for aged peoples and differently

abled persons who can’t walk. By taking this system as a

framework this system can be extended up to a higher level

like real-time face recognition using cloud storage and can

improve the accuracy of identifying the local languages.

VIII. ACKNOWLEDGMENT

Fig. 6.

IFTTT configuration

We are extremely thankful to Anu Prabha RS, Faculty

Associate, Departement of Computer Science and Appli-

cations,Amrita School of Engineering, Amritapuri for her

primary support throughout the project.

R EFERENCES

saying the voice commands given in the triggers, Google

assistant sends that commands to IFTTT. IFTTT interprets

this commands and sends an On/Off signal to the Blynk

app through Blynk server. Blynk sends this signal to ESP32

and then to electrical appliances.After capturing a suitable

image of the person, ESP32 send this image by Post

method through a web server, and the user receives it like

a media message. Then the user can unlock the door using

voice commands.

[1] B. M. Krishna, V. N. Nayak, K. REDDY, B. Rakesh, P. KUMAR,

and N. Sandhya, “Bluetooth based wireless home automation system

using fpga.” Journal of Theoretical & Applied Information Technology,

vol. 77, no. 3, 2015.

[2] S. Amrutha, S. Aravind, S. S. Ansu Mathew, R. Rajasree, and

S. Priyalakshmi, “Speech recognition based wireless automation of

home loads-e home,” International Journal of Engineering Science and

Innovative Technology (IJESIT), vol. 4, no. 1, 2015.

[3] H. AlShu’eili, G. S. Gupta, and S. Mukhopadhyay, “Voice recognition

based wireless home automation system,” in 2011 4th International

Conference on Mechatronics (ICOM), May 2011, pp. 1–6.[4] M. Kumar and S. Shimi, “Voice recognition based home automa-

tion system for paralyzed people,” International Journal of Advanced

Research in Electronics and Communication Engineering (IJARECE),

vol. 4, no. 10, pp. 2278–909, 2015.

[5] R. K. Kodali, V. Jain, S. Bose, and L. Boppana, “Iot based smart security

and home automation system,” in 2016 International Conference on

Computing, Communication and Automation (ICCCA), April 2016, pp.

1286–1289.