MEERA

Multifunctional Event-driven Expert in Real-time Assistance

GitHub release Travis (.org) Website GitHub

MEERA or Multifunctional Event-driven Expert in Real-time Assistance is a general purpose open source Artificially Intelligent Bot framework. MEERA is designed to be used as general purpose bot with expandable skill set that can be extended using easy to use plugin framework

Table of Contents

Features

Supported Platform

External Dependencies

See requirements.txt file for more details.

Getting Started

Setting up MEERA

To setup MEERA:

  1. If your machine has lower configuration, run ./scripts/create-swapfile.sh, before starting installation.
  2. Run ./setup.sh
  3. Add your API keys to .env file
  4. Run ./meera.sh deploy all to deploy the application along with telegram client.

Launch Configuration for Remote Server

apt install git
git clone https://github.com/AmeyKamat/MEERA.git
cd MEERA
chmod +x ./scripts/create-swapfile.sh
chmod +x ./meera.sh
chmod +x ./setup.sh
./scripts/create-swapfile.sh
./setup.sh

Using Terraform

Meera ships with infra/main.tf file that helps deploy application on AWS Lightsail. Here are the instructions to do so:

  1. Add AWS Access Key Id and AWS Secret Access Key to environment.
export AWS_ACCESS_KEY_ID=<Your AWS Access Key Id>
export AWS_SECRET_ACCESS_KEY=<Your AWS Secrect Access Key>
  1. Navigate to infra/ directory.
  2. Add .env file
  3. Generate ssh keys as follows:
ssh-keygen -b 2048 -t rsa -f meera
  1. Run terraform init
  2. Run terraform plan to validate
  3. Run terraform apply

Environment variables in .env file

ENV Variable Description More Details
MEERA_GOOGLE_API_KEY Google API Key link
MEERA_NEWSAPI_API_KEY NewsAPI API key link
MEERA_OPENWEATHER_API_KEY OpenWeatherAPI API key link
MEERA_TIMEZONEDB_API_KEY TimeZoneDB API key link
MEERA_MAILGUN_API_KEY Mailgun API key link
MEERA_MAILGUN_DOMAIN Mailgun domain link
MEERA_TELEGRAM_BOT_TOKEN Telegram bot token link
MEERA_TELEGRAM_AUTHORIZED_USERS List of telegram authorized users  

Usage

Once MEERA is installed on your system, you can start using MEERA from command line utility. Following are the commands supported:

Supported Commands

$ ./meera.sh help
usage: ./meera.sh <command> [<args>]

These are commands used in various situations:

install MEERA as fresh installation
   clean                                Clean the project directory
   pre-install                          Prepare machine for installation
   install                              Install MEERA
   deploy [server|telegram-client|all]  Deploys MEERA. Takes an optional argument to deploy specific component. Defaults to 'all'

manipulate machine learning model
   train [<iterations>]                 Train ML models. Takes optional argument to specify number of training iterations. Defaults to 50
   evaluate                             Evaluate ML models
   download-model                       Downloads ML models
   install-model                        Install backup models stored in download folder during training

code quality and sanity checks
   lint                                 Lint code for code quality
   test                                 Run tests

system info
   version                              Shows version and system info

For more info visit http://www.ameykamat.in/MEERA/

Here is what each of the above commands do:

Enabling Bash Autocomplete for MEERA

Bash autocomplete is enabled in MEERA by default once you run ./meera.sh install sucessfully as root.

Bash autocomplete can be enabled for MEERA temporarily by running source scripts/autocomplete.sh in project root.

It can be disabled by removing meeta.sh file from /etc/bash_autocomplete.d/ folder and running source ~/.bashrc.

Run ./meera.py [TAB][TAB] to see it action.

Client Implementation

This repository provides an implementation of telegram bot that can be used to interact with MEERA. One needs to create a telegram bot, and add the bot token to interface/telegram_bot/telegram.ini.example file for bot to function.

Developers Manual

Concepts

Client

Client is an object that represents external device that is connected to MEERA through websocket API. MEERA generates an id for each of such client which is used to identify the socket to be used to communicate with this device. Client also has a human friendly name attribute and type attribute (currently restricted to mobile_app, web_app or telegram_bot). See this for definition of client object.

Context

Each message that MEERA receives is represent by a Context object. This object contains information such as message, clientId of the client that sent the message, nlpAnalysis performed on the context, conversationId of the conversation the context belongs to, previousContext in the context chain of the conversation, result object generated by the plugins, interaction object generated by the plugins, etc. Each context is uniquely identified by a contextId. See this for the object definition.

Conversation

Conversation is sequence of contexts from a particular client. Each conversation is uniquely identified by a conversationId.

NlpAnalysis

This object stores optput of each of the machine learning models. NlpAnalysis store intent of the message, entities extracted from the message and confidence associated with the intent. See this for the definition.

Result

This object stores the output of plugin executors. Plugin developers can store all the information required to generate dialogue in this object.

Interaction

Interaction object is the output of plugin dialogue generators. it can contains following attributes that can be used in various scenarios in client devices subject to their implementation in client. Note that it is not mandated to implement all of these attributes on client. If a particular attribute is not implemented on client, value of that attribute should be ignored. For example, the telegram client shipped by default with MEERA only supports text attribute. However, plugins are expected to implement atleast text an voice the attributes below.

Attribute Description
text text that can be shown on the client
voice text that can be passed to speech synthesis module in client
link hypertext link that be opened in in browser on client side
sound link to the sound clip that can be played in embeded media player on client
video link to the video clip that can be played in embeded media player on client
youtube link to the youotube video that can be played in embeded youtube player on client

Machine Learning Models

MEERA generates 4 models on ./meera.sh train command. These are listed below:

Model Name Location —- Type Function
en.assistant.requestType.model meera/nlp/models/request_type_model textcat predicts if message is chat or skill
en.assistant.chat.model meera/nlp/models/chat_model textcat generates response for chat message
en.assistant.entity.model meera/nlp/models/entities_model ner extracts entities from skill message
en.assistant.intent.model meera/nlp/models/intent_model textcat predicts intent of skill message

Developing a Plugin

MEERA performs tasks using plugins included. your plugin should be place in this directory for MEERA to load it. Plugins basically consist of 4 components as follows:

plugin.ini

plugin.ini is a property file that describes the plugin. It should have following structure

[NameOfThePlugin]                            # This should match with the name of the executor and dialogue generator class
intent = intentOnWhichPluginWillBeTriggered
# ... any other property you might load in executor or dialogue generator ...

executor.py

executor.py is class where message received by MEERA can be processed.

Following is the basic structure of executor.py

# Any modules you want to import

class NameOfThePlugin(object):                       # This should be same as plugin.ini file

    def __init__(self, config):
        super(NameOfThePlugin, self).__init__()
        self.config = config                         # config object provides all properties from plugin.ini as a dict

    def execute(self, context):
        # process the context here. All attributes in context will be read only. any changes to context object will not be reflected
        result = {}
        # build result object
        return result

If your plugin expects client location, you should first check in self-location key exists in context.nlpAnalysis.entities and if not raise execution.exception.SelfLocationNotFoundException. If the key exists, you will get following object as value of key self-location in context.nlpAnalysis.entities:

{
    "latitude": 25.0340,
    "longitude": 121.5645
}

Also for every human friendly date and location entity identified, MEERA translates them to absolute date and latitude-longitude pair out-of-the-box respectively.

For a date entity, you will find following object in context.nlpAnalysis.entities

{
    "date": "yesterday",
    "parsedDate": "2019-03-10 19:34:15"
}

For location entity:

{
    "location": "taipei",
    "latitude": 25.0340,
    "longitude": 121.5645
}

If you need to access any secrets such as API keys in your executor:

MEERA_THIRD_PARTY_API_KEY=someinterestingapikeyforsomeinterestingservice
key_variable = MEERA_THIRD_PARTY_API_KEY
Status Description
‘success’ sucessful response, generate dialog for the same
‘need-location’ need user location for processing, request client for location
import os
# Any other modules you want to import

class NameOfThePlugin:                               # This should be same as plugin.ini file

    def __init__(self, config):
        self.config = config                         # config object provides all properties from plugin.ini as a dict

    def execute(self, context):

        keyVariable = self.config['key_variable']
        key = os.environ[keyVariable]

        # do something interesting

        result = {...}                               # build result object
        response = {
            'status': 'success',
            'result': result
        }

        return response

dialogue.py

dialogue.py defines how you want to define response of MEERA for a plugin. Output of this class is interaction object. Following is basic structure of dialogue.py

# Any modules you want to import

class NameOfThePlugin(object):                      # This should be same as plugin.ini file

    def __init__(self, config):
        super(NameOfThePlugin, self).__init__()
        self.config = config                        # config object provides all properties from plugin.ini as a dict

    def generate(self, result):
        # extract information from result. Result object is read-only.

        return {
            "text": ...,
            "voice": ...,
            "link": ...
        }

plugin.utterance

plugin.utterance defines examples of utterances that should trigger the plugin as well as guides on what entities to extract from these utterances. Each of the example should be on a new line an should contain an utterance, intent for that utterance and list of entities to be extracted.

Following is a structure of each record

sentence|intent[|entity1,start_index,end_index|entity2,start_index,end_index...]

For example:

where am i|self-location
how far is mumbai from delhi|distance|source-location,11,17|destination-location,23,28

Once you have these all files is expected location, update this plugins.ini file for binding the plugin to MEERA.

Communication with Clients

MEERA exposes a WebSocket API for communication with clients. This API consist for 5 different messages which are exchanged as json objects. this api is exposed on /talk endpoint. Following image shows the exchange of messages:

hello message

This message is the first message that client sends after connection, to register with server.

Message Structure:
{
    "type": "hello",
    "body": {
        "name": "telegram",
        "client_type": "telegram_bot"
    }
}

registration-success message

This message is received by the server as response to successful registration of client.

Message Structure:
{
    "type": "registration-success",
    "body": {
        "client_id": "b110bc34-762b-493b-a4af-9298f255844d",
        "client_name": "telegram",
        "client_type": "telegram_bot"
    }
}

message message

This message is sent by the client to invoke any operation at server.

is_user_authorized is boolean field that is used to allow user to use chargeable plugins*

Message Structure:
{
    "type": "message",
    "context_id": "977ca71c-fbd9-4024-b215-649c45416103",
    "body": {
        "client_id": "b110bc34-762b-493b-a4af-9298f255844d",
        "is_user_authorized": true,
        "message": "Hi"
    }
}

reply message

This message is sent by the server as response to message message.

Message Structure:
{
    "type": "reply",
    "reply_to": "afacbdf6-e7ef-4ba2-8d35-88eb71a91ce6",
    "body": {
        "text": "Hello",
        "voice": "Hello"
    }
}

self-location-request message

This message is sent by the server if it needs location of the client.

Message Structure:
{
    "type": "self-location-request",
    "reply_to": "c8cc450f-e134-46a8-a90f-0a711f078d48"
}

self-location message

This message is sent by the client with location details as response to self-location-request message.

Message Structure:
{
    "type": "self-location",
    "context_id": "c8cc450f-e134-46a8-a90f-0a711f078d48",
    "body": {
        "client_id": "b110bc34-762b-493b-a4af-9298f255844d",
        "latitude": 25.0340,
        "longitude": 121.5645
    }
}

Debugging API

MEERA also exposes following REST endpoints for debugging purpose

GET /status

/status endpoint shows server details and health.

Usage
$ curl -sv -H "Accept: application/json" http://localhost:8000/status | json_pp
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8000 (#0)
> GET /status HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.47.0
> Accept: application/json
>
< HTTP/1.1 200 OK
< Date: Sat, 16 Mar 2019 07:13:29 GMT
< Server: circuits.web/3.2
< Content-Type: application/json
< Content-Length: 179
<
{ [179 bytes data]
* Connection #0 to host localhost left intact
{
   "cpu_utilization" : 6.3,
   "author" : "Amey Kamat",
   "requests_received" : 0,
   "registered_clients" : 1,
   "status" : "UP",
   "email" : "amey@ameykamat.in",
   "memory_utilization" : 44.1,
   "website" : "https://www.ameykamat.in",
   "release_version" : "v1.0,0",
   "conversation_handled" : 0
}

GET /clients

/clients endpoint displays all the clients registered with MEERA.

Usage
$ curl -sv -H "Accept: application/json" http://localhost:8000/clients | json_pp
* Connected to localhost (127.0.0.1) port 8000 (#0)
> GET /clients HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.47.0
> Accept: application/json
>
< HTTP/1.1 200 OK
< Date: Sun, 10 Mar 2019 12:50:22 GMT
< Server: circuits.web/3.2
< Content-Type: application/json
< Content-Length: 113
<
{ [113 bytes data]
* Connection #0 to host localhost left intact
[
    {
        "client_name" : "telegram",
        "client_type" : "telegram_bot",
        "client_id" : "97079591-7bf3-4462-b11f-ac11f659dc00"
    }
]

GET /conversations

/conversations endpoints displays all the conversations in the memory.

Usage
$ curl -sv -H "Accept: application/json" http://localhost:8000/conversations | json_pp
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8000 (#0)
> GET /conversations HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.47.0
> Accept: application/json
>
< HTTP/1.1 200 OK
< Date: Sun, 10 Mar 2019 12:51:53 GMT
< Server: circuits.web/3.2
< Content-Type: application/json
< Content-Length: 115
<
{ [115 bytes data]
* Connection #0 to host localhost left intact
[
    {
        "contexts" : [
            "977ca71c-fbd9-4024-b215-649c45416103"
        ],
        "conversation_id" : "8532fa9f-f8d2-45b5-99e8-c496fb9ed7d2"
    }
]

GET /context/{contextId}

/context endpoint diplays all the data aggregated during the lifetime of the context

Usage
$ curl -sv -H "Accept: application/json" http://localhost:8000/context/70c80f37-8b01-4e74-9c4c-6563f1b84103 | json_pp
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8000 (#0)
> GET /context/70c80f37-8b01-4e74-9c4c-6563f1b84103 HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.47.0
> Accept: application/json
>
< HTTP/1.1 200 OK
< Date: Tue, 19 Mar 2019 16:30:34 GMT
< Server: circuits.web/3.2
< Content-Type: application/json
< Content-Length: 425
<
{ [425 bytes data]
* Connection #0 to host localhost left intact
{
   "client_id" : "fd60cec4-9c08-427d-8d25-81e8ca6cc860",
   "context_id" : "70c80f37-8b01-4e74-9c4c-6563f1b84103",
   "is_user_authorized" : true,
   "nlp_analysis" : {
      "chat_category" : "Hello",
      "request_type_confidence" : 0.99346113204956,
      "chat_category_confidence" : 0.999954581260681,
      "requestType" : "chat"
   },
   "message" : "Hi",
   "interaction" : {
      "text" : "Hello",
      "voice" : "Hello"
   },
   "conversation_id" : "fafda5af-686d-403e-9d87-dd65ec51ad20"
}

Architecture

MEERA banks on various components that communicate with each other through exchange of events. Following figure shows various components of MEERA interacting with each other.

Built with

Contributions to MEERA

Issues

Feel free to submit an issue or request an enhancement here. Ensure that you tag the issue as “Bug” or “Enhancement”

Contribution

This project accepts pull requests from contributors. We usually follow “fork-and-pull” git workflow. If you are fixing an issue or developing an enhancement, please create an issue on this page, before creating a pull request. As guidelines, please ensure that you check following check list before creating pull request:

This project follows

About the Author

Amey Kamat

Feel free to drop your reviews at amey@ameykamat.in.

Contributors

Articles and Discussions

License

© 2019 Amey Kamat

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.