Contextual intelligence using knowledge graphs

Abstract

1. Background

Context is relevant information related to location, time, user, identity, role, activity and nearby devices. In general, context refers to any information that is related to given entity, such as user or device. Contextual Intelligence refers to the ability of a system to gather environmental information at any given time and adapt its behavior accordingly. Contextually intelligent systems automatically gather data from various sensors and sources to determine system state and process the data intelligently to infer or predict useful information, such as providing enhanced personalized experience services. Knowledge graphs are representation to captures entities, attributes, and relationships in graphical form. Ontology is a formal way to describe structure of knowledge for various domains. The goal of this project is to represent contextual data in form of knowledge graphs (ontology) and apply Probabilistic reasoning methods, such as Dynamic Bayesian Networks, to accurately predict user actions based on contextual data.

2. Objectives

Objective of this project is to develop technique and framework for developing contextually intelligent system for enabling predictive analytics by detecting dynamic changes in contextual data. Such system would be very useful in understanding user behavior patterns through web usage or content consumption and there by performing web analytics or enabling personalized recommendation or user targeted advertisement services.

3. Scope of Work

For the scope of this project, I would like to choose either Web usage pattern analysis or program/content viewing analysis, as use-cases to infer user preference and interest. Context awareness is required to perform Predictive Analytics since the circumstances/situation under which decision is being made is dynamically driven through contextual data. In each of these use-cases, context is time, location, user and device. The system would be modeled in form of a user ontology comprising of various entities, relationships and role. The system would be trained using historical data (training phase), after that unseen instances are dynamically received, the Contextual data influences the weights/condition probabilities resulting in prediction of user behavior / action (prediction phase). System would potentially use Dynamic Bayesian Network for inference/prediction.

Acknowledgements

The satisfaction and euphoria that accompanies the successful completion of any task would be incomplete without mentioning the people who made it possible, because success is the epitome of hard work, perseverance, undeterred missionary zeal, steadfast determination and most of all “ Encouraging Guidance”. I express my gratitude to my supervisor Dr. Kaushik Saha and my mentor Mr. Kiho Kim for providing me a means of attaining my most cherished goals. I record my heart full of thanks and gratitude to my additional examiner Mrs. Sheetal Arya for assisting me to carry this project, along with purposeful guidance and moral support extended to me throughout the duration of the project work.

What is Context? Context is information related to a particular situation given the location, time, user, identity, role and activity, etc. In general, context refers to discourse or a set of facts or circumstances that surround a situation or event. It is information that is related to given entity, such as human or device in a specific environment in a particular situation. Humans have intelligence to understand the situations and take action based on their requisite intelligence of understanding the ‘ context’ from similar experiences in the past. For a computer / device, it is possible to do so only by systematically collecting the contextual data and understanding past experiences through prolonged training experience. Even in such case, it may not be possible to do so due to uncertainties involved in situations and dynamic events. What is Contextual Intelligence? Contextual Intelligence refers to the ability of a system to gather environmental information, adapt to its behavior accordingly and infer/predict useful action based on the situation.

Contextually intelligent systems automatically gather data from various sensors and sources to determine system state and process the data intelligently. Goal of such systems is to infer or predict useful information or predict an action, such as providing enhanced personalized experience services or predicting possible action which will be performed by the user in such situation, thereby providing assistance to the user in accomplishing that seamlessly.

However, predicting an action or providing assistance just based on the current context or current situation may not be appropriate. In real life, decision making is done based on combination of experiences gathered from past events and current context. Thus, it is important to model previous events and situations in form of knowledge and establish relationships to be able to track various situations, dynamic events and infer intelligently from the gathered information. What are Knowledge Graphs & Ontologies? Knowledge graphs are representations to capture entities, attributes, and relationships in graphical form. Ontology is a formal way to describe structure of knowledge for various domains.

Objectives

Broadly the objective is to design an Ontology-based Contextual Intelligence system for Smart devices (such as Smart TV, Smart Phone, Smart Watch, etc. ) which can collect data from various sources (such as frequently watched TV Channels, frequently browsed website categories, time and location, content consumption behavior, such as programs of certain genre or particular celebrities, etc. ) extract the context from it, model them appropriately using Ontologies, derive user behavior from it and then infer something from it in terms of predicting the action which user will perform. Major Goals: This can be broken down into the major goals listed as below:

1. To acquire user context from connected devices, external inputs and usage of devices
2. To understand the context semantics with the help of ontological knowledge
3. To apply reasoning for situation tracking based on context-awareness, and
4. To apply reasoning to intelligently recommend appropriate actions which suit the situation

Motivations

It can be argued why we are interested in investigating Contextual Intelligent services on Smart Device. Contextual Intelligence Service means that the service is aware of the current context of the user.

The two main motivations for using a Context aware service are as following:

1. It helps in providing personalized service to the user
2. Connected device environment allows for context to be available across other devices the user profile to be present ubiquitously

Problem Description

As explained above, the objective is to enable Smart, Personalized, Cross domain and unified experience based services on Smart Devices by learning usage patterns, context and suggest multi-domain and cross domain intelligent actions/services to the user. This would be achieved through single profile of usage of various functions, including all types of contents consumed by the user, tracking and understanding various life patterns of user such as viewing time/location/duration, etc. Key technologies which would enable the same are:

User Behavior Analysis (context extraction, pattern learning, context representation)

Context extraction such as

For example, while a smart device may be able to recognize who the user is (perhaps through face recognition methods), but the system would not be able to gather entire context of the user, such as:

1. Who the user is
2. What is user’s interest and preference
3. What activity user is doing, and so on

Understanding the semantics can assist the Contextual Intelligence systems to provide more valuable and relevant service to the user. Thus, we need knowledge-based systems to make sense of semantics and meaning of the user context, by placing it in the right perspective. An ontology, which is technology of the Semantic Web, is an ideal tool to do so. Existing Ontology-based systems mainly use ontology at a low-level mainly for interpreting devices and for actuation based on data source interpretation. Not only the knowledge mining is done at low-level, but crisp reasoning is used to make decisions and provide recommendations. Crisp reasoning is not ideal for multimedia device inputs that really provide the user context. It is better to use probabilistic reasoning as these inputs may have uncertain values. Existing ontology-based applications are also handicapped by the use of existing languages like RDF/OWL, which only allow crisp reasoning with Description Logic and Rules. These languages have no constructs to model events and situations; they cannot encode uncertainties in context observations, and they cannot provide probabilistic reasoning with events. Ontology-based Contextual Intelligent system use Ontology at a higher Semantic level to provide more complex reasoning and adds Semantic to Context. It also adds a layer of Event Recognition over context situations. We propose a framework in which the Ontology can help abstract devices to a data-type on which the reasoning is based. The framework incorporates a probabilistic reasoning framework which takes into account uncertain and partial context observations and can do reasoning with events as well as contradictions.

Related Work

Ontology has been used in many existing Smart systems, but often at a low level – mainly for modeling the domain of sensor devices and for actuation based on sensor interpretation. In [2], the authors have used ontology to formally represent an automatic in Ambient Intelligence (AmI) scenarios. They have used crisp rule-based reasoning to configure the sensory infrastructure and autonomously manage its functioning. This has confined the system to those sensor based observations which are deterministic as required for crisp logic, whereas in real-life some of these observations may be uncertain. There are other IoT systems [3], which have used fuzzy logic and/or Bayesian reasoning to provide Ambient Intelligence while understanding the user needs, but they have failed to utilize ontology-based knowledge of the domain. Several researchers have used an ontology for an IoT domain like Smart Home, but the ontology used encodes static relationship between concepts, and is unable to cope with dynamic context observations. A temporal ontology and rule-based reasoning has been used in [4], while Nyugen et al [5] have not included any reasoning based on dynamic data. Many research efforts have focused on a combination of statistical reasoning and ontology-based knowledge representation. Dynamic Bayesian networks have been used to derive context fragments and deal with contextual ambiguity in [6] and [7]. The latter have also used an ontological rule-based approach along with DBN for reasoning with unambiguous context situations. In this paper, ontology has been used to model a situation of interest and DBN is incorporated for situation prediction, but the system does not handle situation tracking when the environment variables change dynamically to reflect a changing Universe of discourse for the situation.