Acquiring Commonsense Knowledge with Visual Data: Challenges and Solutions

What to achieve in this model?

What to achieve in this model?

In this project, it is required that a model should be trained in such a way that it can acquire “common sense” Through implicit knowledge. Implicit knowledge is a knowledge that requires a machine to apply commonsense to make inference. Hence, it is more difficult to train a machine in this regard as compared to the explicit knowledge where instructions are clearly stated in a sentence format such as on, below, left, right.

For example, when a machine is trained to understand “man riding horse”, it is expected that when the machines sees a man on a donkey that has a similar spatial features with a horse, the machine can easily predicts is “man riding horse” Simply because of the spatial features of the donkey that is similar with the horse even though it has not seen a donkey before and also understands the action between “man” And “horse”.

To achieve this goal, the structured text must be paired with the visual data(images) to train the model. For instance, how would a machine understand what a horse is in a sentence if it has not seen the image of what a horse looks like. This idea is basically based on the cognitive.

What is CSK?

Commonsense knowledge(csk) is an intrinsic part of human behaviour and understanding but exceedingly difficult for machine to acquire. Csk refers to everyday knowledge and facts possessed by most people and spans over a huge portion of human experience such as physical, social, temporal, and psychological aspects of typical everyday life. Artificial intelligence has experienced a long-standing challenge representing commonsense knowledge in a language understandable by machine.

 Csk acquisition is different from the encyclopedic knowledge, which is explicitly stated and based on facts. The encyclopedia knowledge has given machine the intelligent level that is far more than that of human. For example, we can successfully retrieve the details such as age, education, date of birth, occupation et cetera about a person by simply issuing a query on a search engine. It is extremely difficult for human to have details of millions of people in their memories. Despite high level of machine intelligence with encyclopedia knowledge, it still lags humans when it comes to commonsense reasoning. For example, it is still difficult for machine to differentiate between a truck and an overpass.

Csk is more difficult to acquire because it is unlike the encyclopedic knowledge that is specific and detailed but csk deals with general knowledge. Commonsense is hard to comprehend because it is most times implicitly expressed and considered to be bias in reporting. The over fifty years research history of automated commonsense has witnessed slow but steady progress. However, monitoring this progress has become a difficult one since there is no standard means by which researchers evaluate the performance of their models or make any comparison with other researchers. Csk acquisition challenges are basically the elusiveness and context-dependence. Popular projects in csk bases include hand-crafted resources such as wordnet and cyc conceptnet, webchild and visual knowledge bases like visual genome is an electronic database and has been a resource book since 1991 for thousands of researchers in language study. [9] was originally designed by and for linguists and psychologists but has been a very vital tool for computationalists. Similarly, was conceived since machine learning requires core knowledge to learn “common sense”. The cyc system consists of three major components which are all critical for machine learning. The components of are the knowledge base(kb), the inference engine, and the natural language(nl) is based on textual information to acquire “common sense” And is different from and deals with the categorization of lexis and word-similarity, optimizes logical reasoning basically contains triples that connects nouns with adjectives using fine-grained relations. address the cognitive task of training a model to understand the interaction and relationship between objects in an image.

The advancements in pre-trained word, and sentence representations, and automated text process have given systems a great performance over humans. However, there are complex challenges that go with natural language(nl) representation in machine.