Knowledge in Computer Vision
Fully understanding an image involves recognizing the objects in the scene, how they interact with each other, what might happen next, and discovering the context of the image. This has many practical applications such as data mining from images, compiling visual information from the web and creating agents that can intelligently act in the world.
Our CVPR 2017 paper: The More You Know: Using Knowledge Graphs for Image Classification explores using a new graph neural network model to incorporate knowledge graphs into vision. Code for the paper can be found here: https://github.com/KMarino/GSNN_TMYN
One characteristic that sets humans apart from modern learning-based computer vision algorithms is the ability to acquire knowledge about the world and use that knowledge to reason about the visual world. Humans can learn about the characteristics of objects and the relationships that occur between them to learn a large variety of visual concepts, often with few examples. This paper investigates the use of structured prior knowledge in the form of knowledge graphs and shows that using this knowledge improves performance on image classification. We build on recent work on end-to-end learning on graphs, introducing the Graph Search Neural Network as a way of efficiently incorporating large knowledge graphs into a vision classification pipeline. We show in a number of experiments that our method outperforms standard neural network baselines for multi-label classification.
Our CVPR 2017 paper: The More You Know: Using Knowledge Graphs for Image Classification explores using a new graph neural network model to incorporate knowledge graphs into vision. Code for the paper can be found here: https://github.com/KMarino/GSNN_TMYN
One characteristic that sets humans apart from modern learning-based computer vision algorithms is the ability to acquire knowledge about the world and use that knowledge to reason about the visual world. Humans can learn about the characteristics of objects and the relationships that occur between them to learn a large variety of visual concepts, often with few examples. This paper investigates the use of structured prior knowledge in the form of knowledge graphs and shows that using this knowledge improves performance on image classification. We build on recent work on end-to-end learning on graphs, introducing the Graph Search Neural Network as a way of efficiently incorporating large knowledge graphs into a vision classification pipeline. We show in a number of experiments that our method outperforms standard neural network baselines for multi-label classification.