{"id":798,"date":"2020-11-04T18:04:25","date_gmt":"2020-11-05T00:04:25","guid":{"rendered":"https:\/\/sites.imsa.edu\/hadron\/?p=798"},"modified":"2020-11-04T18:21:38","modified_gmt":"2020-11-05T00:21:38","slug":"talking-to-machines","status":"publish","type":"post","link":"https:\/\/sites.imsa.edu\/hadron\/2020\/11\/04\/talking-to-machines\/","title":{"rendered":"Talking to Machines"},"content":{"rendered":"

Written by Gloria Wang<\/em><\/p>\n

In the past decade, the quest for artificial intelligence has taken off at a pace faster than anyone expected. From the deep learning breakthrough in 2012 to beating the Go world champion in 2016, artificial intelligence (AI) has improved by leaps and bounds. In recent years, it seems like this breakneck pace is only further accelerating. But accelerating in what way? AI is a broad topic, covering areas like machine learning, robotics, and even computer vision. Models are constantly being created, improved, and outdated. Work has been done in all areas, making it difficult to determine what the \u201cnext big thing\u201d will be. But with recent advances in models such as RAG, we might have an answer. As a Harvard Business Review article writes, the \u201cnext big breakthrough in AI will be around language\u201d (Wilson 2020).\u00a0<\/span><\/p>\n

Natural Language Processing (NLP) is a branch of artificial intelligence that helps computers understand, interpret, and process human language. It is the technology that enables us to use our voice to ask questions to Siri, Google Assistant, or Alexa. In order for machines to answer questions, they must have the necessary information, thus requiring the model to be pre-trained. Changing what a pre-trained model knows is difficult. It requires retraining the entire model with new information.\u00a0<\/span><\/p>\n

But now, a team at Facebook has developed a technology that can swap out given information with new ones without having to retrain the entire model. Named Retrieval-Augmented Generation (RAG), the model was open-sourced just last month as a component of the Hugging Face transformer library. In a 2020 paper published in the open-access repository arXiv, Lewis <\/span>et al.<\/span><\/i> presented their new architecture (Lewis 2020).<\/span><\/p>\n

RAG combines an Information Retrieval (IR) component with a sequence-to-sequence, abbreviated seq2seq, generator, producing a model who\u2019s internal knowledge can be easily altered in an efficient manner, but also achieve state-of-the-art results. RAG acts like a standard seq2seq model in that a sequence is inputted into the model, an encoder captures the context of the input sequence in the form of a hidden state vector and sends it to the decoder, which then produces the output sequence. The difference is that instead of passing the input directly into the generator, RAG uses the input to retrieve relevant documents from Wikipedia. Facebook explains how when asked \u201cWhen did the first mammal appear on Earth?,\u201d RAG may look for documents with keywords of \u201cMammal,\u201d \u201cHistory of Earth,\u201d and \u201cEvolution of Mammals,\u201d which are then used as context with the original input to be fed into the seq2seq model.<\/span><\/p>\n

\"RAG

An overview of RAG that shows how the Information Retriever passes related documents to the pre-trained encoder-decoder (Generator).<\/p><\/div>\n

 <\/p>\n

Due to RAG\u2019s ability to synthesize a response using separate pieces of information drawn from a variety of sources, it can generate more specific, diverse, and factual text than other state-of-the-art seq2seq models. Its true strength lies not in its superior text generation, but in its flexibility. The knowledge that RAG possesses can be easily changed by swapping out documents used for information retrieval, allowing it to quickly adapt to new information.<\/span><\/p>\n

The world is changing. Minute by minute, facts, or rather our understanding of facts, evolve. In order to keep up with the rapidly changing world of today, it is necessary for AIs to not only have access to vast quantities of information, but also updated, relevant information. This is where RAG truly shines, paving the way for newer models, newer research, and newer breakthroughs.<\/span><\/p>\n

 <\/p>\n

References<\/span><\/p>\n

Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv.\u00a0 arXiv:2005.11401. Retrieved 28 October 2020, from <\/span><\/i>https:\/\/arxiv.org\/pdf\/2005.11401.pdf<\/span><\/i><\/a>\u00a0<\/span><\/i><\/p>\n

Retrieval Augmented Generation: Streamlining the creation of intelligent natural language processing models. (2020). Facebook. Retrieved 28 October 2020, from <\/span><\/i>https:\/\/ai.facebook.com\/blog\/retrieval-augmented-generation-streamlining-the-creation-of-intelligent-natural-language-processing-models\/<\/span><\/i><\/a>\u00a0<\/span><\/p>\n

Wilson, H., Daugherty, R. (2020). The Next Big Breakthrough in AI Will Be Around Language. Harvard Business Review. Retrieved 30 October 2020, from <\/span><\/i>https:\/\/hbr.org\/2020\/09\/the-next-big-breakthrough-in-ai-will-be-around-language<\/span><\/i><\/a>\u00a0<\/span><\/i><\/p>\n","protected":false},"excerpt":{"rendered":"

Written by Gloria Wang In the past decade, the quest for artificial intelligence has taken off at a pace faster than anyone expected. From the deep learning breakthrough in 2012 to beating the Go world champion in 2016, artificial intelligence (AI) has improved by leaps<\/p>\n","protected":false},"author":588,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"ngg_post_thumbnail":0,"footnotes":""},"categories":[13],"tags":[],"class_list":["post-798","post","type-post","status-publish","format-standard","hentry","category-technology"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/posts\/798","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/users\/588"}],"replies":[{"embeddable":true,"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/comments?post=798"}],"version-history":[{"count":5,"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/posts\/798\/revisions"}],"predecessor-version":[{"id":805,"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/posts\/798\/revisions\/805"}],"wp:attachment":[{"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/media?parent=798"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/categories?post=798"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/sites.imsa.edu\/hadron\/wp-json\/wp\/v2\/tags?post=798"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}