An Overview on the Performance of Reasoning Agents in Large Language Models
HTML-код
- Опубликовано: 8 июл 2024
- May 14, 2024 Colloquia
Department of Computer Science & Engineering
An Overview on the Performance of Reasoning Agents in Large Language Models
The recent rise of Large Language Models (LLMs), which are able to generate human-like text, has put a large amount of attention onto AI and its potential uses. However, most LLMs are limited to a one-dimensional/left-to-right method of decision-making that can impede their performance in tasks that require accurate foresight and reference to previous decisions to execute. We hypothesize that various types of LLM reasoning agents have different strengths and weaknesses that allow for applications for different strategic use cases. In our research, we hope to determine the specific use cases and strengths of various reasoning agents, which will allow for the creation of LLMs tailored towards certain tasks with the use of such agents. With the help of reasoning agents, such as symbolic, arithmetic, and chain-of-thought reasoning, LLMs adopt a greater understanding of the context given to them and use a multi-step approach to adequately solve problems. Existing challenges in evaluating reasoning agents within LLMs include issues such as dataset biases and the potential brittleness of the model. These challenges, combined with the ethical concerns surrounding the reasoning agents such as their susceptibility to amplifying biases within a response, offer a rich research area. Using a quantitative analysis of several reasoning agents within a controlled environment, we apply diverse multi-modal and iterative reasoning techniques. Through this analysis, we explore the strengths and weaknesses of these reasoning techniques, resulting in a better understanding of the reasoning capabilities to be applied to real-world scenarios and products.
RESEARCHERS: Nathan M., Valley Christian High School '25
ADVISOR: Subramaniam, Data Science
KEYWORDS: Artificial Intelligence | Large Language Models | Natural Language Processing | Reasoning Agents 
Наука
