推进大型语言模型中的推理：有前景的方法和途径

Avinash Patilavinashpatil@ieee.orgORCID: 0009-0002-6004-370X Reasoning in AI broadly encompasses multiple cognitiveprocesses, including deductive, inductive, abductive, and com-monsense reasoning [5]–[9]. Unlike retrieval-based knowl-edge synthesis, reasoning requires multi-step logical transfor-mations, contextual generalization, and structured problem-solving. Classical AI approaches have addressed reasoning Abstract—Large Language Models (LLMs) have succeededremarkably in various natural language processing (NLP) tasks,yet their reasoning capabilities remain a fundamental challenge.While LLMs exhibit impressive fluency and factual recall, theirability to perform complex reasoning—spanning logical deduc-tion, mathematical problem-solving, commonsense inference, andmulti-step reasoning—often falls short of human expectations.This survey provides a comprehensive review of emerging tech-niques enhancing reasoning in LLMs. We categorize existingmethods into key approaches, including prompting strategies(e.g., Chain-of-Thought reasoning, Self-Consistency, and Tree-of-Thought reasoning), architectural innovations (e.g., retrieval-augmentedmodels,modular reasoning networks,and neuro- Recent research has explored diverse methodologies to en-hance the reasoning abilities of LLMs. These approaches cancategorized into three domains: (1) Prompting Strategies, suchas Chain-of-Thought (CoT) reasoning [12], Self-Consistency[13], and Tree-of-Thought [14] methods, which leverage struc-tured prompts to guide step-by-step reasoning; (2) Architec-tural Innovations, including retrieval-augmented models [15],neuro-symbolic hybrid frameworks [16], and modular reason- Index Terms—Large Language Models (LLMs), Reasoning,LogicalDeduction,Mathematical Problem-Solving,Common-senseInference,Multi-Step Reasoning,Prompting Strategies,Chain-of-Thought Reasoning, Self-Consistency, Tree-of-ThoughtReasoning,Retrieval-Augmented Models,Modular Reasoning Amongrecent advancements,the newly released LLMDeepSeek-R1 [1] has demonstrated superior reasoning per-formance, particularly in complex domains such as math-ematics and coding. By effectively simulating human-likeanalytical thinking, DeepSeek-R1 enhances multi-step rea-soning in mathematical problem-solving, logical inference,and programming tasks, showcasing the potential of fine-tuned architectures and novel training paradigms to improvestructuredreasoning in LLMs.This survey systematicallyarXiv:2502.03671v1 [cs.CL] 5 Feb 2025 The recently released LLM, DeepSeek-R1 [1], excels incomplex tasks such as mathematics and coding, showcas-ing advanced reasoning capabilities. It effectively simulates I. INTRODUCTION Large Language Models (LLMs) have revolutionized thefield of Natural Language Processing (NLP), enabling break-throughs in machine translation, text generation, question-answering, and other complex linguistic tasks. Despite theirremarkable fluency and knowledge retention, these modelsoften struggle with systematic reasoning—an essential capa-bility for tasks requiring logical inference, problem-solving,and decision-making [2]. While LLMs can generate plausible- The paper is structured as follows: Section 2 covers thefoundations of reasoning, while Section 3 explores prompt-based reasoning enhancements. Section 4 discusses architec-tural innovations, and Section 5 examines learning-based ap- their reasoning capabilities differ significantly from traditional II. FOUNDATIONS OFREASONING INAIANDLLM A. Definitions and Types of Reasoning •Statistical Learning vs. Symbolic Logic: Unlike sym-bolic AI, which follows explicit logical rules, LLMslearn probabilistic patterns in language data, making theirreasoning implicit and non-deterministic.•EmergentReasoning Abilities:Studies suggest thatscaling LLMs improves their ability to perform multi-step reasoning tasks despite the lack of explicit logicalconstraints.•Contextual and Prompt-Driven Reasoning: LLMs relyheavily on context windows and external prompt engi- Reasoning is the cognitive process of deriving conclusionsfrom premises or evidence. It can classified into the following •DeductiveReasoning:Drawing specific conclusionsfromgeneral premises.If the premises are true,theconclusion must be true. This method is fundamental informal logic and automated theorem proving.•Inductive Reasoning: Deriving general principles fromspecific examples or observations. This approach is com-mon in machine learning for pattern recognition andforecasting.•Abductive Reasoning: Inferring the most likely expla-nation for a given set of observations, frequently used indiagnostics and hypothesis formation. D. Challenges of Reasoning in LLMs Despite their progress, LLMs face several challenges whenit comes to robust and reliable reasoning [20]–[22]: •Hallucinations: LLMs sometimes generate plausible butincorrect information, leading to unreliable reasoning.•Lack of Explicit Memory: Unlike knowledge graphsor rule-based systems, LLMs lack structure