qourata research data centre

research note

LLM Agent Planning: Does Hierarchical Decomposition Improve Web Task Success?

Does hierarchical task decomposition improve the success rate of LLM-based agents on web navigation tasks compared to flat planning?

Field
AI
Published
Reading
4 min
Evidence
14/14 claims verified · 11 sources

عربي

Direct answer

Hierarchical task decomposition generally improves the success rate of LLM-based agents on web navigation tasks compared to flat planning approaches. While specific frameworks demonstrate measurable performance gains on benchmarks like WebArena, the field currently lacks a direct head-to-head comparison between hierarchical and non-hierarchical methods across all task types. Existing evidence suggests that hierarchical structures help manage long-horizon tasks, though low-level execution remains a primary bottleneck.

The Hierarchical Planning Perspective

Current research into LLM-based web agents often utilizes a hierarchical planning framework that categorizes agent operations into three distinct layers: high-level planning, low-level execution, and replanning [1]. This structure is designed to address the complexities of web navigation, where existing evaluations have historically focused on end-to-end success metrics, providing limited insight into the specific stages where failures occur [1]. Despite the adoption of these hierarchies, low-level execution remains the dominant bottleneck for agents operating in web environments [1].

Performance Gains in Hierarchical Frameworks

Several hierarchical frameworks have demonstrated improved success rates compared to baseline methods. For instance, the CoAct framework employs a global planning agent alongside a local execution agent, achieving superior performance on long-horizon web tasks within the WebArena benchmark [2]. Similarly, WebAgent improves success rates on real websites by over 50% by decomposing instructions into canonical sub-instructions, summarizing HTML documents into task-relevant snippets, and executing actions via Python programs [4]. Other approaches, such as SkillWeaver, have reported relative success rate improvements of 31.8% on WebArena and 39.8% on real-world websites [5]. Furthermore, AdaPlanner has been shown to outperform state-of-the-art baselines by 3.73% on ALFWorld and 4.11% on MiniWoB++ while requiring significantly fewer samples [6].

Optimization and Evaluation Challenges

Optimizing the observation space and planning strategy is critical for agent efficacy. PageDigest has been shown to reduce observation length while simultaneously improving overall task success rates across diverse backbone LLMs and agent methods [3]. Regarding planning strategy, PDDL plans have been found to produce more concise and goal-directed strategies than natural language plans [1]. However, practical implementation faces latency constraints, as generating a plan with an LLM can take tens of seconds, which significantly degrades the user experience [9]. Evaluation remains a rigorous process, as seen in the TREK framework, which utilizes 800 multi-constraint tasks scored by a fully deterministic, rule-based evaluator rather than an LLM judge [7].

Limits and open questions

While hierarchical decomposition is associated with higher success rates in the cited literature, there is no direct head-to-head study comparing hierarchical versus flat planning architectures in isolation. The performance improvements reported are often bundled with other features, such as improved observation summarization or skill discovery, making it difficult to attribute success solely to the hierarchical structure. Additionally, the impact of latency on real-time agent performance remains a significant open challenge for practical deployment.

Practical

How to build it, or how to use it

  1. Define a high-level planning module that decomposes complex user instructions into a sequence of canonical sub-instructions.
  2. Implement a local execution agent that receives these sub-instructions and translates them into executable actions, such as Python scripts or browser commands.
  3. Integrate an observation summarization layer, such as a snippet generator, to reduce the length of HTML documents before they are processed by the LLM.
  4. Establish a replanning loop that monitors the state of the web environment and triggers a new plan if the current execution fails to meet the sub-instruction goal.
  5. Measure the agent's performance against a baseline flat-planning agent using the WebArena benchmark.
  6. Track the success rate and the latency per step to ensure the agent remains responsive.
  7. Compare the number of samples required to reach a target success rate to determine the efficiency of the hierarchical approach versus the baseline.

Our take

What we would build

We would build a "Hierarchical Web Navigator" designed to solve multi-step shopping tasks on a sandbox e-commerce site. This project would demonstrate the efficacy of separating high-level goal decomposition from low-level DOM interaction. The agent would use a global planner to break down a "find and purchase" request into discrete steps, while a local executor handles the specific button clicks and form entries. We would judge the success of this project by comparing its completion rate and average time-to-task-completion against a baseline agent that attempts the same tasks using a flat, end-to-end prompt. Success would be defined by a statistically significant increase in task completion rate and a reduction in the number of failed navigation steps compared to the flat baseline.

Claim record

What this note is based on

  1. resultsupported

    Structured Planning Domain Definition Language (PDDL) plans produce more concise and goal-directed strategies than natural language (NL) plans.

    [1] Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective — abstract arXiv:2603.14248v2
    “Large language model (LLM) web agents are increasingly used for web navigation but remain far from human reliability on realistic, long-horizon tasks. Existing evaluations focus primarily on end-to-end success, offering limited insight into where failures arise. We propose a hier…”
  2. resultsupported

    Low-level execution remains the dominant bottleneck in LLM-based web agents.

    [1] Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective — abstract arXiv:2603.14248v2
    “Large language model (LLM) web agents are increasingly used for web navigation but remain far from human reliability on realistic, long-horizon tasks. Existing evaluations focus primarily on end-to-end success, offering limited insight into where failures arise. We propose a hier…”
  3. methodsupported

    The hierarchical planning framework analyzes web agents across three layers: high-level planning, low-level execution, and replanning.

    [1] Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective — abstract arXiv:2603.14248v2
    “Large language model (LLM) web agents are increasingly used for web navigation but remain far from human reliability on realistic, long-horizon tasks. Existing evaluations focus primarily on end-to-end success, offering limited insight into where failures arise. We propose a hier…”
  4. limitationsupported

    Existing evaluations focus primarily on end-to-end success, offering limited insight into where failures arise.

    [1] Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective — abstract arXiv:2603.14248v2
    “Large language model (LLM) web agents are increasingly used for web navigation but remain far from human reliability on realistic, long-horizon tasks. Existing evaluations focus primarily on end-to-end success, offering limited insight into where failures arise. We propose a hier…”
  5. methodsupported

    The CoAct framework involves a global planning agent and a local execution agent.

    [2] CoAct: A Global-Local Hierarchy for Autonomous Agent Collaboration — abstract S2 e93f1fdaecd6
    “Existing LLMs exhibit remarkable performance on various NLP tasks, but still struggle with complex real-world tasks, even equipped with advanced strategies like CoT and ReAct. In this work, we propose the CoAct framework, which transfers the hierarchical planning and collaboratio…”
  6. resultsupported

    CoAct achieves superior performance over baseline methods on long-horizon web tasks on the WebArena benchmark.

    [2] CoAct: A Global-Local Hierarchy for Autonomous Agent Collaboration — abstract S2 e93f1fdaecd6
    “Existing LLMs exhibit remarkable performance on various NLP tasks, but still struggle with complex real-world tasks, even equipped with advanced strategies like CoT and ReAct. In this work, we propose the CoAct framework, which transfers the hierarchical planning and collaboratio…”
  7. resultsupported

    PageDigest substantially reduces observation length while improving overall task success rate across diverse backbone LLMs and established agent methods on the WebArena benchmark.

    [3] Region4Web: Rethinking Observation Space Granularity for Web Agents — abstract S2 520977177230
    “Web agents perceive web pages through an observation space, yet its granularity has remained an underexamined design choice. Existing work treats observation at the same element-level granularity as the action space, leaving the page's functional organization implicit and forcing…”
  8. methodsupported

    WebAgent plans ahead by decomposing instructions into canonical sub-instructions, summarizes long HTML documents into task-relevant snippets, and acts on websites via Python programs.

    [4] A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis — abstract DOI 10.48550/arxiv.2307.12856
    “Pre-trained large language models (LLMs) have recently achieved better generalization and sample efficiency in autonomous web automation. However, the performance on real-world websites has still suffered from (1) open domainness, (2) limited context length, and (3) lack of induc…”
  9. resultsupported

    WebAgent improves the success on real websites by over 50%.

    [4] A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis — abstract DOI 10.48550/arxiv.2307.12856
    “Pre-trained large language models (LLMs) have recently achieved better generalization and sample efficiency in autonomous web automation. However, the performance on real-world websites has still suffered from (1) open domainness, (2) limited context length, and (3) lack of induc…”
  10. resultsupported

    SkillWeaver achieves relative success rate improvements of 31.8% on WebArena and 39.8% on real-world websites.

    [5] SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills — abstract S2 69768fdcb8c2
    “To survive and thrive in complex environments, humans have evolved sophisticated self-improvement mechanisms through environment exploration, hierarchical abstraction of experiences into reuseable skills, and collaborative construction of an ever-growing skill repertoire. Despite…”
  11. resultsupported

    AdaPlanner outperforms state-of-the-art baselines by 3.73% on ALFWorld and 4.11% on MiniWoB++ while utilizing 2x and 600x fewer samples respectively.

    [6] AdaPlanner: Adaptive Planning from Feedback with Language Models — abstract DOI 10.48550/arxiv.2305.16653
    “Large language models (LLMs) have recently demonstrated the potential in acting as autonomous agents for sequential decision-making tasks. However, most existing methods either take actions greedily without planning or rely on static plans that are not adaptable to environmental …”
  12. factsupported

    TREK comprises 800 multi-constraint tasks over a synthetic knowledge base of 212,530 records across 375 cities and 13 personas.

    [7] TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning — abstract arXiv:2607.26977v2
    “Travel planning is a demanding stress test for tool-using LLM agents: a usable itinerary is a single artifact that must be right along many axes at once - every flight, hotel, and attraction must exist and be bookable, the days must be physically traversable, the total must clear…”
  13. methodsupported

    Every task in TREK is scored by a fully deterministic, rule-based evaluator with no LLM judge.

    [7] TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning — abstract arXiv:2607.26977v2
    “Travel planning is a demanding stress test for tool-using LLM agents: a usable itinerary is a single artifact that must be right along many axes at once - every flight, hotel, and attraction must exist and be bookable, the days must be physically traversable, the total must clear…”
  14. limitationsupported

    The latency for generating a plan with an LLM can reach tens of seconds, significantly degrading user experience.

    [9] A Plan Reuse Mechanism for LLM-Driven Agent — abstract arXiv:2512.21309v2
    “Integrating large language models (LLMs) into personal assistants, like Xiao Ai and Blue Heart V, effectively enhances their ability to interact with humans, solve complex tasks, and manage IoT devices. Such assistants are also termed LLM-driven agents. Upon receiving user reques…”

References

Sources

  1. [1]
    Mohamed Aghzal, Gregory J. Stein, Ziyu Yao. Why Do LLM-based Web Agents Fail? A Hierarchical Planning Perspective. arXiv, 2026.arxiv · primary · https://arxiv.org/abs/2603.14248v2
  2. [2]
    Xinming Hou, Mingming Yang, Wenxiang Jiao, Xing Wang, Zhaopeng Tu, W. Zhao. CoAct: A Global-Local Hierarchy for Autonomous Agent Collaboration. arXiv.org, 2024.semanticscholar · primary · DOI 10.48550/arXiv.2406.13381 · https://doi.org/10.48550/arXiv.2406.13381
  3. [3]
    Donguk Kwon, Dongha Lee. Region4Web: Rethinking Observation Space Granularity for Web Agents. arXiv.org, 2026.semanticscholar · primary · DOI 10.48550/arXiv.2605.07134 · https://doi.org/10.48550/arXiv.2605.07134
  4. [4]
    İzzeddin Gür, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck. A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis. arXiv (Cornell University), 2023.openalex · primary · DOI 10.48550/arxiv.2307.12856 · https://doi.org/10.48550/arxiv.2307.12856
  5. [5]
    Boyuan Zheng, Michael Y. Fatemi, Xiaolong Jin, Z. Wang, Apurva Gandhi, Yueqi Song. SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills. arXiv.org, 2025.semanticscholar · primary · DOI 10.48550/arXiv.2504.07079 · https://doi.org/10.48550/arXiv.2504.07079
  6. [6]
    Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai, Chao Zhang. AdaPlanner: Adaptive Planning from Feedback with Language Models. arXiv (Cornell University), 2023.openalex · primary · DOI 10.48550/arxiv.2305.16653 · https://doi.org/10.48550/arxiv.2305.16653
  7. [7]
    Jinhu Qi, Wentao Zhang, Siu Man Ng, Feiyang Xu, Yanyu Chen, Yaoman Li, Irwin King. TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning. arXiv, 2026.arxiv · primary · https://arxiv.org/abs/2607.26977v2
  8. [8]
    Damien Pellier, Alexandre Albore, Humbert Fiorino, Rafael Bailon-Ruiz. HDDL 2.1: Towards Defining a Formalism and a Semantics for Temporal HTN Planning. arXiv, 2023.arxiv · primary · https://arxiv.org/abs/2306.07353v1
  9. [9]
    Guopeng Li, Ruiqi Wu, Haisheng Tan. A Plan Reuse Mechanism for LLM-Driven Agent. arXiv, 2025.arxiv · primary · https://arxiv.org/abs/2512.21309v2
  10. [10]
    Anurag Ajay, Seungwook Han, Yilun Du, Shuang Li, Abhi Gupta, Tommi Jaakkola, Josh Tenenbaum, Leslie Kaelbling, Akash Srivastava, Pulkit Agrawal. Compositional Foundation Models for Hierarchical Planning. arXiv, 2023.arxiv · primary · https://arxiv.org/abs/2309.08587v2
  11. [11]
    Saurabh Kumar, Pararth Shah, Dilek Hakkani-Tur, Larry Heck. Federated Control with Hierarchical Multi-Agent Deep Reinforcement Learning. arXiv, 2017.arxiv · primary · https://arxiv.org/abs/1712.08266v1