CORTEXA
← Browse
arxivcs.AI2026-06-27

Characterizing Large Language Model Agentic Workflows: A Study on N8n Ecosystem

Yutian Tang, Yuming Zhou, Huaming Chen

Large Language Models (LLMs) are rapidly being adopted in low-code and no-code automation platforms, where non-expert users design workflows that combine natural language understanding with external services and APIs. LLM agents are LLM systems that use LLMs as a core "brain" to reason, plan, and autonomously execute complex, multi-step tasks. In this paper, we present the first large-scale empirical study of LLM agentic workflows in low-code automation platforms. We analyze more than 6,000 publicly available n8n workflows and examine four aspects of their design: task distribution, structural and tool use patterns, reliability mechanisms, and autonomy levels. Our analysis shows that LLM workflows are not merely prompt response pipelines. Instead, LLMs are commonly embedded within broader automation structures involving control logic, external tools, communication services, storage systems, and human review points. We further find that while many workflows include lightweight post-processing or routing logic after LLM execution, explicit reliability mechanisms such as structured fallback paths, repair loops, failure-specific alerts, and human approval gates remain relatively uncommon. These results reveal a gap between the increasing deployment of LLM agents in practical automation ecosystems and the limited engineering support for reliability, safety, and governance. Overall, our study provides ten empirical findings and five research takeaways for researchers, platform developers, and practitioners seeking to understand and improve real-world LLM agentic workflows.

View free PDFSource page

Related papers

arxivcs.NIcs.AI2026-07-20

Human Grounded Evaluation of Large Language Models for Optical Network Automation

Kiarash Rezaei, Omran Ayoub, Paolo Monti, Carlos Natalino

Large language models (LLMs) are increasingly adopted for network automation, yet their output quality and inference cost can vary substantially across LLM families. We present HuGLEN, a stepwise evaluation pipeline that uses an LLM-as-a-judge together with a small set of expert…

View free PDFSource page
arxivcs.AI2026-07-20

Stress Testing Concept Erasure with Large Language Model Agents

Yuyang Xue, Feng Chen, Zhihua Liu, Edward Moroshko, Jingyu Sun, Steven McDonagh, et al.

Concept erasure aims to remove semantic concepts from a trained generative model and is increasingly important for responsible AI deployment. However, verifying whether a model has robustly removed targeted concepts remains a critical challenge. Existing evaluation methods are ty…

View free PDFSource page
arxivcs.AIcs.CLcs.HC2026-07-16

Benchmarking Multimodal Large Language Models for Scientific Visualization Literacy

Patrick Phuoc Do, Chau M. Ta, Chaoli Wang

Multimodal large language models (MLLMs) are increasingly used to interpret visualizations, yet current evaluations remain largely chart-centric and provide limited evidence of understanding of scientific visualization (SciVis). We benchmark six MLLMs on the scientific visualizat…

View free PDFSource page
arxivcs.AI2026-07-17

Behavioral Controllability of Agentic Models for Information Extraction: From Fixed Workflows to Reflective Agents

Lujia Zhang, Xingzhou Chen, Hongwei Feng

Large language model (LLM) agents are increasingly used for complex information-extraction tasks, yet it remains unclear whether agentic components such as reflection and memory lead to observable and controllable improvements over fixed LLM workflows. We study this question thro…

View free PDFSource page
arxivcs.AI2026-07-20

WuYu-EnvLE-Bench: A Benchmark for Evaluating Large Language Models in Environmental Law Enforcement

Ziliang Yang, Yi Zhang, Kaijun Lin, Jiachao Ke, Haihong Xu, Zongguo Wen

Large language models (LLMs) are increasingly considered for environmental enforcement, but their ability to produce traceable enforcement decisions remains unclear. We introduce WuYu-EnvLE-Bench, a benchmark built from real enforcement cases, regulatory standards, and expert rev…

View free PDFSource page