How to Design an Operation-Specific LLM-Based Information System

Abstract

The performance of large language models (LLMs) has improved significantly in recent years, with the result that they are now used in many companies in various industries. However, the design of a company-specific information system involving an LLM is associated with a large number of decisions. This leads to a high level of complexity in the design task. Against this background, companies need a structured approach that methodically supports the planning, development, implementation and long-term maintenance of LLM-based information systems so that domain- and company-specific requirements are taken into account as a result. This article therefore describes a method that supports the design, introduction and maintenance process of an LLM-based information system. The method consists of a process model and a list of design principles, which are also referred to as success factors. The process model developed is based on the proven six-stage REFA planning system. To identify and describe success factors, a systematic literature search was carried out. Based on an analysis of the contents of individual literature sources, success factors for the design of LLM-based information systems were identified. These success factors relate, for example, to the quality of the data provided, data security, user-centered system design and feedback mechanisms for improving information output.

Keywords: Large Language Model, Information System, Retrieval Augmented Generation

DOI: 10.54941/ahfe1006709