Architectural Approaches to Overcome Challenges in The Development of Data-Intensive Systems

Abstract: All computing systems nowadays are collecting and processing data from various sources, ranges, and applications. This, in turn, leads to new kinds of problems for the software engineering community to focus on. As a result, there is a shift from software-intensive systems toward data-intensive systems. More challenges reside in size (e.g., amount of data), complexity, heterogeneity, and velocity. This requires activities that differ from solving traditional software development problems.This article will present the fundamental challenges when designing data-intensive systems and discuss some of the most popular software architectural styles together with their potential to tackle these challenges.When architecting an information system that processes substantial amounts of data, the system architects may face many problems of various kinds. The most important of them, are as follows:•Challenges in data management •Potential issues in system design or implementation•Issues in messaging service •Security and privacy issues.With concern to the development and implementation of data-intensive systems, there exist the following challenges:•Data volumes •Data dissemination •Data curation •Software reuse and use of open-source software •Search of data•Data processing and analysis •Information modelling.To manage the level of complexity in software systems and deal with various quality attributes, one should put efforts into the proper design of software systems architecture. One of the important aspects of software architecture is the definition and usage of architectural styles. They represent successful architectural configurations that recur in different software development projects. Architectural styles define types of software components, the types of connectors between them, and their bindings. Styles may also be used as a standard means to fulfil quality requirements and this way – to resolve typical software design challenges. The paper will outline some essential architectural styles, helping to overcome existing problems in achieving system architectural design. The presented styles will be further scrutinized concerning their potential to tackle the previously identified challenges. For example, particular data-intensive systems qualities may be improved by the application of specific styles:•Concurrency may be improved by Priority queue and Publish-Subscribe styles •Data search may be improved by Publish-Subscribe and Cache styles •Data curation may be solved partially or fully with a Priority queue, Publish-Subscribe and Wrapper •Information modelling is successfully tackled by Shared data and Publish-Subscribe styles and so on.On the other side, professionals in the area may also be interested if by application of a specific architectural style, it may negatively affect system qualities, so the tradeoffs required for each solution will be examined. Like, caching style may be appropriate for dealing with data volume and search but needs additional work when data curation is important.

Keywords: Data-intensive systems, software architecture, architectural styles, software system design

DOI: 10.54941/ahfe1002521

Cite this paper:

​