Exploring Metamorphic Testing for Self-learning Functions with User Interactions

Abstract: Self-learning functions, an evolving field in modern technology, are increasingly being integrated into a multitude of applications. They primarily rely on data-driven learning techniques, such as supervised, unsupervised machine learning and reinforcement learning. In the field of autonomous vehicles, self-learning functions are important for real-time decision-making, as they adapt to dynamic scenarios by collecting extensive data from sensors. Likewise, self-learning functions with user-interaction, a subset of self-learning functions, are asserting their influence in the automotive industry, as they observe driver behavior and recognize user-specific interactions with the system. In addition to data-driven learning, these functions incorporate real-time user interactions, such as activating seat heating or ventilation, and autonomously execute these interactions, enhancing overall comfort of the driver. The growing integration and interaction of self-learning functions underscore the importance of conducting research and refining testing methodologies to ensure their reliability and effectiveness. To meet the growing need for trusted and reliable self-learning features, effective testing methods are essential for validating the accuracy and robustness of self-learning functions with user interaction. In contrast to traditional software, self-learning functions change and adjust themselves based on data and interactions. This causes challenges to predict and verify their intended behavior. Furthermore, each potential user exhibits distinctive individual behavior that differentiates them from other users. As a consequence, attempting to address every potential user interaction with traditional testing methods and predefined test case specifications becomes impractical. Moreover, the behavior of a self-learning function adapts over time to that of the respective user. As a result, the user behavior to be tested evolves, rendering traditional testing through predefined test cases unfeasible. Consequently, adapted testing methods are indispensable to effectively address the test-oracle problem. This paper presents a solution to address the test-oracle problem by leveraging metamorphic testing as a method for validating self-learning functions with user interaction. Metamorphic testing approaches the problem of the test-oracle from a perspective not typically employed by other testing strategies: instead of focusing on individual test cases, metamorphic tests examine the outcomes of multiple test cases within a testing system and their relationships with each other. Metamorphic testing assesses whether the test inputs and outputs fulfill specific metamorphic relationships upon multiple test executions. These metamorphic relationships describe the essential properties of the intended functionality. They transform existing input-output test cases into new follow-up test cases. If the behavior of the self-learning functions deviates from the metamorphic relationship in these original and follow-up test cases, the testing system is considered faulted. The effectiveness of fault detection significantly relies on metamorphic relations. Thus, the analysis of metamorphic relations becomes an essential task and a creative endeavor for the tester. Furthermore, it will be a significant contribution of this publication.The proposed paper offers an analysis of the insights gained from the application of metamorphic testing to a self-learning comfort function. This underscores how effective the testing methodology is in identifying inaccuracies in the self-learning function's interpretation of user behaviors, thus contributing to our understanding of their reliability and adaptability in simulated scenarios. In conclusion, the utilization of metamorphic testing in the context of self-learning functions, with a specific emphasis on user interactions, emerges as a promising and efficient strategy.

Keywords: Self-learning functions, User-interaction, Testing methodologies, Metamorphic testing

DOI: 10.54941/ahfe1004569

Cite this paper:

​