Examining the Role Memory Plays in Cyber Defence Evaluation: Risk and Uncertainty Demystified

Abstract: Attackers’ decision-making processes can be influenced by their past hacking experiences, their knowledge of a target system, which is determined by the method/tool used to explore the system, and how important it is for them to remain undetected, among other factors. In this work, we utilized reinforcement learning (RL) agents equipped with/lack specific human-like capabilities to analyze how they interact with an environment characterized by (un)certain observation spaces. In particular, we investigated how agents with different memory systems and settings interact with reduced, masked, noisy, and baseline observations. First, to investigate whether the models are able to predict human-like tendencies, we analyzed the ability of non-linear models, calibrated to model different memory systems, and a linear model to predict actions taken by healthy humans and pathological gamblers (addicts) performing gambling tasks under incentive-compatible implicit and explicit learning schemes. Our findings show that a model’s ability to predict an individual’s tendency towards advantageous or disadvantageous actions is a function of the model’s characteristics, including the memory system(s) utilized in decision-making and its related attributes (e.g., impairments) and the condition(s) under which a decision is made (i.e., whether the decision is made under uncertainty or risk). Having examined the role memory systems and settings play in predicting real human behaviors using psychological datasets, we augmented RL agents with similar memory systems and settings to examine their influence on RL agents’ original behavior, that is of maximizing a net reward or satisficing a predefined condition. The analysis shows that equipping agents with different memory systems significantly and uniquely influences their approach to (un)certain observation spaces.Main findings:- In a simulated environment, knowledge of deception (or lack thereof) can be modeled in terms of the conditions (decision-under-uncertainty vs. decision-under-risk) under which the decision is made. We demonstrated that both conditions require further consideration when integrating cyber deception techniques into the environment and when evaluating agents using unique neural network architectures against defensive deception techniques. - Populations with distinct learning strategies require unique placement strategies since the thresholds of success may differ.- Strategies used to integrate deception into a network environment, which includes masking or reducing the observation space or adding noise to the observation space, vary in their effectiveness against the different types of agents.

Keywords: cybersecurity, memory systems, decision-making

DOI: 10.54941/ahfe1005595

Cite this paper:

​