Playing Repeated Stochastic Security Games Against Non-Stationary Attackers

Ling Chen; Runfa Zhang

doi:10.3390/math13172697

journal article Open Access Aug 22, 2025

Playing Repeated Stochastic Security Games Against Non-Stationary Attackers

Ling Chen

Runfa Zhang

Mathematics Vol. 13 No. 17 pp. 2697 · MDPI AG

View at Publisher Save 10.3390/math13172697

Abstract

This paper investigates a repeated stochastic security game against a non-stationary attacker. Most of the work to date assumes that the defender has a repeated interaction with a fixed type of attacker. In fact, the defender is more likely to encounter changing attackers in multi-round games. A defender faces an attacker whose identity is unknown. The attacker type changes stochastically over time and the defender cannot detect when these changes occur. We adopt the BPR (Bayesian Policy Reuse) algorithm to detect the switches of the attacker, and the defender could play the accurate policy correspondingly. The experiment results show that BPR algorithm could accurately detect switches and help the defender gain more utilities than the EXP3-S algorithm.

Topics

No keywords indexed for this article. Browse by subject →

References

21

[1]

Kar, D., Fang, F., Delle Fave, F., Sintov, N., and Tambe, M. (2015, January 4–8). “A Game of Thrones” When Human Behavior Models Compete in Repeated Stackelberg Security Games. Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, Istanbul, Turkey.

[2]

Wang "On repeated stackelberg security game with the cooperative human behavior model for wildlife protection" Appl. Intell. (2019) 10.1007/s10489-018-1307-y

[3]

Clempner "Repeated Stackelberg security games: Learning with incomplete state information" Reliab. Eng. Syst. Saf. (2020) 10.1016/j.ress.2019.106695

[4]

Cheng, Z., Chen, G., and Hong, Y. (2023). Zero-determinant strategy in stochastic Stackelberg asymmetric security game. Sci. Rep., 13. 10.1038/s41598-023-38460-8

[5]

Xu, H., Tran-Thanh, L., and Jennings, N.R. (2016, January 9–13). Playing repeated security games with no prior knowledge. Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, Singapore.

[6]

Garnaev "Security games with unknown adversarial strategies" IEEE Trans. Cybern. (2015) 10.1109/tcyb.2015.2475243

[7]

Balcan, M.F., Blum, A., Haghtalab, N., and Procaccia, A.D. (2015, January 15–19). Commitment without regrets: Online learning in stackelberg security games. Proceedings of the Sixteenth ACM Conference on Economics and Computation, Portland, OR, USA. 10.1145/2764468.2764478

[8]

Hammar "Learning Near-Optimal Intrusion Responses Against Dynamic Attackers" IEEE Trans. Netw. Serv. Manag. (2024) 10.1109/tnsm.2023.3293413

[9]

Vorobeychik, Y., and Singh, S. (2012, January 22–26). Computing stackelberg equilibria in discounted stochastic games. Proceedings of the 26th AAAI Conference on Artificial Intelligence, Toronto, ON, Canada.

[10]

Nguyen, T.H., Wang, Y., Sinha, A., and Wellman, M.P. (2019, January 27). Deception in finitely repeated security games. Proceedings of the 33th AAAI Conference on Artificial Intelligence, Honolulu, HI, USA. 10.1609/aaai.v33i01.33012133

[11]

Mckelvey "Quantal Response Equilibria for Normal Form Games" Games Econ. Behav. (1995) 10.1006/game.1995.1023

[12]

Yang, R., Ordonez, F., and Tambe, M. (2012, January 4–8). Computing optimal strategy against quantal response in security games. Proceedings of the 2012 International Conference on Autonomous Agents and Multiagent Systems, Valencia, Spain.

[13]

Hernandez-Leal, P., and Kaisers, M. (2017, January 11–14). Learning against sequential opponents in repeated stochastic games. Proceedings of the 3rd Multi-Disciplinary Conference on Reinforcement Learning and Decision Making, Ann Arbor, MI, USA.

[14]

Chen "Accurate policy detection and efficient knowledge reuse against multi-strategic opponents" Knowl.-Based Syst. (2022) 10.1016/j.knosys.2022.108404

[15]

Zhan "Efficiently detecting switches against non-stationary opponents" Auton. Agents Multi-Agent Syst. (2017) 10.1007/s10458-016-9352-6

[16]

Shen "Comparative DQN-Improved Algorithms for Stochastic Games-Based Automated Edge Intelligence-Enabled IoT Malware Spread-Suppression Strategies" IEEE Internet Things J. (2024) 10.1109/jiot.2024.3381281

[17]

Bozkurt "Learning optimal strategies for temporal tasks in stochastic games" IEEE Trans. Autom. Control (2024) 10.1109/tac.2024.3390848

[18]

Hammar "Adaptive security response strategies through conjectural online learning" IEEE Trans. Inf. Forensics Secur. (2025) 10.1109/tifs.2025.3558600

[19]

Rosman "Bayesian policy reuse" Mach. Learn. (2016) 10.1007/s10994-016-5547-y

[20]

Hernandezleal, P., and Kaisers, M. (2017). Towards a Fast Detection of Opponents in Repeated Stochastic Games. Autonomous Agents and Multiagent Systems, Springer. 10.1007/978-3-319-71682-4_15

[21]

Allesiardo, R., and Féraud, R. (2015, January 19–21). Exp3 with drift detection for the switching bandit problem. Proceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA), Paris, France. 10.1109/dsaa.2015.7344834

Metrics

1

Citations

21

References

Details

Published: Aug 22, 2025
Vol/Issue: 13(17)
Pages: 2697
License: View

Authors

L

Ling Chen

School of Mathematics and Statistics, Taiyuan Normal University, Jinzhong 030619, China

R

Runfa Zhang

School of Automation and Software Engineering, Shanxi University, Taiyuan 030013, China; Hubei Key Laboratory of Applied Mathematics, Hubei University, Wuhan 430062, China

Funding

Fundamental Research Program of Shanxi Province Award: 12426105

Scientific and Technological Innovation Programs (STIP) of Higher Education Institutions in Shanxi Award: 12426105

Open Foundation of Hubei Key Laboratory of Applied Mathematics (Hubei University) Award: 12426105

“Wen Ying Young Scholars” Talent Project of Shanxi University Award: 12426105

Cite This Article

Ling Chen, Runfa Zhang (2025). Playing Repeated Stochastic Security Games Against Non-Stationary Attackers. Mathematics, 13(17), 2697. https://doi.org/10.3390/math13172697

Playing Repeated Stochastic Security Games Against Non-Stationary Attackers

You May Also Like