Step-Wise Dual Dynamic DPSGD: Enhancing Performance on Imbalanced Medical Datasets with Differential Privacy

Xiaobo Huang; Fang Xie

doi:10.3390/e28040409

journal article Open Access Apr 04, 2026

Step-Wise Dual Dynamic DPSGD: Enhancing Performance on Imbalanced Medical Datasets with Differential Privacy

Xiaobo Huang

Fang Xie

Entropy Vol. 28 No. 4 pp. 409 · MDPI AG

View at Publisher Save 10.3390/e28040409

Abstract

The application of differential privacy in deep learning often leads to significant performance degradation on class-imbalanced medical datasets. Methods such as adding noise to gradients for differential privacy are effective on large datasets, like MNIST and CIFAR-100, but perform poorly on small, imbalanced medical datasets, like HAM10000 and ISIC2019. This is because the imbalanced distribution causes the gradients from the few-shot classes to be clipped, resulting in the loss of crucial information, while the majority classes dominate the learning process. This leads the model to fall into suboptimal solutions early. To address this issue, we propose SDD-DPSGD, which uses a step-wise dynamic exponential scheduling mechanism for noise and clipping thresholds to preserve gradient information. By allocating more privacy budget and employing higher clipping thresholds during the initial training phases, the model can avoid suboptimal solutions and improve its performance. Experiments show that SDD-DPSGD outperforms comparable algorithms on the HAM10000 dataset, and the ISIC2019 dataset.

Topics

No keywords indexed for this article. Browse by subject →

References

36

[1]

Farrand, T., Mireshghallah, F., Singh, S., and Trask, A. (2020). Neither private nor fair: Impact of data imbalance on utility and fairness in differential privacy. Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice, Association for Computing Machinery. 10.1145/3411501.3419419

[2]

Reconciling privacy and accuracy in AI for medical imaging

Alexander Ziller, Tamara T. Mueller, Simon Stieger et al.

Nature Machine Intelligence 2024 10.1038/s42256-024-00858-y

[3]

The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions

Philipp Tschandl, Cliff Rosendahl, Harald Kittler

Scientific Data 2018 10.1038/sdata.2018.161

[4]

Rosenblatt, L., Lut, Y., Turok, E., Avella-Medina, M., and Cummings, R. (2024). Differential Privacy Under Class Imbalance: Methods and Empirical Insights. arXiv.

[5]

Chilukoti, S.V., Hossen, M.I., Shan, L., Tida, V.S., and Hei, X. (2023). Auto DP-SGD: Dual Improvements of Privacy and Accuracy via Automatic Clipping Threshold and Noise Multiplier Estimation. arXiv.

[6]

Combalia, M., Codella, N.C., Rotemberg, V., Helba, B., Vilaplana, V., Reiter, O., Carrera, C., Barreiro, A., Halpern, A.C., and Puig, S. (2019). Bcn20000: Dermoscopic lesions in the wild. arXiv.

[7]

Koskela, A., and Honkela, A. (2020). Learning rate adaptation for differentially private learning. Proceedings of the International Conference on Artificial Intelligence and Statistics, PMLR.

[8]

Chen, S., and Liang, W. (2024). A Study on Adaptive Gradient Clipping Algorithms for Differential Privacy: Enhancing Cyber Security and Trust. Proceedings of the 2024 IEEE Cyber Science and Technology Congress (CyberSciTech), IEEE. 10.1109/cyberscitech64112.2024.00026

[9]

Bu, Z., Wang, H., Dai, Z., and Long, Q. (2023). On the convergence and calibration of deep learning with differential privacy. Trans. Mach. Learn. Res.

[10]

Esipova, M.S., Ghomi, A.A., Luo, Y., and Cresswell, J.C. (2022). Disparate impact in differential privacy from gradient misalignment. arXiv.

[11]

Zhang, J., Yang, W., Zhang, Y., Zheng, H., and Zhang, T. (2024). DPAdaMod_AGC: Adaptive Gradient Clipping-Based Differential Privacy. Proceedings of the 2024 27th International Conference on Computer Supported Cooperative Work in Design (CSCWD), IEEE. 10.1109/cscwd61410.2024.10580740

[12]

Andrew "Differentially private learning with adaptive clipping" Adv. Neural Inf. Process. Syst. (2021)

[13]

Xia "Differentially private learning with per-sample adaptive clipping" Proceedings of the AAAI Conference on Artificial Intelligence (2023) 10.1609/aaai.v37i9.26242

[14]

Du, J., Li, S., Chen, X., Chen, S., and Hong, M. (2021). Dynamic differential-privacy preserving SGD. arXiv.

[15]

Bagdasaryan, E., Poursaeed, O., and Shmatikov, V. (2019). Differential privacy has disparate impact on model accuracy. Adv. Neural Inf. Process. Syst., 32.

[16]

Merler, M., Ratha, N., Feris, R.S., and Smith, J.R. (2019). Diversity in faces. arXiv.

[17]

Mireshghallah, F., Taram, M., Vepakomma, P., Singh, A., Raskar, R., and Esmaeilzadeh, H. (2020). Privacy in deep learning: A survey. arXiv.

[18]

Blodgett, S.L., Green, L., and O’Connor, B. (2016). Demographic dialectal variation in social media: A case study of African-American English. arXiv. 10.18653/v1/d16-1120

[19]

Van Horn, G., Mac Aodha, O., Song, Y., Cui, Y., Sun, C., Shepard, A., Adam, H., Perona, P., and Belongie, S. (2018). The inaturalist species classification and detection dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE. 10.1109/cvpr.2018.00914

[20]

Feng "Local differential privacy for unbalanced multivariate nominal attributes" Hum. Centric Comput. Inf. Sci. (2020) 10.1186/s13673-020-00233-x

[21]

Ding "Differentially private and fair classification via calibrated functional mechanism" Proceedings of the AAAI Conference on Artificial Intelligence (2020) 10.1609/aaai.v34i01.5402

[22]

Ghoukasian, H., and Asoodeh, S. (2024). Differentially private fair binary classifications. Proceedings of the 2024 IEEE International Symposium on Information Theory (ISIT), IEEE. 10.1109/isit57864.2024.10619147

[23]

Böcekçi, S.C., and Yıldız, K. (2026, February 20). Adversarial Attack Detection in Resource-Constrained Environments: A Stable and Sequential Federated Learning Architecture with TinyLlama-1.1 B. Available online: https://www.researchsquare.com/article/rs-8613070/v1. 10.21203/rs.3.rs-8613070/v1

[24]

Altinkaya "Federated learning in intrusion detection: Advancements, applications, and future directions" Clust. Comput. (2025) 10.1007/s10586-025-05325-w

[25]

Yakar "DDoS_FL: Federated learning architecture approach against DDoS attack" Pamukkale Üniversitesi Mühendislik Bilim. Derg. (2025)

[26]

Ozturk, O., Büyüktanir, B., Baydogmus, G.K., and Yildiz, K. (2025). Differential Privacy in Federated Learning: Mitigating Inference Attacks with Randomized Response. arXiv.

[27]

Dwork, C. (2008). Differential privacy: A survey of results. Proceedings of the International Conference on Theory and Applications of Models of Computation, Springer. 10.1007/978-3-540-79228-4_1

[28]

Dwork, C., McSherry, F., Nissim, K., and Smith, A. (2006). Calibrating noise to sensitivity in private data analysis. Proceedings of the Theory of Cryptography: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, 4–7 March 2006, Springer. Proceedings 3.

[29]

Harremos "Rényi divergence and Kullback–Leibler divergence" IEEE Trans. Inf. Theory (2014) 10.1109/tit.2014.2320500

[30]

Mironov, I. (2017). Rényi differential privacy. Proceedings of the 2017 IEEE 30th Computer Security Foundations Symposium (CSF), IEEE. 10.1109/csf.2017.11

[31]

Shwartz-Ziv, R., and Tishby, N. (2017). Opening the black box of deep neural networks via information. arXiv.

[32]

Wang, S., Liu, W., Wu, J., Cao, L., Meng, Q., and Kennedy, P.J. (2016). Training deep neural networks on imbalanced data sets. Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), IEEE. 10.1109/ijcnn.2016.7727770

[33]

The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation

Davide Chicco, Giuseppe Jurman

BMC Genomics 10.1186/s12864-019-6413-7

[34]

Yacouby, R., and Axman, D. (2020, January 20). Probabilistic extension of precision, recall, and f1 score for more thorough evaluation of classification models. Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, Virtual. 10.18653/v1/2020.eval4nlp-1.9

[35]

Hussain "Optimal features selection in the high dimensional data based on robust technique: Application to different health database" Heliyon (2024) 10.1016/j.heliyon.2024.e37241

[36]

Arshad "Performance of classification algorithms under class imbalance: Simulation and real-world evidence" IEEE Access (2025) 10.1109/access.2025.3620264

Metrics

0

Citations

36

References

Details

Published: Apr 04, 2026
Vol/Issue: 28(4)
Pages: 409
License: View

Authors

X

Xiaobo Huang

Guangdong Provincial Key Laboratory of IRADS, Beijing Normal-Hong Kong Baptist University, Zhuhai 519087, China

F

Fang Xie

Guangdong Provincial Key Laboratory of IRADS, Beijing Normal-Hong Kong Baptist University, Zhuhai 519087, China

Funding

Guangdong Basic and Applied Basic Research Foundation Award: 2023A1515110469

Guangdong Higher Education Upgrading Plan Award: 2025KTSCX186

Guangdong Provincial Key Laboratory IRADS Award: 2022B1212010006

Cite This Article

Xiaobo Huang, Fang Xie (2026). Step-Wise Dual Dynamic DPSGD: Enhancing Performance on Imbalanced Medical Datasets with Differential Privacy. Entropy, 28(4), 409. https://doi.org/10.3390/e28040409

Step-Wise Dual Dynamic DPSGD: Enhancing Performance on Imbalanced Medical Datasets with Differential Privacy

You May Also Like