Assessing the Performance of GPT-3.5 and GPT-4 on the 2023 Japanese Nursing Examination

Yudai Kaneda; Ryo Takahashi; Uiri Kaneda; Shiori Akashima; Haruna Okita; Sadaya Misaki; Akimi Yamashiro; Akihiko Ozaki; Tetsuya Tanimoto

doi:10.7759/cureus.42924

journal article Aug 03, 2023

Assessing the Performance of GPT-3.5 and GPT-4 on the 2023 Japanese Nursing Examination

Yudai Kaneda

Ryo Takahashi

Uiri Kaneda Shiori Akashima Haruna Okita Sadaya Misaki Akimi Yamashiro Akihiko Ozaki

Tetsuya Tanimoto

Cureus · Springer Science and Business Media LLC

View at Publisher Save 10.7759/cureus.42924

Topics

No keywords indexed for this article. Browse by subject →

References

27

[1]

Lin SY, Mahoney MR, Sinsky CA. Ten ways artificial intelligence will transform primary care. J Gen Intern Med. 2019, 34:1626-30. 10.1007/s11606-019-05035-1 10.1007/s11606-019-05035-1

[2]

Artificial intelligence in healthcare

Kun-Hsing Yu, Andrew L. Beam, Isaac S. Kohane

Nature Biomedical Engineering 2018 10.1038/s41551-018-0305-z

[3]

Akshaya AVR, VS, R Kumar C. Artificial Intelligence is changing health and eHealth care. EAI Endorsed Trans Smart Cities. 2022, 6:e3. 10.4108/eetsc.v6i3.2274 10.4108/eetsc.v6i3.2274

[4]

Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer

Kunal Nagpal, Davis Foote, Yun Liu et al.

npj Digital Medicine 2019 10.1038/s41746-019-0112-2

[5]

Liu Y, Jain A, Eng C, et al.. A deep learning system for differential diagnosis of skin diseases. Nat Med. 2020, 26:900-8. 10.1038/s41591-020-0842-3 10.1038/s41591-020-0842-3

[6]

ChatGPT. (2023). Accessed. July 13, 2023: https://openai.com/blog/chatgpt/.

[7]

Lee P, Bubeck S, Petro J. Benefits, limits, and risks of GPT-4 as an AI chatbot for medicine. N Engl J Med. 2023, 388:1233-9. 10.1056/NEJMsr2214184 10.1056/nejmsr2214184

[8]

Kung TH, Cheatham M, Medenilla A, et al.. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023, 2:e0000198. 10.1371/journal.pdig.0000198 10.1371/journal.pdig.0000198

[9]

Wang X, Gong Z, Wang G, et al.. ChatGPT performs on the Chinese National Medical Licensing Examination. Res Sq. 2023, 2023. 10.21203/rs.3.rs-2584079/v1 10.21203/rs.3.rs-2584079/v1

[10]

GPT-4 is OpenAI’s most advanced system, producing safer and more useful responses. (2023). Accessed. July 13, 2023: https://openai.com/product/gpt-4.

[11]

The 112th National Nursing Examination [in Japanese]. (2023). Accessed. July 31, 2023 : https://www.mhlw.go.jp/seisakunitsuite/bunya/kenkou_iryou/iryou/topics/tp230524-03_04_05.html.

[12]

Act on Public Health Nurses, Midwives, and Nurses [in Japanese]. (1948). Accessed. July 13, 2023: https://www.japaneselawtranslation.go.jp/ja/laws/view/3993.

[13]

National Examination Criteria for Public Health Nurses, Midwives, and Nurses [in Japanese]. (2023). Accessed. 2023 July 13: https://www.mhlw.go.jp/stf/shingi2/0000159020_00001.html.

[14]

Takagi S, Watari T, Erabi A, Sakaguchi K. Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: comparison study. JMIR Med Educ. 2023, 9:e48002. 10.2196/48002 10.2196/48002

[15]

5 ways ChatGPT will change healthcare forever, for better. (2023). Accessed. July 13, 2023: https://www.forbes.com/sites/robertpearl/2023/02/13/5-ways-chatgpt-will-change-healthcare-forever-for-better/?sh=1a1b....

[16]

Revolutionizing healthcare. the top 14 uses of ChatGPT in medicine and wellness. (2023). Accessed: July 13, 2023: https://www.forbes.com/sites/bernardmarr/2023/03/02/revolutionizing-healthcare-the-top-14-uses-of-chatgpt-in-medicine....

[17]

<p>Burnout in Healthcare Workers: Prevalence, Impact and Preventative Strategies</p>

Stefan De Hert

Local and Regional Anesthesia 2020 10.2147/lra.s240564

[18]

Anderson N, Belavy DL, Perle SM, Hendricks S, Hespanhol L, Verhagen E, Memon AR. AI did not write this manuscript, or did it? Can we trick the AI text detector into generated texts? The potential future of ChatGPT and AI in sports & exercise medicine manuscript generation. BMJ Open Sport Exerc Med. 2023, 9:e001568. 10.1136/bmjsem-2023-001568 10.1136/bmjsem-2023-001568

[19]

AI-generated answers temporarily banned on coding Q&A site Stack Overflow. (2022). Accessed. July 13, 2023: https://technopreneurph.wordpress.com/2022/12/05/ai-generated-answers-temporarily-banned-on-coding-qa-site-stack-over....

[20]

Kaneda Y, Tsubokura M, Ozaki A, Saito H, Tanimoto T. Are the issues pointed out by ChatGPT can be applied to Japan? Examining the reasons behind high COVID-19 excess deaths in Japan. New Microbes New Infect. 2023, 53:101116. 10.1016/j.nmni.2023.101116 10.1016/j.nmni.2023.101116

[21]

Perlich C. Learning curves in machine learning. Encyclopedia of Machine Learning. Sammut C, Webb GI (ed): Springer, Boston, MA; 2011. 577-80. 10.1007/978-0-387-30164-8_452 10.1007/978-0-387-30164-8_452

[22]

Kaneda Y. In the era of prominent AI, what role will physicians be expected to play?. QJM. 2023, 10.1093/qjmed/hcad099 10.1093/qjmed/hcad099

[23]

Usage statistics of content languages for websites. (2023). Accessed. July 13, 2023: https://w3techs.com/technologies/overview/content_language.

[24]

Atayoglu T, Buchholz N, Atayoglu AG, Caliskan M. Is there a place for a holistic approach in surgical training?. Arab J Urol. 2014, 12:21-4. 10.1016/j.aju.2013.06.005 10.1016/j.aju.2013.06.005

[25]

Y Zhou, AI Muresanu, Z Han, Paster K, Pitis S, Chan H, Ba J. Large language models are human-level prompt engineers. arXiv:2211.01910. 2022, 10.48550/arXiv.2211.01910 10.48550/arxiv.2211.01910

[26]

The culture map. breaking through the invisible boundaries of global business. (2014). Accessed: July 13, 2023: https://www.acc.com/sites/default/files/2021-08/Presentation%20The%20culture%20map-%20Breaking%20through%20the%20invi....

[27]

Chen L, Zaharia M, Zou J. How is ChatGPT's behavior changing over time? . arXiv:2307.09009. 2023, 10.48550/arXiv.2307.09009 10.48550/arxiv.2307.09009

Metrics

29

Citations

27

References

Details

Published: Aug 03, 2023

Authors

Cite This Article

Yudai Kaneda, Ryo Takahashi, Uiri Kaneda, et al. (2023). Assessing the Performance of GPT-3.5 and GPT-4 on the 2023 Japanese Nursing Examination. Cureus. https://doi.org/10.7759/cureus.42924

Assessing the Performance of GPT-3.5 and GPT-4 on the 2023 Japanese Nursing Examination

You May Also Like