Abstract
Case study asks Copilot users about its impact on their productivity, and seeks to find their perceptions mirrored in user data.
Topics

No keywords indexed for this article. Browse by subject →

References
29
[2]
Austin J. et al. Program synthesis with large language models. CoRR abs/2108.07732 (2021); https://arxiv.org/abs/2108.07732
[4]
Beller, M., Orgovan, V., Buja, S., and Zimmermann, T. Mind the gap: On the relationship between automatically measured and self-reported productivity. IEEE Software 38, 5 (2020), 24--31.
[5]
Chen M. et al. Evaluating large language models trained on code. CoRR abs/2107.03374 (2021); https://arxiv.org/abs/2107.03374
[8]
Hendrycks D. et al. Measuring coding challenge competence with APPS. CoRR abs/2105.09938 (2021); https://arxiv.org/abs/2105.09938
[11]
Kulal, S. et al. Spoc: Search-based pseudocode to code. In Proceedings of Advances in Neural Information Processing Systems 32, H.M. Wallach et al (eds), Dec. 2019, 11883--11894; https://bit.ly/3H7YLtF
[12]
Meyer, A.N., Barr, E.T., Bird, C., and Zimmermann, T. Today was a good day: The daily life of software developers. IEEE Transactions on Software Engineering 47, 5 (2019), 863--880.
[15]
Murphy-Hill, E. et al. What predicts software developers' productivity? IEEE Transactions on Software Engineering 47, 3 (2019), 582--594.
[16]
Peng S. Kalliamvakou E. Cihon P. and Demirer M. The impact of AI on developer productivity: Evidence from GitHub Copilot. arXiv:2302.06590 [cs.SE] (2014)
[19]
Storey, M. et al. Towards a theory of software developer job satisfaction and perceived productivity. In Proceedings of the IEEE Trans. on Software Engineering 47, 10 (2019), 2125--2142.
[22]
Vaithilingam, P., Zhang, T., and Glassman, E. Expectation vs. experience: Evaluating the usability of code generation tools powered by large language models. In Proceedings of the 2022 Conf. on Human Factors in Computing Systems.
[24]
Wagner, S. and Ruhe, M. A systematic review of productivity factors in software development. arXiv preprint arXiv:1801.06475 (2018).
[25]
Wang, D. et al. From human-human collaboration to human-AI collaboration: Designing AI systems that can work together with people. In Proceedings of the 2020 CHI Conf. on Human Factors in Computing Systems (2020), 1--6.
[27]
Winters, T., Manshreck, T., and Wright, H. Software Engineering at Google: Lessons Learned from Programming Over Time. O'Reilly Media (2020).
[28]
PLS-regression: a basic tool of chemometrics

Svante Wold, Michael Sjöström, Lennart Eriksson

Chemometrics and Intelligent Laboratory Systems 10.1016/s0169-7439(01)00155-1
[29]
Zhou W. Kim S. Murali V. and Ari Aye G. Improving code autocompletion with transfer learning. CoRR abs/2105.05991 (2021); https://arxiv.org/abs/2105.05991
Metrics
73
Citations
29
References
Details
Published
Feb 22, 2024
Vol/Issue
67(3)
Pages
54-63
License
View
Cite This Article
Albert Ziegler, Eirini Kalliamvakou, X. Alice Li, et al. (2024). Measuring GitHub Copilot's Impact on Productivity. Communications of the ACM, 67(3), 54-63. https://doi.org/10.1145/3633453
Related

You May Also Like

ImageNet classification with deep convolutional neural networks

Alex Krizhevsky, Ilya Sutskever · 2017

75,550 citations

Random sample consensus

Martin A. Fischler, Robert C. Bolles · 1981

25,028 citations

Generative adversarial networks

Ian Goodfellow, Jean Pouget-Abadie · 2020

12,710 citations

A method for obtaining digital signatures and public-key cryptosystems

R. L. Rivest, A. Shamir · 1978

11,544 citations

MapReduce

Jeffrey Dean, Sanjay Ghemawat · 2008

11,271 citations