Early evidence on the impact of Generative AI on Software Engineer’s Employment Outcomes

In the past 24 months, Generative Artificial Intelligence (GAI) tools, such as ChatGPT and Copilot, have seen rapid technological advancement and adoption. While GAI offers exciting prospects to increase worker productivity, fears remain about workers being displaced by these new tools. Early research has shown promising signs to how GAI may enhance productivity, but when it comes to how the technology impacts workers employment, little evidence exists. In a new working paper I co-authored with my LinkedIn colleagues Mar Carpanelli and Brian Xu, and GitHub’s Kevin Xu, we offer some early evidence of how one specific group of workers—software engineers—are impacted by one specific GAI tool—GitHub Copilot. GitHub and LinkedIn are both owned by Microsoft.

GitHub Copilot is a GAI coding tool that helps software engineers and developers be more productive by autocompleting lines of code based on AI inference of the developer’s intent, as well as large language querying capabilities. These tools are embedded directly within their developer workspace. With over one million paid subscribers and 20,000 organizations, it is the most widely adopted GAI coding tool.  The findings of our research, while specific to one GAI tool and one occupation, are encouraging and provide a glimpse into how GAI may impact workers in this particular segment of the economy. Below are the main findings from our working paper.

Adoption of GitHub Copilot increases hiring of software engineers

Perhaps, one of the concerns about GAI is how it may worker and jobs. While there’s been research on how occupations may be impacted depending on the replicability and augmentation of their top skills by GAI (Kimbrough and Carpanelli 2023), conducting deeper research on a specific segment like software engineers could share more clear insights to the true impact of GAI on workers. Software engineers use many skills which may be augmented by GAI (e.g. writing code and debugging), while also using human-only skills which can not be replicated by GAI (higher-level conceptualization and strategy, teamwork, leadership). Our findings offer encouragement. GitHub Copilot leads firms to hire more software engineers each month, especially entry and senior level workers who are not managers. GitHub firms not using GitHub Copilot hire on average at least one new software engineer in a given month 44.1% of the time. Adopting GitHub Copilot increases that by 2.9 percentage points to around 47% of the time. We find evidence that it is not just that they are more likely to hire software engineers each month, but when they do have hiring, they hire more of them (a 3.9% increase). Importantly, we find no evidence of displacement of software engineers at any level (management or otherwise) arising from GitHub Copilot.

GitHub Copilot increases demand for software engineers

Our research found demand for and hiring of software engineers increases with GitHub Copilot. There is some evidence that this is driven by these firms increasing their likelihood of having job postings for software engineers on LinkedIn (6.9% increase). There is even some evidence that in the US, the likelihood of a firm having a job posting for software engineers that does not require having a college degree increases with GitHub Copilot. This is consistent with the idea that GitHub Copilot complements the skills of the worker that broadens the fields of candidates the firm considers.

GitHub Copilot leads to increases in the number of non-programming skills new hires have

Adopting GitHub Copilot leads firms’ new hires of software engineers to have 13.3% more non-programming skills—such as Microsoft Office, project management, and communication—on the LinkedIn platform, with no change in the number of programming skills—such as JavaScript, SQL, and Python. If GitHub Copilot makes software engineers more productive in their coding, it would not be surprising that hiring firms place increased interest in the set of non-programming skills new workers have—human skills that are not replicated or augmented by GAI. On the other hand, what happens for software engineers already working at a firm when it gets GitHub Copilot? There, we would expect to see the complementary nature of GitHub Copilot on programming accelerate the learning of those types of skills. And we do see some evidence to that effect, with the workers adding more skills overall, but adding fewer soft skills, business skills, and non-programming skills broadly.

These early results are encouraging with respect to how this specific part of the labor market benefit from GAI

The study highlights the positive impact of GitHub Copilot on the labor market for software engineers. Early evidence shows GitHub Copilot increasing hiring, promoting upskilling and higher levels of non-programming skills without decreases in programming skills, and potentially lowering barriers to entry. This should help temper concerns on the impact of this new technology on this and similar segments of the labor market. As AI continues to evolve, tools like GitHub Copilot will play a crucial role in driving innovation and growth in the software engineering field. It also promotes lifelong learning and pathways for more rapid and inclusive skills development

Technical notes

What is GitHub Copilot?
GitHub Copilot is a code completion tool that is directly embedded into the software engineer’s coding editor (e.g. VS Code). It was collaboratively developed by GitHub, Microsoft, and OpenAI. It assists software engineers by auto-suggesting code snippets based on logical patterns observed in the engineer’s code. GitHub Copilot also responds to queries from the coder—questions like, how would I program this task in this language? Or what does this error mean and how do I fix this bug? It's like having a smart assistant that understands coding and can provide helpful suggestions as you work. This enables software engineers to code more quickly, with fewer errors, and to branch out into tasks and languages they have lower experience with. It is not surprising that firms with software engineers have rapidly adopted GitHub Copilot. GitHub is a web-based platform used for version control and collaborative software development. In less than two years since rollout, over 30% of overall GitHub clients have licensed GitHub Copilot. 

How was the study conducted?
The study was conducted by joining proprietary data at the firm level from GitHub and LinkedIn. We implemented a doubly robust difference-in-difference methodology which allows for staggered treatment timing (Callaway Sant’Anna 2011).  We also employed several sensitivity checks on how the sample was formed and how treatment was defined.