Home TECHNOLOGY Artificial Intelligence How Productive Is Generative AI Really?

How Productive Is Generative AI Really?

July 24, 2023

Generative artificial intelligence is invading the corporate suite and boardroom, surrounding the pros and cons of its use in the enterprise. The first fundamental question is whether generative AI is truly a game changer, as evidenced by basic metrics such as productivity gains. We find that it is, but that the benefits vary considerably from case to case, suggesting that managers need to do their homework to define their most favourable position with respect to generative AI.

For Slidor, a French communication company created a few years ago, 50% of its corporate graphic marketing presentation is already done by MidJourney.¹ This intensive use of generative AI is also visible in the same proportion for the codes written by Copilot within GitHub.²

Needless to say, this extensive use of generative AI is invading the enterprise world at a rapid pace. It may also have created its “I-phone moment” in the process: while most digital technologies have focused on “routine” technologies, new generative AI systems such as MidJourney, Stable Diffusion, You, OpenAI’s ChatGPT, and DALL-E are automating creative tasks such as content image generation or software coding that were thought to be largely isolated by the first generation of neuronal AI.

Technology automation is not a zero-sum game. Technology replaces tasks to improve productivity and ultimately generate a bigger pie to enjoy.

In recent scholarly articles in world-renowned journals such as the American Economic Review³ and the Journal of Human Capital⁴ in 2018, star economist Daron Acemoglu had already warned that the traditional assumption that “high-skilled workers are protected from automation because they specialise in more complex tasks requiring human judgement, problem-solving, and analytical skills” might be a dubious narrative. Powerful generative writing tools like ChatGPT could replace certain types of writing (such as press releases and blogs), could easily translate into multiple languages, engage in powerful dialogues with customers, perform medical diagnostics, or debug software code.

Does this mean the end of work, even for highly skilled workers? We doubt it for a variety of reasons.⁵ But the most fundamental reason is that technology automation is not a zero-sum game. Technology replaces tasks to improve productivity and ultimately generate a bigger pie to enjoy. In fact, several studies have concluded that technology often creates more jobs⁶ than it eliminates and that companies that adopt technology can end up growing faster than their competitors.⁷ In other words, technology absorption is ultimately a win-win for workers and shareholders.

Despite all the hype around GPT chat, there are few studies⁸ today that look at productivity improvements in companies that use these technologies. Here, we report on one such possible study and draw important insights for executives considering whether to invest in generative AI.

The Experiment

The experiment is based on the use of an external tool, e.g. ChatGPT for context-based information/dialogue (nlp) and Dall-E or Stable Diffusion for content generation. By focussing on some of the major tools, we neutralise differences in tool choice as a factor in productivity differences. We also examined three contexts: coding (with sub-activities such as re-coding, debugging, and documentation), content generation (for media and advertising), and customer interactions (social media, blogging, email, and customer service). According to various observations,⁹ these activities are the most used ones so far (except for activities such as medical diagnosis, others such as translation, customer research, etc. are not yet common), so we should see some benefits from using generative AI.

We test the productivity gain compared to NOT using generative AI. We also collect data on the level of experience in using these tools, age, occupation, perception of these tools (“enriches work”, “impoverishes work”, “neutral”), and the reason for use (“curiosity”, “peer pressure”, “fun”, “efficiency”).

Three Main Insights

Three main results stand out:

1.Usage. “Business use is already relatively high and is beginning to take hold.”

Approximately 26% of respondents report using the technology as part of their job, and about 16% already use it routinely, at least once a day. Finally, 3% of respondents report having used and abandoned it.

Let’s not forget that these technologies are relatively new and this pace of use, at the enterprise level, is relatively strong — in the range of three to five times faster than the previous wave of so-called Enterprise 2.0.¹⁰ Second, the conversion to an enterprise habit (16% to 26%) is also very strong in a matter of months, which took years for most Enterprise 2.0 technologies, except for messaging and some collaboration tools.¹¹

Adoption leaders. “Curious digital natives are the drivers of adoption.“
We correlated various indicators collected on whether workers used the tool or not. Although we can only explain less than 50% of the adoption rate, we found that the drivers of adoption are (a) age, (b) occupation, (c) curiosity, and (d) efficiency, in order of explanatory power.

Media and software professionals are more likely to benefit from the technology, as these industries have a history of disruption through digitisation and are also the most exposed to the threats and benefits of generative AI, in all likelihood. Let’s not forget that while low-code has simplified coding, coding is still a complex activity that requires time and effort. AI has been used for many years to try to automate code generation, from Amazon’s CodeWhisperer to IBM’s Wisdom. OpenAI is the next evolution for building simple applications from natural language commands.¹²

Curiosity drives experimentation and then use, while more conservative workers tend to oppose technology, both because of the learning burden and the fear that technology could negatively impact their work and status.

Age is negatively correlated with usage, as it is with technology in general,¹³ while curiosity is a significant driver of usage. The latter two factors are more important than the scope of generative AI and are not without consequences. This means that a digital divide can develop between young, digital natives and other workers — and the psychographic characteristics of workers: curiosity drives experimentation and then use, while more conservative workers tend to oppose technology, both because of the learning burden and the fear that technology could negatively impact their work and status.

2. Productivity gains. “They are real, but they take time to unfold, both because of learning and incentives.”

The productivity impact is the product of (a) the users of the technology, (b) the share of activities where AI is applied, and (c) the productivity gains in those activities.

For (a), our results show that 16% of workers use it daily — and that the momentum for expansion is significant — and faster than for any previous technology. Regarding b), we find that AI generative tools converge to account for about 1/3 of the activities of workers who use the technology every day in their workflow. In particular, this rate ranges from 12% to 44% when it comes to marketing activities (including blogging and social media communication); it is about 23% to 36% when it comes to coding (including documentation, code testing, and debugging; incidentally, the usage rate matches the GitHub report that “46% of coding in Copilot-enabled languages is done by the AICodex wizard“).¹⁴ Lastly, we find that the time spent on content production is between 23% and 41% (especially for special effects, branded content, etc.).

Finally, we compile for (c) that the time reduction for AI-assisted tasks is in the range of 30% to 60%, which is consistent with some general experiments,8 but also with some specific studies around GithubCopilot.¹⁵

Taking these three elements together, total productivity at the firm level is between 1% and 4% at a minimum in the occupations covered by the survey. This may not sound like much, but again, it depends on how usage develops, and this productivity gain is already higher than the average labour productivity growth in Europe in recent years.¹⁶

Technology should augment human labour but is still far from replacing it — and that the quality and proper use of these technologies must be worked out at the organisational level.

Outside of usage, the difference in productivity found ranges from 1 to 4, and correlation analysis suggests that productivity depends on the nature of the task (e.g., debugging versus writing new code for software, a special background effect for content versus creating full content, etc.). In addition, productivity accumulates over time — it takes an average of 6 to 8 weeks to achieve stable productivity gains using these tools (so-called learning effects), while productivity is higher for workers associated with companies that have been promoting the use of AI for some time, including generative AI today.

3. Getting Started with Generative AI

We are in a time of ownership of generative AI in the enterprise. Some leaders are deliberately choosing to limit its use, while other companies believe it should be unleashed and tested among employees (see for example JPMorgan vs. Morgan Stanley).

While this is one of the first (relatively basic) studies reporting on productivity related to generative AI, it confirms early studies of different types of AI¹⁷ that AI can be particularly powerful and adds to the parallel evidence that generative AI can provide significant gains, at least in some industries and for some tasks. As such, these technologies can deliver gains that make the returns on investment attractive enough to be considered part of any company’s technology portfolio.

However, as in Gartner’s growth cycle, the future may have some setbacks, so companies need to weigh their play. We see at least three unapologetic moves. The first is experimentation, as there is a learning effect to get the right benefit from technology. The second is to study use cases that are not problematic. These technologies can be used to improve the efficiency of internal human resources communication, to better predict customer reactions in commerce, to speed up information retrieval in service contracts, for the virtual representation of an architectural project, an advertising campaign, or for the redesign of a new website, etc. The third is to work on the right framework for using these technologies: they are still not transparent, they are not always accurate, and they may present biases and risks of copyright infringement. All of this suggests that technology should augment human labour but is still far from replacing it — and that the quality and proper use of these technologies must be worked out at the organisational level. Finally, as general-purpose technologies have shown, technologies can disrupt workflow productivity. Companies must study the disruptions and reinvent themselves accordingly.

This new AI moment may seem a bit chaotic, but the evidence suggests that the companies that are early adopters aren’t necessarily taking risks — theyre aware of the technology’s limitations, they’re working on more explainable AI and source transparency, and they’re working internally to comply with the recent AI law in Europe. They are also preparing for new competition and advantages — see how Salesforce launched EinsteinGPT as part of its data cloud, improving the business insights provided to its customers, and reacting to Microsoft’s investment in OpenAI.

Go to top

About the Author

Jacques Bughin is the CEO of Machaon Advisory and a former professor of Management. He retired from McKinsey as senior partner and director of the McKinsey Global Institute. He advises Antler and Fortino Capital, two major VC/PE firms, and serves on the board of multiple companies.