In the hype-filled world of how generative AI will transform business, a critical theme for the legal profession is emerging: Please show us the value.
For in-house and outside counsel, a periodic performance review found that GenAI isn’t meeting expectations, according to a quarterly survey by Bain & Co.
Legal executives expressed the greatest dissatisfaction with GenAI software tools in the survey, with an 18% decline in the number of lawyers saying GenAI had “met or exceeded expectations” over a five-month period. For lawyers overall, about half (53%) said the tools had met or surpassed their expectations.
Those using GenAI in operations showed a 16% decline, while staff working in sales and sales operations showed a 6% increase in the satisfaction metric.
Among the reasons offered for disappointment, 43% said GenAI could not perform a needed task with “sufficient performance” and 42% cited “poor quality output.”
Among other respondents, 38% said they didn’t understand how to use the tools and 29% said “low quality” of the vendor and its product. The latter finding, a poor view of a vendor or a tool’s quality, showed the largest increase in sentiment compared to the prior survey.
One reason for the disillusionment among in-house and external counsel is that “lawyers have a high bar,” Mary O’Carroll, chief community officer at San Francisco-based Ironclad, which makes contract management software, told Corporate Counsel.
“I strongly believe that there’s an amazing use case for AI here, and the bar hasn’t been met by the crop of early stage startups that are trying to solve this,” she said.
Bain said that more than 60% of businesses surveyed put GenAI among their top three priorities into 2025, with 87% ranking it among their top five priorities for the next three to four years.
Companies have around 100 people spending some time engaging with the new AI technology in some form, with large companies having as many as 240 team members, according to Bain’s data.
“As with other digital transformations, merely automating a process or porting it to a new, more efficient technology doesn’t necessarily deliver enough upside to justify the investment,” the Bain report says. “Real value comes from learning how the new technology changes not just how something gets done but what gets done.”
The results are based on a sample of senior executives at 200 companies in October 2023 and February 2024.