Specific data on the performance impact of AI in the accounting industry is scarce, but a 2023 Harvard Business Review working paper tested 758 management consultants from Boston Consulting Group on 18 "realistic consulting tasks". One group used AI (GPT-4) to help them complete the tasks while the control group had no access to AI tools.
The consultants who used GPT-4 completed 12% more tasks, were 25% faster, and produced results that were 40% higher quality than the control group.
Even more interesting: these improvements were not evenly spread. Those in the bottom half of performers (judged on a baseline test before the experiment) improved by 43%. The top half improved by only 17%.
Which raises a question for any firm: what would it mean if you could raise the quality of work of your bottom 50% of performers to the level of your top 50%?
Three key takeaways
- Individual consultants using AI on realistic consulting tasks were 25% faster and produced 40% higher quality work than those not using AI.
- The bottom half of performers improved by 43% when given an AI tool, bringing them close to the top half, who improved by 17%.
- We are still learning which complex tasks AI is better at. There are still tasks on which humans perform better without it.
Source: Navigating the Jagged Technological Frontier, Dell'Acqua, McFowland, Mollick et al., Harvard Business School working paper, 2023.
