By Jacques Bughin and Duco Sickinghe
As the AI boom continues to accelerate, investors are keenly eyeing the generative AI sector for potential breakthroughs. Jacques Bughin and Duco Sickinghe explore ten key opportunities within the AI value chain that promise to address user challenges and unlock significant market potential.
Since our article of March 2022 (“AI Inside? Five Tipping Points for a New AI-based Business World”),1 which warned about the big boom in AI, artificial intelligence has indeed become red hot, with the birth of many large language models (LLMs), the launch of Apple AI Intelligence,2 and the booming demand for GPUs.
In this mania, investors are not only pouring money into the public AI companies (Nvidia is one of the most valuable public companies, exceeding $3 trillion in market value),3 they are also fighting to invest in the new private AI darlings, with private-equity-backed investment multiplied tenfold in one year.4 But the key, as a wise investor, is not to follow the herd as in a gold rush, but to anticipate the most important future opportunities in the AI value chain. The best opportunities are those that ultimately solve user issues and create a major buy-side market. Here are 10 examples.
Ten Investment Opportunities
1. Low-bit quantization
Why? The high computational and memory requirements of modern LLMs is costing generative AI users a lot by consuming a large amount of energy, in addition to generating significant pollution.
To give a sense of this, training a single 200 billion parameter LLM on AWS p4d instances consumes more energy than a thousand households for a year.5 In some dense areas in the US, datacenters consume 20 per cent of the grid power, endangering its reliability. Based on current GPU usage, the associated gigatons of CO2 emissions, according to figures laid out by the ACM, could well be the equivalent of 5 billion US cross-country flights.6
Any player that optimizes sustainable energy in the context of AI will be a big winner.
The opportunity: If these numbers are correct, any player that optimizes sustainable energy in the context of AI will be a big winner, as energy consumption will also be closely watched by sensitive users and regulators, who are already pushing for much better optimization and transparency in this area (c.f. the EU AI Act).7 Further, such a reduction in energy consumption might open the low end of the enterprise users market, easily doubling the demand for generative AI.
Examples: Low-bit quantization corresponds to the idea that fixed LLM weighting parameters could be optimized instead of their current “standard” of 32/16 bits in traditional LLMs. Players in low-bit quantization include the big names such as Nvidia and its software TensorRT and cuDNN, or Google TensorFlow Lite, which offers support for quantization-aware training and post-training low-bit quantization. More recently, Microsoft has unveiled its BitNet b1.58 (alluding to a 1.58 bit LLM where every single weight of the LLM is ternary {-1, 0, 1} and is able to match the full precision of 16 bits), demonstrating strong gains, up to 70 per cent down, in energy consumption over traditional LLM models.
The world of startups is also pushing ahead, with low-bit quantization startups including DeepAI, OctoML, and SambaNova Systems, which has established a strong moat through its innovative Reconfigurable Dataflow Architecture, and an integrated hardware-software solutions platform for low-bit quantisation to deliver high-performance AI applications.
2. Liquid neural networks
Why? Neural networks (ANN) have been driving the LLM revolution because of their parallel processing capabilities and their ability to model complex relationships, even with unstructured data. The drawback of ANNs is that they require large amounts of data and computing power, exhibit low explainability, and are still in a race of bigger and larger models.
The issue with ANN is thus a mix of energy cost (see above), a race of concentration whereby LLM models become a key bottleneck resource, and a poor reach, as LLM models cannot be incorporated in thin client layers.
The opportunity: Enter liquid neural networks (LNN), a novel type of neural network architecture inspired by neuroscience, which rely on dynamic connections and weights between neurons.
While LLN technology is potentially revolutionary according to its MIT proponents,8 its value is in shifting “the existing big is better“ paradigm to smaller models, where LLN has a potential to replicate fixed ANN performance with at least a 1,000 times lower number of weights. For instance, drones can be guided by a small 20,000-parameter LNN model9 that performs better in navigating previously unseen environments than other neural networks with millions of neurons.
Two other features of liquid neural networks are their ability to adapt in real time to new and evolving data without requiring extensive retraining, and their ability to perform continuous learning. This means they can integrate new information on the fly, which is particularly useful in real-world applications where data patterns can change rapidly, such as automotive (autonomous driving), industrial robotics (logistics), healthcare (real-time monitoring of patients), and customer service (chat interactions).
Examples: Startup companies such as Liquid AI10 and Vicarious11 are already experimenting with liquid neural networks. The latter aims to create more adaptable and efficient robotic control systems.
3. Quantum computing
Why? Many problems remain too complex to resolve with the current state of generative AI. In parallel, generative AI has boosted a major cybersecurity risk,12 resulting in the majority of enterprises resisting leveraging generative AI.
Meanwhile, superconducting technologies are now reaching over 120 qubits with IBM’s latest Eagle processor.
The opportunity: The promise of quantum computing lies in its combination with generative AI to solve problems previously thought to be intractable. The combination has the potential to solve a wide range of optimization problems, from logistics to financial portfolios, with unprecedented efficiency.
Quantum computing can not only destroy classical encryption methods, but also develop new, quantum-resistant cryptographic algorithms. In addition, quantum computing can be used to improve the efficiency and effectiveness of cybersecurity solutions.
Examples: Rigetti Computing is an example of a company exploring quantum computing applications in drug discovery. A competitor, PasQal, is also redefining energy efficiency standards in quantum computing through neutral atomic quantum.
4. Knowledge distillation
The promise of quantum computing lies in combining it with GenAI to solve problems previously thought to be intractable.
Why? Besides the energy power issue, the size of LLMs makes companies desperate for GPUs. In this evolution, Nvidia is definitely the big winner, with massive excess demand waiting in the wings. However, in the long run, this slows down the evolution of the market and makes the cost of generative AI too high for a large number of companies.
Opportunities: These lie in optimizing training and usage to reduce the number of GPUs. As we said earlier, liquid neural networks are a major breakthrough if they deliver on their promise, as training will be much easier and models will be much less expensive.
Meanwhile, in the short term, knowledge distillation is a technique that makes ANN run much more efficiently through model shrinking. As first introduced by Geoffrey Hinton in 2015,13 the latter technique transfers knowledge from a teacher LLM model to a much lighter student model. During the learning process, a complex neural network is taught to generate meaningful and helpful representations of data. The distillation processes are based on these thorough representations of data, or knowledge, stored by the teacher network in its hidden layers.
Example: distilBERT is a lightweight compressed counterpart of a larger BERT language model, with 60 per cent of the size of the original BERT model while retaining 97 per cent of its performance and being 60 per cent faster.
5. Synthetic data
Why? There are many problems with data, not only because they are increasingly scarce for AI training, but also because data quality can be questionable. There are two possibilities.
The opportunity: Synthetic data. We witness that synthetically trained models are becoming quite powerful and can even outperform models trained on real data.14 Successful applications are emerging in financial services, healthcare, and retail. In addition, the advantage of synthetic data is its privacy clearance.
Examples: We have already reported on the Nvidia simulator application15 in its industrial metaverse that successfully leverages synthetic data to train robots. In general, synthetic data can also be used to rebalance samples when the required prediction concerns rare events such as financial fraud or manufacturing defects.
Startups that rely on the power of synthetic data include SBX Robotics,16 whose generated synthetic data are built 10 times faster and more cheaply than annotation services teaches robots to see; or MOSTLY AI, a leading synthetic data platform with a proprietary GenAI model architecture that results in the highest-accuracy synthetic data, and enables sophisticated AI/ML use cases, including multi-variate time series data and relational databases.
6. AI agents
Why? In our current work environment, there has been a lot of talk about how AI can steal our work.17 In a typical AI model, tasks are well described and automation can be done if the value productivity of humans was lower than the task performed by AI. However, by the end of 2023, the most capable Generative AI could learn many other skills through this process of next token prediction – for example, translation between languages, math and reasoning skills, and much more.
But the most interesting capability is the ability of LLMs to use software tools. ChatGPT, for example, can now browse the web, use a code interpreter plugin to run code, or perform other actions enabled by a developer.18
Opportunities: In this new environment, AI agents represent a leap from traditional automation in that they are designed to think, adapt, and act independently, rather than simply follow a set of instructions. The assertion is that agentic AI systems could dramatically increase users’ abilities to get more done in their lives with less effort,19 and with significantly better effectiveness, especially in complex dynamic tasks.
The second opportunity coming from AI agents is real-time data. We are creating more data every year than has been accumulated in the past. One can argue indeed that the collection of data from AI agents will play at the center of task workflows of enterprise processes.
The rush to adopt AI should not be reckless; the use of AI technologies leads to significant risks.
Examples: Microsoft’s Project AutoGen20 demonstrates a multi-agent framework that simplifies building workflows and applications with LLMs. It features specialized agents that can be configured with different LLMs and enables seamless human interaction. Another prime case of the value of AI agents through generative AI relates to the analysis of real-time traffic data from numerous sources, including road cameras, GPS in vehicles, and social media, to optimize traffic flow dynamically. These AI agents analyze real-time traffic data from numerous sources, including street cameras, in-car GPSs, and social media, to dynamically optimize traffic flow. In Hangzhou, China, the system built by Alibaba has reduced traffic congestion by 15 per cent and sped up emergency response times by 49 per cent .21
7. Responsible AI
Why? The rush to adopt AI should not be reckless. In particular, the use of AI technologies leads to significant risks due to inaccurate data, privacy and copyright violations, algorithmic bias, or even fake and harmful content. Moreover, agents have specifically been shown to be less robust, prone to more harmful behaviors,22 and capable of generating stealthier content than LLMs, highlighting significant safety challenges. Finally, Europe is clearly moving ahead by passing its EU AI Act, which is rather stringent regarding trustworthy AI compliance.
The risks attached to generative AI are such that the largest bottleneck to date in enterprises adopting generative AI is misbehaving models and agents,23 on top of cybersecurity risks.
Opportunities: The development of the EU AI law would continue to make companies comply with a large number of responsible practices, for which a large number of them are not ready. In terms of the metrics, the auditing of practices is a significant regtech opportunity, as it is for other regulatory push areas, such as sustainability.
In addition, the intersection of generative AI and regtech presents significant opportunities to improve regulatory compliance processes. The ability to automate, analyze, and predict through AI provides significant value in terms of cost savings, accuracy, and efficiency.
Examples: Key startups in this space, such as Behavox, are pioneering generative AI solutions to help organizations monitor and manage compliance risks. Their flagship product, Behavox Quantum AI, includes monitoring of text and voice communications, reduction of alert volumes and false positives, and high recall rates for detecting compliance violations. In a recent test, ChatGPT detected less than 20 per cent of the intentionally planted phrases, compared with more than 80%24 for Behavox Quantum AI.
8. ML ops
Why? The majority of generative AI bottlenecks inside firms adopting the technology occur at the transition from prototype to production. In fact, research25 has found that 87 per cent of AI projects never make it into production.
Opportunities: MLOps (machine learning operations) is essential for managing the complex life cycle of generative AI models, and hence delivers multiple advantages over scaling. By standardizing workflows and automating repetitive tasks, MLOps ensures consistent performance and reliability of models in a time of strong compliance linked to generative AI.
MLOps tools provide robust capabilities for monitoring model performance, detecting drifts, and triggering alerts for anomalies. This ensures that generative AI models remain accurate and effective over time. Finally, MLOps fosters better collaboration between data scientists, developers, and operations teams by providing a unified framework and tools for managing the entire machine learning life cycle.
Examples: Robust Intelligence is an automated intelligence platform that integrates with ML models to “stress test” AI models prior to deployment, detect vulnerabilities, highlight erroneous data, and identify data issues that could compromise ML integrity.
Comet’s machine learning platform integrates with your existing infrastructure and tools, allowing you to manage, visualize, and optimize models – from training runs to production monitoring.
9. Cybersecurity
While generative AI is a problem, it is also a powerful solution for cybersecurity development.
Why? From former star Cisco buying Splunk to SentinelOne buying Attivo, the M&A race in cybersecurity is a symptom of the rush to provide a global platform for all cybersecurity issues. These issues have only grown exponentially in the last few years, and they are experiencing a major uptick through LLM. Currently, it reflects email phishing attacks, but with the rise of multimodal LLM, the threats will only expand and diversify. With quantum AI, the cryptographic elements may also be at risk.
Opportunities: While generative AI is a problem, it is also a powerful solution for cybersecurity development. Some specific opportunities in cybersecurity software and management that leverage generative AI are: a) automated threat detection and response (AI models can generate threat signatures and response strategies in real time, enabling faster and more effective incident response, while AI-powered tools can automatically filter out phishing emails and generate alerts for suspicious messages, reducing the risk of successful phishing attacks); b) vulnerability management (AI models can generate reports on potential vulnerabilities and suggest patches or mitigations, helping organizations to proactively address security weaknesses); c) malware analysis and generation (AI can assist in creating honeypots and decoys that mimic vulnerabilities, attracting and analyzing malware to better understand and mitigate threats).
Examples: There are a number of startups emerging from Jericho that are looking at better security, from governance to security solutions such as providing AI firewalls and threat detection and response solutions to detect malicious behavior that attacks generative AI models. For example, Vectra AI uses AI to identify and stop cyberattacks by analyzing network traffic and detecting malicious behavior.
While there are many companies in the space, opportunities remain, especially for more real-time risk detection and more sophisticated detection models.
10. Thin-client generative AI
Why? By 2024, barely 10 per cent of smartphones are GenAI capable26 but, as in the first wave of the internet, the market truly exploded when the internet migrated to mobile and became the reference access for everyone. The battle is on,27 with Samsung taking the lead in the GenAI smartphone market and Apple’s recent entry into the artificial intelligence space.
Opportunities? Generative AI is not yet widespread on thin clients, due to significant computational, storage, and latency challenges, on top of mobile network infrastructure interfacing with typical other IP networks. Finally, the user experience on mobile is tied to the multimodal experience.
Regarding mobile networks, it seems that many standardization activities currently exist for AI functionality in RAN and core parts of mobile networks.28 Yet, the user experience may require the roll-out of 5G or more. The opportunity will thus manifest itself as a mix of the model and architecture optimization we have touched upon earlier, such as low-bit quantization, and LNN, on top of advances in edge computing, for more feasible deployment of generative AI on mobile and other lightweight devices. Opportunities are not only consumer-based, but also enterprise-based. Digital twinning29 for training, and mobility is, for instance, a major opportunity under inspection by many players in the mobile ecosystem.
Examples: Chip set manufacturers like ARM and Qualcomm have already made significant strides in launching thin-client GenAI-powered chip sets. But startups are also making inroads in this large market: Syntiant, on the other hand, develops ultra-low-power AI chips for edge devices that use advanced quantization techniques to enable efficient AI processing in power-constrained environments.
Conclusions
The world of generative AI is only just opening. It is still complex and faces major hurdles to being deployed. Still, these hurdles imply a major fix opportunity to scale the market significantly. Investors should pay close attention to how solutions will impact the market, and whether solution providers offer an opportunity to be exploited in exchange for a large market (TAM) in the making.
