Strategic Excellence: Steps to Maximise ROI in GEN AI Implementations

By Stephan Kudyba and Agnel D’Cruz

The power of generative artificial intelligence (GEN AI) has organisations of all types intrigued and clamouring to leverage its functionality to enhance productivity, improve their financial bottom line and maintain market share or possibly achieve a competitive advantage. Increasing the speed and robustness of information assets presents ample opportunities for process applications. However, the jury is still out on a few issues for implementing this technology to achieve truly valuable results. Issues involving the data that must be accessible for large language models (LLMs), verifying the output generated and where to apply the platform to operationalise it for a sustained production environment introduces difficulties in its adoption.

Individuals who understand GEN AI capabilities and best applications in a given area need to collaborate in order to optimise potential roll-outs.

The following steps provide a high-level methodology on how to best approach the implementation of GEN AI to produce effective results that will justify the cost of the strategic initiative.

Throughout the step-wise approach, a major factor must be emphasised, and this is the involvement of expert knowledge of workers at all levels. It is an essential element to achieving success. A phrase that sums up the process is best stated….true value in Generative AI involves a collaborative environment integrating top-down strategy and bottom-up implementation.

1. Establish knowledge transfer to strategic management and SMEs regarding the functionality of GEN AI.

This goes beyond simply presenting generic prompting and content creation but involves prominent process applications.

In other words, knowledge experts within a given organisation must be informed of what GEN AI can actually do. Examples include:

• Devising marketing correspondence for a production environment
• Examining and extracting information from complex and lengthy documents
• Creating computer code and reverseengineering legacy systems
• Creating imagery
• Identifying gaps and new, evolving elements in existing content

Possible GEN AI functionality can help maintain parity with the market given its adoption by competitors. Competitive advantage could be achieved through faster time to market of custom information creation (e.g. combining internal data with that of open source).

2. Conduct collaborative knowledge transfer among informed knowledge experts along with GEN AI technicians (e.g. vendors) to achieve optimal organisational applications.

Knowledge creation is best achieved through an open collaboration of knowledge assets (e.g. SMEs). Individuals who understand GEN AI capabilities and best applications in a given area need to collaborate in order to optimise potential roll-outs. Diverse perspectives and cross-functional input can uncover both routine and innovative opportunities to create value with LLMs.

For example, leverage customer feedback from voice and online sources to enhance product attributes according to consumer (sizing, delivery, price, etc.).

This step not only entails the identification of best areas of GEN AI applications but also should involve estimations of potential returns. In the case above, adjusting product attributes yields increased customer satisfaction, sales, repeat buying, etc. Or in the case of a more routine application (streamlining marketing resources), estimate the expected reduction in labour and the value of re-allocated labour to more productive activities in the firm.

3. Once realistic GEN AI applications that prove tangible value have been identified, cost estimates need to be generated to measure implementation versus potential gains.

Technology And Technology Labour Costs

Data resources that must be accessed by LLMs must be identified, which involves the incorporation of data engineers when considering internal data or the combination of open source and internal data for competitive advantage. These engineers must estimate the time for alignment and optimisation of required data (e.g. cloud-based, internal repository based).

Computer processing costs entailed in LLMs utilisation must be included.

Editing And Verification Costs

True value in Generative AI involves a collaborative environment integrating top-down strategy and bottom-up implementation.

Costs required for the selection of editorial staff time must be initiated. This entails labour resources required to authenticate created content (e.g. safeguarding against adverse, incorrect, outdated content or hallucinations) and potential copyright infringement. This mitigates the risk of costly legal or accountability issues.

Total cost must then be weighed against the expected value created by GEN AI incorporation. The estimation of value must be recognised with variances, given the uncertainty of outcomes achieved, where probabilities should be considered.

4. Value estimation consideration: 

“Reduction” in cost (e.g. savings in a reduction in labour hours through streamlined processes).

“Enhanced” market share through competitive advantage in producing timely custom content. This may include knowledge-enhancing content for internal processes (e.g. adjusting product attributes to consumer needs) or creating custom content of timely information for sources external to the organisations (e.g. customers, suppliers).

“Reduction” in risk of losing market share to competitor activities who adopt GEN AI .

Risk Management is Key

As is the case with many innovative technologies available to the entire market, organisations must not only consider if the functionality is a fit for their organisation but whether they are exposed to loss of market position by competitors effectively adopting it. One of the main advantages of GEN AI is speed, or how quickly content can be created. This speed element includes the ability to generate ideas (identify critical content initially not thought of etc.). Speed also entails vast processing capabilities of information resources to streamline operations. Both of these can increase the exposure of losing market share.

Risk also involves the exposure and accountability of missing adverse content created (e.g. a missed hallucination or simply including inaccurate sources for creating content). This opens exposure to legal ramifications and loss of market share by producing content violating copyright laws, or releasing inaccurate content to customers.

5. Prioritise projects according to value and risk reduction.

After consideration of the previous steps, organisations must answer a major question. Is the project really worth it?

This considers the company’s risk profile and cost/value analysis. If the answer is YES, then move on.

It is time to select the application that best fits the organisation’s situation. What this involves is an examination of some previous steps. Prioritisation of a GEN AI implementation should focus on the following elements:

1. How exposed is the company to loss of market share by not taking action?
2. How significant is the process that will be augmented with GEN AI (will the implementation make a real difference to its performance?)

To illustrate these points, consider a hypothetical case in the insurance industry.

The application of GEN AI may augment an insurance provider’s ability to identify the elements required for custom coverage to an entity (individual, group, etc.) in a timelier manner than industry standards.

The ability to provide a quote for custom coverage faster than the industry norms may increase market share, given the augmentation of the customer experience. The risk exposure of not moving on GEN AI may be high. However, an additional risk that must be considered is that of devising coverage elements with inaccurate information.

6. The final stage is the roll-out of GEN AI for the application.

At this stage, data sources should be aligned, LLMs trained through the engagement of data scientists and prompters, and SMEs should be in place to measure accuracy. An overall assessment of the entire performance of the platform must take place, where the critical elements to focus on are the accuracy of content and time to production. The new process must evolve regarding new data sources and LLM accuracy.

About the Authors

Stephan Kudyba is a professor of analytics and information systems at the Martin Tuchman School of Business, New Jersey Institute of Technology. He has held senior management positions at prominent organisations and has been a researcher, professor, and practitioner of AI applications in business for over 20 years. Email: [email protected]

Agnel J D’Cruz is a seasoned data professional who is currently a principal of data strategy and partnerships at ZoomInfo. Agnel has been published by HBR for his outstanding work at Honeywell International where he led large-scale data and AI implementations.