How to Use LLMs with Your NLU: The Power of a Hybrid Approach
Table of contents
LLMs have been leaving a notable impact and what was once just a small presence in conversational AI, is now gaining immense attention.
With LLMs capable of engaging convincingly on a wide range of topics, many are questioning the necessity of NLUs. Why invest time and resources in refining intents, training data, and entities when an LLM can chat effortlessly for hours without such specifications? Moreover, weren’t NLU-based bots too limiting, confined to predetermined paths and unable to assist users with unanticipated needs?
The reality isn’t a binary choice between the two and it’s crucial to explore the potential synergy between them. Each approach has its own strengths and weaknesses, and by integrating both, many longstanding challenges in the conversational AI industry can be addressed.
Here are three strategic ways to leverage LLMs alongside your NLU.
#1: Utilizing an LLM-powered bot for better semantic understanding
Unlike NLUs, which require meticulous training to categorize user inputs into specific intents, LLMs leverage their training on extensive datasets to predict language behavior. This allows them to interpret diverse user queries, from straightforward ones like “what’s my balance” to more colloquial expressions such as “have I any dough,” without the need for predefined rules or examples.
The potential of LLMs as front-end assistants in conversational AI is substantial. They excel at analyzing user input to discern their underlying needs and accurately route them to the appropriate intent. Cathal showcased a demo involving an embassy inquiry about visa applications. The LLM correctly identified semantically similar phrases like “how much does it cost for a visa” and “how much does it cost to apply for a visa” as inquiries about pricing. In contrast, the NLU, weighted towards certain keywords, misinterpreted the latter query as an inquiry about the application process instead of focusing on cost.
While NLUs could be updated to accommodate new utterances and refine intent matching, Cathal highlights the dual benefits of employing LLMs. Firstly, they grasp meaning inherently, alleviating the need to explicitly instruct the bot on semantic nuances. Secondly, they minimize reliance on training data; instead, clear language defining intents suffices, with the LLM intuitively triggering relevant actions.
Although creating intents remains essential for guiding user interactions, integrating LLMs in this manner can reduce the necessity for extensive training data, as Cathal suggests.
#2: Establishing guardrails for an LLM with a pre-designed flow
Flowcharts are conventional tools utilized in crafting conversational AI assistants. Essentially, they provide a roadmap for the beginning, middle, and end of a conversation. Initially, you outline the parameters of the interaction (the bot’s identity and capabilities), then the middle phase involves the exchange or collection of crucial information by the bot, and finally, the various outcomes represent resolutions to different user inquiries.
Traditionally, flowcharts dictated the potential conversation paths, with NLUs ensuring functionality during live interactions by capturing user inputs and directing them based on training. An alternative approach is utilizing a flowchart to define the interaction while bypassing the NLU. Instead, user inputs are processed by ChatGPT to generate responses.
This approach incorporates design guardrails to restrict the LLM’s responses, addressing concerns like potential exploitation or “jailbreaking” of LLMs by malicious entities seeking unauthorized information disclosure.
This underscores the shift in mindset required when employing LLMs in conversational AI. Rather than meticulously designing every aspect of the bot’s responses, the focus shifts to providing a comprehensive information base and instructing the bot on what to exclude from its replies.
Despite the potential benefits, such as multilingual response generation, this method entails foregoing NLU training in favor of defining constraints for the LLM. While it may seem time-saving initially, ongoing updates to the guardrails around the LLM are necessary as new issues emerge, raising questions about long-term efficiency gains.
#3: Using LLMs for bot testing and training
NLUs are in a perpetual state of refinement, as they require substantial amounts of data to function effectively, typically hundreds or thousands of utterances per intent. However, as more data is added to address interpretation issues, the risk of model confusion with false positives and negatives increases.
Continuous refinement of NLU training is standard practice but can be labor-intensive. Identifying confusion, augmenting data to address it, training, testing, and analyzing outcomes are iterative tasks prone to unintended consequences. LLMs offer a potential solution by serving as vast repositories of varied language usage. They can assist in testing NLUs with semantically similar utterances to gauge their performance and augmenting training data where weaknesses are identified.
Automating NLU testing and data generation using LLMs could streamline management significantly. As interactions with the bot increase, NLU training data should expand, reflecting observed user interactions. However, managing this growth becomes increasingly complex over time. Leveraging LLMs in this capacity can help maintain oversight of the intricate relationships between intents and training data, ensuring ongoing NLU effectiveness.
Summary
Decades of accumulated expertise in NLU design and maintenance highlight its efficacy in addressing user needs when their intentions and communication patterns are well understood. A proficiently trained NLU is generally robust enough to cater to the majority of user requirements. Hence, discarding a functional system prematurely seems unnecessary.
Despite the extended tenure of some individuals in the LLM domain, the technology remains somewhat enigmatic. As exemplified by Cathal, there exist numerous innovative approaches to integrate LLMs alongside NLUs to harness the advantages of both. LLMs can be particularly valuable in assisting users with unconventional needs or expressions, occurrences that are commonplace in interactions with most bots.
Why limit oneself to a singular option? Combining NLUs and LLMs expands the scope of assistance, accommodating a broader spectrum of users and their diverse requirements. Ultimately, the objective is to serve users optimally. Therefore, it’s important to weigh the benefits of both technologies and consider how they collectively serve the varied needs of users.