Function Calling with LLMs

What is Function Calling with LLMs?

In the realm of large language models (LLMs), Function Calling refers to the ability of a LLM to impute, from the user prompt, the correct function to execute from a set of available functions and the correct parameters to pass to that function. Instead of generating standard text responses, a LLM for function calling is typically fine-tuned to return structured data responses, typically JSON objects. The returned structured data can be used to execute predefined functions, such as retrieving data from a data warehouse or feature store, retrieving real-time data, or calling 3rd party APIs. Function Calling has emerged as a significant technique, particularly in models like OpenAI's GPT models, in how these models interact with users and the external world, offering a structured approach to handling complex queries and tasks. In particular, function calling is a promising technique for opening up enterprise data directly to LLMs without the need for a vector database.

Function Calling significantly expands the capabilities of LLMs by bridging the gap between natural language understanding and practical tasks. With Function Calling, these models can seamlessly integrate with external systems, perform complex operations, and provide more accurate and contextually relevant responses.

Using Function Calling in AI Systems

You should look into function calling if you are designing an AI system that allows users to not only get text-based responses, but also executes tasks on your behalf. For example, let’s consider the task of sending an email. Instead of simply generating text-based responses, the LLMs can use Function Calling to generate a structured response (such as a JSON object) that is used to execute the email-sending task

The user initiates a conversation with the virtual assistant, expressing their intent to send an email.
The virtual assistant will first use the LLM (e.g., a LLM fine-tuned for Function Calling), to process the user's request. Based on the context of the conversation, the model detects the need to call a specific function for sending an email.
The model then generates a function call response, e.g., a JSON object, specifying the details of the email as the arguments of the function call, such as the recipient's email address (‘to’) and the content of the email (‘body’). This request is then passed to the pre-defined function in the system that is responsible for sending emails.‍

send_email(to: "colleague@example.com", body: "Hi [Colleague's Name], I hope this email finds you well. Can we reschedule our meeting tomorrow to next Monday? Best regards, [Your Name]")

In the backend, the system receives the function call request from the LLM. Using the provided arguments, the system executes the necessary steps to send the email(s).
Once the email has been sent, the system prepares a response for the LLM. This response may include confirmation of the email being sent or any relevant details related to the task. The LLM can then generate natural language text to the users about this request.

Why is Function Calling Useful?

By incorporating Function Calling into LLM-based AI systems, the interactivity and utility can be largely enhanced, enabling the AI system to perform real-world tasks on behalf of users. Furthermore, Function Calling streamlines the execution of tasks, eliminating the need for users to switch between multiple applications or interfaces. On the other hand, Function Calling provides flexibility to users to define custom functions tailored to specific use cases, allowing for offering personalized experience.

Function Calling in ML Frameworks

OpenAI supports function calling and has been fine-tuned for function calling. The open-source Instructor library supports function calling with the help of Pydantic objects. OpenHermes is a fine-tuned LLM that supports function calling. Together with Instructor, they can be used by the DSPy library to build compound AI systems that optimize the chains of commands that make up the system. Both Instructor and DSPy can be installed as Python libraries in Hopsworks. OpenHermes can be downloaded for free from Hugging Face. Hopsworks additionally supports RAG through its feature store with approximate nearest neighbor indexing. You can write DataFrames to Hopsworks that are both available for querying with either function calling or similarity search.

In summary, Function Calling represents a significant new capability for LLMs to interact with Enterprise Data and Enterprise Systems, empowering developers to create more interactive and functional applications. By seamlessly integrating with external applications, systems, and APIs, LLMs become even more versatile, capable of performing a diverse range of tasks with ease.

Examples of using Function Calling for LLMs

In this online session we explore the capability of extending RAG for LLMs to include the ability to query structured data and API calls using function calling.