Working with ChatGPT Functions on Heroku

How to Build and Deploy a Node.js App That Uses OpenAI’s APIs

Near the end of 2023, ChatGPT announced that it had 100M weekly users. That’s a massive base of users who want to take advantage of the convenience and power of intelligent question answering with natural language.

With this level of popularity for ChatGPT, it’s no wonder that software developers are joining the ChatGPT app gold rush, building tools on top of OpenAI’s APIs. Building and deploying a GenAI-based app is quite easy to do—and we’re going to show you how!

In this post, we walk through how to build a Node.js application that works with OpenAI’s Chat Completions API and uses its function calling feature. We deploy it all to Heroku for quick, secure, and simple hosting. And we’ll have some fun along the way. This project is part of our new Heroku Reference Applications, a GitHub organization where we host different projects showcasing architectures to deploy to Heroku.

Ready? Let’s go!

Meet the Menu Maker

Our web application is called Menu Maker. What does it do? Menu Maker lets users enter a list of ingredients that they have available to them. Menu Maker comes up with a dish using those ingredients. It provides a description of the dish as you’d find it on a fine dining menu, along with a full ingredients list and recipe instructions.

This basic example of using generative AI uses the user-supplied ingredients, additional instructional prompts, and some structured constraints via ChatGPT's functions calling to create new content. The application’s code provides the user experience and the data flow.

Menu Maker is a Node.js application with a React front-end UI that talks to an Express back-end API server. The Node.js application is a monorepo, containing both front-end and back-end code, stored at GitHub. The entire application is deployed on Heroku.

Here’s a preview of Menu Maker in action:

Let’s briefly break down the application flow:

1. The back-end server takes the user’s form submission, supplements it with additional information, and then sends a request to OpenAI’s Chat Completions API.
2. The back-end server receives the response from OpenAI and passes it up to the front-end.
3. The front-end updates the interface to reflect the response received from OpenAI.

Prerequisites

Note: If you want to try the application first, deploy it using the “Deploy to Heroku” button in the reference application’s README file.

Before we dive into the code let’s cover the prerequisites. Here’s what you need to get started:

1. An OpenAI account. You must add a payment method and purchase a small amount of credit to access its APIs. As we built and tested our application, the total cost of all the API calls made was less than $1*.
2. After setting up your OpenAI account, create a secret API key and copy it down. Your application back-end needs this key to authenticate its requests to the OpenAI API.
3. A Heroku account. You must add a payment method to cover your compute costs. For building and testing this application, we recommend using an Eco dyno, which has a $5 monthly flat fee and provides more than enough hours for your initial app.
4. A GitHub account for your code repository. Heroku hooks into your GitHub repo directly, simplifying deployment to a single click.

Note: Every menu recipe request incurs costs and the price varies depending on the selected model. For example, using the GPT-3 model, in order to spend $1, you'd have to request more than 30,000 recipes. See the OpenAI API pricing page for more information.

Initial Steps

For our environment, we use Node v20.10.0 and yarn as our package manager. Start by cloning the codebase available in our Heroku Reference Applications GitHub organization. Then, install your dependencies by running:

yarn install

Build the Back-End

Our back-end API server uses Express and listens for POST requests to the /ingredients endpoint. We supplement those ingredients with more precise prompt instructions, sending a subsequent request to OpenAI.

Working with OpenAI

Although OpenAI’s API supports advanced usage like image generation or speech-to-text, the simplest use case is to work with text generation. You send a set of messages to let OpenAI know what you’re seeking, and what kind of behavior you expect as it responds to you.

Typically, the first message is a system message, where you specify the desired behavior of ChatGPT. Eventually, you end up with a string of messages, a conversation, between the user (you) and the assistant (ChatGPT).

Call Functions with OpenAI

Most users are familiar with the chatbot-style conversation format of ChatGPT. However, developers want structured data, like a JSON object, in their ChatGPT responses. JSON makes it easier to work with responses programmatically.

For example, imagine asking ChatGPT for a list of events in the 2020 Summer Olympics. As a programmer, you want to process the response by inserting each Olympic event into a database. You also want to send follow-up API requests for each event returned. In this case, you don’t want several paragraphs of ChatGPT describing Olympic events in prose. You’d rather have a JSON object with an array of event names.

Use cases like these are where ChatGPT functions come in handy. Alongside the set of messages you send to OpenAI, you send functions, which detail how you use the response from OpenAI. You can specify the name of a function to call, along with data types and descriptions of all the parameters to pass to that function.

Note: ChatGPT doesn’t call functions as part of its response. Instead, it provides a formatted response that you can easily feed directly into a custom function in your code.

Initialize Prompt Settings with Function Information

Let’s take a look at src/server/ai.js. In our code, we send a settings object to the Chat Completions API. The settings object starts with the following:

const settings = {
  functions: [
    {
    name: 'updateDish',
    description: 'Generate a fine dining dish based on a list of ingredients',
    parameters: {
        type: 'object',
        properties: {
        title: {
            type: 'string',
            description: 'Name of the dish, as it would appear on a fine dining menu'
        },
        description: {
            type: 'string',
            description: 'Description of the dish, in 2-3 sentences, as it would appear on a fine dining menu'
        },
        ingredients: {
            type: 'array',
            description: 'List of all ingredients--both provided and additional ones in the dish you have conceived--capitalized, along with measurements, that would be needed to make 8 servings of this dish',
            items: {
            type: 'object',
            properties: {
                ingredient: {
                type: 'string',
                description: 'Name of ingredient'
                },
                amount: {
                type: 'string',
                description: 'Amount of ingredient needed for recipe'
                }
            }
            }
        },
        recipe: {
            type: 'array',
            description: 'Ordered list of recipe steps, numbered as "1.", "2.", etc., needed to make this dish',
            items: {
            type: 'string',
            description: 'Recipe step'
            }
        }
        },
        required: ['title', 'description', 'ingredients', 'recipe']
    }
    }
  ],
  model: CHATGPT_MODEL,
  function_call: 'auto'
}

We’re telling OpenAI that we plan to use its response in a function that we call updateDish, a function in our React front-end code. When calling updateDish, we must pass in an object with four parameters:

1. title: the name of our dish
2. description: a description of our dish
3. ingredients: an array of objects, each having an ingredient name and amount
4. recipe: an array of recipe steps for making the dish

Send Settings with Ingredients Attached

In addition to the functions specification, we must attach messages in our request settings, to clearly tell ChatGPT what we want it to do. Our module’s send function looks like:

const PROMPT = 'I am writing descriptions of dishes for a menu. I am going to provide you with a list of ingredients. Based on that list, please come up with a dish that can be created with those ingredients.'

const send = async (ingredients) => {
  const openai = new OpenAI({
    timeout: 10000,
    maxRetries: 3
  })
  settings.messages = [
    {
      role: 'system',
      content: PROMPT
    }, {
      role: 'user',
      content: `The ingredients that will contribute to my dish are: ${ingredients}.`
    }
  ]
  const completion = await openai.chat.completions.create(settings)
  return completion.choices[0].message
}

Our Node.js application imports the openai package (not shown), which serves as a handy JavaScript library for OpenAI. It abstracts away the details of sending HTTP requests to the OpenAI API.

We start with a system message that tells ChatGPT what the basic task is and the behavior we expect. Then, we add a user message that includes the ingredients, which gets passed as an argument to the send function. We send these settings to the API, asking it to create a model response. Then, we return the response message.

Handle the POST Request

In src/server/index.js, we set up our Express server and handle POST requests to /ingredients. Our code looks like:

import express from 'express'
import AI from './ai.js'

const server = express()
server.use(express.json())

server.post('/ingredients', async (req, res) => {
  if (process.env.NODE_ENV !== 'test') {
    console.log(`Request to /ingredients received: ${req.body.message}`)
  }
  if ((typeof req.body.message) === 'undefined' || !req.body.message.length) {
    res.status(400).json({ error: 'No ingredients provided in "message" key of payload.' })
    return
  }
  try {
    const completionResponse = await AI.send(req.body.message)
    res.json(completionResponse.function_call)
  } catch (error) {
    res.status(500).json({ error: error.message })
  }
})

export default server

After removing the error handling and log messages, the most important lines of code are:

const completionResponse = await AI.send(req.body.message)
res.json(completionResponse.function_call)

Our server passes the request payload message contents to our module’s send method. The response, from OpenAI, and then from our module, is an object that includes a function_call subobject. function_call has a name and arguments, which we use in our custom updateDish function.

Testing the Back-End

We’re ready to test our back-end!

The openai JavaScript package expects an environment variable called OPENAI_API_KEY. We set up our server to listen on port 3000, and then we start it:

OPENAI_API_KEY=sk-Kie*** node index.js
Server is running on port 3000

In a separate terminal, we send a request with curl:

curl -X POST \
  --header "Content-type:application/json" \
  --data "{\"message\":\"cauliflower, fresh rosemary, parmesan cheese\"}" \
  http://localhost:3000/ingredients

{"name":"updateDish","arguments":"{\"title\":\"Crispy Rosemary Parmesan Cauliflower\",\"description\":\"Tender cauliflower florets roasted to perfection with aromatic fresh rosemary and savory Parmesan cheese, creating a crispy and flavorful dish.\",\"ingredients\":[{\"ingredient\":\"cauliflower\",\"amount\":\"1 large head, cut into florets\"},{\"ingredient\":\"fresh rosemary\",\"amount\":\"2 tbsp, chopped\"},{\"ingredient\":\"parmesan cheese\",\"amount\":\"1/2 cup, grated\"},{\"ingredient\":\"olive oil\",\"amount\":\"3 tbsp\"},{\"ingredient\":\"salt\",\"amount\":\"to taste\"},{\"ingredient\":\"black pepper\",\"amount\":\"to taste\"}],\"recipe\":[\"1. Preheat the oven to 425°F.\",\"2. In a large bowl, toss the cauliflower florets with olive oil, chopped rosemary, salt, and black pepper.\",\"3. Spread the cauliflower on a baking sheet and roast for 25-30 minutes, or until golden brown and crispy.\",\"4. Sprinkle the roasted cauliflower with grated Parmesan cheese and return to the oven for 5 more minutes, until the cheese is melted and bubbly.\",\"5. Serve hot and enjoy!\"]}"}

It works! We have a JSON response with arguments that our back-end can pass to the front-end’s updateDish function.

Let’s briefly touch on what we did for the front-end UI.

Build the Front-End

All the OpenAI-related work happened in the back-end, so we won’t spend too much time unpacking the front-end. We built a basic React application that uses Material UI for styling. You can poke around in src/client to see all the details for our front-end application.

In src/client/App.js, we see how our app handles the user’s web form submission:

const handleSubmit = async (inputValue) => {
  if (inputValue.length === 0) {
    setErrorMessage('Please provide ingredients before submitting the form.')
    return
  }
  try {
    setWaiting(true)
    const response = await fetch('/ingredients', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({ message: inputValue })
    })
    const data = await response.json()
    if (!response.ok) {
      setErrorMessage(data.error)
      return
    }

    updateDish(JSON.parse(data.arguments))
  } catch (error) {
    setErrorMessage(error)
  }
}

When a user submits the form, the application sends a POST request to /ingredients. The arguments object in the response is JSON-parsed, then sent directly to our updateDish function. Using ChatGPT’s function calling feature significantly simplifies the steps to handle the response programmatically.

Our updateDish function looks like:

const [title, setTitle] = useState('')
const [waiting, setWaiting] = useState(false)
const [description, setDescription] = useState('')
const [recipeSteps, setRecipeSteps] = useState([])
const [ingredients, setIngredients] = useState([])
const [errorMessage, setErrorMessage] = useState('')
const updateDish = ({ title, description, recipe, ingredients }) => {
  setTitle(title)
  setDescription(description)
  setRecipeSteps(recipe)
  setIngredients(ingredients)
  setWaiting(false)
  setErrorMessage('')
}

Yes, that’s it. We work with React states to keep track of our dish title, description, ingredients, and recipe. When updateDish updates these values, all of our components update accordingly.

Our back-end and front-end pieces are all done. All that’s left to do is deploy.

Not shown in this walkthrough, but which you can find in the code repository, are:

• Basic unit tests for back-end and front-end components, using Jest
• ESLint and Prettier configurations to keep our code clean and readable
• Babel and Webpack configurations for working with modules and packaging our front-end code for deployment

Deploy to Heroku

With our codebase committed to GitHub, we’re ready to deploy our entire application on Heroku. You can also use the Heroku Button in the reference repository to simplify the deployment.

Step 1: Create a New Heroku App

After logging in to Heroku, click “Create new app” in the Heroku Dashboard.

Next, provide a name for your app and click “Create app”.

Step 2: Connect Your Repository

With your Heroku app created, connect it to the GitHub repository for your project.

Step 3: Set Up Config Vars

Remember that your application back-end needs an OpenAI API key to authenticate requests. Navigate to your app “Settings”, then look for “Config Vars”. Add a new config var called OPENAI_API_KEY, and paste in the value for your key.

Optionally, you can also set a CHATGPT_MODEL config var, telling src/server/ai.js which GPT model you want OpenAI to use. Models differ in capabilities, training data cutoff date, speed, and usage cost. If you don’t specify this config var, Menu Maker defaults to gpt-3.5-turbo-1106.

Step 4: Deploy

Go to the “Deploy” tab for your Heroku app. Click “Deploy Branch”. Heroku takes the latest commit on the main branch, builds the application (yarn build), and then starts it up (yarn start). With just one click, you can deploy and update your application in under a minute.

Step 5: Open Your App

With the app deployed, click “Open app” at the top of your Heroku app page to get redirected to the unique and secure URL for your app.

With that, your shiny, new, ChatGPT-powered web application is up and running!

Step 6: Scale Down Your App

When you’re done using the app, remember to scale your dynos to zero to prevent incurring unwanted costs.

Conclusion

With all the recent hype surrounding generative AI, many developers are itching to build ChatGPT-powered applications. Working with OpenAI’s API can initially seem daunting, but it’s straightforward. In addition, OpenAI’s function calling feature simplifies your task by accommodating your structured data needs.

When it comes to quick and easy deployment, you can get up and running on Heroku within minutes, for just a few dollars a month. While the demonstration here works specifically with ChatGPT, it’s just as easy to deploy apps that use other foundation models, such as Google Bard, LLaMA from Meta, or other APIs.

Are you ready to take the plunge into building GenAI-based applications? Today is the day. Happy coding!