Want to create a chatbot that listens to your voice and responds with the power of AI? This beginner-friendly guide shows you how to set up a voice-controlled chatbot on a Raspberry Pi 4 using OpenAI’s GPT-3.5 model. Your chatbot will hear commands, process them with GPT-3.5, and reply through speakers—all for just the cost of an OpenAI API key (~$0.002 per 1K tokens). Perfect for hobbyists and makers, this project is fun, affordable, and a great way to dive into AI. Let’s build something awesome!

Why Build a Voice Chatbot on Raspberry Pi?

The Raspberry Pi 4 is a widely used, low-cost computer perfect for AI projects. With OpenAI’s GPT-3.5, your chatbot can:

• Understand Natural Speech: Respond to questions, jokes, or commands with human-like intelligence.
• Run Locally: Operate on your Raspberry Pi with minimal cloud dependency (except for OpenAI API calls).
• Be Customised: Add your wake word, voice, or features.
• Cost Little: Requires only a Raspberry Pi, a microphone, speakers, and an OpenAI API key.

Prerequisites

Before you start, gather these items:

• Raspberry Pi 4: Running Raspberry Pi OS (Raspbian, latest version recommended).
• Microphone and Speakers: A USB microphone and speakers (or a headset) connected to the Raspberry Pi.
• Internet Connection: Stable Wi-Fi or Ethernet for API calls and updates.
• OpenAI API Key: Sign up at platform.openai.com and generate an API key (~$0.002 per 1K tokens for GPT-3.5-turbo).
• Basic Tools: A keyboard, mouse, and monitor (or SSH access) for setup.

 

Step 1: Set Up the Raspberry Pi

Get your Raspberry Pi ready for the chatbot project.

Open the Terminal

1. Open the terminal by clicking the terminal icon in the taskbar or pressing Ctrl+Alt+T.

Update and Upgrade the System

2. Run these commands to update and upgrade your system packages:

   sudo apt update
   sudo apt upgrade -y
   

   Tip: This ensures your Raspberry Pi has the latest software, which prevents compatibility issues.

Step 2: Install Required Libraries and Dependencies

Install the software needed for speech recognition, text-to-speech, and OpenAI integration.

Install Python Packages

1. Run this command to install essential Python libraries:

   python3 -m pip install python-dotenv openai SpeechRecognition pyttsx3 gtts numpy
   

   • Libraries:
     • python-dotenv: Loads environment variables (e.g., API key).
     • openai: Connects to OpenAI’s GPT-3.5 model.
     • SpeechRecognition: Converts speech to text.
     • pyttsx3: Provides offline text-to-speech.
     • gtts: Google Text-to-Speech for alternative voice output.
     • numpy: Handles random greeting selection.

Install System Dependencies

2. Install audio-related dependencies:

   sudo apt install python3-pyaudio flac espeak -y
   

   • Dependencies:
     • python3-pyaudio: Enables microphone input.
     • flac: Supports audio processing.
     • espeak: Provides basic text-to-speech functionality.

Step 3: Configure Audio Hardware

Ensure your microphone and speakers work correctly.

Check Connected Audio Devices

1. List audio devices:
   • For the microphone:

     arecord --list-devices
     

   • For speakers:

     speaker-test -t wav -c 2
     

   • Output Example:

     **** List of CAPTURE Hardware Devices ****
     card 1: Device [USB Audio Device], device 0: USB Audio [USB Audio]
     

   • Note the card and device numbers (e.g., plughw:1,0).

Test the Microphone

2. Record a 5-second audio clip and play it back:

   arecord -D plughw:1,0 -d 5 test.wav
   aplay test.wav
   

   • Adjust -D plughw:1,0 Based on your arecord --list-devices output.
   • Tip: If you hear your voice, the microphone is working.

Adjust Audio Settings

3. Use alsamixer to set volume levels:

   alsamixer
   

   • Press F6 to select your audio device.
   • Use the arrow keys to adjust the microphone and speaker volumes.
   • Ensure devices are not muted (press M to toggle mute).
   • Press Esc to exit.

Step 4: Set Up the Environment

Securely store your OpenAI API key and prepare the project directory.

Create a Project Directory

1. Navigate to your home directory and create a project folder:

   cd ~
   mkdir chatbot_project
   cd chatbot_project
   

Create a .env File

2. Create a .env file to store your API key:

   nano .env
   

3. Add this line, replacing your_openai_api_key_here With your actual OpenAI API key:

   OPENAI_API_KEY=your_openai_api_key_here
   

4. Save and exit:
   • Press Ctrl+X, then Y, then Enter.

Secure the .env File

5. Restrict file permissions:

   chmod 600 .env
   

   • This ensures only you can read the file.

Step 5: Verify OpenAI GPT-3.5 Model

Test your OpenAI API key and GPT-3.5 model access.

Create a Test Script

1. Create a test script:

   nano test_openai_model.py
   

2. Add this code:

   import openai
   from dotenv import load_dotenv
   import os
   
   # Load environment variables
   load_dotenv()
   openai.api_key = os.getenv('OPENAI_API_KEY')
   
   # Test GPT-3.5 model
   def test_gpt_model():
       try:
           response = openai.ChatCompletion.create(
               model="gpt-3.5-turbo",
               messages=[
                   {"role": "user", "content": "Hello, how are you?"}
               ]
           )
           print("Model response:", response.choices[0].message.content)
       except openai.error.OpenAIError as e:
           print("An error occurred:", e)
   
   if __name__ == "__main__":
       test_gpt_model()
   

3. Save and exit (Ctrl+X, Y, Enter).

Run the Test Script

4. Execute the script:

   python3 test_openai_model.py
   

   • Expected Output: A response like “Model response: I’m doing great, thanks for asking!”
   • Error Handling: If you see an error (e.g., “Invalid API key”), check your .env file or OpenAI account.

Step 6: Create the Chatbot Script

Build the main Python script for your voice-controlled chatbot.

Create the Script

1. Create a new file:

   nano chatbot.py
   

2. Add this code:

   import openai
   from dotenv import load_dotenv
   import time
   import speech_recognition as sr
   import pyttsx3
   import numpy as np
   import os
   import logging
   
   # Set up logging
   logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
   
   # Load environment variables
   load_dotenv()
   openai.api_key = os.getenv('OPENAI_API_KEY')
   
   # Model name
   model = 'gpt-3.5-turbo'
   
   # Initialize speech recognition and text-to-speech
   r = sr.Recognizer()
   engine = pyttsx3.init()
   
   # Configure voice
   voices = engine.getProperty('voices')
   if len(voices) > 1:
       engine.setProperty('voice', voices[1].id)
   else:
       engine.setProperty('voice', voices[0].id)
   
   # User personalization
   user_name = "Friend"  # Change this to your name
   greetings = [
       f"What's up, {user_name}?",
       "Hey, what's good?",
       f"Hello, {user_name}! Ready to chat?",
       "Yo, what's the vibe today?",
       "Hi there! How can I help you?"
   ]
   
   # Listen for wake word
   def listen_for_wake_word(source):
       logging.info("Listening for 'hey'...")
       while True:
           try:
               audio = r.listen(source, timeout=5)
               text = r.recognize_google(audio)
               if "hey" in text.lower():
                   logging.info("Wake word detected.")
                   engine.say(np.random.choice(greetings))
                   engine.runAndWait()
                   listen_and_respond(source)
                   break
           except sr.WaitTimeoutError:
               logging.info("No sound detected, still listening...")
           except sr.UnknownValueError:
               pass
           except sr.RequestError as e:
               logging.error(f"Speech recognition error: {e}")
               engine.say("Sorry, I couldn't connect to the speech service.")
               engine.runAndWait()
   
   # Listen and respond with GPT-3.5
   def listen_and_respond(source):
       logging.info("Listening for your command...")
       while True:
           try:
               audio = r.listen(source, timeout=5)
               text = r.recognize_google(audio)
               logging.info(f"You said: {text}")
               if not text:
                   continue
   
               # Call OpenAI API
               response = openai.ChatCompletion.create(
                   model=model,
                   messages=[{"role": "user", "content": text}]
               )
               response_text = response.choices[0].message.content
               logging.info(f"GPT-3.5 response: {response_text}")
   
               # Speak response
               engine.say(response_text)
               engine.runAndWait()
               os.system(f"espeak '{response_text}'")
   
               listen_for_wake_word(source)
           except sr.WaitTimeoutError:
               logging.info("Silence detected, returning to wake word mode...")
               listen_for_wake_word(source)
               break
           except sr.UnknownValueError:
               logging.info("Could not understand audio, listening again...")
           except sr.RequestError as e:
               logging.error(f"Speech recognition error: {e}")
               engine.say("Sorry, I couldn't connect to the speech service.")
               engine.runAndWait()
               listen_for_wake_word(source)
               break
           except openai.error.OpenAIError as e:
               logging.error(f"OpenAI API error: {e}")
               engine.say("Sorry, there was an issue with the AI service.")
               engine.runAndWait()
               listen_for_wake_word(source)
               break
   
   # Handle graceful exit
   import signal
   import sys
   
   def signal_handler(sig, frame):
       logging.info("Exiting chatbot gracefully...")
       sys.exit(0)
   
   signal.signal(signal.SIGINT, signal_handler)
   
   # Start chatbot
   with sr.Microphone() as source:
       logging.info("Adjusting for ambient noise...")
       r.adjust_for_ambient_noise(source, duration=5)
       listen_for_wake_word(source)
   

3. Save and exit (Ctrl+X, Y, Enter).

Code Explanation

• Imports: Libraries for OpenAI, speech recognition, text-to-speech, logging, and environment variables.
• Logging: Tracks events and errors for debugging.
• Environment Variables: Loads the OpenAI API key from .env.
• Speech Setup: Initialises SpeechRecognition for audio input and pyttsx3 for text-to-speech.
• Personalisation: You can set user_name (e.g., “Alex”) for custom greetings.
• Functions:
  • listen_for_wake_word: Listens for “hey” to activate the chatbot.
  • listen_and_respond: Captures speech, sends it to GPT-3.5, and speaks the response.
• Graceful Exit: Handles Ctrl+C to stop the script cleanly.
• Ambient Noise Adjustment: Calibrates the microphone for better accuracy.

Personalise the Script

• Change user_name = "Friend" to your name (e.g., user_name = "Alex").
• Modify greetings list to add fun phrases (e.g., "What's cooking, buddy?").

Step 7: Run the Chatbot

Start your voice-controlled chatbot.

Navigate to the Project Directory

1. Ensure you’re in the project folder:

   cd ~/chatbot_project
   

Run the Script

2. Execute the chatbot script:

   python3 chatbot.py
   

   • The script adjusts for ambient noise, then listens for “hey.”
   • Say “hey” followed by a command (e.g., “Hey, tell me a joke”).
   • The chatbot responds with GPT-3.5’s output.

Sample Conversation

• You: “Hey, what’s the weather like today?”
• Chatbot: “I don’t have real-time weather data, but I can tell you it’s probably sunny somewhere! Want me to make up a forecast?”
• You: “Hey, tell me a joke.”
• Chatbot: “Why did the computer go to art school? Because it wanted to learn how to draw a better ‘byte’!”

Step 8: Set Up as a Background Service

Run the chatbot automatically on boot.

Create a Systemd Service

1. Create a service file:

   sudo nano /etc/systemd/system/chatbot.service
   

2. Add this content:

   [Unit]
   Description=Voice-Controlled Chatbot Service
   After=network.target
   
   [Service]
   ExecStart=/usr/bin/python3 /home/pi/chatbot_project/chatbot.py
   WorkingDirectory=/home/pi/chatbot_project
   StandardOutput=inherit
   StandardError=inherit
   Restart=always
   User=pi
   
   [Install]
   WantedBy=multi-user.target
   

3. Save and exit (Ctrl+X, Y, Enter).

Enable and Start the Service

4. Run these commands:

   sudo systemctl daemon-reload
   sudo systemctl enable chatbot.service
   sudo systemctl start chatbot.service
   

5. Check the service status:

   sudo systemctl status chatbot.service
   

   • Look for “active (running)” to confirm it’s working.

Step 9: Customise Your Chatbot

Make your chatbot unique with these advanced options.

Change the Wake Word

• Edit listen_for_wake_word to use a different wake word (e.g., “robot”):

  if "robot" in text.lower():
  

Add Custom Responses

• Modify the greetings list or add a dictionary for specific commands:

  custom_responses = {
      "hello": "Hey, nice to hear from you!",
      "joke": "Why did the scarecrow become a coder? He was outstanding in his field!"
  }
  

Integrate Additional APIs

• Add weather or news APIs (e.g., OpenWeatherMap):

  import requests
  def get_weather(city):
      api_key = "your_weather_api_key"
      url = f"http://api.openweathermap.org/data/2.5/weather?q={city}&appid={api_key}"
      response = requests.get(url).json()
      return f"It's {response['weather'][0]['description']} in {city}."
  

Adjust Voice Settings

• Change the voice speed or pitch:

  engine.setProperty('rate', 150)  # Speed (words per minute)
  engine.setProperty('pitch', 0.8)  # Pitch (0.5 to 2.0)
  

Step 10: Troubleshoot Common Issues

Fix problems you might encounter.

• Microphone Not Detected:
  • Check arecord --list-devices and ensure the correct device is used.
  • Verify USB connections and run lsusb to confirm the microphone is recognised.
• No Sound Output:
  • Test speakers with speaker-test -t wav -c 2.
  • Adjust alsamixer to unmute and increase volume.
• Speech Recognition Fails:
  • Ensure a stable internet connection for Google’s speech API.
  • Reduce background noise or adjust r.adjust_for_ambient_noise(source, duration=5).
• OpenAI API Errors:
  • Verify your API key in .env.
  • Check OpenAI account for billing issues or rate limits (~$0.002 per 1K tokens).
• Script Crashes:
  • Review logs in the terminal or /var/log/syslog.
  • Ensure all dependencies are installed (pip install and apt install commands).
• Service Not Starting:
  • Check sudo systemctl status chatbot.service for errors.
  • Verify the ExecStart path in chatbot.service matches your script location.

Conclusion

You’ve built a voice-controlled chatbot on your Raspberry Pi 4 using OpenAI’s GPT-3.5 model! It listens for “hey,” processes your commands with AI, and responds through speakers. Customise it with new wake words, voices, or APIs to make it your own. You’ve created a powerful AI project for just the cost of an OpenAI API key (~$0.002 per 1K tokens). Keep experimenting and share your chatbot creations!

Explore more Raspberry Pi projects at Nicrobit.

Resources

• OpenAI API Documentation
• Raspberry Pi Documentation
• SpeechRecognition Library
• Pyttsx3 Documentation
• Nicrobit Learning Lab

FAQs

Q: How much does the OpenAI API cost for this chatbot?
A: GPT-3.5-turbo costs ~$0.002 per 1K tokens. A short conversation (e.g., 100 tokens) costs ~$0.0002.

Q: Can I use a different wake word?
A: Yes, edit the listen_for_wake_word function to replace “hey” with your preferred word (e.g., “robot”).

Q: Why is my microphone not working?
A: Check arecord --list-devices, verify USB connections, and adjust alsamixer settings.

Q: What if I get an OpenAI API error?
A: Verify your API key in .env, check your OpenAI account for billing, and ensure internet connectivity.

Q: Can I run the chatbot without internet?
A: No, the OpenAI API and Google speech recognition require the internet. Offline alternatives exist, but are less powerful.

Q: How do I stop the chatbot service?
A: Run sudo systemctl stop chatbot.service or press Ctrl+C in the terminal.

Q: Can I use a different AI model?
A: Yes, replace gpt-3.5-turbo with another OpenAI model (e.g., gpt-4), but check pricing and availability.

Q: Why is the chatbot’s voice robotic?
A: The default espeak voice is basic. Try gtts for smoother speech or adjust pyttsx3 voice settings.