IoT Private Chatbot

Developed a Raspberry Pi based IoT system as a private chatbot with face and speaker recognition

Overview

Developed an IoT system using Raspberry Pi 4 as a private chatbot with face detection and speaker recognition to guarantee privacy and personal conversations as well as interactions.

System Architecture

System architecture showing the integration of face recognition, speech processing, and AI assistant components on Raspberry Pi 4.
GitHub Repository
Raspberry Pi Smart Assistant

Technical Details

  • Implemented MTCNN with ResNet and dlib-based face recognition, achieving better performance with the latter; trained the system with one hundred face images for live recognition.
  • Built a custom residual neural network with Keras for speaker recognition, achieving 96% accuracy.
  • Integrated a server-client architecture using Google Cloud for accelerated processing and implemented speech recognition and TTS for user interaction.

Key Features

  • Real-time face detection and recognition
  • Speaker recognition for enhanced security
  • Natural language processing with ChatGPT integration
  • Text-to-speech and speech-to-text capabilities
  • Cloud-based processing for improved performance

Technical Stack

  • Raspberry Pi 4
  • Python
  • TensorFlow/Keras
  • OpenCV
  • dlib
  • Google Cloud Services
  • OpenAI API
  • PyAudio
  • gTTS

Project Advisor

Prof. Deming Chen, UIUC

Demo of our ECE479 Final Porject -- "IoT Private Chatbot" (Right click on the cover above to view the video)