Real Time Demo that allows natural conversations

[freddyaboulton]

Overview

This PR adds an interactive demo that enables natural, continuous conversations with Qwen2-Audio. Users can engage in fluid dialogue with the model through their microphone. Responses are automatically generated when they finish speaking. This enhancement makes the model more accessible and natural to interact with.

Key Features

• Real-time audio streaming using WebRTC
• Automatic speech detection and processing
• Support for both local and cloud deployment

Dependencies

Added requirements:

• gradio-webrtc (gradio custom component that enables real time audio/video streaming). Disclaimer - I am the author of this extension.
• twilio (optional, for cloud deployment)

Demo

qwen2-audio.mp4

There is some delay in processing the response due to acquiring the shared GPU on HuggingFace spaces. On dedicated hardware it should be much faster but I don't have the GPUs to verify myself.

[robinnarsinghranabhat]

Hi. Sorry to ask here ..

I am trying to run Qwen in my Apple M3 Pro (18 gigs combined memory). The basic inference snippet in hugging face examples takes too long (5 mins). I thought mps device would be fast.

Any suggestions what could be done ?