OpenAI Adds Live Video and Screen Sharing to Advanced Voice Mode

ChatGPT's Advanced Voice Mode has just leveled up with new video and screenshare features! This addition, which had been eagerly anticipated since the launch of GPT-4 in May, is now live, allowing users to chat with ChatGPT through a phone camera—so the model can actually see what you see.

During a recent livestream, OpenAI's Chief Product Officer, Kevin Weil, along with other team members, demonstrated the new capabilities by using ChatGPT to help make pour-over coffee. By simply pointing the camera at the coffee maker, ChatGPT was able to recognize the process, understand how the equipment works, and guide the team through each step of brewing. The demo also included a screensharing feature, where ChatGPT could interpret a message displayed on a phone screen, even with Kevin Weil sporting a Santa beard!