OpenAI’s ChatGPT, the AI-powered language model, is unveiling a set of exciting new features, allowing users to “see, hear, and speak.” These enhancements are designed to make ChatGPT more user-friendly and versatile, offering a variety of ways for users to interact with the AI model.
OpenAI has announced a phased rollout of voice and image capabilities within ChatGPT over the next two weeks. These features are intended to empower users to engage in voice conversations and visually convey their queries to ChatGPT, making the AI experience even more interactive and accessible.
The primary goal behind these updates is to enhance the utility and user-friendliness of ChatGPT. According to MIT Technology Review, OpenAI has been diligently refining its technology with the aim of providing a comprehensive AI solution through the ChatGPT Plus app. This puts it in direct competition with virtual assistants like Siri, Google Assistant, and Alexa.
OpenAI emphasized the significance of these new features, stating, “Voice and image give you more ways to use ChatGPT in your life. Snap a picture of a landmark while traveling and have a live conversation about what’s interesting about it.” The voice feature will be available on both iOS and Android platforms, with the option to opt-in through your settings, while the image feature will be functional across all platforms.
OpenAI went on to explain how users can leverage these capabilities: “You can now use voice to engage in a back-and-forth conversation with your assistant. Speak with it on the go, request a bedtime story for your family, or settle a dinner table debate.”
The image feature had been hinted at earlier in March when GPT-4, the model powering ChatGPT, was introduced. However, it was not accessible to the general public at the time. Now, users can upload images to the app and inquire about the content of those images, expanding the AI’s versatility.
MIT Technology Review also noted that this announcement follows the recent integration of DALL-E 3, OpenAI’s image-generation model, into ChatGPT. This integration allows users to instruct the chatbot to generate images based on their input.
Additionally, OpenAI has partnered with Be My Eyes, enabling users to ask ChatGPT questions based on images, further expanding its practical applications.
Powering the voice feature of ChatGPT, OpenAI utilized Whisper, its speech-to-text model, to convert spoken words into text, which ChatGPT can then process, enabling voice interactions with the AI software. Joanne Jang, a producer manager at OpenAI, mentioned that synthetic voices were created by training the text-to-speech model on the voices of hired actors. OpenAI is also considering the possibility of allowing users to create their own custom voices in the future.
OpenAI is taking privacy, safety, and accessibility concerns seriously with the introduction of these features. They have outlined a multifaceted approach to address these issues, including content moderation, responsible data handling, clear user guidelines, restrictions on sensitive topics, and a strong focus on ethical software use. Furthermore, OpenAI is actively collaborating with external organizations, researchers, and experts to conduct audits and assessments of the system, ensuring that ChatGPT remains a responsible and reliable tool for users.
