At Digital Scribbler, we’re always on the lookout for emerging technologies and the incredible potential they hold to assist individuals with special needs. So, when the opportunity arose to attend Google I/O last week, I didn’t hesitate. I wanted to share some of my key takeaways and explore how these new advancements can propel our work forward.
AI at the Forefront: Gemini’s Transformative Potential
Unsurprisingly, most of the keynote and breakout sessions at Google I/O centered on advancements in AI, which was definitely a highlight. It’s clear that Google is pouring immense resources into making Gemini the ultimate AI assistant. Their progress in integrating Gemini’s capabilities into products like Gemini Live is particularly exciting, especially its new leverage of camera and screen sharing.
For those with special needs, Gemini Live’s ability to understand context through visual input opens up revolutionary avenues for understanding and communication. Imagine someone struggling to express a need; they could simply point their device at an object, and Gemini could help formulate options for them to choose from, even more dynamically than our current Quick Talk AI. This allows for initiating conversations based on visual prompts, not just responding within them.
During the keynote, I had the serendipitous opportunity to sit next to the Product Manager for DeepMind’s R&D products. I enthusiastically shared Digital Scribbler’s mission, and he was genuinely interested and inspired. He revealed how Gemini Live and other products they’re actively developing are working to translate sign language into text or audio, which is truly monumental. His encouragement affirmed that our path—bridging the gap between the products and services large companies create and the opportunities to use them in assistive technologies—is indeed a strong one.
The advancements in Gemma, Google’s on-device model, were also incredibly encouraging. We’ve already tested this model with QuickTalk AI, and it effectively provided speech options based on conversation context. Hearing about the ongoing advancements, we’re confident that these mobile models will soon be powerful enough to enable reliable offline access to communication.
A Glimpse into the Future: Google XR Glasses and Google Beam
The Google XR glasses truly stole the show during the keynote. These prototype glasses, running Android XR, are designed to work seamlessly with Gemini. Equipped with a camera and microphones, they can “see” and “hear” your world, understand your context, and provide information precisely when you need it. At Digital Scribbler, we’ve already been working on using the Meta Ray-Ban glasses to assist individuals with dyslexia in reading. Google XR glasses would allow us to customize the experience for each child and fine-tune it with unprecedented precision. I was delighted to learn from the Google XR team that we can begin development for these devices now.
With the Google XR glasses, I can envision an integration with Quick Talk AI to provide a hands-free method for selecting response options. Years ago, our team developed a prototype of this with Google Glass during a Google Make-a-thon. The integration possibilities for assistive technologies with this new form factor seem truly endless.
The demo for Google Beam, formerly known as Project Starline, was nothing short of amazing. Google Beam is Google’s new AI-first 3D video communication platform. It’s designed to make remote conversations feel as though you’re actually in the same room, with realistic 3D representations of participants that maintain eye contact and subtle cues, all without the need for specialized glasses or headsets. For individuals with special needs who might struggle with traditional video calls, this could be a game-changer, as it genuinely feels like the person is right there with you.
Creative AI: Revolutionizing Social Stories and More
Then there were the creative advancements with Imagen 4 and Veo. Imagen 4 is a game-changer for image generation, producing stunning visuals with incredible detail and better text rendering. But Veo truly blew me away. This video generation model isn’t just creating scenes; it’s adding native support for sound effects, background noises, and even character dialogue, all from simple text prompts. I can see these technologies being the future of social stories. As someone who has created my share of these to help my son prepare for a new appointment or experience, the ability to write a prompt and produce a video with integrated audio would be an absolutely amazing tool.
Looking Ahead
Attending Google I/O left me profoundly inspired about the future possibilities. The rapid advancements in AI, particularly with Gemini, the innovative potential of Google XR glasses, the immersive communication offered by Google Beam, and the creative power of Imagen and Veo, all point towards a future where technology can more seamlessly and intuitively support individuals with special needs. We at Digital Scribbler are more motivated than ever to harness these incredible tools and continue our mission of bridging the gap between cutting-edge innovation and practical assistive technologies. The future is bright, and we’re excited to be a part of shaping it.
We're always on the lookout for emerging technologies and the incredible potential they have and Google I/O did not disappoint.
We had the privilege of joining innovators, educators, entrepreneurs, policy makers, and learners at Stanford University
At our recent “Tech Day” at Hope Technology School, we presented two exciting projects that are already making a meaningful impact.
