As blind honor Apple accessibility pioneer my son shows far more work is ahead

It didn’t shock me that MojoVision (a Silicon Valley startup making augmented reality contact lenses) brought a big percentage of their team and had a table right in the middle of Sight Tech’s event honoring Mike Shebanek for his work on Apple’s VoiceOver functionality that enables blind people and those who have vision impairments use iPhones. All around the audience were blind people.

MojoVision’s CEO, Drew Perkins, had cataracts and eye surgery, and has long sought to build a bionic eye. So, it makes sense MojoVision would align themselves with the blind community. But all around were others working on augmented reality products. Meta, Apple, and others.

While Shebanek’s speech will be interesting to any Apple fan (he gives lots of stories about building an accessibility team at Apple, including lots of Steve Jobs stories) I don’t want you to miss what happens about 57 minutes into my video: several of the blind people around the room were called on to tell what Apple’s VoiceOver meant to them.

The stories are heartwarming but the job isn’t done. Why do I say that? My 14-year-old son is a special needs kid and has speech that is hard to understand by many and is autistic. None of the AI voice systems understand him and you should hear his frustration at not being able to communicate with computers like his brother can by talking to Alexa or Siri. He’s had Apple devices since he was two years old.

He can’t use systems like Apple’s Siri, Amazon’s Alexa, or even Google’s Assistant with his voice. They just don’t understand him.

As we move into augmented reality devices, which could greatly help him live his, and those who are like him, life, these technology walls grow more daunting. Why? Five years from now we will be talking to AIs far more frequently than today.

At his public school his special needs classmates have similar problems. Some can’t hold their hands still enough to type on a keyboard. Many have a tough time with speech.

Will my son and his fellow students be included in the next paradigm shift? The paradigm shift of moving to 3D computing and new user interfaces for using your real voice and real hands in. For some users, like my son, this will be a frustrating paradigm shift.

It was an honor hearing Mike Shebanek’s stories. He’s a real pioneer who has had a deep mark on many companies (he now is working at Meta). He gives me hope that my son, and his fellow students, will be included in the computing platform of the future.

Thanks to the Vista Center for inviting me.

The Vista Center empowers individuals who are blind or visually impaired to embrace life to the fullest through evaluation, counseling, education, and training. Learn more: https://vistacenter.org

It has a conference coming in December, 2022, for developers who are shaping new technologies to create a more accessible world for people who are blind. Details on that here: https://sighttechglobal.com

Future proof your playlists with this HUGE Dolby Atmos music list

As Apple puts the finishing touches on its new augmented reality headset, expected later this year, I’ve been tracking innovation in music. Spatial Audio/Dolby Atmos. Why? Dolby Atmos will be a huge part of the announcements Apple is going to make. It will also be very important in the future of the “metaverse.”

Last year we got a new Sonos system that plays Dolby Atmos (new spatial audio/surround sound/better quality). Since I sold audio gear in the 1980s it’s amazing to me that you can feel like you are in the middle of a concert now. Apple’s headphones, which I also have, also support Dolby Atmos but don’t really get you the surround sound or the bass of our $3,800 Sonos system.

While watching group forums on Facebook and elsewhere I see lots of others are getting new audio systems that play Dolby Atmos. Movies have played Atmos for years, but music services started sharing Atmos less than a year ago.

The problem is finding Dolby Atmos music.

For instance, Apple’s “Rock Spatial Audio” list has 99 songs. Nice start, but I got bored very quickly. So I started collecting my own. My rock list has 1,131 songs and my hard rock list has 166 songs. Finding these are very difficult. Why? Some albums only have one song done in Dolby Atmos. So you gotta go one by one through each song and you need to know where to look to find new ones.

None of the services are doing Dolby Atmos fans, like me, justice. I’m on all of them that support Atmos (Tidal, Amazon, and Apple) and even some that don’t support Atmos (like Spotify and YouTube Music).

It makes you wonder why the music industry is hiding its biggest technology advance in decades? When it comes to Apple, I’m pretty sure it is readying their own Dolby Atmos music service for its new headset. But Amazon? Its UI is horrid. Worse, all services have really shitty search engines.

Anyway, Dave Winer regularly writes that blogs let authors route around big companies. This is exactly what is going on here. Now, I know 99.99% of people don’t care. That’s fine. You will when you get new surround sound headphones next year. If you still are reading, just remember that this post exists so when you do start to care about music quality you have a resource to go to.

Anyway, here’s the master list of my 60 playlists. I’m breaking them into two sections: “curated” and “catalog.” Curated means I built the list after listening to every song. I built these for my own home and are what I listen to every day. Catalog means it’s just a list of everything I can find (like my rock lists) without any concern about the quality).

If you use these, you must see the Dolby Atmos logo. If you don’t see a logo when playing then you aren’t getting the full Atmos experience (you might need to turn it on in your phone’s settings, or upgrade your equipment).

So, let’s start with “curated.” The first link is to Apple Music. Amazon has a lot less music in Dolby Atmos format and I have only moved over some of my playlists (they take hours to move over because Amazon has far less Atmos).

I include the link here because Amazon sounds better than Apple. Even on Apple’s own headphones. Why? Because it is using a new version of Dolby Atmos that Apple and Tidal aren’t yet using.

1. Chill Together. 161 songs. This is music that Maryam (I’m her husband) and I like listening together to. Nice and calm music.

2. Dolby Atmos Nightclub. 452 songs. The opposite of Chill Together. Lots of explicit language and mostly hip hop/rap. Loud, obnoxious. Rattles the subwoofers. (Amazon)

3. Dolby Atmos Party. 160 songs. None of the explicitness of the nightclub, but still fun beats to get people dancing. (Amazon)

4. Dolby Atmos Radio. 1,472 songs. Music that’s great to listen to all day long. No explicit stuff, but a wide variety of songs. (Amazon)

5. Dolby Atmos Speaker Demonstrations. 84 songs. The best of the best. I did this list to show family and friends what Dolby Atmos is all about but I found it’s great to keep going back to whenever the software in my speaker system upgrades. (Amazon)

6. Explicit Favorites. 694 songs.

7. Explicit Subwoofer Shakers. 1,373 songs. I have two subwoofers so look for recordings that get them working harder than usual.

8. Subwoofer Shakers. 550 songs. Same as #7 but no explicit stuff. (Amazon)

9. Favorite Bollywood. 32 songs. I need to work on this more, I love the upbeat and happy music from India.

10. Favorite Classical. 343 songs. Anytime I find classical music that’s better than the average it goes here. (Amazon)

11. Favorites. 1,254 songs. Similar to Dolby Atmos Radio but with a little higher quality level.

12. Holiday Party. 102 songs. My favorite Christmas/holiday music. (Amazon)

13. Quiet Beauty. 165 songs. Very quiet instrumental music. Great for having in the background while working or reading. (Amazon)

14. Timeless. 275 songs. The music that we can listen to for decades and not get tired of listening to. (Amazon)

15. Unusual. 23 songs. Sometimes I find music that just makes me go “huh?” That goes here.

16. Vibe Alignment. 335 songs. Nice songs to listen to with other people in the room. (Amazon)

The rest is what I call “catalog.” In other words, genres or other things that don’t have editorial input from me. Here I go for completeness, not quality. Usually I try to stay with Apple’s own categorization.

17. African. 33 songs.

18. Alternative. 1,400 songs. (Amazon)

19. Alternative Folk. 14 songs.

20. Blues. 36 songs.

21. Bollywood. 247 songs.

22. Catalog A. (everything I can find that has last name of “A”). 785 songs. (Amazon)

23. Catalog A-M. (Explicit). 1,624 songs. (Amazon)

24. Catalog B-C. 2,408 songs. (Amazon)

25. Catalog D-F. 1,299 songs. (Amazon)

26. Catalog G-J. 2,344 songs. (Amazon)

27. Catalog K. 477 songs. (Amazon)

28. Catalog L-M. 2,439 songs. (Amazon)

29. Catalog N-P. 1,372 songs. (Amazon)

30. Catalog N-Z Explicit. 1,117 songs. (Amazon)

31. Catalog Q-S. 1,832 songs. (Amazon)

32. Catalog T-V. 1,097 songs. (Amazon)

33. Catalog W-Z. 978 songs. (Amazon)

34. Children’s Music. 68 songs.

35. Chinese Pop. 98 songs.

36. Chorale. 429 songs. Choirs and classical. (Amazon)

37. Church. 184 songs. Religious.

38. Classical. 6,182 songs. (Amazon #1. Amazon #2)

39. Coachella 2022. 390 songs. Music in Dolby Atmos from artists performing this year at Coachella, big music festival. (Amazon)

40. Country. 783 songs. (Amazon)

41. Dance. 197 songs. (Amazon)

42. Drinking. 36 songs.

43. Electronic. 408 songs. (Amazon)

44. Funk, R&B, & Soul. 747 songs. (Amazon)

45. Hard Rock. 166 songs. (Amazon)

46. Hip-Hop & Rap. 2,022 songs. (Amazon)

47. Holiday. 647 songs.

48. Jazz. 365 songs. (Amazon)

49. Latino. 463 songs. (Amazon)

50. Meditation. 30 songs.

51. New Dolby Atmos Friday. 357 songs. (Changes every day as new music comes out. I keep music here for about a week).

52. Organ. 55 songs.

53. Pop. 2,626 songs. (Amazon)

54. Reggae. 12 songs.

55. Rock. 1,131 songs. (Amazon)

56. Scary. 13 songs.

57. Singer/Songwriter. 137 songs. (Amazon)

58. Soundtrack. 539 songs. (Amazon)

59. Worldbeat. 87 songs.

New startup mixes reality with computer vision and sets the stage for an entire industry

About 11 years ago I was standing outside in the snow in Munich, Germany with the CTO of a small company, Metaio. He was showing me monsters on the sides of buildings. Apple later bought his company. It got me interested in augmented reality and its uses to make people’s lives more fun and more interesting. The way that first demo happened? The building was turned into an invisible digital twin that the virtual monsters could attach themselves to and move around the building.

Snapchat has, in the past few years, finally brought that tech to consumers with its augmented reality lenses. Things that can turn your world into complete augmented reality scenes, way better than what Metaio showed me 11 years ago.

While SnapChat’s invisible digital twin lets developers do very cool things, it leaves me wanting. Why? So far the Snap platform doesn’t give us many computer vision capabilities. Doesn’t really let us do a whole range of things we want our augmented reality worlds to do (like keep track of your keys).

Today Perceptus, from Singulos Research, gives us an important answer to the future of what humans might do with augmented reality, beyond Snap’s filters, which are really designed to make your selfies much more interesting and it does it with a new kind of computer vision that can catalog physical items in your home or factory.

Augmented Reality has so much more potential into changing EVERYTHING in our homes and factories and Perceptus shows us just how we can use digital twins, computer vision, and AI to make our world better than it was before augmented reality arrives.

CEO Brad Quinton buried the lede when we talked yesterday. Journalism term for hiding an important fact until late in the conversation. About an hour in he let drop “you can play virtual chess without even a chess board.” Then he said you don’t need all the physical pieces, either! The invisible digital twin strikes again.

First, why do I have the credibility to make such a statement? 1. I created the first augmented reality app on Apple’s store on this blog. 2. I wrote major parts of four books about technology. Qualcomm’s head of its augmented reality efforts (Qualcomm makes the inside of Meta’s VR headset), Hugo Swart, tells people my latest “The Infinite Retina” is a must read. 3. Siri was launched in my son’s bedroom. That was the first AI app. 4. I had the first video of a Google self-driving car on YouTube.

I also am not paid for, nor am connected with this company. Although I would work for it for free if asked. Why? This platform has a major impact on the future of things developers will be able to do in our homes.

This does NOT require a headset or glasses. If you watch the video you will see founder Brad Quinton demoing it on a simple iPad. But, of course, this really will rock when we get wearable devices that will enable us to use this kind of augmented reality without holding a device in our hands.

“So, Scoble, what is it?”

It lets you do magic. Just watch the demo in the video I posted or watch the more professional video on Perceptus’ website. It shows the power of digital twins and computer vision in our homes.

Soon devices will augment everything with this kind of computer vision and an invisible digital twin. That’s how Snap’s filters work, they add a digital twin of your face and of the city around you into its database which developers can then manipulate.

Digital twins are like a 3D copy of the real world.

In the demo Quinton shows, the table is a digital twin. It looks like the real table, doesn’t it? But it isn’t. In this case the digital twin is mostly invisible. Perceptus uses this digital twin to keep track of things like chess pieces and Lego pieces on the table top (and a few other things too, but I’m trying to keep it simple here). Think of it as a new kind of database: one that is laid out on top of the real world.

The magic here is that Perceptus quickly makes this digital twin and figures out what is on top of it. Computer vision running inside says “hey, there’s a rook in a chess game” and it keeps track of that rook from then on and, from then on, developers can perform their own magic (for instance a developer might want to change that rook into something crazy, like a Sponge Bob character).

Don’t focus too much on the two examples Brad demos. The game and Lego organizer aren’t the secret sauce here. They are just examples of things that COULD be built on top of this platform. As I walk around the home I see dozens of things developers could do with Perceptus, from making new musical instruments on top of Coke cans to making all our board games far more interactive and interesting. It could be used for non entertainment purposes too, like suggesting recipes, or keeping track of things around your home. All my lights have computers in them and they are hard to control. A developer could use this platform to make that much easier. Or could make our Heinz ketchup bottle play games against the salt shaker next to it.

Some details behind the company?

So far it’s self funded. He says he isn’t raising funds right now but I’ve learned many times that entrepreneurs are always raising funds even when they say they aren’t. The valuation just hasn’t gotten interesting enough yet! (My words, not his).

This isn’t his first company, he cofounded Veridae Systems, Invionics, and Abreezio (acquired by Qualcomm) and has 28 patents to his name.

You can learn more at https://www.perceptusai.com/

More in depth about my thinking

If you have read this far you are really crazy about augmented reality. My kind of people! So, why am I so strongly excited by this company?

Well, we all know Apple is coming with some sort of head mounted display (technical term for something like a headphone that has screens for you to look at). There have been tons of rumors, lots of ideas of what is coming.

I’m not playing that game anymore, except that whatever comes will be the most expensive product launch of all time in any industry. So expectations are extremely high and whatever Apple will do will change the opportunities for developers.

This is a big example of just what kinds of startups are about to come. I’m expecting that over the next 24 months we’ll see hundreds of startups, like this one, created. That has me excited, but even better, I’ve been doing investor research lately and a huge percentage of them are waiting to see what Apple is doing before even considering whether to invest in virtual or augmented reality. I need more time to finish off that research to present real numbers, but it’s already clear after asking 100 investors.

Major companies are already talking to me about augmenting their products in our homes. Tonight I was talking to an employee at Samsung about its appliances and how they would augment them. Then there are consumer products companies like Procter and Gamble. If someone there wants to augment a Tide bottle it’ll need a computer vision platform. Does this work for every use case? The market will decide but it shows me that magic can be brought by developers to EVERY object in our homes. Forks? Yes. Refrigerators? Yes. Board games? Yes. Spices? Yes.

I’m tracking companies that will have a unicorn potential in this augmented reality world. This certainly is one.

Anyway, the next 16 months will be huge for augmented reality and this is just another example why and it demonstrates that machine learning/computer vision and digital twins are about to become much more useful to consumers.