Meta Connect has been the big event of the week, and as it is a tradition of this blog, I’m going to write today for you a super article where you can find inside ALL the news announced during the event! This way, by reading this article, you are sure you have not missed any one of the announcements made there. I truly hope this can be useful for all of you in the XR community…
Just a quick notice before starting. Some years ago, I was used to writing in the Connect roundup article every detail I could find online about the news announced. This year I’m still writing a very long and comprehensive post, but I’m going to skip some details, that anyway you can read in the other articles I’m linking to let you can go deeper on the topics. I’ll do this for two reasons: first of all, there have been so many pieces of news, that if I had to write every detail of all of them, you would need 4 hours to read this post (and ain’t no one got time fo that); and then, I’m currently too busy with my dev work that I don’t have two full-time days to write a post of that kind anymore. I’m sure you will appreciate my choice.
Ok, enough with this boring premise, let’s dig into the Connect!
Meta Quest Pro was what everyone was expecting from this event, and Zuck has not disappointed us with this point. It has especially not disappointed the analyst Brad Lynch, because basically, the device is exactly as it was leaked by him. In fact, for all of us in the XR ecosystem, the announcement was not a surprise at all, because it was exactly how we were expecting it to be.
Let me list here the full specs of the Quest Pro, courtesy of Road To VR:
The specifications are more or less as it was leaked, with two exceptions:
The Quest Pro is not a huge bump over the Quest 2. It’s a step forward, but not a big one, if not in terms of ergonomics and installed sensors. If you look at the specs above, it even looks quite underwhelming: the resolution and battery duration are for instance inferior to the one of Pico 4, which costs one-third of the price of the Pro. Headsets anyway have not to be just evaluated on paper: Quest Pro introduces a lot of innovations and polishes, which all together create a headset that is for sure not revolutionary, but a noticeable step forward from the Quest 2.
Talking about visuals, the display resolution is 1800 × 1920 per eye, which is even less than the one of the Quest 2, but here there are two displays, one per each eye, so there are fewer “wasted pixels”. Plus the lenses are not Fresnel ones, but pancake lenses studied to give you a bigger eye box. This means that on the Quest Pro you can actually move your eyes and read a text in a normal way, by just scrolling the text with your eyes without having to rotate your head. The clarity is much more on this device, with “more pixels” concentrated into the center of vision, that is exactly where your eyes perceive better. This has been reported by all the reviewers. According to Meta, Pro has a 25% improvement in sharpness at the center of the field-of-view, and a 50% improvement across the periphery. The field of view is just a bit wider than the one of Quest 2, but enough so that it is noticeable. Meta also claims Quest Pro includes a 500-element (Mini-LEDs) local dimming backlight which enables improved contrast up to 75% compared to Quest 2, while also allowing for a 1.3 times larger range of color. This means that blacks are “more black” than in a standard LCD display, but still not as black as in a OLED one. Refresh rate tops at 90hz, and it is unlikely it will go at 120 after launch.
The chipset is the new Qualcomm Snapdragon XR2+, which moves slightly the RAM on the chip: XR2 has the RAM stacked on top of the chip, whereas XR2+ has the RAM to the side to enable better thermal dissipation and thus higher sustained performance. It offers 50% more performance than its predecessor… but most of this performance boost is used by the additional sensors this device mounts, so very little arrives to the final application. So, don’t expect a huge bump of performance of your application on this device.
As for the comfort, the device is lightweight (722g), compact, and comfortable, with the battery on the back which guarantees a good balancing of the weight . Pancake lenses help in making it smaller and lighter than Quest 2. The only disadvantage of the Pro reported by the reviewers is that it uses a halo design for the headband for which is usually difficult to find a perfect fit on the head. It’s good news by the way that there is continuous IPD adjustment (and with a wider range than before) and that thanks to eye tracking sensors, the headset can tell you if you have positioned it well on your head. There is also an eye-relief dial, so you can put the lenses very close to your eyes, for added field of view.
Talking about eye tracking, there is another feature that it brings to the table: dynamic foveated rendering, so that the headset can render in full resolution only the part of the screen you are looking at. I was pretty excited about this, but John Carmack clarified that it doesn’t bring a massive performance boost with regard to the fixed foveated rendering of Quest 2 (less than 2x, which is anyway good, but not impressive). Anyway, at least it makes the presence of foveated rendering less visible: with FFR, you can see the visuals at the border of the lenses becoming jagged, while here, wherever you look at, you see that part of the scren in full resolution.
Audio still works via integrated speakers. It is a bit better than on Quest 2: it should provide slightly better basses (half-octave deeper bass tones) and improved mid-tones. Since the speakers are more directional, they reduce sound leakage by 10 decibels and people outside you hear less what you are hearing in VR. Quest Pro has a 3 microphone array, 1 more than other standalone headsets. Meta claims this enables 20% better noise canceling and “advanced beamforming” compared to Quest 2.
The Pro will use the same store of the standard Quest. Air Link and tethered Link will still be provided on it, so that you can also enjoy PCVR content. Since Quest Pro has 2x higher Wi-fi bandwidth, Air Link should have much better performances.
The real bummer is that the battery lasts just 1-2 hours, which is a quite ridiculous time, especially if you aim this headset at the enterprise market.
As you can read, until now we are talking about a device that is slightly better than Quest 2 in all areas. But there are some features in which the The Quest Pro makes a true jump forward:
Face and eye tracking introduce obvious privacy concerns. Meta has clarified that the images of the tracking cameras never leave the device and that the user is free to enable and disable these sensors whenever he wants, so it should be safe privacy-wise. BUT if you read the privacy policies of these two services, you can discover that Meta can send some of your eye tracking data to Meta servers, and while this may just be to guarantee the correct visualization of expressions on your avatar face in multiplayer experiences, it opens a backdoor to Meta to get some precious data about our eyes. I think we need better clarifications on this side.
The battery is the true real concern of Quest Pro: while the battery of the controllers has a decent duration (around 4-5 hours, with a maximum of 8), the headset lasts for just 1-2 hours. John Carmack has said that if you use the Quest Pro without using all its new features (i.e. the new sensors) it can last even more than the Quest 2, but since if you bought a Pro, it is to use its special features, it is most probable that you will have less than 2 hours of usage.
This is ridiculous, not only because the Pro costs a lot of money, but also because it is sold as a “work device”. I don’t know how things work where Zuck lives, but here in Italy, a typical workday is 8 hours, not 2. If at Meta they just work 2 hours, I want to be hired there immediately. Now I understand why they take so long to review applications on App Lab…
Jokes apart, the duration is a bit small, so I envision that 3rd-parties power banks are going to be announced soon for this device.
In the meantime, the company advises recharging the device often by leaving it on the dedicated charging dock which is provided together with the whole Quest Pro set.
The hands-on reviews have highlighted more or less what we could imagine. The device is very good, but not a huge jump forward if compared with Quest 2. Reviewers were usually impressed by the wide eye box and the clarity of the visuals even at resolution parity with Quest 2. Even the peripheral part of the screen has increased clarity if compared to similar headsets. Anyway, Ben Lang said that the text on the device is readable but still not perfectly readable.
Also, the controllers were reported to be great. Ben Lang said that the handles feel comfortable like the ones of the Rift CV1 controllers (which are the most ergonomic controllers I have ever tried), and David Heaney of Upload VR reported the haptics to be so well defined that they were be even better than the ones on PSVR 2!
Quest Pro costs $1500/€1800 (we have VAT in Europe). It can be pre-ordered on the Meta website and will ship from October, 25th in all the 22 countries where Quest 2 is already shipping (sorry, still no Germany, then). After that date, it will be possible also to buy it in dedicated physical stores in some countries.
Meta is also offering demos of the device in physical stores to let people try the headset before buying it.
You can also buy various (overpriced) accessories for it. Let me copy paste what they are:
A thing that made everyone get crazy is that the Quest ships with a partial blinder, and to buy the full one, that covers all your field of view, you need to spend $50.
The Touch Pro controllers can also be bought separately: they are compatible with the Quest 2, and you can buy them for $300 (so it is more or less like buying a new Quest 2). Prices of everything related to this device are very high: like all prosumers/enterprise gadgets, everything is sold at a premium. Do you remember when everyone was mocking HTC for its super-high prices and people were praising Meta for its cheap costs? How are the turntables… now the Focus 3 is cheaper than the Quest Pro!
Zuck has launched the Pro as a secondary line of business. If the Quest is the device for consumers, the Quest Pro is the device for prosumers and companies. The Quest Pro is not sold at loss, even if Zuck clarified that if you consider all the R&D that there is behind it, it is sold at loss too, even if it costs so much (remember that Meta spends $10B/year for its metaverse efforts…)
This is not a device for gamers. It is not a device for consumers. So if you just want to play VR games, you had better buy a Quest 2 or a Pico 4. This is a device for those who work with VR, a device for tech enthusiasts that wants the latest gadget, or a device for those who have a company to which the Quest Pro can make spare money. In all the other cases, you should not buy it: it has no sense spending $1500 to then play Gorilla Tag (also, playing Gorilla Tag, you would destroy the expensive controllers in a second).
If you need it for work, I think it is worth a shot. Not only the device is very well designed and polished, but Meta is building around it a very complete ecosystem (as you will see in the next paragraphs). It is the same that Meta did on the consumers’ side with Quest 1. The first Quest was not that superior to the Vive Focus Plus, but the ecosystem it built around it over time (e.g. all the games, the integration with an external keyboard, the passthrough, the runtime updates, etc…), and the level of polish it offered, was much superior. And I’m sure Meta is going to do the same with Quest Pro: I bet that one year from now, the Quest Pro will be much more valuable than it is now because there will be more features and a richer ecosystem.
Unless Apple kills it, of course…
Meta has worked hard behind the curtains to create a valuable ecosystem for the Quest Pro. In particular, the company has identified “productivity” as the third vertical where to push virtual reality, after gaming and fitness. The Quest Pro fits very well this strategy since it is a headset made for professionals. Productivity for Meta means in particular making the job done, that is giving the tools to professionals to do work on the headset. In the long run, this means substituting your computer with the VR headset or integrating your computer with a VR headset (like it happens in Horizon Workrooms).
Meta has prepared a lot of announcements in this sense, and some of them have been incredibly relevant.
Horizon Workrooms is the enterprise social VR space created by Meta, which de facto aims to emulate in VR the life of a real office. At Meta Connect many pieces of news have been announced about it.
Let me copy-paste the press release about them:
What I’m most excited about is the solo room with three virtual screens. It means that you will be able to enter Horizon Workrooms alone, connect your headset to your computer, and then start working with three big virtual headsets in front of you. Think about it: maybe like me you like to work in mobility, so you just have a laptop with a small screen. But now, connecting it to the Quest Pro, you can have your laptop with three huge virtual screens to use, to do your work better. I would love that because sometimes I truly feel the need to have more screen space. So this will be for sure one of the first things I will test with my Quest Pro. Unluckily the battery time is a true bummer, but if it can give me three big displays and it is actually comfortable in my head, I could think completing some tasks in mixed reality every day. I’m a bit skeptical, but let’s see…
Meta has partnered with other two tech giants: Adobe and Autodesk, to bring their professional tools to Quest Pro.
Autodesk is updating its collaborative design review app to take advantage of the new possibilities unlocked by Meta Quest Pro. This should give architects and designers a new way of reviewing 3D models immersively.
For what concerns Adobe instead, next year, Adobe’s Substance 3D apps for professional 3D creators, designers, and artists are coming to Meta Quest Pro and Meta Quest 2, so anyone can model 3D objects and join collaborative reviews inside VR. And Adobe will bring Adobe Acrobat to the Meta Quest Store.
I guess that Meta knows that most creatives (e.g. video editors) use Apple computers, and for sure Apple will launch its device with a lot of functionalities for prosumers and professionals… so it has established partnerships to announce a similar type of content before Apple itself, to beat it in time.
There are many other productivity applications on Quest, and according to Zuck, even big companies are already using them. For instance, Logitech is using ShapeXR to evaluate its designs; Puma used GravitySketch; Novartis is using Nanome to better visualize the molecules of its products.
Many of these applications are coming to Quest Pro: for instance, my friends of ShapeXR already prepared an update with a bunch of features that are ideal for the Quest Pro. Also Arkio has released an update for the Pro.
Meta has announced some very relevant initiatives to make it tackle better the B2B ecosystem.
The biggest surprise of the day has been seeing Satya Nadella and Mark Zuckerberg on the same stage together to announce a partnership. This was totally unexpected, and in my opinion was huge, because they are the heads of two of the biggest tech companies in the world, and together they can make a lot of things.
Meta and Microsoft worked together to integrate Quest Pro into the Microsoft ecosystem. This means they will release:
The result of all of this is great both for companies and individuals. For company is great that the Quest Pro is integrated in the frameworks they already use: for instance, they can manage the Quest Pro of their employees with the Mobile Device Management solution they were already using. As an individual, you can be happy that you will be able to do on Quest Pro some useful work, like editing a presentation in Powerpoint, maybe on a big virtual screen. Even more relevant, you will be able to stream Windows 11 via cloud rendering, and so use your Quest as a PC for real!
Microsoft 365 is coming to Quest Pro!#virtualreality #vr #metaconnect #metaquestpro pic.twitter.com/pE16Rc1PPn
It’s funny that when talking about the “killer app of VR” many people said that for VR to become mainstream, we needed “the Word of VR” and “the Powerpoint of VR” and now Meta is bringing Word and Powerpoint to VR. This is surely great, but honestly, this is not what we were looking for: we were (and are) looking for useful tools that are made from the ground up for VR. Honestly, I feel much better using Word on my laptop without something on my head than on whatever Quest. But I appreciate the willingness on enriching the productivity ecosystem.
Even more relevant than the actual software brought by Microsoft, I think that it is important to understand what is the true value of this partnership. Microsoft is very strong in the enterprise world, and in fact, many companies (mine included) use Microsoft Teams and Microsoft solutions in general. Meta partnering with Microsoft means that Microsoft can open many doors for Meta to enter into the enterprise world with a credible partner.
And actually… the credible partners are two. On the virtual stage of the event, we have also been able to see Julie Sweet, the CEO of Accenture. She announced that Accenture will be working alongside Meta and Microsoft in the coming years to help companies use VR to transform the way they engage employees, interact with customers, or create products and services in the metaverse. Long story short, the three companies will work together to bring bespoke solutions to big corporates that need XR integrations. Most probably Meta is bringing the hardware, Microsoft the platform (e.g. Azure), and Accenture the development workforce for the customizations. Together they can offer a lot to corporates.
This strategic partnership is very important and I‘m sure it will make Meta perform huge steps forward in the enterprise world, where until now companies like Pico and HTC seemed to have the leadership. At least, this will happen for what concerns collaborating with huge enterprises.
Accenture in its keynote also said an interesting thing: it has already onboarded 150,000 employees with virtual reality, and it has noticed that its employees retain +33% of the information when they are in virtual reality. You can use this stat in your next Powerpoint presentation about the “metaverse”.
One of my predictions for Meta Connect was that Meta was going to define better its enterprise strategy. And this is exactly what happened: contextually to the definition of these important partnerships, it has announced the revival of a business program for its hardware.
So next year, Meta will launch Meta Quest for Business, a subscription bundle for Meta Quest Pro and Meta Quest 2 that includes essential admin features like device and application management, premium support, and more. This is the ideal solution for big corporates that want to integrate Quest devices in their production processes and need dedicated services to relate to that (e.g. dedicated service if one of the products breaks).
Quest 3 was the great absent of the show. I was expecting a mention to it, so that to entice people not to buy the Pico 4, but Zuckerberg didn’t even hinted at it. I guess that it was made on purpose so that to focus the keynote on the launch of Quest Pro.
Anyway, soon after the keynote, in an interview, Zuck officially spoke about the Quest 3, hinting to the fact that it won’t be released this year and that its price range will be in the $300-500 interval. This means that Meta is still very interested in tackling the consumer sector… a good news for everyone.
Zuck and Boz have talked about the Quest 2, defining it as the first headset to become mainstream. After that, they mentioned the Quest Store and released a few interesting statistics about it.
Meta revealed that of the over 400 apps in the Quest Store, roughly 1 in 3 are making revenue in the millions. Even more, 55 games made more than $5 million in gross revenues, and 33 titles made over $10 million.
These are all great numbers, and kudos to Meta for having created such a profitable ecosystem. BUT never forget the risk of falling for survivorship bias: these titles are making millions, but what about the ones we are not talking about? If 1 out of 3 on the official store makes $1M, it means that 2 out of 3 are not making $1M. If making a VR application costs hundreds of thousands of dollars, it means that 66% of applications are probably not making enough money to make their studio scale. And we are talking about the official Quest Store. Now imagine how the situation is even worse on the B-league store App Lab!
This is not to say that the Quest Store isn’t good (it is VERY good), but to show you that being an indie VR developer, and trying to survive by just making VR games, is still terribly hard.
Zuck has said that social platforms are the ones that are most successful in the store, and that data analysis shows that the majority of time is spent in general by people in multiplayer experiences. This is totally in line with Meta’s philosophy of “defying distance” and making people meet together thanks to the power of XR.
This is for sure good, but it has also a downside: John Carmack has stated that one of the reasons why Meta had to rise the price of the Quest 2 from $299 to $399 is that there were too many people using the Quest 2 just to use free social VR apps like VRChat or Rec Room. The Quest 2 is sold under cost, and Meta expects to gain back part of the money via software sales… but if people just use free software this system doesn’t work. So Meta had to raise the price of the Quest to make the financials work again. As JustDaven made me notice, Rec Room replied to this saying that actually, RecRoom pays a 30% cut on all the In-App Purchases to Meta, so it actually brings a lot revenues, and Carmack replied that it is true, but his argument is still true for the other free apps.
Sorry, you are right — those old arguments against free apps don’t apply to IAP apps now, but do still come up with non-monetized apps and media.
Slowly, Horizon Home is truly becoming the personal social hub that Meta has envisioned for a long time. After giving you the possibility of meeting your friends in your home and watching 360 videos with them, Meta is now unlocking the possibility for a group of friends to watch together videos that are on Youtube VR. Youtube VR has proven to be the place to go to find interesting 2D and 360 videos to enjoy on a headset, and Meta has recognized this, so it is partnering with Google to bring it inside Horizon Home in a social way.
I still think it’s getting confusing to have a personal social home space both on Horizon Worlds and Horizon Home… but I guess Meta is trying different strategies to see what sticks better.
Meta has announced very important updates for Horizon Worlds, its main social hub.
First of all, Horizon Worlds is coming to the web. Entering it won’t require a mandatory VR headset anymore, and it will be possible to enter a room by just opening a link. This is going to reduce a lot of the friction in its usage.
The result of all of this is that when these updates will roll out, it will be possible to create visually stunning worlds with quite complex logic in Horizon Worlds. It may switch from a toy tool to something where it is possible to create valuable content. I’ll re-evaluate it for the virtual concerts that we do at Vrroom in a few months from now for sure.
The fourth and final thing that is relevant is that it will be possible to share videos shot in Horizon Worlds as reels on Instagram. Meta says that there will be always more connections between the social VR worlds and the traditional social, and this is just the start. And Meta is not even the first company in doing this: when I interviewed Nikk Mitchell, he told me that it is possible to share content from Pico Video directly to TikTok in China. I think this communication between the digital and the virtual world will be one of the trends of the upcoming months, that will mark the transition between the Internet and the M-word.
Meta has announced that new customizations are coming to its avatars so that to make them even more inclusive (e.g. there will be wheelchairs and cochlear implants, plus various shapes of the body). Then it announced even more uses for the avatars, like that you will be able to do video calls using your avatar instead of your camera feed (if you want other people to think you are cringe).
Then Zuck told that he wanted to tease us with something special, and so he entered a virtual world where he became his avatar and with him there was another Meta person. His avatar had new graphics, it was not the usual Meta avatar: it was more realistic, having better materials and texture. I found it super cringy, but many people online told me that they actually liked that.
Oh no, cringe! pic.twitter.com/R0Glp1WgSg
And… surprise surprise, the avatars had legs! Zuck and his colleague started doing things like jumping and touching knees with their hands, and I was genuinely astonished by the full body tracking achieved with such high quality by just using a standalone headset and its controllers. Then, a few days after the event, Meta clarified that that section of the event had been recorded with a standard mocap tool and was just made “to convey the idea”. I was incredibly disappointed to discover that, to be honest. It turns out that yes, there will be some sort of body tracking, but it is the same as shown in an old video, which infers some sort of “believable pose” for your body and let only the other people see that, while you don’t see your own legs. Lies, deception.
Anyway, an Avatar SDK will be released next year, so that you can integrate the full body feature in your applications. In the meantime, Meta will integrate legs first of all in Horizon Worlds, and then in other apps next year.
On the plus side, Meta is going to open up an official Store for Meta Avatars, so that to trigger the creators’ economy about them. And many special avatars made in partnership with Netflix and other brands will be released to promote these brands and their products. I can’t wait to see the avatar customizations taht will be created by the community.
Quest 2 is still prominently a gaming device, so every one of us was expecting great news on the gaming side. We had quite a few announcements:
I think that more than what is being announced, it is relevant to what has NOT been announced. We had no info about GTA VR or Assassin’s Creed VR, and not even a sneak peek of a new upcoming great game. The gaming part of the conference was very underwhelming. Probably this has been done on purpose: being the great day of the Quest Pro and the productivity sector, entertainment has been put a bit in the background.
Meta has announced it has acquired three gaming studios:
These companies will join Oculus Studio and will work on still-unannounced projects. On one side it is good seeing some startups acquired in the field (it gives Venture Capitalists faith in VR), but on the other side, if all the best developers get acquired by the major hardware manufacturers, this ecosystem risks becoming too fragmented. I’m not a big fan of exclusive titles, you know.
I’m also wondering if the FTC will be happy about this…
Meta acquires 3 #virtualreality game studios. The FTC: pic.twitter.com/nrgCvNUVzA
The partnership with Microsoft resulted also in a new feature for the consumer: soon, it will be possible to use Xbox Cloud Gaming on Quest and Quest Pro. Users will so be able to play their favorite Xbox games with their Xbox controller on a big screen in virtual reality!
This is a piece of great news for the Quest ecosystem (which becomes even richer), but I know some indie VR developers are not happy because they think that now their games on Quest have not only to compete with the high-budget VR games (like Resident Evil 4 VR) but also with 2D Xbox Games.
Of course, Zuck is expecting many upcoming games to exploit the high-quality mixed reality of the Pro. There are already some interesting projects in the work:
The future is now – Meta Quest 2 and Quest Pro players will soon be able to play Demeo Passthrough! 🤩 pic.twitter.com/AHaYOltpOz
I think that it will be important for game studios to understand if mixed reality is relevant to their game or not. Beat Saber is cool in VR because you have all those effects around you, while the video of the mixed-reality mode looked a bit lame. Demeo, instead, since it is a tabletop game, fits very well with the AR mode, where you can play the game on your real table. Mixed reality doesn’t fit all games: it is not just that you remove the environment and put the passthrough in its place and you have made a mixed reality game. MR requires a special design. I think that over time studios will understand how to use it properly.
Meta has just started a multi-year partnership with NBC Universal. As a result of that, the video streaming app Peacock is coming to Quest 2 and Quest Pro.
And starting next year, Meta and NBCU will co-create experiences around The Office, Universal Monsters, DreamWorks, Blumhouse, Halloween Horror Nights, and other iconic IPs.
I’m very intrigued by this, but I also wonder how it will be implemented: I mean, it’s the dream of every one of us to enter into “The Office”, but only if it is made in the right way and it doesn’t result just into a cash grab.
Fitness remains a sector where Meta wants to keep investing money. Meta has announced that it is going to release a fitness pack for Quest 2 (with elastic handles for the controllers and a waterproof facemask) on October, 25th. It is also going to partner with other companies to let them produce dedicated accessories for VR fitness.
Meta is also going to open soon its fitness APIs, at first to dedicated partners, and then to all developers, so it will be possible to create applications that share real-time data, compute custom stats, and share your fitness data with your friends so that to motivate each other thanks to the power of gamification. This is good for teams that are making fitness games like us of HitMotion: Reloaded (play our game, by the way 😛 )
On the software side, Supernatural is going to release a new incredible feature: knee tracking, which you can see in the GIF here below.
Considering that it is using the plain Quest, I have no idea how this is possible, but I’m very intrigued…
Meta also announced a few interesting features for us developers:
We expected some news on Project Nazare, but we have been all disappointed. Zuck has just said that AR glasses are the final goal and the company is working on all the technology stack to make them happen.
He also dedicated a brief part of his video to the Rayban Stories smartglasses made in partnership with Exilor-Luxottica. Meta and Luxottica announced that they are working on the next version of the smartglasses, and confirmed that their partnership is going to last for many years.
The current version of Rayban Stories is going to get more customization options. On the software side, soon will come the possibility of making phone calls, writing text messages, and also using Spotify Tap, which is Spotify with a special interface that relies on the tap on the glasses frames.
Like last year, Zuck has had a brief visit to Michael Abrash, to talk about the R&D that is happening at Meta.
The first topic has been the EMG (Electromyography) wristband that Meta is building as the next-generation input for mixed reality. Abrash reiterated some concepts already expressed in the past. The EMG bracelet reads the input that your brain is sending to your hands and learns what happens on your nerves when you want to send an impulse to your hand (e.g. to do the pinch gesture). After this learning phase, it is able to detect your willing of doing that gesture even if you just think about doing it, but you are not performing it with your fingers. This magic is possible because your brain sends anyway some signals to the hand even before sending the actual order of moving the hand, and the bracelet can detect this “preorder”. You can see this in action in this video below from CTRL+Labs, the startup acquired by Meta to create this magical wristband.
Abrash said that it is important that this wristband can adapt to the wearer because every one of us does the gestures in a different way: my thumbs-up gesture is for sure different from yours, for instance. He called this feature of adapting the particular gestures of every user “coadaptation”.
Then they showcased the video of a prototype of the system in action on some prototype AR glasses, and you can see it here below:
Upload VR has been able to visit Meta R&D labs and have even better insights on this bracelet. It seems that Meta sees this as the next-generation input for XR (something like the mouse of XR), and it is working on two sides related to it. One, the most immediate, is to use it for “activation”. That is, the AR glasses thanks to AI can learn your behavior, and then analyze what you have around you, and so can predict what you want to do, and show you just that option, to which you just have to “click” yes with the thought of moving your hand. The goal here is not to have many complicated menus, but just one single option that is already the correct one. So, Zuck wants to remove the friction that you have now, when for instance if you receive a message, you have to take out your phone from the pocket, unlock it, then answer the message, and then put it again in the pocket. He hopes that in the future, an AI system can directly put the message on the AR glasses in front of your eyes, with a suggestion to answer something, and you have just to tap to accept the suggested answer.
The second feature is “bandwidth”, which is using this bracelet to convey more complex data than just yes/no. Here the clear use case is a virtual keyboard: you just think about what you want to write and can write it. Considering how it is difficult to write in VR, that would simply be amazing. But typing is just one of the use cases, there are many others.
Zuck believes that in the next 5 years, these wristbands could surpass a standard keyboard in terms of throughput. It seems that some technicians inside Meta already can write well using them.
There have not been huge updates on Project Aria, but Meta showed that they created a mobile app that is able to do environment understanding and tracking in a good enough way, that they can help visually impaired people in navigating inside indoor environments. The reconstruction of the map of the Pittsburgh airport where the application is usable has been performed by people wearing Project Aria smartglasses. This proves that with Project Aria, Meta can reconstruct the full mesh of whole environments (hence can build its AR Cloud).
Since there is a lot of talking about Apple and its use of Neural Radiant Fields (NeRF), Meta wanted absolutely to show it can do them itself, too.
Zuck and Abrash showcased the technology they have internally to scan 3D objects and bring them to virtual and mixed reality. They talked about two methods:
Meta gave us an update on the status of the realistic avatars it is working on, both in the conference and by letting journalists (like the ones of Upload) enter its R&D lab.
Meta has currently three interesting technologies related to realistic avatars:
Now Meta has two challenges in front of it: first of all, try to unify all these three types of avatars to obtain ultra-realistic full-body avatars that can be obtained with commercial hardware. And then optimize them so that they can be enjoyed on an off-the-shelf headset. It’s not easy, but I truly hope they can deliver because the result would be having true telepresence, the ability to feel that a person that is very distant away from us is actually very close to us.
We all know that Meta, like many other tech giants, is undergoing a spending review because of the adverse economical conditions. For this reason, the management has identified four main research areas where to put effort for defining the future of XR: the work on this areas will go full steam ahead, while on the other ones, the budget will be cut considerably, or they will be removed altogether.
The four main areas will be:
John Carmack has given his usual very honest unscripted talk… and as usual he has made some statements that for sure have not made Meta executives happy. For instance he said that he usually keeps eye tracking disabled, he is not a big advocate for mixed reality, and that he is not “much a Horizon guy”. So basically he doesn’t care about everything it has been announced lol.
He also said that the open casing of Quest Pro is not disturbing much his experience with the device. He said a lot of other interesting info which I already added to the reminder of the article.
Here below you can find the video of the whole Carmack’s unscripted talk:
You can read a very long and interesting interview with Mark Zuckberg performed by Alex Heath of The Verge at this link: https://www.theverge.com/23397187/mark-zuckerberg-quest-pro-metaverse-interview-decoder
You can watch the full integral keynote of Meta Connect 2022 here:
Of course, there have been some memes made about this Connect, so let me share a few of them with you:
I made some memes myself, and was even appreciated by Meta CTO!
Zuck closed the event by talking about the two foundational values of Meta.
The first one is that “people are at center” of Meta’s business. Meta’s purpose is to connect people, and with XR technology there is an immense potential for taking people’s connections to the next level. Zuck always mentioned that making people meet in virtual worlds has been a dream of his since when he was very young, and this may explain his “obsession” with the metaverse.
Then he added that the second foundational value of Meta is “openness” and that Meta is doing everything to promote an open metaverse. He clearly opposed this to “other closed approaches”: he said that the history of computing has always been about open vs closed ecosystems. At first, it was Windows vs Mac, and then Android vs iOS. He never said that explicitly, but he clearly hinted that the upcoming battle will be Meta vs Apple and that Meta is the white knight which will promote openness in the metaverse. The partnerships with Microsoft and other companies should be proof of that according to Zuck.
He even finished his speech by saying “Our role is that the open ecosystem wins out in the next generation of the internet”.
That is very inspiring. BUT I should remind you all that the Quest Store is a closed platform, that the Quest is not rootable, that sideloaded applications on Quest are treated as dangerous stuff, that until a few months ago you needed a Facebook account to use the Quest, that Meta makes both hardware and software of its headsets exactly like Apple, that Meta takes the best ideas of the apps on the store and implements them itself, that as a developer you can’t access all camera streams, etc… The Quest is NOT an open ecosystem. It is a “not closed” ecosystem, which is completely different. Apple will surely be worse, but it is not possible to compare the Quest Runtime to Android, which is completely open source, or even to the (Windows) PC ecosystem, which is totally open to every kind of customization. With much respect for what Meta has done for the development of the XR ecosystem… if Meta is our champion for promoting openness in the metaverse, then we are completely fucked.
And that’s it for this huge post! I hope it has been useful for you. If you like these detailed articles of mine, subscribe to my newsletter not to miss any of the future ones! And please, since I spent a lot of time to write this post, would you mind helping me by sharing it around in the VR community? This way it can inform also all your VR peers! Thank you!
See you at the next Connect!
AR/VR developer, startupper, zombie killer. Sometimes I pretend I can blog, but actually I've no idea what I'm doing. I tried to change the world with my startup Immotionar, offering super-awesome full body virtual reality, but now the dream is over. But I'm not giving up: I've started an AR/VR agency called New Technology Walkers with which help you in realizing your XR dreams with our consultancies (Contact us if you need a project done!)
After I published my detailed article about the launch of the Pico 4 in Europe, many developers reached out to me to understand how to apply for the announced $12M fund for developers, dubbed Developer Incentive Program. I reached out to Pico to have further explanations, so now I can tell you! Pico Developer Incentive […]
Meta Connect keynote has just finished: for 90 minutes, Mark Zuckerberg and Andrew Bosworth guided us in discovering the upcoming products and partnerships that Meta is developing on its road to the Metaverse. Like every year, in a few days I will write a super big roundup of the event with EVERYTHING announced (you can […]
Donate for The Ghost Howls on Patreon! Scientific studies certify that if you donate for this blog on Patreon, your happiness will increase by 55%, your productivity by 32% and the average chance of meeting a Panda in your city by 77%. Never say no to Pandas, donate even $1 a month for this blog to keep it alive!
This blog express my personal opinions about various stuff. None of the content of this website express the opinion of the company or the groups I belong.