Stacking the Optical Deck: Introducing Infinite Display + a Primer on Measuring Visual Quality in VR

When it comes to VR, seeing is believing. Sound and haptics both play an important role, but for most people, it’s the visuals that seal the deal. And beautifully rendered low-poly art can be just as breathtaking as a hyper-realistic environment. There’s just nothing quite like being fully enveloped in a visually stunning virtual world, regardless of whether it’s steeped in realism, completely abstracted, or somewhere in between.

But VR’s still an emerging technology, and there isn’t a lot of shared understanding of what goes into creating that visual experience and the complex technologies that must come together to produce what you see inside the headset. There’s also some inconsistency across the industry in the way visual quality in VR is being defined today.

When thinking about visual quality, many people apply the same principles and standards that we traditionally use to think about 2D displays, like mobile phones and TVs. But VR is vastly different from these other consumer devices. Instead of looking at a large screen at a farther distance, VR users are looking at a smaller screen, much closer to their eyes and magnified by a set of lenses within an optical stack. It’s like looking at a TV through a camera lens—what you’ll see isn’t just determined by the resolution of the screen, but also by the optical properties of the lens, like magnification and sharpness.

Also unlike TVs and mobile phones, the display essentially follows you as you move your head—and because it’s so close to your eyes, which can gaze across a large range, the optics must provide a consistent geometrical and optical performance across those viewing angles to provide a quality visual experience. This points to the increasingly important role that eye tracking technology will play when it comes to visual performance and experience in VR in the future.

We believe the industry needs to align on VR-specific performance metrics that better show the full picture. Rather than focusing on display resolution alone, we must consider how display and optics work together as a system in a VR headset. Visual clarity in VR is determined by a number of factors, the most important of which are system resolution (measured in pixels per degree, or PPD), sharpness (measured by a lens’s modulation transfer function, or MTF), contrast ratio, and color. In this post, we’ll take a closer look at all four and how they relate to Meta Quest Pro’s newly introduced Infinite Display optical stack.

System Resolution

It’s tempting to focus solely on display resolution when thinking of VR headset quality, since that’s the metric by which we tend to judge 2D displays. But instead, we should evaluate the full optical system’s resolution, which is measured in PPD—a combined metric that takes into account the display and optics working together. An angular measurement, PPD measures the number of pixels that are packed within 1° of the field of view (FOV). The higher the PPD, the better the system resolution of the VR headset.

We look closely at PPD in the design and manufacturing of our VR headsets to ensure a quality experience. And thanks to the higher pixel density of Meta Quest Pro’s display and its subsequent magnification by the headset’s pancake optics, we were able to increase overall system resolution for Meta Quest Pro (22 PPD) by 10% compared to Meta Quest 2 (20 PPD). While it’s not the highest PPD of any VR headset on the market today, this is an important milestone in launching high-quality VR and mixed reality-enabled headsets at scale.

And we’re investing to push this even further. Our Display Systems Research team at Reality Labs Research recently unveiled its Butterscotch prototype, which limits the field of view to about half that of Meta Quest 2 and uses 3K LCD panels with a new type of hybrid lens to achieve an overall system resolution of 55 PPD—approaching retinal resolution (generally considered to be around 60 PPD, which is sufficient to depict the 20/20 line on an eye chart) and potentially paving the path to a new industry standard. While Butterscotch is a research prototype with a technical configuration that may not make it into a future product, we’re developing high-density displays and high-resolution optical systems in tandem to achieve retinal resolution on our product roadmap.

Sharpness

While PPD sets a baseline for overall system resolution, taking into account the components of the optical system, there’s another crucial metric that doesn’t get enough attention in today’s discussions about VR visual clarity: sharpness.

Measured by MTF, which combines resolution and contrast, sharpness determines how much detail can be reproduced by an optical system. The higher the MTF, the finer the image detail. Because VR displays are viewed through and thus magnified by a lens, our team must optimize the design and fabrication of our optics to avoid artifacts caused by the lens. It’s also important to acknowledge that, with a flat 2D display like a TV, sharpness is uniform. In VR, it’s not—you have varying degrees of sharpness at the center and edge of the lens. The goal, then, is to achieve a higher MTF at both the center and periphery.

On Meta Quest Pro, center sharpness is 0.98 and edge sharpness is 0.85*—a 25% and 50% improvement over Meta Quest 2, respectively.

Contrast

The contrast ratio is the ratio between the luminance of the brightest white and the darkest black that a display optical system can produce. The higher the ratio, the better the contrast.

To maximize contrast, Meta Quest Pro’s high pixel density display is equipped with specialized local dimming technology that can control over 500 individual LED blocks independently. This results in 75% more contrast, providing deeper blacks and a much enhanced viewing experience.

Additionally, Meta Quest Pro’s pancake optics use a unique design of three polarization-film stacks—one on the display and one on each of the two lens elements—to minimize ghost images, especially from the mid-periphery to the edge of the FOV. This also contributes to its overall improved contrast.

Color

Finally, the quality of a VR headset’s color is measured by color gamut and color accuracy. Color gamut is the range of colors within a spectrum that can be reproduced by the overall system. Color accuracy refers to the visual system’s ability to reproduce colors and shades as intended. Together, they’re vital to provide a lifelike immersive experience.

Meta Quest Pro has a wide DCI-P3 color gamut, which is 1.3 times that of Meta Quest 2. We’ve also developed advanced color calibration technology in Meta Quest Pro to ensure the color accuracy and brightness disparity across two eyes. With rich and vibrant colors, Meta Quest Pro is capable of providing premium visual clarity across the full field of view.

Shortening the Optical Stack

Meta Quest Pro features a two-element lens and 10+ layers of functional optical films. The original concept was developed by Reality Labs Research over the course of about two years, and it represents a highly successful hardware tech transfer from research to product in VR. However, there was more work to be done to design pancake lenses and polarization optics that would work in product—a process that took roughly four years, including more than a year during which the product and research teams worked as a single integrated team to ensure a successful and efficient transfer.

While our research teams are able to build time machines—prototypes that show what might be possible five to 10 years or more into the future, without regard for constraints like cost, weight, and more—our product teams must deliver a viable design that can be produced at scale. Far from off-the-shelf products, the product team designed truly unique lenses and polarization stacks. While pancake lenses and polarization optics are in themselves nothing new, their application for VR poses unique challenges in productization at scale. There was no manufacturer that made the parts, which required incredibly precise lens and polarization film fabrication and integration, so our product teams had to work closely with the industry over the course of years to create the supply chain from scratch and advance their state-of-art manufacturing capabilities. We were among the first to work with tier 1 industrial partners to create a shortened optical stack and pancake optics like Meta Quest Pro’s—and we did it at scale.

All told, Meta currently holds 350+ patents related to optical technologies, including those focused on minimizing size while optimizing visual performance.

Adding It All Up

When you combine all this innovation and technology into a single consumer device, you get a VR headset with a visual system that allows for sharper text, clearer visuals, more vibrant colors, and an overall more comfortable experience—whether you’re reading emails at your desk in Immersed, creating a physics game in Figmin XR, or fighting monsters with friends in Demeo.

It’s a system optimization design challenge—requiring our product teams to take the VR visual system as a whole, rather than focusing solely on individual components. At the end of the day, our goal is to maximize the holistic user experience. It may not be perfect, but the tradeoffs we’ve made are intended to preserve the best possible visual experience. And this is just the beginning. As our Infinite Display technology continues to improve and we’re able to combine Meta Quest Pro’s innovative pancake lenses and polarization optics with even higher-resolution displays, we should see significant leaps in visual quality—as measured by PPD and beyond.

*MTF here is measured at 5 lp/mm (line pairs per millimeter).