It has been announced that Oculus VR as bought The Eye Tribe, a company known for its eye tracking hardware and software technologies, earlier in December. Facebook’s VR division does not say how it plans to use the acquired company, but the two primary reasons for the takeover could be The Eye Tribe’s gaze tracking IP along with the company’s experience with foveated rendering. Theoretically, the buyout may indicate that foveated rendering could become an important feature of the upcoming VR platforms.
The Eye Tribe was established in 2011 with the aim to develop software and hardware technologies for eye tracking. By 2013, the company introduced its eye-tracking device for PCs and tablets that cost only $99 along with an SDK to develop software for the product. The company actually started shipping the units to developers in 2014, but the devices have never been integrated into actual tablets or computers shipped to end-users. Eventually, the company designed eye tracking technologies for smartphones and earlier this year demonstrated an eye tracking tech for VR headsets.
Eye tracking itself could enable a more intuitive control for virtual reality interactions when combined with gestures, voice and other methods. However, this might not be the only reason why Oculus VR was interested in The Eye Tribe. Usage of gaze tracking also enables developers to use foveated rendering, which allows optimizing the use of GPU horsepower. The method is relatively simple on paper: each scene is rendered first in low resolution and with the least amount of polygons and then the specific part of the screen where the user is currently looking at is rendered in high resolution. Developers may play with such effects like depth of field, blur and antialiasing to further optimize performance and/or improve quality, but all of those tricks require gaze tracking.
The foveated rendering technique is already supported in Unity3D engine and is enabled by the Tobii EyeX tracking device. According to the developers, usage of the tech helped to improve performance of a laptop from 11 FPS to 42 FPS. Microsoft and NVIDIA are also researching foveated rendering and the latter has even come up with new ideas on the matter. The company says that foveated rendering per se is not a panacea for VR (at the end of the day, you can improve performance of any platform by using foveated rendering with gaze tracking) because rendering in lower resolution for peripheral vision results in flickering if the foveation is too aggressive, meanwhile blur itself reduces contrast and introduces a sense of tunnel vision. NVIDIA has a number of suggestions how to minimize negative effects of foveated rendering (e.g., by introducing contrast preservation), but currently this is a research project.
For VR applications, foveated rendering makes sense, especially if we are dealing with a headset based on a smartphone that has a limited amount of GPU horsepower. To enable it, Oculus VR needed gaze tracking IP and a team that understands how things work here. This does not mean that foveated rendering will be a part of Oculus VR’s next-gen VR headset because its current form requires some additional work. However, with eye tracking IP and appropriate team, Oculus VR could implement the technology in the future.
Related Reading:
ncG1vNJzZmivp6x7orrAp5utnZOde6S7zGiqoaenZH5xhZNvZqibpaHCtHnVq2Sam6GqtrOx0maroZ1dmsamedOroJudXZmyt7HLqKeeql2ks26x2J5kraqRmLiqusZmq56bmKO8rbvGopys