Apple has been granted a patent that covers a new method of controlling Macs and future television sets using gaze and point gestures. The technology uses 3D depth mapping, similar to that Apple is now using to create depth of field effects with the iPhone 7 Plus’ new camera.
Apple’s patent describes how a 3D map of a user — or at least parts of their body — could be used by future Macs or TVs to read gaze and point gestures, allowing the user to take control of the device without using a remote or other physical controls.
The 3D map is combined with a 2D image of the user, which the software then uses to establish what on-screen controls they are interacting with. Apple’s illustrations depict a radial menu that can be used to access things like email, the internet, and games using gestures.
In another example, Apple explains a method that tracks a user’s eyes to identify what they’re looking at on the display. The direction of their gaze, coupled with gesture controls, can then be used to interact with the computer.
For instance, a user could scroll through a webpage in Safari simply by looking at the page, then pointing towards the bottom of it. They could then gaze at another window and point at something else to perform an action there.
This patented was originally filed by Apple in Q3 2013, and published today by the U.S. Patent and Trademark Office. Like all Apple patents, it’s far from a guarantee of things to come, but it does give us some insight into the things Apple is working on behind closed doors.[via Patently Apple]