Apple’s Craig Federighi Explains How iPadOS 14’s Scribble Feature Was Developed

Scribble Anywhere

Earlier this month, Apple released iPadOS 14 to the public. Apple has always been very keen to position the iPad as a full-fledged computing machine. The iPadOS and powerful iPad Pro series have helped the iPad shape up as a computer replacement. One of the powerful features of iPadOS 14 is Scribble. The Scribble feature lets you input text and performs other operations like control and navigation without a keyboard.

In an interview with Popular Mechanics, Craig Federighi has explained the process of developing Scribble for iPadOS 14. Apple used data gathering and first understood how people across the world write. This helped the company bake in different writing styles into Scribble. Furthermore, they also analyzed various aspects of users’ stroke while writing things, which allowed them to understand how each letter is formed.

When it comes to understanding [handwriting] strokes, we do data-gathering. We find people all over the world, and have them write things,” says Craig Federighi, senior vice president of software engineering at Apple. “We give them a Pencil, and we have them write fast, we have them write slow, write at a tilt. All of this variation.

That methodology is distinct from the comparatively simple approach of scanning and analyzing existing handwriting. Federighi says that for Apple’s tech, static examples weren’t enough. They needed to see the strokes that formed each letter. “If you understand the strokes and how the strokes went down, that can be used to disambiguate what was being written.”

Our Take

Scribble is a mighty useful feature. All you need to do is write things using Apple Pencil, and the software will transform it into text. The most significant advantage is the system-wide implementation. Whatever you write using Apple Pencil will automatically be turned into legible text across the iPadOS 14. Craig explains how Apple uses advanced machine learning and other processing mechanisms to translate the written text.

Handwriting recognition is not something new. Companies like Palm and HP employed stylus as a primary way to interact with touch screens. Truth be told, the handwriting recognition has always had its flaws and was replaced by touchscreen phones. Apple seems to have done a great job in understanding how people write, and perhaps Scribble will mark the return of handwriting feature on other devices as well.

[via Popular Mechanics]