Apple goes in-depth on Smart Script and Math Notes for iPadOS 18 in new interview

Screenshot

Math Notes and Smart Script are two of the most impressive features included as part of iPadOS 18 this year. In a new interview with Chris Lawley, Apple has provided more detail on how these features work with a deep dive into some of the tech behind them.

As a refresher, Math Notes is a new feature in iPadOS 18 that lets you type or write out mathematical expressions and see them solved instantly in your own handwriting.

In the interview, Apple’s Ty Jordan, system experiences product manager, elaborated on Math Notes and why Apple finally decided to bring the Calculator app to iPad with iPadOS 18:

We think very deeply when we’re bringing these types of experiences to iPad about how those experiences can be designed for the unique capabilities of iPad and its unique accessories like Apple pencil.

We could have just brought a basic or scientific calculator to iPad, but instead we took the time to really reimagine how you could even do math with Math Notes and Calculator. It sort of works like magic.

It combines the natural input of pencil with the amazing on-device machine learning models that [Jenny Chen, Apple’s engineering manager for input experiences] and her teams work on and the power of Apple Silicon. So you just write math like on a piece of paper and like magic, it just gives you the answer.

Jenny Chen, Apple’s engineering manager for input experiences, shared more details on how you can insert a graph for your data with just a single tap with Math Notes:

One of the great things first is, how are we even able to recognize that this equation is even graphable? So if we have two variables that are not defined, first of all, we use the amazing recognition engine under the hood to be like, “Hey, can we graph this?”

And then for the charts, it actually builds on all the years of work that we’ve done for the Swift charts API and what that provides. It’s great that we’re both enhancing that, but also using it in our own products. And so some of the fun things that came out of that is the fun animations that make it feel nice and smooth and fancy.

The ability to kind of scrub along the graph so that you can see the value as the value changes along the X axis.

Meanwhile, Smart Script in iPadOS 18 is a new feature that aims to improve your handwriting when taking notes. The feature smooths and straightens your handwritten text as you’re writing with Apple Pencil. It also lets you edit handwritten text just as easily as you edit typed text.

Jordan explained how Smart Script learns more about you:

Smart Script is an on-device machine learning model that learns continuously, but also works very quickly. So you’ll actually notice, even if you start with a fresh brand new iPad, as soon as you start writing words, we’re going to refine them. We know how to make that neater for you.

We’ll also have spell check working straight away because we can see what you wrote, like Jenny said, understand it, and we can offer those suggestions for you.

There are some features like “paste as handwriting” that get better over time as you write more. So really to like dial in your handwriting style takes like one to two paragraphs of that style. But it’ll continuously learn and it’s very contextual.

So for example, if you were to go to a previous paragraph where you had some bad handwriting, we’re going to match the handwriting that was refined there and not sort of try to replicate what you’ve had in other notes.

Check out the full interview below: