A look at Apple’s new Transformer-powered predictive text model:

Apple wants a model that can run very quickly and very frequently, without draining much of your device’s battery. When I was testing the predictive text feature, suggestions appeared almost instantly as I typed, making for a great user experience. While the model’s limited size means it wouldn’t be very good at writing full sentences or paragraphs, when it exhibits very high confidence in the next word or two, they’re likely to be good enough to suggest to the user.