Reprinted from TidBITS by permission; reuse governed by Creative Commons license BY-NC-ND 3.0. TidBITS has offered years of thoughtful commentary on Apple and Internet topics. For free email subscriptions and access to the entire TidBITS archive, visit http://www.tidbits.com/ How Mail May Be Anything but IDLE when Pushing Email Michael E. Cohen So much happens behind the scenes on our computers that at times technology seems magical. For example, when a new email message arrives, the Notification Center of both my Mac and iPad alert me simultaneously. As with most daily technological magic, I didn't think much of it until I was helping to edit the manuscript of a new edition of Joe Kissell's immensely helpful [1]Take Control of Apple Mail, when Joe and I had an enlightening discussion that took place in the manuscript's margin comments. At issue was the Notification Center in OS X 10.8 Mountain Lion and how mail notifications would work with it. I proffered the suggestion that the Notification Center in OS X 10.8 Mountain Lion would hook up with the same push service that iOS devices used, and was surprised when the manuscript came back to me with a comment from Joe saying that Apple Mail on the Mac did not use push services in earlier versions of OS X and, more importantly, would not be using them in Mountain Lion. I was doubly surprised because, just as I was reading his comment, I heard the new mail sound simultaneously from both my Mac (running Mac OS X 10.7 Lion) but also from my nearby iPad. I knew my iPad was using push services, but what was Mail on my Mac using that could account for the simultaneous new mail sound? It may not have been push email on my Mac, but it sure seemed to act the same as push email on my iPad to me! A little bit of research provided the answer: Apple Mail in OS X uses an optional bit of the IMAP (Internet Mail Access Protocol) specification ' the IDLE command. The clue that quickly lead me to the answer was an option in the Advanced pane of Mail's Account preferences: Use IDLE command if the server supports it. For my iCloud email account, this option was checked by default. A little IMAP mail background -- IMAP is an advance over POP (Post Office Protocol), which was the way email used to work in the deep dark past of the Internet, and which, to the dismay of some, still is used by many email services today. With POP, incoming messages arrive at the mail server and remain there until the client (that is, your email program), fetches them from the server and, optionally, removes them from the server. The mail server only keeps track of the messages on it; it doesn't record whether a particular mail user has actually read or downloaded the messages: those tasks are the responsibility of the mail client. Thus, you can fetch messages from a POP server with your mail client, and, if you don't delete the messages from the server with that client, you can fetch them again using another mail client (on the same or on a different computer), and those messages appear in that client as brand-new unread messages. With IMAP, the mail client displays the messages as they exist on the server. The client may or may not download the messages to the local device ' that's not important. What is important is that the IMAP server knows when the client has read a message. This means that when you read IMAP mail with one mail client, and then attempt to read the same message with a different client (on the same or on a different computer) you can see which messages you have already read, no matter the client with which you have read them. The advantage if you own more than one Apple device is synchronization of which messages have been read or deleted ' for example, you can read a message on your iPhone and then find it marked as read on your Mac later. IMAP mail clients and IMAP mail servers can have much more complex 'conversations' with each other than those that the more basic POP protocol allows. Back to IDLE -- What does all this have to do with arriving email notifications? After all, whether the protocol is POP or IMAP, the mail client still has to connect to a mail server and send a request to it asking for mail. Except, that is, for when an IMAP server supports the IDLE command. When a mail server lets a mail client know that it can respond to the IDLE command (Apple's iCloud servers do, though many others don't), the client can send the IDLE command to the server with the following result: the server keeps a connection open between it and the client so that when new mail arrives the server can immediately let the client know about it; the client can then inform the user that new mail is available. (This description of how a client and server use the IDLE command, of course, is almost criminally over-simplified; you can read all the geeky details of how the IDLE command works in [2]RFC 2177.) So, the reason I know that I have new mail available on my Mac is because Mail and the iCloud server are in constant contact with one another: the server says, "You have mail," and the client says, "Oh, goodie, let me see it." And, in Mountain Lion, Mail also lets the Notification Center know about it, too. But What about Push? -- IDLE in IMAP is relatively old: it was first specified late in the last century, a decade before the iPhone first saw the light of day. IMAP IDLE assumes that the mail client is running and can maintain an open network connection to the mail server. That's great for devices plugged into a power source or with great big batteries; not so good for a pocket-sized device with a tiny battery that has to maintain a connection to a cell-phone network as well. That's where push comes in. In iOS, there isn't a whole lot of memory to support simultaneously running apps, nor enough battery power to keep a bunch of them open at any one time. There is, however, enough of each to keep a small program semi-awake and aware in the background that can listen to the mobile network. That's how Apple's push system works. With push, an iOS app registers with that small background program in iOS, informing it that the app expects to receive network notification messages ' even when the app isn't running! That small background program in iOS listens to the network and fields incoming communications, sending the appropriate notifications to the apps that have requested them. On the other end, remote applications, such as mail servers, can register with an Apple Push Notification (APN) server, passing that server a token that identifies what device out there on the Internet is interested in receiving notification messages. It's Apple's APN servers that send the notification messages to the iOS device from the remote application. The path is something like this: the mail server gets new mail for you and sends a notification message and a device token to the APN server. The APN server then sends the appropriate data to the small background program running on the iOS device so that you can hear the new mail sound, see the badge number change on the Mail app's icon, and read a brief summary of the incoming mail's content (the push system can send a notification payload of no more than 256 characters). When you open the Mail app on your iOS device, the Mail app then establishes a full IMAP connection with the remote mail server and you can read your mail. (There's yet another push technology Apple devices can use: Microsoft's ActiveSync; curious readers can find out more about how that works in the Microsoft article, '[3]Understanding Direct Push.') Subtle differences -- These two systems, IDLE and push, account for the multiple notifications I simultaneously received on my iPad and on my Mac when the new message arrived in my iCloud account. But there are subtle differences in behavior between the two systems. On the Mac, you only receive new mail notifications when the Apple Mail application is running, because it is the conversation between Mail and the mail server that allows notifications to be made. When Mail is not running, or your Mac is asleep (and isn't using the Power Nap feature in Mountain Lion), your Mac has no way of knowing when new mail arrives, so you don't get notifications. On an iOS device, you always receive notifications as long as your device is on and has a network connection ' even when the device is asleep, that little background program is awake enough to receive incoming push notifications. (By the way, that's why you can eke out a little more battery life if you disable push on your iOS device: that little background program doesn't use much power, but it uses some.) However, the Apple push system only provides for notifications from the APN service to the device; it doesn't send messages back the other way. So, if you get a notification on your iOS device telling you that you have new mail, and you read that mail on a different device (like, say, your Mac), the iOS device won't know you have read it (and won't adjust the number in the badge on the Mail app icon) until you actually open the Mail app and establish the IMAP connection between your device and the remote mail server. So there you have it: two completely different notification systems that produce very similar results. And all so you can get your singing cat emails as soon as possible, no matter what device you are using. Technology is magical! References 1. http://www.takecontrolbooks.com/mountain-lion-apple-mail 2. http://tools.ietf.org/html/rfc2177 3. http://technet.microsoft.com/en-us/library/aa997252.aspx .