[Download the Read ActiveSync Sent Items script]
Exchange users with new iPhones are re-discovering an old bug where messages sent through ActiveSync for mobile devices do not appear properly in both Entourage 2004 and Entourage 2008. The messages sometimes look like a mix of HTML, E-mail headers and what's left of the original message or they look like an unintelligible jumble of letters and numbers. The recipient and the subject are also empty.

This problem was addressed in the knowledgebase article "The header information is not displayed in Entourage 2004 for an e-mail message that is sent by using Exchange ActiveSync". However, the workaround mentioned in the article, which is to view the message in Outlook Web Access (OWA), does not work in either the full version that Internet Explorer for Windows users would see or the light version that Mac users would see.
What's happening and how can you read these "corrupt" messages?
Who's the culprit?
Exchange users assume this must be a bug in both Entourage 2004 and Entourage 2008 because they do not see this problem in Outlook for Windows. However, if they also try to view the files in OWA they will find the messages missing altogether.
What do Entourage and OWA have in common? They both connect to Exchange via WebDAV. So, this problem isn't necessarily a bug in Entourage. In fact, it's a bug in ActiveSync for Exchange. ActiveySync is the Microsoft technology that allows mobile devices such as Windows Mobile phones and the iPhone connect to Exchange. Furthermore, it's specifically a bug in ActiveSync for Exchange Server 2003. Entourage users connecting to Exchange Server 2007 servers should not see this problem.
What causes the problem?
So, what is ActiveSync doing to mail messages as they pass through it and why doesn't Outlook or any other E-mail client have this problem?
ActiveSync is munging the headers of the mail messages. Notice in the earlier screenshot that the mail message includes X-MimeOLE, Received, MIME-Version and other lines of information you normally wouldn't see. These are the headers, or the envelope of your mail message. They are suppose to be in a specific format so that mail applications can translate them and make the message presentable.
But look at what ActiveSync is doing to the message you send. The headers are appearing in the body of the message, which is not normal:

If you look at the source, or the full message with headers and body combined, by selecting Source from the Message menu (select Source from the View menu in Entourage 2004), you'll see it is identical to the body of the message except for two (sometimes more) additional headers at the top:

Technically, a mail message header is a sequence of non-empty lines at the beginning of the mail message. But if we are to believe that with this example message then the entire header would simply be the first two lines because they are followed by an empty line:
MAIL FROM: <William.Smith@domain.com>
RCPT TO: <talkingmoose@gmail.com>
ActiveSync is appending these bogus headers and causing the real headers to fall into the body of the message. But why isn't Outlook for Windows susceptible to this problem? Most likely it has simply been programmed to realize these two headers are bogus and to continue reading the rest of the headers properly.
Are the messages still readable?
Yes! We simply need to remove the bogus headers and then view the messages in their correct format. To say that these messages are "corrupt" is a misnomer. They are not irreparably damaged. They can be properly viewed with the help of AppleScript. I've written a script to do just that.
Download the script and follow the instructions to install it into the Entourage Script Menu Items folder in your Microsoft User Data folder. Once installed it will be immediately available in the Script menu in Entourage.
To use the script simply select the unreadable message in the message list, or open it, and then select the Read ActiveSync Sent Item script from the Script menu. If the script detects the message has the improper headers then it will open the message in a new window and it should be readable. So the example message above should now display as:

The new message is not really in Entourage but just a temporary file. If you wish to save it to your Desktop then you'll have that option:

The new message file will be saved with the Subject as its name.
Due to limits with Entourage's AppleScript capabilities, the message cannot be automatically saved back into Entourage. You can, however, drag and drop the message into the original folder and delete the old message. Sent date and time will be preserved so the message will properly sort within the message list. Only one message can be read and saved at a time.
[Note: The download file contains an Example.eml file, which I used for the examples in this post. I've included it for anyone interested in examining this problem in detail.
Special thanks to Jon Connell in the Microsoft Entourage newsgroup for his insight that this problem could possibly be solved with AppleScript!]

Comments (40)
Would you be able to tweak the script that it will process multiple messages at once. I'd love to be able to batch a group of messages every month until Microsoft fixes this.
I hope to.
The script started simple and then grew more complex as I threw in the ability to run it with the message open or closed, threw in error checking and threw in some cleanup routines in case something went wrong.
I only got it posted just last night so I'll take your feedback and feedback from others for the next tweaks that folks need.
Great! The end of my frustration. Batch conversion would be a nice addition.
Thank you!
The workaround in the KB article does work, I can view the messages fine (in Sent Items folder, which were sent thru my EAS based device) without any issue in Outlook and Outlook Web Access using any browser on any platform.
Excellent, thank you William!
For me, my iPhone sent messages do not appear at all in Entourage nor Outlook Web Access (OWA).
Let's talk in a few weeks, RageGuy, and I'll try to show you my examples. ;-)
I've noticed that once the emails are converted and everything re-synced, that the converted messages now have the conversion date and time (actually only when viewed on my iPhone. Entourage and Outlook do the right thing). It isn't tragic, just a little weird...and possibly only seen on my setup...
Hey William, thanks for the script! This was getting frustrating for me since I'm the only iPhone user in my office and was trying to vet the phone for full network integration.
Thanks william, worked for me... will be a big help for others in my organisation. Thanks again.
I would say this is a bug in exchange for not stripping the headers correctly in the protocol before presenting it to users via webdav
is it not possible to make a filter on exchange to correct this ?
regards
John Jones
http://www.johnjones.me.uk
I've experienced this exact problem with messages sent from my iPhone 3G and viewed in Entourage 2004. Thanks for the script which I will try. I'm also wondering if another issue I am having is related. This morning I launched Entourage and got an error message that my database was corrupted and needed verifying. It then asked me if I wanted to rebuild it, which I did. At the end of the process, numerous old e-mails appeared in my outbox and got sent to several recipients confusing them and prompting them to write to me asking what was going on? i never had this problem prior to buying the iPhone a couple weeks ago. Comments welcome.
My sent emails show uo fine in OWA but comes out corrupt in Entourage 2004 and 2008. Doesn't that imply that the bug is in Entourage?
According to one MacBU developer, the workaround in the Microsoft KB article, which is to read the messages in OWA, does work. However, I've seen in two different Exchange Server 2003 systems where the messages simply don't appear in OWA. I'll be visiting MacBU in a little more than a week and hope to discuss this with him more. I'm hoping he can explain why this sometimes works and sometimes doesn't.
However, for reasons I've already explained in this post, the problem is not a bug with Entourage. The problem is with munged headers. Entourage has not been programmed to interpret these munged headers as I suspect Outlook has been.
has anyone seen a fix for this yet?
this is a exchange 2003 problem. i read somewhere where they say in the exchange 2007 this is fixed
Nice work. Thank you very much for this. As some others mentioned, a method to batch process these messages would be fantastic.
Thanks so much for this script. But just one further comment: why doesn't MS fix this!!??
Thanks for the background info on how the messages are mishandled. I was not able to get the script to work within entourage (2004) - it did not display the corrected message and when opting to save to the desktop, an error (...can't make desktop type constant) occurs. The script runs fine in the script editor, correcting and displaying the selected message, but when opting to save to the desktop, it errs again. Same behavior on two different machines (10.4.11, ent2004; must be something simple). Was able to edit the messages 'by hand', though, after seeing what was causing all the trouble.
This script is no longer working well for me. Worked fine at first, but now takes forever to execute. Entourage also becomes very sluggish in the meantime.
I did try to add a key shortcut through Quickeys so that I didn't have to use the mouse to press the No button on the Save to Desktop screen. Maybe that did it, but getting rid of the shortcut didn't help with performance.
Any ideas?
Hi Michael!
I have no idea why the script would be getting slower. It simply reads what you've selected and acts only on that information.
You can also open the script in the Script Editor found in /Applications/AppleScript and run it directly from there. Saving it again may help.
Hope this helps!
In case anyone is interested, I have modified this script to be able to be run with many selected items. If the email does not need to be fixed, it is skipped, otherwise it is processed using the logic from William Smith's original AppleScript. After the first message is processed, it prompts you if you would like the entire batch of email messages to be saved to the desktop or not, similar to how the original version did for a single email. I am a bit of a hack with AppleScript, so I apologize in advance for the clumsiness of the code! Here is the URL:
http://www.sopriswest.com/downloads/Read_Selected_ActiveSync_Sent_Items.zip
Very welcome changes indeed, Karl! A lot of folks have been asking for this capability.
Thanks for your work! I'm sure folks will be appreciative. :-)
Hi,
Inspired by William's research I wrote an Applescript that works better for me. You can run the script in any way you please but putting it in the Entourage script menu is convenient. The script makes a new 'readable' outgoing message for every 'iPhone originated' outgoing message, while deleting the latter.
It works fine for me. But use it at your own risk:
tell application "Microsoft Entourage"
set theExchangeAccount to item 1 of every Exchange account
set theSendbox to sent items folder of theExchangeAccount
set theMessages to every message in theSendbox
repeat with theMessage in theMessages
set theSource to source of contents of theMessage
if theSource begins with "MAIL FROM: set the AppleScript's text item delimiters to "Received: from"
set theNewSource to "Received: from" & text item 2 of theSource
set the AppleScript's text item delimiters to ""
set theExchangeAccount to item 1 of every Exchange account
set theSendbox to folder "Verzonden items" in theExchangeAccount
set theEntourageMessage to make new outgoing message with properties {source:theNewSource}
move theEntourageMessage to theSendbox
delete theMessage
end if
end repeat
end tell
Have Fun,
Gijs
Hi,
Pasting the script as text in the previous forum message wasn't such a good idea after all, as some characters got lost. I did some more work on the script, removed a bug that prevents the script from running on non-Dutch version of Entourage, and uploaded it to Scriptbuilder.net
You can find MakeSentFromiPhoneReadable 1.0 at
http://scriptbuilders.net/files/makesentfromiphonereadable1.0.html
Hope it works for you too!
Gijs
Hey Gijs,
thanx for your investigation in this issue. I tried to use your script, copying the file in the "msuserdata" folder, but when executing the script nothing happens?
I user the german version - does this ma any difference?
Thanx in advance
benny
I've downloaded the script and it works fine on my Mac. However I also use Outlook on a Windows pc (same server) and in the Outlook Sent items folder the item shows up with the date (i.e. today) that it was re-imported into Entourage when the mail item was actually sent six months ago and shows the six months ago date in Entourage
Any advice. Many thanks.
@Mike Flint
Probably like Entourage, Outlook can display two dates: Sent and Received.
The Sent date is part of the original message and is timestamped by the Sender. The Received date is the message creation date timestamped by your own E-mail program.
Make sure you're viewing by Sent and not Received.
Bill,
So where do we complain about this. This is nuts to have this broken this long.
@ Kevin Walsh
Because the problem lies with Exchange and it is primarily an enterprise product, the first place you should consider is your company's Microsoft representative if you have one.
I doubt seriously this will get fixed, however, since it was resolved in Exchange Server 2007. Exchange Server 2003 is now in maintenance mode. It'll get patched for security issues but I'd imagine little else.
I noticed a few things while I tried to use the script provided by Gijs:
1. the initial dialog causes an error (on German Entourage 08). Removed it and it works.
2. the scripts looks for "iPod mail" in text. Needs to be updated to "iPhone mail" to work.
3. "iPhone mail" header was introduced by a newer iPhone firmware. Older mails do not have it
4. script searches for "Received: from" which is wrong for older mails (older firmware). Here you will find two blanks between ":" and "from"!
Otherwise I was able to "recover" all mails ;-)
Hope this helps!
Thomas
Here is my version of this script. This script sequentially does the following:
a) Read each item that is received in my Sent Mail folder,
b) Remove the erroneous headers,
c) Copy the rest of the message to a new file placed in the Sent Mail folder, and
d) Delete the original message
Usage instructions are at the end of the script.
tell application "Microsoft Entourage" set myExchangeAccount to item 1 of every Exchange account set selectedMessages to (current messages) repeat with suspectMessage in selectedMessages set theSource to source of suspectMessage's contents set lineCount to 1 set aParagraph to "not empty" repeat until aParagraph is "" set aParagraph to paragraph lineCount of theSource set lineCount to lineCount + 1 end repeat set newSource to "" repeat with i from lineCount to count of paragraphs of theSource set newSource to newSource & paragraph i of theSource & return end repeat set theNewMessage to make new outgoing message with properties {source:newSource} move theNewMessage to sent items folder of myExchangeAccount set read status of theNewMessage to read delete suspectMessage end repeat end tellHow to install -
a) Copy this file (FixActiveSyncSentItems.scpt) to
~/Documents/Microsoft User Data/Entourage Script Menu Items
b) Create a rule in Entourage - called FixBlackBerrySentItems with the following criteria
Folder is "Sent Items"
Message body contains "X-rim-org-msg-ref-id:"
then
Run AppleScript "FixBlackBerrySentItems"
Note: This script searches for a BlackBerry specific header that is stuck in what appears to be the message body. If you use an iPhone, just replace this header with an equivalent iPhone header.
Hope this help!
Ashok
So, the scripts posted here work, but unfortunately they run very slowly on large messages when message sizes get larger than 50k or so. I guess it is a performance issue reading lines in Applescript. Anyway, I reworked the script where the heavy lifting is done by sed since all we need to do is strip off the offensive headers until the real headers start. The modified script runs orders of magnitude faster. Here it is:
tell application "Microsoft Entourage"
set theProcessedItemsCounter to 0
set theFaildItemsCounter to 0
set theExchangeAccount to item 1 of every Exchange account
set theSentbox to sent items folder of theExchangeAccount
set theSentMessages to current messages
set tmpfolderpath to (path to temporary items folder from user domain) as Unicode text
repeat with theMessage in theSentMessages
set theSource to source of theMessage
set theheader to headers of theMessage
if theheader contains "Subject" then
--return -- do not continue if this is a properly formatted message
else
try
set sourceMacPath to (tmpfolderpath & "entourage-message-" & theProcessedItemsCounter)
set sourcePosixPath to POSIX path of sourceMacPath
set fd to open for access sourceMacPath with write permission
set eof of fd to 0
write theSource to fd
close access fd
-- translate mac to unix line endings
set targetPosixPath to (sourcePosixPath & ".unix")
do shell script "tr '\\r' '\\n' " & targetPosixPath
-- get rid of the malformed headers
set fixedMessagePath to (sourcePosixPath & ".fixed")
do shell script "sed -e '1,/^$/d' " & targetPosixPath & " > " & fixedMessagePath
-- read in the corrected message
tell application "Finder"
set fd to POSIX file fixedMessagePath as alias
set newSource to (read fd)
end tell
set theEntourageMessage to make new outgoing message with properties {source:newSource}
move theEntourageMessage to theSentbox
set properties of theEntourageMessage to {color:{0, 0, 0}, priority:normal, read status:read, delivery status:sent, replied to:false, redirected:false, forwarded:false, connection action:keep on server, flagged:false}
-- clean up
do shell script "rm " & sourcePosixPath
do shell script "rm " & targetPosixPath
do shell script "rm " & fixedMessagePath
delete theMessage
set theProcessedItemsCounter to theProcessedItemsCounter + 1
on error
set theFaildItemsCounter to theFaildItemsCounter + 1
end try
end if
end repeat
-- give feedback
set theText to "Microsoft Entourage:" & return & return & theProcessedItemsCounter & " iPhone messages have been made readable" as text
if the theFaildItemsCounter > 0 then
set theText to theText & return & theFaildItemsCounter & " messages failed to process" as text
end if
display dialog theText buttons {"OK"} default button "OK"
end tell
I have uploaded my version of the script to
http://files.macscripter.net/ScriptBuilders/Entourage/fix%20iphone%20sent%20mail.scpt.zip
Basically, it "borrows" from the other scripts posted here, and dramatically improves performance by using "tr" and "sed" to do the actual message fixing.
Thanks to everyone for the discussion here. This thing was driving me crazy.
The last posted script works great! The previous scripts would not work on all my messages, it looks as though they were looking for a specific start of message, and my iphone doesn't always have that start.
This is ridiculous. There is no reason a fix for Entourage hasn't been issued to address this. Just another drop in the bucket of "why I hate Entourage" issues.
I am new to applescript...how do I get Ujwal Setlur's script above installed and working within Entourage? I was able to copy the script into a new script and save it to the ~/Documents/Microsoft User Data/Entourage Script Menu Items folder. Then I added a rule like in one of the above examples, but I get zero results when I run the rule.
Help, step by step instructions would be great. I have iPhone, not Blackberry.
Thanks!
I set up a rule with the script from Ujwal Setlur by:
If:
Specific Header-MAIL FROM:-Contains-
Then:
Run AppleScript-Script-Fix iPhone Sent Mail.scpt (or whatever you named the script)
That did the trick for me - it is now automatic. I hope this helps the thread. Jeff
The fix iphone sent mail script from Ujwal Setlur works perfect. Thanks a lot!!!
Thanks guys
Ujwal's script is great and the notion of using the rule makes it better. Great collaborative effort!!
Thanks Guys!!
Thank you! Ujwal's script works perfectly for me.
I had a little trouble understanding and implementing Jeff's rule because I understood it to mean leaving the contains field blank, which Entourage won't let me do. But I saw a similar rule on another blog (RickyCampbell.com) that was a bit clearer:
Specific Header > Subject > Does not exist
Does the same thing - I just stupidly hadn't realised there was a 'does not exist' from the dropdown!
Thanks for all your hard work :)