Advertisement
 

Gmail, Google Apps Changing Email Headers and Messing up Email Display

User Rating: / 2
PoorBest 
Thursday, 26 April 2012

Google is Messing with SMTP Headers and it is screwing up HTML Emails (SOLUTION BELOW)

Our web server sends out tons of emails to customers and employees.  We use Google Apps for our email and iPhones at times to read them too.  Something strange started happeningon 4/25/2012.  HTML Emails started not working on 4/25/2012.  Nothing has changed on our servers, emails juststarting displaying headers and not looking right.  Keep reading, we know why it is happening...

 

http://www.evdoinfo.com/images/stories/header1_list.jpg http://www.evdoinfo.com/images/stories/header_mail_ok.jpg  http://www.evdoinfo.com/images/stories/header_showing.jpg

 

The first image show a daily email that started displaying funny on 4/25/2012.  Notice that the "From Address" shows "<>".  The middle screen grab show what the email looked like before 4/25/2012, email looks fine.  Then for the last few days, the email looks like the image on the right.  Starts off showing the header: "Content-type: text/html; charset=iso-8859-1" and then a bunch of email headers.  If I scroll down, I will see the raw HTML of the email, but why?  Nothing changed on our end.  Something did change on Google's end.

 

Email Headers before 4/25/2012

To: xxx@3gstore.com
Subject: Router Advisor Click Log 2012-04-23
MIME-Version: 1.0
From: 3Gstore.com <xxx@3Gstore.com>
Message-Id: <20120424050102.8871E4B05A3@mdg-web.localdomain>
Date: Tue, 24 Apr 2012 00:01:02 -0500 (CDT)
X-pstn-neptune: 0/0/0.00/0
X-pstn-levels: (S: 1.35367/99.90000 CV:99.9000 FC:95.5390 xxx )
X-pstn-dkim: 0 skipped:not-enabled
X-pstn-settings: 2 (0.5000:0.5000) s cv gt3 gt2 gt1
X-pstn-addresses: from <xxx@3Gstore.com> forward (user good) [2172/88]
X-Original-Sender: xxx@3gstore.com
X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain
of xxx@3gstore.com designates 74.125.149.61 as permitted sender) smtp.mail=xxx@3gstore.com
Precedence: list
Mailing-list: list xxx@3gstore.com; contact xxx@3gstore.com
List-ID: <xxx.3gstore.com>
X-Google-Group-Id: 725066998195
List-Help: <http://www.google.com/support/a/3gstore.com/bin/static.py?hl=en_US&page=groups.cs>,
<mailto:xxx+help@3gstore.com>
Content-type: text/html; charset=iso-8859-1
X-pstn-neptune: 0/0/0.00/0
X-pstn-levels: (S:59.71620/99.90000 CV:99.9000 FC:95.5390 LC:95.5390 R:xxx )
X-pstn-dkim: 1 skipped:not-enabled
X-pstn-nxpr: disp=neutral, envrcpt=xxx@3gstore.com
X-pstn-nxp: bodyHash=df69e09be9f37e07b8380517f9e5f7c11537df73,
headerHash=62a12f02680bb6d95f35bf5420a5a43fbcb7e920, keyName=4,
rcptHash=01f0a189360825ca14f163766b98db238208c87a, sourceip=74.125.149.244, version=1


<strong>Total Routers Clicked: 8</strong><table border='0' cellpadding='0' cellspacing='3' width='100%'><tr>


Here is exactlty the same email that is having the display issues:

 

To: xxx@3gstore.com
Subject: Router Advisor Click Log 2012-04-25
MIME-Version: 1.0
X-Original-Sender: xxx@3gstore.com
X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain
of xxx@3gstore.com designates 74.125.149.55 as permitted sender) smtp.mail=xxx@3gstore.com
Precedence: list
Mailing-list: list xxx@3gstore.com; contact xxx+owners@3gstore.com
List-ID: <xxx.3gstore.com>
X-Google-Group-Id: 725066998195
List-Help: <http://www.google.com/support/a/3gstore.com/bin/static.py?hl=en_US&page=groups.cs>,
<mailto:xxx+help@3gstore.com>
X-pstn-neptune: 0/0/0.00/0
X-pstn-levels: (S: 3.77844/99.90000 CV:99.9000 FC:95.5390 LC:95.5390 R:xxx )
X-pstn-dkim: 0 skipped:no-from
X-pstn-nxpr: disp=neutral, envrcpt=xxx@3gstore.com
X-pstn-nxp: bodyHash=a4ea0efb001cfaf7335141c3df807a506f7bbc3d,
headerHash=3151a48c99d25e8180b6aa61d8fdd1ba72e93b47, keyName=4,
rcptHash=01f0a189360825ca14f163766b98db238208c87a, sourceip=74.125.149.240, version=1

Content-type: text/html; charset=iso-8859-1


From: 3Gstore.com <xxx@3Gstore.com>


Message-Id: <20120426050102.9E5BE4B03E9@mdg-web.localdomain>

Date: Thu, 26 Apr 2012 00:01:02 -0500 (CDT)
X-pstn-levels:     (S: 4.43673/99.90000 CV:99.9000 FC:95.5390 LC:95.xxx )
X-pstn-dkim: 0 skipped:not-enabled
X-pstn-settings: 2 (0.5000:0.5000) s cv gt3 gt2 gt1
X-pstn-addresses: from <xxx@3Gstore.com> forward (user good) [2172/88]
X-pstn-nvpr: envrcpt=xxx@3gstore.com,disp=neutral
X-pstn-nvp: sourceip=208.67.207.101,bodyHash=xxx,headerHash=xxx,sig=xxx


<strong>Total Routers Clicked: 9</strong><table border='0' cellpadding='0' cellspacing='3' width='100%'><tr>


Notice that the "Content-type" and "From" headers are in different positions and there is an extra carriage return/line feed after the line (ie. a blank line).  According to the RFC for sending emails, when you have a double CRLF, that is the end of the headers.  For some reasong, Google is re-arranging the headers and adding extra returns.  WHY?

When the exact same email is sent to Hotmail or others, the headers are received exactly as they are sent.

Anyone else seing this same issue?   We have confirmed with another user that started experiencing this same issue yesterday as well.

 

Friday April 27, 2012 11:45am CST UPDATE WE HAVE A FIX, NO NEED TO WAIT ON GOOGLE TO FIX

Something changed with Google's Email servers and it seems to impact incoming email that is generated from the PHP MAIL() call.  Many servers use this (including the one at 3Gstore.com).  Emails that are sent in plain text are fine.  Emails that are sent as HTML emails are not displayed properly - all the HTML code is displayed.  However, the same email sent to Yahoo or Hotmail or others is fine, so the problem would appear to be something on Google's end, since emails have been sending perfectly for a few years.

This information is for the programmer/developer at your site.

They can fix the issue in 5 minutes.  In a nutshell, you have to replace the PHP MAIL() call with a direct SMTP Call.

More info on Mail call at http://php.net/manual/en/function.mail.php

 

Here are the steps to following:

  1. Download SMTP mail package from http://3gstore.com/smtp-fix.php
    • This package is slightly upgraded (and simplified) version of the email package originally developed by Manuel Lemos. The original package can be downloaded from http://poss.sourceforge.net/email/get/
  2. Unzip and open test.php in a text editor.
  3. Modify line numbers 4 to 7 with appropriate "from", "to", "subject" and "message".
  4. Save and upload all the files to your web server.
  5. Run test.php
  6. Enjoy!
  7. PS: The debug mode is enabled by default, you can disable it in class.smtp.php on line number 24.


This is what the php mail() called looked like before:

$from="info@domain.com";
$to="you@domain.com";
$subject="This is an HTML email";
$message="<h1>Sending HTML email via local server using SMTP method</h1><b>This</b> <i>is</i> <a href='http://google.com'>a</a> <u>test</u>";
$headers  = 'MIME-Version: 1.0' . "\r\n";
$headers .= 'Content-type: text/html; charset=iso-8859-1' . "\r\n";
$headers .= "From: ".$from . "\r\n";
mail($to,$subject,$message,$headers);


Here is the updated php to use the new SMTP mail package:

require("class.smtp.php");
$SMTP=new SMTP();
$from="info@domain.com";
$to="you@domain.com";
$subject="This is an HTML email";
$message="<h1>Sending HTML email via local server using SMTP method</h1><b>This</b> <i>is</i> <a href='http://google.com'>a</a> <u>test</u>";
$SMTP->send($from,$to,$subject,$message);



So, when sending emails from SMTP vs PHP MAIL, Google does not change the headers and emails will be delivered without any issue.  We have spent the last 24 hours tracking this down and testing and we hope that this helps others.

4/27/2012 4:37pm CST - Email we just received from Google:

 

Sorry for the regression, though the fault doesn't lie entirely with Gmail.

The problem is with the PHP mail class.  If you specify headers to add, then they get added with the wrong line ending.  Then, your helpful local mailer you're using to send (postfix, for example, in our testing).  SMTP mail is supposed to all be sent with CRLF line endings, but apparently postfix only checks the first header to see whether or not it needs to do the conversion.  This means that if the first line has only an LF, and some other line has a CRLF, you'll wind up with CRCRLF on the wire.

Our code interprets CRCRLF as CRLFCRLF, ie two lines, which will end your headers early.

GIGO, as they say.  Unless there's something in one of the RFCs I missed about how to correctly handle this type of messed up data.

Anyways, we've had a long time bugfix for this broken condition, but we recently had a regression where the bugfix wasn't being applied.  Its already been fixed and should be out in the next release next week.

I should point out that this obviously doesn't require PHP, it just requires sending mixed ending headers to some sendmail clones.

Sorry for the trouble.

 

So, it appears that Google will fix this next week - THANKS GOOGLE! 




Last Updated ( Friday, 27 April 2012 )
 
< Prev   Next >

EVDO News, Tips, Products, Reviews, Verizon and Sprint Experts.
Welcome to the #1 source for EVDO Information. Search our EVDO forums, read our EVDO Blogs, check EVDO coverage and when ready, buy your EVDO products from us, your EVDO Experts. Call us @ 1-866-3GSTORE.
 
The image “http://www.evdoinfo.com/images/stories/evdo_easy_button.jpg” cannot be displayed, because it contains errors.
 

CrawlTrack: free crawlers and spiders tracking script for webmaster- SEO script -script gratuit de dÔøΩtection des robots pour webmaster