ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum IndexWebSphere Message Broker SupportEmailOutput Node MIME Binary Attachment Encoding Issues

Post new topicReply to topic Goto page 1, 2  Next
EmailOutput Node MIME Binary Attachment Encoding Issues View previous topic :: View next topic
Author Message
eammnsh
PostPosted: Sat Dec 31, 2016 9:55 am Post subject: EmailOutput Node MIME Binary Attachment Encoding Issues Reply with quote

Apprentice

Joined: 22 Aug 2016
Posts: 29

We have IIB V10.0.0.7 running on Linux Redhat X86-64.

we are trying to send an Email through EmailOutput node with MIME attachment, the main issue is that we are reeving the attachment but corrupted, and that's because we can see extra X'0A' and X'0D' are added to our attachment.

The extra X'0A and X'0D' are not affecting the tex, csv, ..etc files, but for sure corrupt the Images, docs, pdfs, etc..

Below are our code, and the comparison between the original file and the received one by Email:
Code:


   BEGIN
      CALL CopyMessageHeaders();
      
      SET OutputRoot.EmailOutputHeader.To = InputRoot.XMLNSC.MSG.Body.sendEmailRq.emailTo;
   SET OutputRoot.EmailOutputHeader.Cc = InputRoot.XMLNSC.MSG.Body.sendEmailRq.emailCC;
   
   SET OutputRoot.EmailOutputHeader.Subject = InputRoot.XMLNSC.MSG.Body.sendEmailRq.subject;

 -- Create a new MIME message body, which will be sent as the main text of the email,
     -- including an attachment.
    CREATE FIELD OutputRoot.MIME TYPE Name;
    DECLARE M REFERENCE TO OutputRoot.MIME;

     -- Create the Content-Type child of MIME explicitly to ensure the correct order. If we set
     -- the ContentType property instead, the field could appear as the last child of MIME.
      CREATE FIELD M."Content-Type" TYPE NameValue VALUE 'multipart/Mixed; boundary=myBoundary';
     CREATE FIELD M."Content-ID"   TYPE NameValue VALUE 'new MIME document';

     CREATE LASTCHILD OF M TYPE Name NAME 'Parts';
    CREATE LASTCHILD OF M.Parts TYPE Name NAME 'Part';
     DECLARE P1 REFERENCE TO M.Parts.Part;

     -- First part:
     --   Create the body of the email.
     --   The body of the email has the text 'This is the main body of the email.'.
     CREATE FIELD P1."Content-Type" TYPE NameValue VALUE 'text/plain; charset=utf-8';
     CREATE FIELD P1."Content-Transfer-Encoding" TYPE NameValue VALUE '8bit';
    CREATE LASTCHILD OF P1 TYPE Name NAME 'Data';
    CREATE LASTCHILD OF P1.Data DOMAIN('BLOB') PARSE(CAST(InputRoot.XMLNSC.MSG.Body.sendEmailRq.body AS BLOB CCSID 1208));
    
     CREATE LASTCHILD OF M.Parts TYPE Name NAME 'Part';
     DECLARE P2 REFERENCE TO M.Parts.Part[2];

     -- Second part:
     --   Create the attachment of an email.
   
     CREATE FIELD P2."Content-Type" TYPE NameValue VALUE InputRoot.XMLNSC.MSG.Attachment.*[<].MIME_Headers."Content-Type";
     CREATE FIELD P2."Content-Transfer-Encoding" TYPE NameValue VALUE InputRoot.XMLNSC.MSG.Attachment.*[<].MIME_Headers."Content-Transfer-Encoding";
     CREATE FIELD P2."Content-Id" TYPE NameValue VALUE InputRoot.XMLNSC.MSG.Attachment.*[<].MIME_Headers."Content-Id";
      
     CREATE LASTCHILD OF P2 TYPE Name NAME 'Data';
     CREATE LASTCHILD OF P2.Data DOMAIN('BLOB');
     SET P2.Data.BLOB.BLOB = InputRoot.XMLNSC.MSG.Attachment.*[<].BLOB.BLOB;

      RETURN TRUE;
   END;



To see the image, use the below HTML
Code:
<img src='data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAABRoAAAJOCAYAAADYnbtEAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAA
DsMAAA7DAcdvqGQAAP+lSURBVHhe7L2/byU7ri7qP+YGN+zkBA5f+NJJnE5wYKCzB+zQyTXQOKkxkeHAwE09B9gwMIAjw5lvMMD1P2A0OtvxROckG6gnUlIVxa
JYlFbV+uHFb/BNe5UkiiJV1eK3a7UvhhXxr3/9C/m//tf/Gn92Op1Op9PpdDqdTqfT6XQ6nV+fLjQ6nU6n0+l0Op1Op9PpdDqdzp3pQqPT6XQ6nU6n0+l0Op1Op9Pp
3JkuNDqdTqfT6XQ6nU6n0+l0Op3OnelCo9PpdDqdTqfT6XQ6nU6n0+ncmS40Op1Op9PpdDqdTqfT6XQ6nc6d6UKj0+l0Op1Op9PpdDqdTqfT6dyZxys0/udfh4uLv
w7/Sa79518vwrXIf/uPj7I/EMb89T/Dzx/Df/zb1Pev/8n6mGwo82k2BL+Bk61/G/7jo2wbWRlbvU7J+nz8x7+l+Si5DTlOtrH/Ofy1aKfrSm0pjpzRvhKHHu4Qd97Hsn
7s82//MXyQa5Gw9pXXtil5Htn9sjmVeH38x/Bvq8ayttZTy5nT6XQ6e6meC+AsQc8us7OFcr7MJDaO5hyZWJ5vcj/DmgLlsYHaWpD8717qf2o7gfNi8/pNcVl5bZuydo
baF5V4+XnR6XQ6j45HKDSmA89f/xoe5uQv8vCXyH+Mf6FJD3oYl6/95/CfuS/+5ZPtwDj6s2KjOl/NRsXvQDi8yIeMzNrYus3lsSXhgDT3oRankrWxNH54ABuFt9j2b
2M+KKEt+DuLfS93ibutz3z9aQ2rCY09Y9bivufm8+1z/tpch4y/0+l0OvdF/e98OE......
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Sat Dec 31, 2016 3:38 pm Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19921
Location: LI,NY

Looks like you define the image as hex encoded whereas the content is defined as per the image tag as base 64 encoded....
Base 64 has certain rules as to the length of a line... and will introduce as many line breaks as necessary. Reading the data with a base64 decoder can then drop the new lines and reassemble the data as one stream...

You need to define the container according to the data it contains and not to your wishes...
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
eammnsh
PostPosted: Sun Jan 01, 2017 11:29 am Post subject: Reply with quote

Apprentice

Joined: 22 Aug 2016
Posts: 29

No, The image is in binary, but the image attached in this post is a screenshot of the hex comparison between the original image and the received one by email, dont mix things:)
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Sun Jan 01, 2017 12:09 pm Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19921
Location: LI,NY

eammnsh wrote:
No, The image is in binary, but the image attached in this post is a screenshot of the hex comparison between the original image and the received one by email, dont mix things:)

Never said that the image wasn't in binary. However the MIME part you defined is defined as
Code:
CREATE FIELD P1."Content-Type" TYPE NameValue VALUE 'text/plain; charset=utf-8';
     CREATE FIELD P1."Content-Transfer-Encoding" TYPE NameValue VALUE '8bit';

whereas it should probably be defined as 'application/64bit encoded'...

This is probably the reason why the CRLF(0d0a) that base64 encoding inserts when representing the binary values. Defining the content correctly as 64 bit encoded would then strip the inserted CRLF before rendering.

Have fun
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
eammnsh
PostPosted: Tue Jan 03, 2017 3:29 am Post subject: Reply with quote

Apprentice

Joined: 22 Aug 2016
Posts: 29

Part 1 of MIME tree contains the Email body, which you are referring to in your reply, not the attachment. Part 2 does contain the attachment with the proper information.

But let me ask my question in different way based on your reply, does the EmailOutput node only support Base64 content type for Binary attachments?

Below is my MIME attachment values for Part 2 (attachment)

Content-Type: application/octet-stream; name=IMAGE.PNG

Content-Transfer-Encoding: binary

Content-ID: <IMAGE.PNG>

Content-Disposition: attachment; name="IMAGE.PNG"; filename="IMAGE.PNG"

Thanks.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Tue Jan 03, 2017 6:28 am Post subject: Reply with quote

Grand Poobah

Joined: 18 Nov 2003
Posts: 19921
Location: LI,NY

eammnsh wrote:
Part 1 of MIME tree contains the Email body, which you are referring to in your reply, not the attachment. Part 2 does contain the attachment with the proper information.

But let me ask my question in different way based on your reply, does the EmailOutput node only support Base64 content type for Binary attachments?

Below is my MIME attachment values for Part 2 (attachment)

Content-Type: application/octet-stream; name=IMAGE.PNG

Content-Transfer-Encoding: binary

Content-ID: <IMAGE.PNG>

Content-Disposition: attachment; name="IMAGE.PNG"; filename="IMAGE.PNG"

Thanks.

Well this definition is fine for strict binary... I'd go with Base64 encoded though. With Mime this is a nice way to display binary as characters and a little bit more compact than strict hex...
What happens when you try it?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
eammnsh
PostPosted: Tue Jan 03, 2017 7:44 am Post subject: Reply with quote

Apprentice

Joined: 22 Aug 2016
Posts: 29

Ok, this is the hex representation, I am expecting :

Code:
89 50 4E 47 0D 0A 1A 0A 00 00 00 0D 49


and this is what I am getting

Code:
89 50 4E 47 0D 0A 1A 0D 0A 00 00 00 0D 0A 49


As you can notice, There is an Extra X'0D' and X'0A' in what I get.

I tried to send it as Base64

Content-Type: text/plain; charset=Cp1252; name=IMAGE.PNG
Content-Transfer-Encoding: base64
Content-ID: <IMAGE.PNG>

but still I am getting the same response with the corrupted data above.
_________________
WMB 7 and 8 IIB 9 and 10
MQ 7, 7.5, and 8
Since 2012
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Jan 03, 2017 7:48 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

It looks like you are only getting an extra X'0A' as the X'0D' is in both expected and received.

This is unix-windows line endings.

Are you sure the ' charset=Cp1252' should be Cp1252 and not UTF-8 or etc?
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
Vitor
PostPosted: Tue Jan 03, 2017 7:51 am Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 25552
Location: Ohio, USA

eammnsh wrote:
As you can notice, There is an Extra X'0D' and X'0A' in what I get.


You're using the wrong CCSID. You're not getting an extra X'0D', you're getting an extra X'0A'. Unix platforms (and their CCSIDs) represent end-of-line with X'0D', Windoze represents end-of-line with a X'0D' X'0A' pair.

Functioning as coded.
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
timber
PostPosted: Tue Jan 03, 2017 8:06 am Post subject: Reply with quote

Guardian

Joined: 25 Aug 2015
Posts: 947

I'm not convinced yet. 0x0D and 0x0A are separate characters. Windows just happens to use a pair of characters to represent an end-of-line. Using the wrong CCSID can cause individual characters to be parsed/written incorrectly but it does not magically cause extra characters to appear.

@eammnsh: This looks like some kind of EOL character normalization. Is there anything in the processing pathway that could be doing that?
Back to top
View user's profile Send private message
eammnsh
PostPosted: Tue Jan 03, 2017 8:08 am Post subject: Reply with quote

Apprentice

Joined: 22 Aug 2016
Posts: 29

Nop, this is not true here is is another sample of the data:

original data:
Code:
F5 F1 0A 5F CC 05 7F 1B C4 8D 08 CC D6 17


What I am receiving
Code:
F5 F1 0D 0A 5F CC 05 7F 1B C4 8D 08 CC D6 17


In this sample you will find extra '0D' not '0A' )
_________________
WMB 7 and 8 IIB 9 and 10
MQ 7, 7.5, and 8
Since 2012
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Jan 03, 2017 8:14 am Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

... This ESQL is confusing
Code:
SET P2.Data.BLOB.BLOB = InputRoot.XMLNSC.MSG.Attachment.*[<].BLOB.BLOB


Does the XML document really have a field named "MSG.Attachment.<something>.BLOB.BLOB" ?

Or are you building that BLOB.BLOB from somewhere ?
_________________
chmod -R ugo-wx /
Back to top
View user's profile Send private message
eammnsh
PostPosted: Tue Jan 03, 2017 8:21 am Post subject: Reply with quote

Apprentice

Joined: 22 Aug 2016
Posts: 29

@mqjeff: I am receiving the attachment in MIME part with SOAP request, so I am just taking the incoming BLOB and putting it inside my XMLNSC tree:

Code:


CREATE LASTCHILD OF  OutBodyRef.MSG.Attachment.*[<] DOMAIN 'BLOB';
      
SET OutBodyRef.MSG.Attachment.*[<].BLOB.BLOB = InBodyRef.Attachment.*[<].BLOB.BLOB;
   


@timber: even when I leave my content encoding to Binary, and I dont touch the incoming BLOB within the SOAP request, I get the same results.
Do you think SOAP-UI client can be doing this to the attachment?
For your information my server is linux redhat x86-64 and I am using SOAP-UI from a windows machine.
_________________
WMB 7 and 8 IIB 9 and 10
MQ 7, 7.5, and 8
Since 2012
Back to top
View user's profile Send private message
adubya
PostPosted: Tue Jan 03, 2017 8:54 am Post subject: Reply with quote

Partisan

Joined: 25 Aug 2011
Posts: 377
Location: GU12, UK

Have you verified that the extra characters are being added in Broker and not by SOAP-UI ?
_________________
Independent Middleware Consultant
andy@knownentity.com
Back to top
View user's profile Send private message Send e-mail
timber
PostPosted: Tue Jan 03, 2017 2:47 pm Post subject: Reply with quote

Guardian

Joined: 25 Aug 2015
Posts: 947

That was going to be my next question. At this point, I would be putting Trace nodes into the flow and inspecting the byte arrays that are being received/produced.
Back to top
View user's profile Send private message
Display posts from previous:
Post new topicReply to topic Goto page 1, 2  Next Page 1 of 2

MQSeries.net Forum IndexWebSphere Message Broker SupportEmailOutput Node MIME Binary Attachment Encoding Issues
Jump to:



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP


Theme by Dustin Baccetti
Powered by phpBB 2001, 2002 phpBB Group

Copyright MQSeries.net. All rights reserved.