| Author |
Message
|
| Buzz Lightyear |
 Posted: Tue Jun 13, 2017 3:32 am Post subject: Zip file as an email attachment. |
 |
|
Newbie
Joined: 29 May 2017
Posts: 9
|
Reviving a very old thread here and hoping to find the answer.
I am trying to send out an email with a zip attachment through my message flow application deployed on WMB 7.0.3. The issue that I face is that the zip file attachment is sent through the email; I can open the zip file; but inside it, I do not see any files.
Below is the code snippet that I use to create the zip file. This is an external Java procedure that I call through from my ESQL code.
| Code: |
public static void createZipFile(MbElement[] fileSetRef) {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ZipOutputStream zos = new ZipOutputStream(baos);
byte[] packedData = null;
MbElement child = fileSetRef[0].getFirstChild(); // File element.
while(true) {
// navigate through the MbElement tree
String fName = (String) child.getValueAsString(); // File Name
byte[] data = (byte[]) nxtChild.getValue(); // BLOB
ZipEntry ze = new ZipEntry(fName);
zos.putNextEntry(ze);
zos.write(data, 0, data.length);
zos.closeEntry();
// loop through all child nodes
} // loop ends.
zos.finish();
zos.close();
packedData = baos.toByteArray();
MbElement packedDataElement = fileSetRef[0].getFirstChild().createElementAfter(MbBLOB.PARSER_NAME);
packedDataElement.setName("PackedData");
packedDataElement.setValue(packedData);
} // method ends
|
Below is the ESQL code to call the above Java function.
| Code: |
DECLARE refFileSet REFERENCE TO Environment.Variables.FileSet; -- contains File fields. File field contains FileName and Data (BLOB) fields.
CALL CreateZipFile(refFileSet);
CREATE LASTCHILD OF OutputRoot.MIME.Parts TYPE Name NAME 'Part';
DECLARE mimePart REFERENCE TO OutputRoot.MIME.Parts.Part[<];
CREATE FIELD mimePart."Content-Type" TYPE NameValue VALUE 'application/zip; name=log.zip';
CREATE FIELD mimePart."Content-Transfer-Encoding" TYPE NameValue VALUE '8bit';
CREATE LASTCHILD OF mimePart TYPE Name NAME 'Data';
CREATE LASTCHILD OF mimePart.Data DOMAIN('BLOB');
SET mimePart .Data.BLOB.BLOB = refFileSet.PackedData;
|
I can confirm that the Java code works. Writing the byte array "packedData" to file output file stream (all this within the Java code), results in a file with the zipped contents. Its only then the control returns back to ESQL module and the BLOB is set in the output MIME tree that the details are lost.
What am I doing wrong here ? |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Tue Jun 13, 2017 3:59 am Post subject: Re: Zip file as an email attachment. |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Buzz Lightyear wrote: |
| Reviving a very old thread here and hoping to find the answer. |
Perhaps a moderator will split your question into a new thread, where it should be... you could then edit it to have a link to this old thread.
Otherwisee, what does it show using the debugger?
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jun 13, 2017 4:38 am Post subject: Re: Zip file as an email attachment. |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| mqjeff wrote: |
| Buzz Lightyear wrote: |
| Reviving a very old thread here and hoping to find the answer. |
Perhaps a moderator will split your question into a new thread, where it should be... |
Done.
| mqjeff wrote: |
| you could then edit it to have a link to this old thread. |
Including this at no extra charge.
| mqjeff wrote: |
| Otherwisee, what does it show using the debugger? |
Better still, what does the user trace (in conjunction with a Trace node just before the EmailOutput node) say about the message tree?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jun 13, 2017 4:52 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
I would look at your static Java function. I'd expect it to may be return an MbElement?
It looks to me that you may be just modifying the one being passed but then it needs to be passed by reference and not by value... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Buzz Lightyear |
| Posted: Tue Jun 13, 2017 5:19 am Post subject: |
|
|
Newbie
Joined: 29 May 2017
Posts: 9
|
@Vitor: Thank you for spawning a new thread.
Well, the debugger shows the BLOB data in the reference that was passed to the Java function. This is later set in the MIME which works out well as the email carries that attachment. Its a zip ! It opens ! But no file contents, which is weird !
@fjp_saper: Yes, the CreateZipFile works on the MbElement. Below is the ESQL declaration for it.
| Code: |
CREATE PROCEDURE CreateZipFile (INOUT fileSet REFERENCE)
LANGUAGE JAVA
EXTERNAL NAME "com.examples.utils.ZipFileHelper.createZipFile";
|
Below is the way I call this function.
| Code: |
DECLARE refFileSet REFERENCE TO Environment.Variables.FileSet;
CALL CreateZipFile(refFileSet);
|
Subsequent code simply uses this refFileSet reference variable. I didn't quite understand the "passed by reference and not by value" remark.
Do I need to do anything differently ? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jun 13, 2017 5:28 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Java parameters are only passed by value. So your INOUT has no meaning if there is not a return (MbElement ) parmname statement in Java.
When passing MbElement you're in fact passing an non mutable pointer to MbElement. That means that you can manipulate the content of the class by using its method, however you cannot change the pointer or instantiate the variable by passing it a new pointer...
So myobject.someMethod(parms) will change the content of the class as seen from the calling program...
myobject = xyz will have no effect to the content or pointer of myobject as far as the calling program is concerned...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Buzz Lightyear |
| Posted: Tue Jun 13, 2017 8:11 am Post subject: |
|
|
Newbie
Joined: 29 May 2017
Posts: 9
|
Not sure I understood. Indeed Java parameters are pass by value. But those values are addresses. Hence something like the below should work.
| Code: |
public static void createZipFile(MbElement[] fileSetRef) {
...
...
...
MbElement packedDataElement = fileSetRef[0].getFirstChild().createElementAfter(MbBLOB.PARSER_NAME);
packedDataElement.setName("PackedData");
packedDataElement.setValue(packedData);
} // method ends
}
|
And it does work. Below is the before and after of the reference passed between ESQL and Java function as seen in the debugger.
Before:
| Quote: |
refFileSet
File
FileName:CHARACTER:file1.xml
DataBinary:BLOB:[B@7c127c12
File
FileName:CHARACTER:file2.xml
DataBinary:BLOB:[B@3140314
File
FileName:CHARACTER:file3.xml
DataBinary:BLOB:[B@e5e0e5e
File
FileName:CHARACTER:file4.xml
DataBinary:BLOB:[B@19a619a6
File
FileName:CHARACTER:file5.xml
DataBinary:BLOB:[B@24f024f0
|
After:
| Quote: |
refFileSet
File
FileName:CHARACTER:file1.xml
DataBinary:BLOB:[B@55845584
PackedData:BLOB:[B@5a825a82
File
FileName:CHARACTER:file2.xml
DataBinary:BLOB:[B@645a645a
File
FileName:CHARACTER:file3.xml
DataBinary:BLOB:[B@6e1a6e1a
File
FileName:CHARACTER:file4.xml
DataBinary:BLOB:[B@79627962
File
FileName:CHARACTER:file5.xml
DataBinary:BLOB:[B@4b804b8
|
Note that the BLOB addresses have changed.
I surely cannot have a returns (MbElement ) parameter name statement in the Java code as this would require me to modify my function signature to
| Code: |
| public static MbElement createZipFile(MbElement[] fileSetRef) { |
Doing so would return the following error.
| Code: |
RecoverableException
File:CHARACTER:/build/S700_P/src/DataFlowEngine/ImbRdl/ImbRdlExternalJava.cpp
Line:INTEGER:1123
Function:CHARACTER:ESQL2JavaMethodResolver::decodeReturnStatus
Type:CHARACTER:
Name:CHARACTER:
Label:CHARACTER:
Catalog:CHARACTER:BIPmsgs
Severity:INTEGER:3
Number:INTEGER:2928
Text:CHARACTER:The Java return type does not match the ESQL return type
Insert
Type:INTEGER:5
Text:CHARACTER:com.examples.utils.ZipFileHelper.createZipFile
|
Beats me ! |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jun 13, 2017 9:22 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
What I meant to say is that you have defined Filesetref as INOUT whereas it is an IN parameter... There is no out...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Buzz Lightyear |
| Posted: Tue Jun 13, 2017 9:50 am Post subject: |
|
|
Newbie
Joined: 29 May 2017
Posts: 9
|
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jun 13, 2017 12:42 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
The point here is that you don't return it as you did not change it. What you did was manipulate it's contents... So an IN would have been sufficient...
Same as using a procedure in ESQL and passing a reference. As long as you never "MOVE" the reference...
For INOUT you should have used MbElement[] as signature for the return type... 
[edit] He did use the correct signatures however there is no reference change of MbElelment[0] and as such clarification should have shown that it is an IN parameter only and not INOUT and have given a different signature to Java. [/edit]
And the point was that you're not returning anything from your Java function as shown by the void in your signature.
[edit]The point was that the pointer MbElement[0] never got changed... so why an INOUT? [/edit]
_________________
MQ & Broker admin
Last edited by fjb_saper on Thu Jun 15, 2017 2:18 am; edited 2 times in total |
|
| Back to top |
|
|
| Buzz Lightyear |
| Posted: Wed Jun 14, 2017 10:17 am Post subject: |
|
|
Newbie
Joined: 29 May 2017
Posts: 9
|
Changing the signature to the one below resulted in an exception. The method was not found.
| Code: |
public static MbElement[] createZipFile(MbElement[] fileSetRef) {
...
...
}
|
I am really sorry about this, but can you please illustrate with an example ? By adding a new element "PackedData" of BLOB domain to the reference tree am I not changing the reference ? Or is it manipulating the tree ? I don't seem to understand the nuance there-in !
Are you suggesting that I create a PackedData element in the ESQL code before passing the reference to the Java function ?
[/quote] |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Jun 14, 2017 11:44 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
Despite what my esteemed colleague says, I'm going to state that your original procedure definition and method call were correct.
You should make that call either a) in a separate compute node or b) with a propagate statement that points to an out terminal that goes to a trace node.
Basically, pass the contents of your message tree after the java procedure, but before anything else, to a trace node - ideally pointing to user trace.
I would also do a trace node output without the attachment to make sure you like the MIME tree.
Then you can see what you get back.
You might also be having CCSID issues, where the zip file data is being transformed into text inappropriately.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Jun 14, 2017 7:40 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Buzz Lightyear wrote: |
Changing the signature to the one below resulted in an exception. The method was not found.
| Code: |
public static MbElement[] createZipFile(MbElement[] fileSetRef) {
...
...
}
|
I am really sorry about this, but can you please illustrate with an example ? By adding a new element "PackedData" of BLOB domain to the reference tree am I not changing the reference ? Or is it manipulating the tree ? I don't seem to understand the nuance there-in !
Are you suggesting that I create a PackedData element in the ESQL code before passing the reference to the Java function ?
|
Agreed the nuance is a little bit pedantic. What I meant was
A) Don't touch your Java procedure
B) Keep the ESQL signature as INOUT
C) Verify that it works as expected.
D) Specify in the comments that you are not changing the pointer for the reference.
The reason that you are asked to pass the INOUT parameter as MbElement[] is that you would now be able to change the content of the array and put a different instance of MBElement into the array, thus enabling the MbElement to be changed (i.e. reference to be Moved).
For what goes on beyond you need like Jeff said to add some logging to look at the tree. You may also have to byte64 encode your zip BLOB... before creating the attachment. This would be to mitigate any CCSID problem as I'd expect all to be UTF-8 (email protocol) ...
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Buzz Lightyear |
| Posted: Thu Jun 15, 2017 5:03 am Post subject: |
|
|
Newbie
Joined: 29 May 2017
Posts: 9
|
Thank you mqJeff for the intervention.
I rolled-back my CreateZipFile function signature and ESQL declaration to the one I started with. The Java function returned a BLOB as before. But this time around I was able to store it in a file on the file system using the FileOutputNode (I modified my flow a bit for this). The file created was a valid zip file. Its opened and I could see the contents in there. What did the trick were these two seemingly innocuous statements.
| Code: |
SET OutputRoot.Properties.Encoding = 273;
SET OutputRoot.Properties.CodedCharSetId = 1208;
|
However the mail attachment continues to be an empty zip. Below is the trace details (edited for brevity!).
| Quote: |
(0x01000000:Name):EmailOutputHeader = ( ['EMAILHDR' : 0x11bb815f0]
(0x03000000:NameValue):To = 'john.doe@who.where.com' (CHARACTER)
(0x03000000:NameValue):Cc = 'jane.doe@who.where.com' (CHARACTER)
(0x03000000:NameValue):From = 'doe@who.where.com' (CHARACTER)
(0x03000000:NameValue):Subject = 'Hello World !' (CHARACTER)
)
(0x01000000:Name):MIME = ( ['MIME' : 0x11bb84970]
(0x03000000:NameValue):Content-Type = 'multipart/related; boundary=bf2c9c7a-51c7-11e7-83a2-000000000000' (CHARACTER)
(0x03000000:NameValue):Content-ID = 'bf2ca45e-51c7-11e7-83a2-000000000000' (CHARACTER)
(0x01000000:Name ):Parts = (
(0x01000000:Name):Part = (
(0x03000000:NameValue):Content-Type = 'text/html; charset=UTF-8' (CHARACTER)
(0x03000000:NameValue):Content-Transfer-Encoding = '8bit' (CHARACTER)
(0x01000000:Name ):Data = (
(0x01000000:Name):BLOB = ( ['none' : 0x11bb85cb0]
(0x03000000:NameValue):BLOB = X'3c68746d686561643e047970620636f687474702d6c3e0a3c65717569766c3e0a3c3d22436f642d547970
... more bytes
' (BLOB)
)
)
)
(0x01000000:Name):Part = (
(0x03000000:NameValue):Content-Type = 'application/zip; name=application-logs.zip' (CHARACTER)
(0x03000000:NameValue):Content-Transfer-Encoding = '8bit' (CHARACTER)
(0x01000000:Name ):Data = (
(0x03000000:NameValue):BLOB = X'504b03041400080008004091cf4a000000000000000000000000280000004149535f534150494e5f4d42535f4d465f32303133313232325f303132333031353332312e786d
... more bytes
00040058010000fe1d00000000' (BLOB)
)
)
)
)
)
|
I confirmed that the attachment BLOB produced the valid zip file (using the FileOutputNode). Looks like I still need do something with the BLOB encoding specifically for email as indicated by fjb_saper.
I will get back to this thread with a solution or after losing a bit more of my hair ! |
|
| Back to top |
|
|
| mqjeff |
| Posted: Thu Jun 15, 2017 6:32 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
It kinda looks like the BLOB data is in a basic conversion ("X'....") instead of being in a real translation.
Try ASBITSTREAM with the CCSID/Encoding you found, like
| Code: |
SET mimePart .Data.BLOB.BLOB = ASBITSTREAM(refFileSet.PackedData ... ); |
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
|
|
