| Author |
Message
|
| mqjeff |
 Posted: Tue Oct 02, 2012 11:50 am Post subject: |
 |
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| lancelotlinc wrote: |
| mqjeff wrote: |
| At 7.0.0.3 you should be able to release the file handles just as easily by undeploying the flow that contains the trace node - without needing to restart the EG. |
rekarm intimates in the above quote that the EG holds the handle. I agree with you, the design should be at the flow level, but my experience and rekarm's quote point to the EG level. |
You're misreading rekarm's quote. He's saying that if you DO NOT CHANGE the deploy configuration, but merely DELETE THE TRACE FILE, then the file is still being written to. He's talking about changes to the file at the file system level, not changes to the broker configuration.
In particular, he says
| rekarm01 wrote: |
| The net effect of redeploying a barfile is to close the trace file, and then re-open (re-create) it. |
If you simply undeploy the flow, it will close the trace file and free the file handle. It won't delete the trace file, but it will stop writing to it. |
|
| Back to top |
|
 |
| simon.starkie |
| Posted: Tue Oct 02, 2012 12:49 pm Post subject: |
|
|
Disciple
Joined: 24 Mar 2002
Posts: 180
|
We are running WMB6, sorry, I should have mentioned that at the outset.
Let me try a test I considered while walking at lunch in 100*F temps.
I'll create a brand new flow with just MQInput, Compute and MQOutput nodes.
Plus an unconnected Trace of ${ExceptionList} to a brand-new file in /tmp.
And see what happens. |
|
| Back to top |
|
|
| simon.starkie |
| Posted: Tue Oct 02, 2012 1:24 pm Post subject: |
|
|
Disciple
Joined: 24 Mar 2002
Posts: 180
|
Well, no file was created/written by the unconnected Trace node.
This is the expected result.
But doesn't explain why my original flow misbehaved.
I'll keep digging. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Oct 02, 2012 1:24 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Again, I suspect the original file misbehaved because it wasn't deployed with the full changes. |
|
| Back to top |
|
|
| simon.starkie |
| Posted: Tue Oct 02, 2012 1:35 pm Post subject: |
|
|
Disciple
Joined: 24 Mar 2002
Posts: 180
|
Tested with the Trace node connected and the expected ExceptionList Trace files was created in /tmp.
Disconnected the Trace node and re-tested and observed no additional entries in the existing Trace file in /tmp.
Not sure what could have misbehaved with the original BAR file.
But I am sure that removing all the Trace nodes solves the problem.
This has been an interesting problem.
But it's time for me to let go and move on.
Thanks everyone for the outstanding help.  |
|
| Back to top |
|
|
| mqjeff |
| Posted: Tue Oct 02, 2012 1:40 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| simon.starkie wrote: |
| But it's time for me to let go and move on. |
Hopefully to getting them to upgrade prod to some version that doesn't start with a '6'. |
|
| Back to top |
|
|
| simon.starkie |
| Posted: Tue Oct 02, 2012 1:59 pm Post subject: |
|
|
Disciple
Joined: 24 Mar 2002
Posts: 180
|
| mqjeff wrote: |
| simon.starkie wrote: |
| But it's time for me to let go and move on. |
Hopefully to getting them to upgrade prod to some version that doesn't start with a '6'. |
Yep, they are in the process of doing a WMB8 conversion.
It's quite a trick to migrate all of their legacy flows to a new version of WMB.
Especially with the custom HL7 parser.
Tougher than woodpecker lips!
Last edited by simon.starkie on Tue Oct 02, 2012 5:28 pm; edited 2 times in total |
|
| Back to top |
|
|
| kimbert |
| Posted: Tue Oct 02, 2012 2:58 pm Post subject: |
|
|
Jedi Council
Joined: 29 Jul 2003
Posts: 5542
Location: Southampton
|
Hmmm...wet hens and woodpecker lips. You must be an ornithologist.
re: the custom HL7 parser...you should make them aware of the Healthcare Connectivity pack in WMB8. It's based on WMB parsers ( MRM for now, DFDL in the future ) and does a lot more than just parsing and writing of HL7 message. |
|
| Back to top |
|
|
| simon.starkie |
| Posted: Tue Oct 02, 2012 3:09 pm Post subject: |
|
|
Disciple
Joined: 24 Mar 2002
Posts: 180
|
|
| Back to top |
|
|
| Esa |
| Posted: Tue Oct 02, 2012 11:16 pm Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| simon.starkie wrote: |
This was a brand new flow with unique file names in PROD.
I was astonished that /tmp eventually filled to 98% after running for about a month.
And that was because of these unconnected Trace nodes.
The fact that it happened in PROD has made alot of people madder than a wet hen  |
You did not explain your deployment process. How do you move a flow in production? Do you deploy a bar file that has been tested in your test environment or do you create a brand new bar file from the version control system, or something else?
If you deployed a tested bar file in production it is possible that the last version of the flow had not compiled and you were actually deploying an older version - with the trace nodes attached... In that case you can probably find the trace files in /tmp of your test environment, too. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed Oct 03, 2012 1:49 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| lancelotlinc wrote: |
| rekarm intimates in the above quote that the EG holds the handle. |
The above quote was from a different topic, for a somewhat different issue; the point there was that once something has an open file handle, (whether it's the EG process or message flow thread), it could keep writing to a "deleted" file, until it closes the file handle, (either by undeploying, redeploying, or stopping the EG, etc.).
| kimbert wrote: |
| re: the custom HL7 parser...you should make them aware of the Healthcare Connectivity pack in WMB8. |
If merely migrating "all of their legacy flows" from WMB6 to WMB8 is "quite a trick", then replacing a custom HL7 parser with WMB parsers in all those legacy flows might take a while ... |
|
| Back to top |
|
|
| simon.starkie |
| Posted: Wed Oct 03, 2012 6:22 am Post subject: |
|
|
Disciple
Joined: 24 Mar 2002
Posts: 180
|
| Esa wrote: |
| simon.starkie wrote: |
This was a brand new flow with unique file names in PROD.
I was astonished that /tmp eventually filled to 98% after running for about a month.
And that was because of these unconnected Trace nodes.
The fact that it happened in PROD has made alot of people madder than a wet hen |
You did not explain your deployment process. How do you move a flow in production? Do you deploy a bar file that has been tested in your test environment or do you create a brand new bar file from the version control system, or something else?
If you deployed a tested bar file in production it is possible that the last version of the flow had not compiled and you were actually deploying an older version - with the trace nodes attached... In that case you can probably find the trace files in /tmp of your test environment, too. |
We do not create a new BAR for each environment. We export the BAR from DEV and check it into Star Team with a particular Label. Our deploy tool uses the Label to take the BAR file from StarTeam and deploys it to QA or UAT or PROD and also creates an additional Star Team Label showing which environment the particular Label was deployed to. In this case, the additional Labels show the same BAR was deployed seperately to each of the three environments as the testing process proceeded. And it turns out the problem is occurring in all three environments but unfortunately nobody noticed that until now which is why the code was eventually promoted to PROD. |
|
| Back to top |
|
|
| simon.starkie |
| Posted: Wed Oct 03, 2012 6:27 am Post subject: |
|
|
Disciple
Joined: 24 Mar 2002
Posts: 180
|
| rekarm01 wrote: |
| kimbert wrote: |
| re: the custom HL7 parser...you should make them aware of the Healthcare Connectivity pack in WMB8. |
If merely migrating "all of their legacy flows" from WMB6 to WMB8 is "quite a trick", then replacing a custom HL7 parser with WMB parsers in all those legacy flows might take a while ... |
Yes, replacing the custom HL7 parser with WMB parsers in all the legacy flows is huge, time consuming and expensive. |
|
| Back to top |
|
|
| Esa |
| Posted: Wed Oct 03, 2012 7:36 am Post subject: |
|
|
Grand Master
Joined: 22 May 2008
Posts: 1387
Location: Finland
|
| simon.starkie wrote: |
We do not create a new BAR for each environment. We export the BAR from DEV and check it into Star Team with a particular Label. Our deploy tool uses the Label to take the BAR file from StarTeam and deploys it to QA or UAT or PROD and also creates an additional Star Team Label showing which environment the particular Label was deployed to. In this case, the additional Labels show the same BAR was deployed seperately to each of the three environments as the testing process proceeded. And it turns out the problem is occurring in all three environments but unfortunately nobody noticed that until now which is why the code was eventually promoted to PROD. |
If I remember corrcectly, in V6 you did not get any warning if the flow did not compile when you rebuilt the bar file. You had to check the compile timestamp to make sure that the flow really compiled. Otherwise the old compilation was left in place. V7 removes the old compiled flow from the bar file before recompilation and it gives you a warning in a pop-up window if the compilation fails.
So it is obvious the compilation of the new version - with trace nodes disconnected - failed and as a result the old version was redeployed.
That must be how it went, don't you agree?
If you have the bar files archived somewhere, you can check the compilation time to verify this. |
|
| Back to top |
|
|
| simon.starkie |
| Posted: Wed Oct 03, 2012 8:45 am Post subject: |
|
|
Disciple
Joined: 24 Mar 2002
Posts: 180
|
| Esa wrote: |
| simon.starkie wrote: |
We do not create a new BAR for each environment. We export the BAR from DEV and check it into Star Team with a particular Label. Our deploy tool uses the Label to take the BAR file from StarTeam and deploys it to QA or UAT or PROD and also creates an additional Star Team Label showing which environment the particular Label was deployed to. In this case, the additional Labels show the same BAR was deployed seperately to each of the three environments as the testing process proceeded. And it turns out the problem is occurring in all three environments but unfortunately nobody noticed that until now which is why the code was eventually promoted to PROD. |
If I remember corrcectly, in V6 you did not get any warning if the flow did not compile when you rebuilt the bar file. You had to check the compile timestamp to make sure that the flow really compiled. Otherwise the old compilation was left in place. V7 removes the old compiled flow from the bar file before recompilation and it gives you a warning in a pop-up window if the compilation fails.
So it is obvious the compilation of the new version - with trace nodes disconnected - failed and as a result the old version was redeployed.
That must be how it went, don't you agree?
If you have the bar files archived somewhere, you can check the compilation time to verify this. |
Here's a snapshot of the BAR file showing the compile date/time for the flow was the same as all the other included files. This is expected because the project was rebuilt on that date/time.
[img] [/img] |
|
| Back to top |
|
|
|
|
