ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Unconnected Trace nodes ${ExceptionList} filling up /tm?

Post new topic  Reply to topic Goto page Previous  1, 2, 3  Next
 Unconnected Trace nodes ${ExceptionList} filling up /tm? « View previous topic :: View next topic » 
Author Message
mqjeff
PostPosted: Tue Oct 02, 2012 11:50 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

lancelotlinc wrote:
mqjeff wrote:
At 7.0.0.3 you should be able to release the file handles just as easily by undeploying the flow that contains the trace node - without needing to restart the EG.


rekarm intimates in the above quote that the EG holds the handle. I agree with you, the design should be at the flow level, but my experience and rekarm's quote point to the EG level.


You're misreading rekarm's quote. He's saying that if you DO NOT CHANGE the deploy configuration, but merely DELETE THE TRACE FILE, then the file is still being written to. He's talking about changes to the file at the file system level, not changes to the broker configuration.

In particular, he says
rekarm01 wrote:
The net effect of redeploying a barfile is to close the trace file, and then re-open (re-create) it.


If you simply undeploy the flow, it will close the trace file and free the file handle. It won't delete the trace file, but it will stop writing to it.
Back to top
View user's profile Send private message
simon.starkie
PostPosted: Tue Oct 02, 2012 12:49 pm    Post subject: Reply with quote

Disciple

Joined: 24 Mar 2002
Posts: 180

We are running WMB6, sorry, I should have mentioned that at the outset.
Let me try a test I considered while walking at lunch in 100*F temps.
I'll create a brand new flow with just MQInput, Compute and MQOutput nodes.
Plus an unconnected Trace of ${ExceptionList} to a brand-new file in /tmp.
And see what happens.
Back to top
View user's profile Send private message
simon.starkie
PostPosted: Tue Oct 02, 2012 1:24 pm    Post subject: Reply with quote

Disciple

Joined: 24 Mar 2002
Posts: 180

Well, no file was created/written by the unconnected Trace node.
This is the expected result.
But doesn't explain why my original flow misbehaved.
I'll keep digging.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Oct 02, 2012 1:24 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Again, I suspect the original file misbehaved because it wasn't deployed with the full changes.
Back to top
View user's profile Send private message
simon.starkie
PostPosted: Tue Oct 02, 2012 1:35 pm    Post subject: Reply with quote

Disciple

Joined: 24 Mar 2002
Posts: 180

Tested with the Trace node connected and the expected ExceptionList Trace files was created in /tmp.
Disconnected the Trace node and re-tested and observed no additional entries in the existing Trace file in /tmp.

Not sure what could have misbehaved with the original BAR file.
But I am sure that removing all the Trace nodes solves the problem.

This has been an interesting problem.
But it's time for me to let go and move on.
Thanks everyone for the outstanding help.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Tue Oct 02, 2012 1:40 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

simon.starkie wrote:
But it's time for me to let go and move on.

Hopefully to getting them to upgrade prod to some version that doesn't start with a '6'.
Back to top
View user's profile Send private message
simon.starkie
PostPosted: Tue Oct 02, 2012 1:59 pm    Post subject: Reply with quote

Disciple

Joined: 24 Mar 2002
Posts: 180

mqjeff wrote:
simon.starkie wrote:
But it's time for me to let go and move on.

Hopefully to getting them to upgrade prod to some version that doesn't start with a '6'.


Yep, they are in the process of doing a WMB8 conversion.
It's quite a trick to migrate all of their legacy flows to a new version of WMB.
Especially with the custom HL7 parser.

Tougher than woodpecker lips!


Last edited by simon.starkie on Tue Oct 02, 2012 5:28 pm; edited 2 times in total
Back to top
View user's profile Send private message
kimbert
PostPosted: Tue Oct 02, 2012 2:58 pm    Post subject: Reply with quote

Jedi Council

Joined: 29 Jul 2003
Posts: 5542
Location: Southampton

Hmmm...wet hens and woodpecker lips. You must be an ornithologist.

re: the custom HL7 parser...you should make them aware of the Healthcare Connectivity pack in WMB8. It's based on WMB parsers ( MRM for now, DFDL in the future ) and does a lot more than just parsing and writing of HL7 message.
Back to top
View user's profile Send private message
simon.starkie
PostPosted: Tue Oct 02, 2012 3:09 pm    Post subject: Reply with quote

Disciple

Joined: 24 Mar 2002
Posts: 180

Thanks.
Back to top
View user's profile Send private message
Esa
PostPosted: Tue Oct 02, 2012 11:16 pm    Post subject: Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

simon.starkie wrote:


This was a brand new flow with unique file names in PROD.
I was astonished that /tmp eventually filled to 98% after running for about a month.
And that was because of these unconnected Trace nodes.
The fact that it happened in PROD has made alot of people madder than a wet hen


You did not explain your deployment process. How do you move a flow in production? Do you deploy a bar file that has been tested in your test environment or do you create a brand new bar file from the version control system, or something else?

If you deployed a tested bar file in production it is possible that the last version of the flow had not compiled and you were actually deploying an older version - with the trace nodes attached... In that case you can probably find the trace files in /tmp of your test environment, too.
Back to top
View user's profile Send private message
rekarm01
PostPosted: Wed Oct 03, 2012 1:49 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 1415

lancelotlinc wrote:
rekarm intimates in the above quote that the EG holds the handle.

The above quote was from a different topic, for a somewhat different issue; the point there was that once something has an open file handle, (whether it's the EG process or message flow thread), it could keep writing to a "deleted" file, until it closes the file handle, (either by undeploying, redeploying, or stopping the EG, etc.).

kimbert wrote:
re: the custom HL7 parser...you should make them aware of the Healthcare Connectivity pack in WMB8.

If merely migrating "all of their legacy flows" from WMB6 to WMB8 is "quite a trick", then replacing a custom HL7 parser with WMB parsers in all those legacy flows might take a while ...
Back to top
View user's profile Send private message
simon.starkie
PostPosted: Wed Oct 03, 2012 6:22 am    Post subject: Reply with quote

Disciple

Joined: 24 Mar 2002
Posts: 180

Esa wrote:
simon.starkie wrote:


This was a brand new flow with unique file names in PROD.
I was astonished that /tmp eventually filled to 98% after running for about a month.
And that was because of these unconnected Trace nodes.
The fact that it happened in PROD has made alot of people madder than a wet hen


You did not explain your deployment process. How do you move a flow in production? Do you deploy a bar file that has been tested in your test environment or do you create a brand new bar file from the version control system, or something else?

If you deployed a tested bar file in production it is possible that the last version of the flow had not compiled and you were actually deploying an older version - with the trace nodes attached... In that case you can probably find the trace files in /tmp of your test environment, too.


We do not create a new BAR for each environment. We export the BAR from DEV and check it into Star Team with a particular Label. Our deploy tool uses the Label to take the BAR file from StarTeam and deploys it to QA or UAT or PROD and also creates an additional Star Team Label showing which environment the particular Label was deployed to. In this case, the additional Labels show the same BAR was deployed seperately to each of the three environments as the testing process proceeded. And it turns out the problem is occurring in all three environments but unfortunately nobody noticed that until now which is why the code was eventually promoted to PROD.
Back to top
View user's profile Send private message
simon.starkie
PostPosted: Wed Oct 03, 2012 6:27 am    Post subject: Reply with quote

Disciple

Joined: 24 Mar 2002
Posts: 180

rekarm01 wrote:

kimbert wrote:
re: the custom HL7 parser...you should make them aware of the Healthcare Connectivity pack in WMB8.

If merely migrating "all of their legacy flows" from WMB6 to WMB8 is "quite a trick", then replacing a custom HL7 parser with WMB parsers in all those legacy flows might take a while ...


Yes, replacing the custom HL7 parser with WMB parsers in all the legacy flows is huge, time consuming and expensive.
Back to top
View user's profile Send private message
Esa
PostPosted: Wed Oct 03, 2012 7:36 am    Post subject: Reply with quote

Grand Master

Joined: 22 May 2008
Posts: 1387
Location: Finland

simon.starkie wrote:


We do not create a new BAR for each environment. We export the BAR from DEV and check it into Star Team with a particular Label. Our deploy tool uses the Label to take the BAR file from StarTeam and deploys it to QA or UAT or PROD and also creates an additional Star Team Label showing which environment the particular Label was deployed to. In this case, the additional Labels show the same BAR was deployed seperately to each of the three environments as the testing process proceeded. And it turns out the problem is occurring in all three environments but unfortunately nobody noticed that until now which is why the code was eventually promoted to PROD.


If I remember corrcectly, in V6 you did not get any warning if the flow did not compile when you rebuilt the bar file. You had to check the compile timestamp to make sure that the flow really compiled. Otherwise the old compilation was left in place. V7 removes the old compiled flow from the bar file before recompilation and it gives you a warning in a pop-up window if the compilation fails.

So it is obvious the compilation of the new version - with trace nodes disconnected - failed and as a result the old version was redeployed.
That must be how it went, don't you agree?

If you have the bar files archived somewhere, you can check the compilation time to verify this.
Back to top
View user's profile Send private message
simon.starkie
PostPosted: Wed Oct 03, 2012 8:45 am    Post subject: Reply with quote

Disciple

Joined: 24 Mar 2002
Posts: 180

Esa wrote:
simon.starkie wrote:


We do not create a new BAR for each environment. We export the BAR from DEV and check it into Star Team with a particular Label. Our deploy tool uses the Label to take the BAR file from StarTeam and deploys it to QA or UAT or PROD and also creates an additional Star Team Label showing which environment the particular Label was deployed to. In this case, the additional Labels show the same BAR was deployed seperately to each of the three environments as the testing process proceeded. And it turns out the problem is occurring in all three environments but unfortunately nobody noticed that until now which is why the code was eventually promoted to PROD.


If I remember corrcectly, in V6 you did not get any warning if the flow did not compile when you rebuilt the bar file. You had to check the compile timestamp to make sure that the flow really compiled. Otherwise the old compilation was left in place. V7 removes the old compiled flow from the bar file before recompilation and it gives you a warning in a pop-up window if the compilation fails.

So it is obvious the compilation of the new version - with trace nodes disconnected - failed and as a result the old version was redeployed.
That must be how it went, don't you agree?

If you have the bar files archived somewhere, you can check the compilation time to verify this.


Here's a snapshot of the BAR file showing the compile date/time for the flow was the same as all the other included files. This is expected because the project was rebuilt on that date/time.

[img][/img]
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page Previous  1, 2, 3  Next Page 2 of 3

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Unconnected Trace nodes ${ExceptionList} filling up /tm?
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.