ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » MBFE FileOutput node fails on AIX on writing 26th record

Post new topic  Reply to topic Goto page Previous  1, 2
 MBFE FileOutput node fails on AIX on writing 26th record « View previous topic :: View next topic » 
Author Message
mqjeff
PostPosted: Thu Sep 23, 2010 5:11 pm    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

fjb_saper wrote:
mqjeff wrote:
Java is a fine and decent language when written by well trained and experienced programmers.

I do not, mind you, make that same claim about Java EE


I would be so bold as to extend the claim to Java EE but with the same caveat.


How many versions of the EJB standard have there been so far? Which of those versions are actually useful?

Java EE directly and indirectly encourages programmers of all skill levels to design 12 layers of interface when they need one.
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Thu Sep 23, 2010 6:52 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY

mqjeff wrote:
fjb_saper wrote:
mqjeff wrote:
Java is a fine and decent language when written by well trained and experienced programmers.

I do not, mind you, make that same claim about Java EE


I would be so bold as to extend the claim to Java EE but with the same caveat.


How many versions of the EJB standard have there been so far? Which of those versions are actually useful?

Java EE directly and indirectly encourages programmers of all skill levels to design 12 layers of interface when they need one.


But the skill of a good J2EE programmer is to recognize the strengths and weaknesses of the specification and use them to good advantage.
It is also to avoid anti-patterns instead of trying to force the programing to go against the explicit pattern it was designed for... to comply with a requirement expressed by somebody who had no idea about the pattern's usability for what he/she asked for.

A good J2EE programmer does not mindlessly bend the purpose and pattern the standard endorses and sometimes enforces.
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Herbert
PostPosted: Fri Sep 24, 2010 4:14 am    Post subject: Reply with quote

Centurion

Joined: 05 Dec 2006
Posts: 146
Location: Leersum, The Netherlands

mgk wrote:
Quote:
It's news to be that the File Output node has any Java code; all of WMB & it's nodes are written in C.


This is no longer the case. Several of the newer built-in nodes (including the File nodes) are written in Java. And the rest are C++


Indeed, when errors are not written to the broker log and putting the flow under trace does not give any information in the trace file then the node is written in java

hgj
Back to top
View user's profile Send private message Visit poster's website
mqjeff
PostPosted: Fri Sep 24, 2010 4:38 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Herbert wrote:
mgk wrote:
Quote:
It's news to be that the File Output node has any Java code; all of WMB & it's nodes are written in C.


This is no longer the case. Several of the newer built-in nodes (including the File nodes) are written in Java. And the rest are C++


Indeed, when errors are not written to the broker log and putting the flow under trace does not give any information in the trace file then the node is written in java


I think you are kidding, and if so

But to be clear, this is not in fact true. All of the built-in nodes should and do log things to the broker log and will log events of interest to the user trace, and more events of less interest to the service trace.
Back to top
View user's profile Send private message
Herbert
PostPosted: Fri Sep 24, 2010 5:24 am    Post subject: Reply with quote

Centurion

Joined: 05 Dec 2006
Posts: 146
Location: Leersum, The Netherlands

mqjeff wrote:
Herbert wrote:
mgk wrote:
Quote:
It's news to be that the File Output node has any Java code; all of WMB & it's nodes are written in C.

This is no longer the case. Several of the newer built-in nodes (including the File nodes) are written in Java. And the rest are C++

Indeed, when errors are not written to the broker log and putting the flow under trace does not give any information in the trace file then the node is written in java

I think you are kidding, and if so

But to be clear, this is not in fact true. All of the built-in nodes should and do log things to the broker log and will log events of interest to the user trace, and more events of less interest to the service trace.

Ok as a generic rule it's much to sharp, it's friday However in the case of the EmailOutput node it's true (at least with 6.1). When it goes wrong you really need stdout/stderr to find out what you did wrong. And even with User Trace Debug it does not write to the user trace.

hgj
Back to top
View user's profile Send private message Visit poster's website
smdavies99
PostPosted: Fri Sep 24, 2010 5:28 am    Post subject: Reply with quote

Jedi Council

Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.

mqjeff,
The MBFE nodes are not 'built-in' ones though. If I recall they have a whole directory structure of their own to use as temporary storage etc. I seem to remember some log stuff there as well.

going back to the original problem:-
IMHO, the problem lies in the design. I revert to what I said before, by all means get the messages using the MQGet node but doing this in a loop is near the top of Broker's 'DO NOT DO THIS' list unless you are very careful indeed. I have done it and have made the flow loop limited. i.e. when the loop limit is hit, it sends a message to itself and thus kicks itself off again with a new set of resources in a new thread.

I'd also put the output data onto a Queue rather than the MBFE FileOutput Node. Then use another flow to pick up the data, prepare the output params and then finally pass it to the MBFE Node. This way, any issues with the MBFE Nodes and JVM heap and other things are minimised as each message is in its own UOW/thread and has its own resourses.

Granted that my approach is possibly overkill but I all ways err on the side of robustness & simplicity in these cases.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Sep 24, 2010 5:45 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

smdavies99 wrote:
mqjeff,
The MBFE nodes are not 'built-in' ones though. If I recall they have a whole directory structure of their own to use as temporary storage etc. I seem to remember some log stuff there as well.


Yes, Stephen, I know.

Getting back, as you say, to the issue at hand, we've not heard back much from the original poster. We don't even know how they have coded the loop at all.

Perhaps they've resolved the issue simply by using a propagate/return false loop rather than using node wires.
Back to top
View user's profile Send private message
padleywoods
PostPosted: Fri Sep 24, 2010 11:51 am    Post subject: Reply with quote

Novice

Joined: 12 Dec 2005
Posts: 12
Location: UK

Well, the original poster got a bit peeved with the arguments over Java programming!!

However, it seems that there is some credence in not looping back - although the labs advise me it should not be a design issue. However, what has become apparent is the thread stack is being exhausted which results in the JVM (EG) crashing.

So, the suggestion to process n messages before triggering the same flow is a very good one and one that we shall adopt.

Many thanks for everyone's input - especially on the relative merits of coding in C, C++ & Java!!

Back to top
View user's profile Send private message
Vitor
PostPosted: Fri Sep 24, 2010 11:56 am    Post subject: Reply with quote

Grand High Poobah

Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA

padleywoods wrote:
Well, the original poster got a bit peeved with the arguments over Java programming!!


Sorry about that. Never a competent moderator round when you need one....
_________________
Honesty is the best policy.
Insanity is the best defence.
Back to top
View user's profile Send private message
mqjeff
PostPosted: Fri Sep 24, 2010 11:59 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

padleywoods wrote:
Well, the original poster got a bit peeved with the arguments over Java programming!!

Those weren't *arguments*. Comradely pontificating, maybe, but certainly FJ and I weren't at all arguing.

padleywoods wrote:
However, it seems that there is some credence in not looping back - although the labs advise me it should not be a design issue. However, what has become apparent is the thread stack is being exhausted which results in the JVM (EG) crashing.

So, the suggestion to process n messages before triggering the same flow is a very good one and one that we shall adopt.

Many thanks for everyone's input - especially on the relative merits of coding in C, C++ & Java!!



26 records is fairly small and really does suggest you're using wire-level looping rather than propagate/return false. The use of a propagate loop should "unwind" or "close" the thread stack after each execution through.

Other than that, yes it's a good practice to not try and execute 10,000 iterations through multiple nodes in a single message flow instance if possible. Using smdavies99's suggestion of limiting the number of repeats in a single instance is a good idea.
Back to top
View user's profile Send private message
WMBDEV1
PostPosted: Sat Sep 25, 2010 12:38 am    Post subject: Reply with quote

Sentinel

Joined: 05 Mar 2009
Posts: 888
Location: UK

padleywoods wrote:
Well, the original poster got a bit peeved with the arguments over Java programming!!


Its good to see that my response to your original question didnt go unnoticed then!
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Goto page Previous  1, 2 Page 2 of 2

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » MBFE FileOutput node fails on AIX on writing 26th record
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.