| Author |
Message
|
| santy |
 Posted: Fri Jun 19, 2009 2:19 am Post subject: Problem while File Writing from Java Compute Node to UNIX |
 |
|
Centurion
Joined: 03 Nov 2006
Posts: 141
|
Hi,
We have code written in WMB 6.0 under Java Compute Node to write file to UNIX directory.
We are getting more that 10000's at a time.
While writing file to UNIX we have written a thread.sleep (2000) for 2 seconds so that each file gets written after 2 seconds to UNIX.
But when we are checking timestamp on UNIX system, the timestamp looks same for all the files written to UNIX directory.
when we tried to retrive the logs, it shows the 2 second's difference.
Any idea why it is happening so? |
|
| Back to top |
|
 |
| zpat |
| Posted: Fri Jun 19, 2009 2:26 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Why not use the FileOutput node?
WMB is not meant to be used as a Java application hosting platform when there are builtin nodes.
It is very scarey that people are coding Java in WMB - often totally unnecessarily - this will become a maintenance nightmare. |
|
| Back to top |
|
|
| santy |
| Posted: Fri Jun 19, 2009 2:38 am Post subject: |
|
|
Centurion
Joined: 03 Nov 2006
Posts: 141
|
| Because we are not using WMB 6.1, So we are not using FileOutput node |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Jun 19, 2009 2:54 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| Upgrade to WMB 6.1 - don't write your own version of the IBM product. |
|
| Back to top |
|
|
| ovasquez |
| Posted: Fri Jun 19, 2009 9:28 pm Post subject: |
|
|
Centurion
Joined: 09 Dec 2005
Posts: 141
Location: Lima, Peru
|
| zpat wrote: |
| Upgrade to WMB 6.1 - don't write your own version of the IBM product. |
Upgrade to WMB 6.1. is solution?...sure? 
_________________
Oscar Vásquez Flores |
|
| Back to top |
|
|
| Vitor |
| Posted: Sat Jun 20, 2009 12:50 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| ovasquez wrote: |
| zpat wrote: |
| Upgrade to WMB 6.1 - don't write your own version of the IBM product. |
Upgrade to WMB 6.1. is solution?...sure? |
It's a solution, not the solution.
It can be supposed that at some point the site in question will move to v6.1. If there is a specific feature in the new version you need (like the File Output node) then this might inform the decision of how quickly this is done. It might not be planned to upgrade for a year, as part of a rolling cycle, but the new need might bring that forward.
It's also generally better to use inbuilt features than roll your own in Java. You wouldn't write TCP/IP code, or queue manipulation nodes.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| mymq |
| Posted: Sat Jun 20, 2009 6:45 pm Post subject: |
|
|
Centurion
Joined: 01 Mar 2007
Posts: 101
Location: US-Greenwille
|
just curious, how you naming your file?
what is the logic r u using it?
_________________
--SRK-- |
|
| Back to top |
|
|
| mqpaul |
| Posted: Mon Jun 29, 2009 2:52 am Post subject: Re: Problem while File Writing from Java Compute Node to UNI |
|
|
Acolyte
Joined: 14 Jan 2008
Posts: 66
Location: Hursley, UK
|
| santy wrote: |
Hi,
We have code written in WMB 6.0 under Java Compute Node to write file to UNIX directory.
We are getting more that 10000's at a time.
While writing file to UNIX we have written a thread.sleep (2000) for 2 seconds so that each file gets written after 2 seconds to UNIX.
But when we are checking timestamp on UNIX system, the timestamp looks same for all the files written to UNIX directory.
when we tried to retrive the logs, it shows the 2 second's difference.
Any idea why it is happening so? |
I expect UNIX sets the time stamp when the flow closes the file. If your Java doesn't close it explicitly, it may be that Broker end-of-flow housekeeping closes all the files at the same time, which would give you the same time stamps.
Out of interest, do you mean 10000 files each written 2 seconds apart in the same flow? If so, your Java node will be executing for over 5 hours - which is a bit long for a flow!
_________________
Paul |
|
| Back to top |
|
|
| bpmikaelj |
| Posted: Tue Nov 20, 2012 5:59 am Post subject: TCP/IP in a JavaCompute node |
|
|
Newbie
Joined: 08 Nov 2012
Posts: 2
|
I found this old post where
| Vitor wrote: |
| It's also generally better to use inbuilt features than roll your own in Java. You wouldn't write TCP/IP code, or queue manipulation nodes. |
I find solace in this statement since I'm in the unhappy position of having inherited several WMB solutions with implementation of proprietary TCP/IP protocols JavaCompute nodes.
I've been argumenting that "you cannot do that" and "it doesn't work" but that's been based mostly on gut feeling and I haven't been able to properly explain why it's such a bad idea.
Thus I'd love to get an explanation that can help me to argument for doing it the right way! |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 20, 2012 6:14 am Post subject: Re: TCP/IP in a JavaCompute node |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bpmikaelj wrote: |
| I've been argumenting that "you cannot do that" and "it doesn't work" but that's been based mostly on gut feeling and I haven't been able to properly explain why it's such a bad idea. |
There is a difference between "you cannot do that" and "it's such a bad idea". There's also a difference between "it doesn't work" and "it's a bad idea".
Take TCP/IP. You can write custom code to do this; you're looking at it. So the comment "it's doesn't work" falls. Is the custom code as good as the inbuilt nodes produced by a room full of very smart IBM employees working dilligently for years? Is the custom code supported by something the size of the IBM support organization? Does your organization have the resources to investigate all the future changes in TCP/IP, leverage them and apply them to your custom code?
If so, why exactly did you spend a very, very large amount of money on WMB? Why are you still paying a large amount of money to maintain your ownership?
The point of buying something with the price tag of WMB is to avoid all the costs and problems of which I list examples above. If all your site wants to do is is write custom Java then WAS is much more cost effective.
Now let's take queue manupulation nodes. You can write Java code to get and put to a queue outside of WMB's facilties. This will work, in the sense messages will move onto and off of the queue. It will do terrible things to WMB's transactional control, which is predicated on the assumption it's controlling queue activity.
Let's wrap up with XML. It's perfectly possible to take data into WMB as a BLOB and parse it in a Java compute node with any of the Java faciltities you'd use in WAS. Is that better or more efficient than the XMLNSC parser kimbert has spent years patiently polishing, perfecting and optimizing? Easier to use than the message tree it produces?
Cheaper than just using the XMLNSC than comes with the package and which you've paid for? 
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| McueMart |
| Posted: Tue Nov 20, 2012 8:04 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
| After having read all this valuable advice, if the OP is willing to post the java code which is writing the files, we might be in a better spot to help them with their initial query. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 20, 2012 8:07 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| McueMart wrote: |
| if the OP is willing to post the java code which is writing the files, we might be in a better spot to help them with their initial query. |
If the OP has not found a solution in the 3 years since their initial query, I suspect they've learned to live with it.
I was directly responding to the more recent query.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| McueMart |
| Posted: Tue Nov 20, 2012 8:19 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
| Bah - I only looked at the day part of the date, and saw a 19 and thought that it was pretty close to 20! |
|
| Back to top |
|
|
| bpmikaelj |
| Posted: Tue Nov 20, 2012 9:34 am Post subject: |
|
|
Newbie
Joined: 08 Nov 2012
Posts: 2
|
| Vitor wrote: |
If so, why exactly did you spend a very, very large amount of money on WMB? Why are you still paying a large amount of money to maintain your ownership?
The point of buying something with the price tag of WMB is to avoid all the costs and problems of which I list examples above. If all your site wants to do is is write custom Java then WAS is much more cost effective. |
You've got perfectly valid points there! From where I stand, however we're micro managing things. Which means that the people that need to fund the reimplementation of the TCP/IP connections, are not directly affected by the price of the WMB. In fact, if we were to implement a couple of TCP/IP-to-WMB bridges for these wire protocols, those would have to be hosted somewhere and thus incurr extra costs (not to the organisation as a whole but to these peoples budgets).
We've experienced a lot of problems with the TCP/IP connections and still we're only trying to patch it up. I've used a lot of arguments along the lines of it's bad design, it's hard to maintain, there is operational risk involved, etc. The consultant who implemented them still argues that it's java so it should work! But then again, creating a thread could be done easily enough in java, but one of IBM's WMB developers from Hursley explicitly said that that's a bad idea.
Therefore, I was hoping for some technical reasons why this is a bad approach, something along the same line as
| Vitor wrote: |
| Now let's take queue manupulation nodes. You can write Java code to get and put to a queue outside of WMB's facilties. This will work, in the sense messages will move onto and off of the queue. It will do terrible things to WMB's transactional control, which is predicated on the assumption it's controlling queue activity. |
|
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Nov 20, 2012 9:45 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bpmikaelj wrote: |
| The consultant who implemented them still argues that it's java so it should work! But then again, creating a thread could be done easily enough in java, but one of IBM's WMB developers from Hursley explicitly said that that's a bad idea. |
This is an often held belief within the Java community; that everything which runs Java is some kind of app server. WMB is not WAS, nor any other kind of app server. It's a piece of software that happens to include a JVM and has the ability to run Java.
I'll leave it to my more Java literate associates to throw technical reasons into the pot. I'll throw these 2 points:
- WMB controls thread use like it controls queue access. Spawning your own threads from inside the execution group process which the process don't expect & doesn't know about can't be good
- If one of the Hursley people has explicitly told you it's a bad idea, how much more techincal do you want? If you'd fitted a perfectly good turbo charger to your car, and the guy who designed the engine said it was a bad idea, would you then brush off his objections and go for a spin?
If you did, muttering to yourself that "if he can't come up with a specific issue there's no real issue", I would hope you'd remember about intellectual property, confidentiality and the ability of employees to speak freely at some point before red-hot metal came crashing through your body.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
