| Author |
Message
|
| isaw |
 Posted: Tue May 29, 2007 7:02 am Post subject: 'mqsicreatebar' fails referencing Java proj- known issue? |
 |
|
Apprentice
Joined: 09 May 2007
Posts: 45
|
Hey guys,
I'm trying to manually create a bar where a message flow uses a Java node which references a Java project.
| Code: |
TEST_PROJECT (msg flow project)
- BasicFlow.msgflow
- JavaFlow.msgflow (has JavaFlow.java file ref'd in following project)
TEST_PROJECTJava (java project)
- JavaFlow.java
- JavaFlow.class |
When I isssue the manual command:
| Code: |
| mqsicreatebar -data "C:/Test" -b "C:/Test/TEST_PROJECT/MyTestBar.bar" -p TEST_PROJECT TEST_PROJECTJava -o TEST_PROJECT/BasicFlow.msgflow TEST_PROJECT/JavaFlow.msgflow |
It fails everytime suggesting there is an error in the JavaFlow.msgflow project:
| Code: |
ERROR:
Processing file JavaFlow.msgflow
Failed to add file JavaFlow.msgflow to archive file.
/TEST_PROJECT/JavaFlow.msgflow is in a project that has errors.
Please correct the errors and try again.
BIP0988W Error occurred while completing mqsicreatebar command. |
I have cleaned & rebuilt the project, I have removed the 'TEST_PROJECTJava' reference from the -p flag but no luck. If I remove the JavaFlow.msgflow file from the -o flag it seems to work fine, only because I have asked it to create bar without the msgflow containing the Java Node I need.
Am I beating a dead horse here? Is this a known issue? Or will this mqsicreatebar allow me to create a bar file that ref's a project that isn't a MsgFlow or MsgSet project type and I'm missing something obvious?
Thanks all! |
|
| Back to top |
|
 |
| jefflowrey |
| Posted: Tue May 29, 2007 10:36 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
What version?
Are you sure the java project builds successfully?
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| isaw |
| Posted: Wed May 30, 2007 12:37 am Post subject: 'mqsicreatebar' fails referencing Java proj- RESOLVED |
|
|
Apprentice
Joined: 09 May 2007
Posts: 45
|
<ugh!>
Ok. Stupidly, while I was ensuring my "code" was working fine and checked into source-control (as my builds are using extracts from sc), I hadn't updated my .project file of the main Message Flow Project which had (or i should say didnt have) a reference to the Java project!
That done, it compiled into a bar fine.
Sorry for that guys... |
|
| Back to top |
|
|
| roysterdoyster |
| Posted: Mon Mar 17, 2008 6:14 am Post subject: mqsicreatebar command missing Java jar files |
|
|
Apprentice
Joined: 26 Oct 2007
Posts: 25
Location: Glasgow, UK
|
This seems like a similar problem. I have 3 projects, project A containing flows, project B containing library sub flows and project C and Java project. One of the flows in A references a subflow in project B which in turn uses a Java compute node with Java code held in project C.
When I create a bar file using the toolkit, the Java project is jar'd up and added to the bar and so deploys correctly. When I use mqsicreatebar with all projects referenced with -p switch, the bar file builds successfully but when I check the contents the Java jar file has not been included in it and therefore it fails to deploy.
Can anyone explain why this is and what I am doing wrong here. I am taking all the same files from ClearCase before running the mqsicreatbar command so the .project files etc are identical in both cases.
I've checked the service.log of the resulting bar file and I see the entry
"Resource xxx_JAVA_LIBRARY.jar was not added to the BAR file because it was not found in the
workspace nor found built in metadata. This resource may be required by the
broker because it contains java dependencies for a flow to be deployed in this
BAR. If actually required and java project is to be found in worskpace, you can
force a build on this project and then retry adding the flow to this BAR. You can
also deploy a JAR resource containing these dependencies from the tooling
adding it manually to a BAR file or copying it directly to the broker."
It is required but this message is particularly unlcear. I've tried including in the workspace and still get the same message. |
|
| Back to top |
|
|
| roysterdoyster |
| Posted: Thu Mar 27, 2008 1:31 am Post subject: |
|
|
Apprentice
Joined: 26 Oct 2007
Posts: 25
Location: Glasgow, UK
|
Bit of an update to this. If I build the java jar project separately and include the resulting jar file in the Java project it does get picked up and added to the bar file. Now however, when I try to deploy this bar file, I get a deployment error:
BIP4500E: Failed to deploy Java code resource 'JAR'. The Java stack trace is: 'com.ibm.broker.javacompute.MbRuntimeJavaComputeNode.createJavaComputeNode(Native Method) com.ibm.broker.javacompute.MbRuntimeJavaComputeNode.updateNodeClass(MbRuntimeJavaComputeNode.java:251) com.ibm.broker.javacompute.MbRuntimeJavaComputeNode.updateNodeClass(MbRuntimeJavaComputeNode.java:197) com.ibm.broker.javacompute.MbRuntimeJavaComputeNode.setJavaClass(MbRuntimeJavaComputeNode.java:179)'
The Java JAR file deployment to the broker failed.
This is an internal error. Contact your IBM support center.
There is something very wrong with this process if the mqsicreatebar command behaves differently from the toolkit. The toolkit is not a suitable tool for automating or scripting builds/deployments so some clear IBM doco should be provided around the difference. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Mar 27, 2008 1:44 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| roysterdoyster wrote: |
| This is an internal error. Contact your IBM support center. |
Did you?
| roysterdoyster wrote: |
| There is something very wrong with this process if the mqsicreatebar command behaves differently from the toolkit. The toolkit is not a suitable tool for automating or scripting builds/deployments so some clear IBM doco should be provided around the difference. |
Or you should raise a PMR to advise IBM it behaves differently. I suspect (from the message) it's not supposed to be behaving differently......
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| roysterdoyster |
| Posted: Thu Mar 27, 2008 3:56 am Post subject: |
|
|
Apprentice
Joined: 26 Oct 2007
Posts: 25
Location: Glasgow, UK
|
Not raised one yet but will do (although it would be useful if there was more information around the error which could help me identify the root cause).
Having trawled the forums, there are a lot of people struggling with automation of build and deploy processes with WMB.
It's about time that IBM adequately documented some of this stuff (and I don't mean with small simple examples which don't actually tackle the complexity of a production environment)
1. Backing up the brokers (not just the databases)
2. deploying across multiple brokers
3. rolling back deployments
4. full version control with clearcase
5. failover with MQ clustering
6. adequate error logging
etc etc
seems like the real info developers need is not available. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Mar 27, 2008 4:07 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
This http://publib.boulder.ibm.com/infocenter/wmbhelp/v6r1m0/topic/com.ibm.etools.mft.doc/an20141_.htm
Is very clear, very straight forward, and is "not just the database".
Deploying across multiple brokers is easily done - one broker at a time, scripted with mqsideply if you like. Or you can use IS02.
There is no "rollback" of deployments.
At all.
Never has been. That's why you keep your flows & sets in version control, so you can recreate a BAR file at the previous level. Or you also keep your BARs under version control, so you can redo a particular deploy exactly.
You should look to ClearCase for "how do I use ClearCase with Broker". Ideally, they should have an article somewhere under http://www.ibm.com/developerworks/rational/products/clearcase/
There is no "failover" with MQ Clustering, so I don't see what you mean to try to connect this with Broker?
As for adequate error logging, I really don't know what you mean by this. It could be that you're trying to do error logging in the same way you do it in WAS or etc. - which is an invalid proposition, as Broker is *not* WAS.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Mar 27, 2008 4:17 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| roysterdoyster wrote: |
| Not raised one yet but will do (although it would be useful if there was more information around the error which could help me identify the root cause). |
You should always consider a PMR when you get an actual exception from the product (like a stack dump). That should never happen.
Moving to your other points
| roysterdoyster wrote: |
| 1. Backing up the brokers (not just the databases) |
You shouldn't back up brokers. You should back up their configuration (e.g. bar files) and redeploy them in the event of problems. Rebuilding first in the event of serious problems.
| roysterdoyster wrote: |
| 2. deploying across multiple brokers |
Easily done with the command line tools. I've seen an ANT script used to build and deploy to brokers.
| roysterdoyster wrote: |
| 3. rolling back deployments |
See my response to 1. above
| roysterdoyster wrote: |
| 4. full version control with clearcase |
Available in WMBv6
| roysterdoyster wrote: |
| 5. failover with MQ clustering |
MQ clustering is NOT an HA solution - it's a workload balancing solution and always has been.
| roysterdoyster wrote: |
| 6. adequate error logging |
What's inadequate about the current logging?
| roysterdoyster wrote: |
| seems like the real info developers need is not available. |
Pretty much it is
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| roysterdoyster |
| Posted: Thu Mar 27, 2008 5:44 am Post subject: |
|
|
Apprentice
Joined: 26 Oct 2007
Posts: 25
Location: Glasgow, UK
|
Thanks for your replies guys. It's good to know there is a lot of knowledge out there. I've read these various guides before and there are some really useful ones out there but I'd like to point out a few things.
1. when deploying on a typical production system it would not be acceptable to redeploy every existing flow every time a new new flow is deployed, so you would opt for incremental deploys as new flows are created. This incremental build of a live system brings with it some serious backup issues. If, as was suggested we backed up every bar file we ever deployed then if we needed to rebuild the environment we would need to redeploy these in the same order they were originally done. This would be time consuming and error prone.
2.or assume we decide we wish to back up the broker fully prior to deployment, step 1. of the guide states "stop the broker", in a live environment this brings a full service outage which is also unacceptable (even in a Live - Live environment there is the possibility of each broker getting out synch) during this process as well as reducing capacity by 50%
3. using clearcase to store the source code is fine (although we currently have a PMR raised against clearcase as the merge tool does not adequately merge the xml that make up the message flow files resulting in a requirement to regression test all code before and after merges) so it is possible for us to pull out the message flows at the correct level, but as mqsicreatebar does not behave the same as the toolkit bar building process, we cannot rely on scripted build processes to recreate the bar files exactly as any built from the toolkit.
4. no I don't consider having to unzip a bar file and read throught the service log to be adequate logging in terms of helping identify when a required Java library has been missed from a bar file.
5. If I was to raise a PMR everytime I had seen a Stack Trace IBM would have no time to do any work.
I know this sounds like a bad tempered rant and I am in no way trying to offend anyone. It's just that, working in a high demand financial service industry we have spent far too much time trying to find ways to perform what would seem to me to be standard production procedures with lack of suitable documentation dealing with real world scenarios. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Mar 27, 2008 6:17 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| roysterdoyster wrote: |
| 1. when deploying on a typical production system it would not be acceptable to redeploy every existing flow every time a new new flow is deployed, so you would opt for incremental deploys as new flows are created. This incremental build of a live system brings with it some serious backup issues. If, as was suggested we backed up every bar file we ever deployed then if we needed to rebuild the environment we would need to redeploy these in the same order they were originally done. This would be time consuming and error prone. |
You've got version control issues. You can reconstruct a bar file pulling the components out by ClearCase label. So to rebuild a broker at a given release level you reconstruct the bar file at that level using label information.
| roysterdoyster wrote: |
| 2.or assume we decide we wish to back up the broker fully prior to deployment, step 1. of the guide states "stop the broker", in a live environment this brings a full service outage which is also unacceptable (even in a Live - Live environment there is the possibility of each broker getting out synch) during this process as well as reducing capacity by 50% |
How do you back up databases in your environment? They have to be stopped to provide a consistent point-in-time recovery. I repeat that backing up brokers is unnecessary as the contents of the broker is stored elsewhere.
| roysterdoyster wrote: |
| 3. using clearcase to store the source code is fine (although we currently have a PMR raised against clearcase as the merge tool does not adequately merge the xml that make up the message flow files resulting in a requirement to regression test all code before and after merges) so it is possible for us to pull out the message flows at the correct level, but as mqsicreatebar does not behave the same as the toolkit bar building process, we cannot rely on scripted build processes to recreate the bar files exactly as any built from the toolkit. |
I've commented above on the need for a PMR. I've seen a scripted build run through ANT that worked just fine (see also below)
| roysterdoyster wrote: |
| 4. no I don't consider having to unzip a bar file and read throught the service log to be adequate logging in terms of helping identify when a required Java library has been missed from a bar file. |
This ties back to the build / version control issue. You should be able to rely on the process, or at least the manifest to obtain this information except in cases of error. I'm thinking here of a developer not mentioning a new file needs to be included.
| roysterdoyster wrote: |
| 5. If I was to raise a PMR everytime I had seen a Stack Trace IBM would have no time to do any work. |
This is the below I was talking about - what maintance level are you on? I've never seen this kind of error from a WMBv6 installation. My experience is it's much more stable than that!
| roysterdoyster wrote: |
| It's just that, working in a high demand financial service industry we have spent far too much time trying to find ways to perform what would seem to me to be standard production procedures with lack of suitable documentation dealing with real world scenarios. |
Your location says Glasgow - I think I know where you mean....! 
You certainly seem to be having more trouble with the product than is usual. IMHO some of this is procedureal and political (version control et al) and I accept unresrvedly that some or all of this is beyond your control & unchangeable in the short to medium term.
FWIW the real world scenarios you're describing can be done and have been done. Locally to you if I'm right (how close to the M8 are you?).
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| roysterdoyster |
| Posted: Thu Mar 27, 2008 7:54 am Post subject: |
|
|
Apprentice
Joined: 26 Oct 2007
Posts: 25
Location: Glasgow, UK
|
Hey Vitor,
Cheers again and without mentioning names, yes there are only a few of us up here using the product at the moment but I'm not aware of anyone so heavily dependant on it.
The ClearCase PMR is being raised against 'recommended' ClearCase UCM (not base clearcase) so using labels for pulling out specific flows is not really feasible as UCM hides the config spec from you so it's not straightforward to pull out flows by labels. Again though, it is not feasible to completely redeploy a number of full bar files containing old and new flows when what is required is a one flow change. If this is a version control issue, what's the solution, so far, having engaged IBM we've had no guidance as to a solution.
You also mention that you don't need to back up the broker as the information is held elsewhere. Where do you mean? To reconstruct the environment would you not need to back up config manager, db's and brokers to be able to restore this or is there another way of doing it?
We have Ant scripted build and deploy scripts which to date have worked ok. Now we are using Java compute nodes we've had a number of issues. And also trying to introduce Cruise we've hit even more (over a week was wasted on trying to identify why the scripted build kept hanging... the answer: every time you run mqsicreatebar you need to delete the metadata directory or it will hang in headless mode.. not mentioned in any doco)
W're running fixpack 3 on V6. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Mar 27, 2008 8:02 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| roysterdoyster wrote: |
| HI'm not aware of anyone so heavily dependant on it. |
I am. About 10 miles down the M8 if I've got my geography straight (not a financial institution), and utterly reliant on WMBv6.
| roysterdoyster wrote: |
| The ClearCase PMR is being raised against 'recommended' ClearCase UCM (not base clearcase) so using labels for pulling out specific flows is not really feasible as UCM hides the config spec from you so it's not straightforward to pull out flows by labels. |
I'll take your word as my ClearCase is insufficient to disagree.
| roysterdoyster wrote: |
| Again though, it is not feasible to completely redeploy a number of full bar files containing old and new flows when what is required is a one flow change. |
I suppose if you can't generate a point-in-time by labels then yes you're stuck (odd to have version control that can't recreate how something looks by label but see above re: ClearCase)
| roysterdoyster wrote: |
| You also mention that you don't need to back up the broker as the information is held elsewhere. Where do you mean? To reconstruct the environment would you not need to back up config manager, db's and brokers to be able to restore this or is there another way of doing it? |
It's all in ClearCase or, at a push, the Config Manager. Because the Config Manager manages the config of the brokers, backing up the brokers is redundant.
| roysterdoyster wrote: |
| We have Ant scripted build and deploy scripts which to date have worked ok. Now we are using Java compute nodes we've had a number of issues. And also trying to introduce Cruise we've hit even more (over a week was wasted on trying to identify why the scripted build kept hanging... the answer: every time you run mqsicreatebar you need to delete the metadata directory or it will hang in headless mode.. not mentioned in any doco) |
No it doesn't. Or doesn't when we did it.
| roysterdoyster wrote: |
| W're running fixpack 3 on V6. |
So were we, and it didn't dump / abend / wobble as much as you seem to indicate. Never raised a PMR (just checked with a contact & it's still humming away) and the ANT builds are still chugging.
Odd.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| roysterdoyster |
| Posted: Thu Mar 27, 2008 8:46 am Post subject: |
|
|
Apprentice
Joined: 26 Oct 2007
Posts: 25
Location: Glasgow, UK
|
Would be good to share the knowledge of these scripts amongst the community. In terms of the metadata issue. Looks like others (myself included) have also seen this issue
http://www.mqseries.net/phpBB2/viewtopic.php?t=28618&sid=c9e1bd6876a621d9cca1a82ed357992f
IMHO ClearCase UCM is odd as baselines are a 'label in time' across all components rather than on individual elements. We're working through this at the moment.
If we were to stop the brokers, back up the config manager and db's. And then restore both have we restored the environment fully then?
And back to my original issue. I've built a simple jar file from within the toolkit and the resulting bar consists of:
1. the compiled flow
2. the message set for this flow
3. the Java project jar file
4. a library folder containg the 3rd party libraries from the java project
and it deploys fine.
When I use mqsicreatebar, the resulting bar consists of:
1. the compiled flow
2. the message set for this flow
3. the Java project jar file
and it fails at deploy time.
I have no idea how/why the library folder is created in the toolkit version as there are no project references to it and we are using identical files in the toolkit and via command line.
I'll get onto the PMR first thing.. |
|
| Back to top |
|
|
| jefflowrey |
| Posted: Thu Mar 27, 2008 8:48 am Post subject: |
|
|
Grand Poobah
Joined: 16 Oct 2002
Posts: 19981
|
| roysterdoyster wrote: |
| W're running fixpack 3 on V6. |
Of course... in MB v6.0, mqsicreatebar is a toolkit command, so the runtime FP level doesn't matter...
Make sure the associated Toolkit install is at 6.0.2 and at least iFix 11.
_________________
I am *not* the model of the modern major general. |
|
| Back to top |
|
|
|
|
