ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » mqsipackage bar bloat

Post new topic  Reply to topic
 mqsipackage bar bloat « View previous topic :: View next topic » 
Author Message
lancelotlinc
PostPosted: Tue Jul 02, 2013 5:43 am    Post subject: mqsipackage bar bloat Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

WMBT 8.0.0.2

We had some Ant build scripts written to include specified files into a bar. These scripts have been working for a few months now, no problems. We explicitly did not create an 'Application' from our project. We simply built the bar by including all the needed files, file-by-file, and used command line like:

Code:
mqsipackagebar -w xyz -a zyx -o file1 file2 ... file49 file50 file51


Someone down the line converted the Message Flows into an Application. Now, the mqsipackagebar fails with

Quote:
BIP1868W: Ignoring 'file1.esql', which is in application '<projectname>'. Use the -k option to include the application.


We have thousands of xml files that make up test case payloads located in a subdirectory of the project. We never want to include these xml files in the bar files.

When putting the -k option in the mqsipackagebar command, the mqsipackagebar scans the project and includes all files specified by

http://publib.boulder.ibm.com/infocenter/wmbhelp/v8r0m0/index.jsp?topic=%2Fcom.ibm.etools.mft.doc%2Fbc31720_.htm

Quote:
By default files with only the following extensions are included in the BAR file: .xsdzip, .tblxmi, .xsd, .wsdl, .dictionary, .xsl, .xslt, .xml, .jar, .inadapter, .outadapter, .insca, .outsca, .descriptor, .php, .idl, .map, .esql, .msgflow, and .subflow. You can specify the -i option to include files with any extension.


which is incorrectly including also all the thousands of xml payload files.


How to de-Applicationify the project?

We don't want to use the Application construct.

Or else how to change the default so that xml files are not included, or specified subdirectories are ignored?
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
smdavies99
PostPosted: Tue Jul 02, 2013 5:48 am    Post subject: Reply with quote

Jedi Council

Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.

I think that a directory named 'Docs&Testing' gets ignored.
Worth a try...
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Back to top
View user's profile Send private message
lancelotlinc
PostPosted: Tue Jul 02, 2013 5:56 am    Post subject: Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

smdavies99 wrote:
I think that a directory named 'Docs&Testing' gets ignored.
Worth a try...


Good suggestion, however, - by default - all subdirectories are scanned for files within the filename extension specified in the command reference, which includes 'Docs", 'docs', 'Testing', etc.

The InfoCentre leads the reader to believe that the default can be changed. The question is: how to change the default behavior of mqsipackagebar.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Tue Jul 02, 2013 6:37 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

You can change an Application project back into a Message Flow project - but you have to manually edit the .project file
Code:
<natures>
      <nature>com.ibm.etools.msgbroker.tooling.applicationNature</nature>
      <nature>com.ibm.etools.msgbroker.tooling.messageBrokerProjectNature</nature>
      <nature>com.ibm.etools.mft.bar.ext.barnature</nature>
   </natures>


Otherwise, I'd look into modifying your source code procedures to either not check out certain folders or to delete certain folders after checkout but before build.

The way to "modify" the default behavior of mqsipackagebar is not what you want - you use the -i to include "non-deployable" artifacts.


Last edited by mqjeff on Tue Jul 02, 2013 6:43 am; edited 2 times in total
Back to top
View user's profile Send private message
lancelotlinc
PostPosted: Tue Jul 02, 2013 6:40 am    Post subject: Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

mqjeff wrote:
You can change an Application project back into a Message Flow project - but you have to manually edit the .project file
Code:
<natures>
      <nature>com.ibm.etools.msgbroker.tooling.applicationNature</nature>
      <nature>com.ibm.etools.msgbroker.tooling.messageBrokerProjectNature</nature>
      <nature>com.ibm.etools.mft.bar.ext.barnature</nature>
   </natures>


Ok, thats one solution, and I appreciate your comment.

Is there a way to set mqsipackagebar -k option to exclude certain subdirectories? The InfoCentre leads one to believe that changing this option is possible since it says "By default..."
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Tue Jul 02, 2013 6:44 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

You might be able to use both -o and -k together to only include named files from the application.
Back to top
View user's profile Send private message
lancelotlinc
PostPosted: Tue Jul 02, 2013 6:49 am    Post subject: Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

mqjeff wrote:
You might be able to use both -o and -k together to only include named files from the application.


When doing this, mqsipackagebar rejects the -o args with the error above, and then includes those files, along with all the other unwanted files, as part of the application.

We need a way to scope the file inclusion somehow. Simply including all files inside the project filesystem is not desirable. There are files we want to be included in the project filesystem so that they get recorded into the version control on that project but we don't want to deploy those thousands of XML files to the Broker.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Tue Jul 02, 2013 6:52 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Yeah. There doesn't seem to be an option you need.

The -i option only lets you expand the list, not shorten the list of things that are included.

But it's worth a quick check to see if it changes the behavior of -o and -k together.

Otherwise, again, look at modifying your build process to delete known subfolders before build.
Back to top
View user's profile Send private message
lancelotlinc
PostPosted: Tue Jul 02, 2013 6:58 am    Post subject: Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

Thanks for your comments.

-i has no impact on the unwanted files on any combination of -o or -k.

Deleting the directories and files is not viable because that impacts the source code control system (Jazz - RTC).

How long would it take to create an APAR on mqsipackagebar to exclude various directories, or check each subdirectory folder for an .exclude file :RFE: ?

We're scheduled to go to Production in 30 days. For now, we're building the bar with all the unwanted files. We cannot delete the XML files from the broker once deployed because they get included in the appzip file.

I suppose we'll open a PMR and see where it takes us, quickly or not.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
mqjeff
PostPosted: Tue Jul 02, 2013 7:04 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

I'm not talking about changes that would affect source control.

You should be able to check out the whole shebang and then either move or delete files and then package the bar file without affecting the source control repository's understanding.

At a bare minimum, something like
  1. check out to directory A
  2. copy directory A to directory B
  3. delete files from dir B
  4. package from directory B
  5. copy bar file back to directory A
  6. sync/commit/etc


Alternately, mqsipackagebar produces a bar file which is a zip file which can be unzipped. The appzip inside it can be unzipped. Any libzips inside that can be unzipped. Or java based tools, perhaps invoked from Ant, can be used to edit the zip file contents "in place".
Back to top
View user's profile Send private message
goffinf
PostPosted: Tue Jul 02, 2013 11:56 am    Post subject: Re: mqsipackage bar bloat Reply with quote

Chevalier

Joined: 05 Nov 2005
Posts: 401

lancelotlinc wrote:
WMBT 8.0.0.2
Someone down the line converted the Message Flows into an Application.
....

We have thousands of xml files that make up test case payloads located in a subdirectory of the project. We never want to include these xml files in the bar files.



Yeah we noted this when we were working thru our migration strategy from v6.1 to v8. If you stick with MBPs then removing what you want from a BAR created in the toolkit or (as in your case) specifying exactly what gets included is straight-forward.

What we decided on was to refactor our existing projects and split them into separate projects, one (the v8 Application/Library or MBP) for the integration solution we want to deploy to production, and another (an MBP) to contain all of the test resources (following a standard naming convention). In our case the 'test' project includes and flows and ESQL that are just used for the purpose of unit/e2e testing. We also have a further Java project in where we keep the JUnit test classes (that's what we use to test all our flows) and all of the associated resources, including request and expected response messages, schemas and such like.

Ther are a few issues with referencing Application projects from an MBP but they are resolvable.

It is a bit of a nuisance and it would probably be better to be able to selectively choose the contents of the BAR, but I guess the key concept is that an Application is intended to be an indivisible unit of deployment.

In the end it didn't turn out to be much of a problem.

HTHs

Fraser.
Back to top
View user's profile Send private message
lancelotlinc
PostPosted: Tue Jul 02, 2013 12:13 pm    Post subject: Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

Submitted RFE 36740 in case anyone wants to vote on it.

The RFE asks for an APAR that allows proper scoping of files. We don't want to maintain our test payloads outside of our project. We want our test payloads to be recorded in the Version Control system of choice. We want our test payloads to be within the project filesystem hierarchy. We do not want our thousands of XML file test payloads to be deployed to a Broker runtime.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
smdavies99
PostPosted: Tue Jul 02, 2013 10:16 pm    Post subject: Reply with quote

Jedi Council

Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.

lancelotlinc wrote:
We do not want our thousands of XML file test payloads.


Perhaps now would be a good time to 'zip' them all up?
I do that with many of my non time dependent test messages.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995

Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions.
Back to top
View user's profile Send private message
lancelotlinc
PostPosted: Wed Jul 03, 2013 3:14 am    Post subject: Reply with quote

Jedi Knight

Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA

smdavies99 wrote:
lancelotlinc wrote:
We do not want our thousands of XML file test payloads.


Perhaps now would be a good time to 'zip' them all up?
I do that with many of my non time dependent test messages.


Hi smdavies - thanks for your comment. Our automated test tool which regression tests the message flows uses the payloads often. Zipping them up would be a workaround, not a solution. For a long-term solution, our IBM Accelerated Value Leader (AVL) representative is in discussions with IBM support for an APAR to mqsipackagebar.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » mqsipackage bar bloat
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.