| Author |
Message
|
| Abhinay185127 |
 Posted: Mon Jul 08, 2013 8:38 pm Post subject: File Nodes : Need some pointers |
 |
|
Acolyte
Joined: 20 May 2013
Posts: 58
|
Hi,
There is a requirement where we need to archive our messages in a file. Now the messages have to be archived in the same file(append) till the size reahes a particular limit (say 10 MB). After that a new file needs to be created which would be following similar pattern. Can anyone tell me how to achieve this if someone has worked on this. |
|
| Back to top |
|
 |
| zpat |
| Posted: Mon Jul 08, 2013 10:21 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Easy enough - just keep appending the message data to the same file and keep a count of the cumulative amount of data (e.g. in an external variable).
When you reach the limit finalize the file after that message. The next one can start a new file (set a new name obviously). |
|
| Back to top |
|
|
| Abhinay185127 |
| Posted: Mon Jul 08, 2013 10:36 pm Post subject: |
|
|
Acolyte
Joined: 20 May 2013
Posts: 58
|
| Quote: |
Easy enough - just keep appending the message data to the same file and keep a count of the cumulative amount of data (e.g. in an external variable).
When you reach the limit finalize the file after that message. The next one can start a new file (set a new name obviously). |
Basically we will be getting a message every 5 mins. Now second time when the message comes (say we have kept the count of the data in an external variable) we have to compare the cumulative data size with the 10 MB size. So the external variable will work or do we need to use global cache for storing the size information. |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Jul 08, 2013 10:39 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
Why would I give you a solution that does not work?
Perhaps read the infocenter. |
|
| Back to top |
|
|
| Abhinay185127 |
| Posted: Mon Jul 08, 2013 10:41 pm Post subject: |
|
|
Acolyte
Joined: 20 May 2013
Posts: 58
|
Actually have not worked with external variable so don't know the functionality.
Will read about it.
Thanks for the help.  |
|
| Back to top |
|
|
| Abhinay185127 |
| Posted: Mon Jul 08, 2013 10:43 pm Post subject: |
|
|
Acolyte
Joined: 20 May 2013
Posts: 58
|
Actually have not worked with external variable so don't know the functionality.
Will read about it.
Thanks for the help. |
|
| Back to top |
|
|
| zpat |
| Posted: Mon Jul 08, 2013 10:57 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
You will also need to consider what happens if you want to stop the the EG or broker before when a partial file has been written.
The value in the external variable will not survive a broker, or EG restart. |
|
| Back to top |
|
|
| Abhinay185127 |
| Posted: Mon Jul 08, 2013 11:09 pm Post subject: |
|
|
Acolyte
Joined: 20 May 2013
Posts: 58
|
| Quote: |
You will also need to consider what happens if you want to stop the the EG or broker before when a partial file has been written.
The value in the external variable will not survive a broker, or EG restart. |
External variable we will be setting as a UDP. So in this case we will define this external variable at the flow level by calculating the size of input message and storing it.
So once the flow terminates and we get a new request does this means that the value(size) that we store in external variable will still be there because we have to compare the size.
I am asking this because we are defining it at flow level so after the flow successfully terminates the scope of the external variable should end.
Kindly correct my understanding if wrong. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Mon Jul 08, 2013 11:25 pm Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
How does defining it as a UDP cover a
- a redeploy of the flow (i.e. a new version)
- a broker restart
- failing the broker over to another node (if in an HACMP environment)
You really need to hold the accumulated file size in a table.
IMHO, Archiving messages to files like this is a seriously 'old-school' habit. There are far better ways these days.
I'd go back to the person who gave you the requirement and ask them the following:-
What sort of file maintenance and archiving are you putting in place?
What happens when the target disk becomes full?
Dare I say it, perhaps the requirement is wrong in the first place?
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Abhinay185127 |
| Posted: Mon Jul 08, 2013 11:42 pm Post subject: |
|
|
Acolyte
Joined: 20 May 2013
Posts: 58
|
But since the requirement is such and we already suggested the record and replay mechanism for archiving but the clients want to go with the file archiving only.
| Quote: |
How does defining it as a UDP cover a
- a redeploy of the flow (i.e. a new version)
- a broker restart
- failing the broker over to another node (if in an HACMP environment) |
So in that case external variable won't work. Can we use shared variable for achieving this scenario or should we go for global cache.
Once we have archived the files they will be deleted after 7 days based on the timestamp. So the disk should not be full as the traffic coming is not much. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jul 09, 2013 12:20 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
A shared variable won't survive a broker restart.
Unless you are already using a Global Cache setup then I'd go for storing the data in a table.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| lancelotlinc |
| Posted: Tue Jul 09, 2013 3:11 am Post subject: |
|
|
Jedi Knight
Joined: 22 Mar 2010
Posts: 4941
Location: Bloomington, IL USA
|
| Abhinay185127 wrote: |
| should we go for global cache. |
Before you go for anything else, you should go for training.
_________________
http://leanpub.com/IIB_Tips_and_Tricks
Save $20: Coupon Code: MQSERIES_READER |
|
| Back to top |
|
|
| Abhinay185127 |
| Posted: Tue Jul 09, 2013 8:14 pm Post subject: |
|
|
Acolyte
Joined: 20 May 2013
Posts: 58
|
| Quote: |
| Unless you are already using a Global Cache setup then I'd go for storing the data in a table. |
I tried using the global cache but even global cache couldn't survive a broker restart.
So this means to achieve the scenario using a database is the only possibility.
Kindly guide. |
|
| Back to top |
|
|
| Abhinay185127 |
| Posted: Tue Jul 09, 2013 8:15 pm Post subject: |
|
|
Acolyte
Joined: 20 May 2013
Posts: 58
|
@lancelotlinc
Yes sure i'll be going for a training to learn the wider aspects of WMB. |
|
| Back to top |
|
|
| zpat |
| Posted: Tue Jul 09, 2013 11:48 pm Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5866
Location: UK
|
| You could also put it into a message on a queue. I know a queue is not a database, but a single message does no harm and it can be a lot easier to arrange than getting a new database table set up. |
|
| Back to top |
|
|
|
|
