| Author |
Message
|
| hughson |
 Posted: Sun Aug 27, 2017 3:39 am Post subject: SERVICE object on Windows - STDOUT and STDERR |
 |
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
In answer to another thread,
| gbaddeley wrote: |
I use something like the following to set up a trigger monitor on Windows:
| Code: |
define service('MY.TRIGMON') replace +
control(qmgr) servtype(server) descr('Meaningful description') +
startcmd('+MQ_INSTALL_PATH+bin\runmqtrm.exe') +
startarg('-m +QMNAME+ -q MY.INITQ') +
stopcmd('+MQ_INSTALL_PATH+bin\amqsstop.exe') +
stoparg('-m +QMNAME+ -p +MQ_SERVER_PID+') +
stdout('D:\MyTrigMonLogs\stdout.log') +
stderr('D:\MyTrigMonLogs\stderr.log') |
|
I am curious Glenn, on Windows, can you see the contents of said log files while the SERVICE is running?
In my recent experiments, I can see the contents on Linux, but not on Windows, and was wondering if you had the same? Once I stop the SERVICE on Windows, then I can see the contents of the file, but not when it is running.
Cheers
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Sun Aug 27, 2017 6:22 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
That sounds like the windows background process that's running the MQ service doesn't do either a flush or a close on the log file periodically. Or that the service is doing a flush, and windows is staging the data in another file waiting for the file to be closed. some kind of temp file.
Do you see something like that in the directory, Morag? Like MS Word does a ~filewhatever until the file is closed?
Either way it sounds like an interaction between the mq service daemon process and Windows.
It would be interesting to see if the STARTCMD did something different when it did a windows "start". Makes it harder to make the STOPCMD useful, though.
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| hughson |
| Posted: Sun Aug 27, 2017 2:33 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
| mqjeff wrote: |
| That sounds like the windows background process that's running the MQ service doesn't do either a flush or a close on the log file periodically. |
My thoughts exactly. Was wondering if anyone has actually seen the contents of these files (where the service is running) or whether this is a product defect.
| mqjeff wrote: |
| Do you see something like that in the directory, Morag? Like MS Word does a ~filewhatever until the file is closed? |
Nope, only the file names with 0 size (even though I know there is content in one of them since the messages are written as timestamped).
| mqjeff wrote: |
| It would be interesting to see if the STARTCMD did something different when it did a windows "start". |
Adding start to the front of the STARTCMD results in the following error, so non-starter (no pun intended!)
| Code: |
AMQ5019: Unable to access program 'start C:\mqm9030\bin64\runmqtrm'.
EXPLANATION:
A request was made to execute the program start C:\mqm9030\bin64\runmqtrm,
however the operation was unsuccessful because the program could not be found
in the specified location.
ACTION:
Check the definition of the service specifies the correct and full path to the
program to run. If the path is correct then verify that the program exists in
the specified location and that IBM MQ userid has permission to access it. |
Cheers,
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sun Aug 27, 2017 3:58 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
... isn't start a standard msdos shell command? maybe it needs to be run inside a shell, or as another argument to cmd.exe...
hrm. Not sure it's worth doing anything more than just running a shell script... or opening a PMR...
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Sun Aug 27, 2017 4:53 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| hughson wrote: |
| mqjeff wrote: |
| That sounds like the windows background process that's running the MQ service doesn't do either a flush or a close on the log file periodically. |
My thoughts exactly. Was wondering if anyone has actually seen the contents of these files (where the service is running) or whether this is a product defect. |
I did some testing. Set up a trigmon service as per above. Put 10000 messages on the init q to generate some stdout. The output file shows as 0 size in Windows Explorer and empty when opened. I put another 10000 messages. The file size jumped to 1054KB and it had the expected content, although the last line of the file was incomplete. It looks like flushing is occurring, but using a fairly large buffer size.
_________________
Glenn |
|
| Back to top |
|
|
| hughson |
| Posted: Mon Aug 28, 2017 9:55 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
| gbaddeley wrote: |
| I did some testing. Set up a trigmon service as per above. Put 10000 messages on the init q to generate some stdout. The output file shows as 0 size in Windows Explorer and empty when opened. I put another 10000 messages. The file size jumped to 1054KB and it had the expected content, although the last line of the file was incomplete. It looks like flushing is occurring, but using a fairly large buffer size. |
Prompted by this, I also had a play and found that 10 triggering application invocations seems to be enough to flush the file. My file showed size 8246 bytes when it was flushed.
So is it size that causes the flush? or perhaps 'n' activities?
At least I can see something now! 
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Tue Aug 29, 2017 5:33 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
Morag, I couldn't cause an immediate flush with just a few messages (even 100's). However, I checked again after about 10 minutes of no more trigger messages, and a flush had occurred, though the last line was incomplete.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa364218
Unless runmqtrm.exe calls fflush(), physical writes to the log file are at the mercy of Windows?
_________________
Glenn |
|
| Back to top |
|
|
| hughson |
| Posted: Tue Aug 29, 2017 6:03 pm Post subject: |
|
|
Padawan
Joined: 09 May 2013
Posts: 1959
Location: Bay of Plenty, New Zealand
|
I agree it is no doubt at the mercy of Windows. It also strikes me that applications designed to write to stdout and stderr wouldn't expect to need to call fflush().
The SERVICE object wrapper is what is getting in the way here.
The link you provide also has the following to say:-
| MSDN wrote: |
| Some applications, such as virus-checking software, require that their write operations be flushed to disk immediately; Windows provides this ability through write-through caching. A process enables write-through caching for a specific I/O operation by passing the FILE_FLAG_WRITE_THROUGH flag into its call to CreateFile. With write-through caching enabled, data is still written into the cache, but the cache manager writes the data immediately to disk rather than incurring a delay by using the lazy writer. |
It occurs to me that the SERVICE object wrapper when it creates the files configured in the STDOUT and STDERR attributes, should perhaps be using this flag.
Should I be raising an RFE?
Cheers
Morag
_________________
Morag Hughson @MoragHughson
IBM MQ Technical Education Specialist
Get your IBM MQ training here!
MQGem Software |
|
| Back to top |
|
|
| mqjeff |
| Posted: Wed Aug 30, 2017 3:55 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| hughson wrote: |
| Should I be raising an RFE? |
If it works one way on one platform, and not on another, this seems like a PMR issue and not an RFE..
And just think of the fun you can have working through a PMR...  
_________________
chmod -R ugo-wx / |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Aug 30, 2017 5:26 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| gbaddeley wrote: |
Morag, I couldn't cause an immediate flush with just a few messages (even 100's). However, I checked again after about 10 minutes of no more trigger messages, and a flush had occurred, though the last line was incomplete.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa364218
Unless runmqtrm.exe calls fflush(), physical writes to the log file are at the mercy of Windows? |
Not quite. I believe a flush occurs when you stop the trigger monitor. So bouncing the trigger monitor should give you a current file to look at.
I usually deal with troubleshooting those problems by running the trigger monitor in the foreground instead of the background. The information then sails right across the screen. Just plan for enough screen buffer. And of course do this in a very strictly controlled environment: i.e. the only queue using this trigger monitor, one message only (if possible).... mostly this reveals an unexpected error or some foreground stuff that had not been catered for... 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Wed Aug 30, 2017 4:17 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2538
Location: Melbourne, Australia
|
| hughson wrote: |
| It occurs to me that the SERVICE object wrapper when it creates the files configured in the STDOUT and STDERR attributes, should perhaps be using this flag. |
It would be interesting to know exactly how the SERVICE object implements these attributes, and if there is any OS specific code. It could be as primitive as building a command line containing "1>>stdoutfile.txt 2>>stderrfile.txt &" and feeding it to the system() function to spin off a new process. In which case, the OS opens the files and won't be using the flag.
_________________
Glenn |
|
| Back to top |
|
|
| tczielke |
| Posted: Wed Aug 30, 2017 5:22 pm Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
| fjb_saper wrote: |
| gbaddeley wrote: |
Morag, I couldn't cause an immediate flush with just a few messages (even 100's). However, I checked again after about 10 minutes of no more trigger messages, and a flush had occurred, though the last line was incomplete.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa364218
Unless runmqtrm.exe calls fflush(), physical writes to the log file are at the mercy of Windows? |
Not quite. I believe a flush occurs when you stop the trigger monitor. So bouncing the trigger monitor should give you a current file to look at.
I usually deal with troubleshooting those problems by running the trigger monitor in the foreground instead of the background. The information then sails right across the screen. Just plan for enough screen buffer. And of course do this in a very strictly controlled environment: i.e. the only queue using this trigger monitor, one message only (if possible).... mostly this reveals an unexpected error or some foreground stuff that had not been catered for... |
Another trick is to do "strmqtrc -t all -p runmqtrm". You can then let the trigger monitor run as normal, and the trace will collect information like if a trigger happened, return codes back to the runmqtrm program when it tried to exec the triggered script, etc.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
|
|
