| Author |
Message
|
| Sam Uppu |
 Posted: Mon Jul 20, 2009 3:38 pm Post subject: core dump |
 |
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
Hi Guys,
we have few scripts for mq administration. In version earlier to 7.0, all these scripts were running fine. One of the script is throwing core dump error when I gets a message from queue.
Environment:
Solaris 86: MQ v 7.0.0.2
Do we need to set any environment variables? let me know your thoughts
Thankis |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Mon Jul 20, 2009 10:19 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
What is the generated FDC?
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| exerk |
| Posted: Mon Jul 20, 2009 11:19 pm Post subject: Re: core dump |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| Sam Uppu wrote: |
| ...Do we need to set any environment variables? let me know your thoughts... |
How would we know, given such sparse information? Without knowing the function of the scripts, no real advice can be given, other than what fjb_saper has already suggested.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Tue Jul 21, 2009 6:54 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
No FDC files are generated.
The scripts we are using for administration activities. Like removing the message from dead letter queue, monitoring channels, listeners, removing expired messages etc.
I am wondering why other scripts are working fine.
The issue comes when removing expired messages from queue. The same script works fine in version lower to MQ v7. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 21, 2009 7:06 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Sam Uppu wrote: |
I am wondering why other scripts are working fine.
The issue comes when removing expired messages from queue. The same script works fine in version lower to MQ v7. |
I am wondering why you use a script for this. The queue manager removes them automatically.
So how does this script remove these messages?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Tue Jul 21, 2009 8:02 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| Vitor wrote: |
| Sam Uppu wrote: |
I am wondering why other scripts are working fine.
The issue comes when removing expired messages from queue. The same script works fine in version lower to MQ v7. |
I am wondering why you use a script for this. The queue manager removes them automatically.
So how does this script remove these messages? |
THis script is executed because if there are any expired msgs on the queue and when we look at the current depth of the queue, we dont know whether they are expired or live msgs sitting on the queue. We dont want to have expired msgs sitting on the queues and filling up the queues. For this reason we are removing the expired msgs with this script.
The script was running fine until we were running on 7.0.0.0 and lower versions of MQ. We recnetly upgraded to 7.0.0.2 and since then we are having this issue.
Not sure whether it is the issue with fixpack 7.0.0.2 or something else.
Your thoughts please.
thanks. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 21, 2009 8:10 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Sam Uppu wrote: |
| THis script is executed because if there are any expired msgs on the queue and when we look at the current depth of the queue, we dont know whether they are expired or live msgs sitting on the queue. We dont want to have expired msgs sitting on the queues and filling up the queues. For this reason we are removing the expired msgs with this script. |
You still have not said what this script does; how it is actually doing this task. This makes it very hard to offer advice. What command/app/etc is the script trying to run? Why is it not simply browsing the queue?
Also expired messages do not "sit" on queues; the queue manager removes them. I accept that you could examine a queue and get a "false positive" on depth but wonder how much of a problem this could cause.
Likewise if your queues are close to filling, you need deeper queues. Disc space is cheap.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Jul 21, 2009 8:17 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
Curious: are you running this script once a day? once an hour? once every second? 100 times a second?
How exactly does your script delete expired messages from queues?
Since an application can not get an expired message (refer to the WMQ Application Programming Reference and WMQ Application Programming Guide), how is your script deleting expired messages?
Have expired messages caused you any problem?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Tue Jul 21, 2009 8:49 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
| bruce2359 wrote: |
Curious: are you running this script once a day? once an hour? once every second? 100 times a second?
How exactly does your script delete expired messages from queues?
Since an application can not get an expired message (refer to the WMQ Application Programming Reference and WMQ Application Programming Guide), how is your script deleting expired messages?
Have expired messages caused you any problem? |
We are running it daily once. The script is a binary and we dont see amqsget or amqsbcg when we do a strings <scriptname> and its not written by me and dont know what is inside this binary.
The script will remove the expired msgs which are older than 1 day.
Thanks for your insight on this. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Tue Jul 21, 2009 9:40 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9469
Location: US: west coast, almost. Otherwise, enroute.
|
I'll take a guess here that the message in the queues do not have expiry set in the MQMD; but are logically expired due to date/time business rules. And that the script executes an application that browses through the queues, comparing the date/time the message was created (in the MQMD).
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jul 21, 2009 12:12 pm Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| bruce2359 wrote: |
| I'll take a guess here that the message in the queues do not have expiry set in the MQMD; but are logically expired due to date/time business rules. And that the script executes an application that browses through the queues, comparing the date/time the message was created (in the MQMD). |
It sounds like whatever this application is doing (presumably a browse & delete under cursor) it doesn't work so well under the latest fix pack. This would explain the core dump and lack of FDC - it's the application abending not the queue manager.
I think this is about the point where we stop being a lot of help. As it's a home-grown binary then it could be doing anything, there are no clues from the WMQ side so the core dump is your only clue.
If you can find the source code that will give you and us more to work with. As a simple plan recompiling the source against the v7 libraries is a solid start.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jul 21, 2009 8:25 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Vitor wrote: |
| If you can find the source code that will give you and us more to work with. As a simple plan recompiling the source against the v7 libraries is a solid start. |
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Sam Uppu |
| Posted: Fri Jul 24, 2009 7:27 am Post subject: |
|
|
Yatiri
Joined: 11 Nov 2008
Posts: 610
|
Hi Guys,
When the core was debugged, it was found that the application function printfmqftstringlist has called strcpy() which has suffered a SIGSEGV exception. No application exception handler is found and so the MQ exception handler takes the default action of the exception, which is to call abort(). Calling abort() results in the core dump being produced.
I think there is problem with linking program to MQ libraries. I am wondering if there are any Header files/ Libraries needed to sopport MQ C clients are missing?
Do I need to include any references to addressibility modules in CLASSPATH and LD_LIBRARY_PATH variables?
Let me know your thoughts.
Thanks |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Jul 25, 2009 2:38 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
 programing manual V7  tells you how to compile and link your source... and look at the examples to see if your source needs some changes too... Not rocket science... And remember you must link to MQ libraries first...
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Sat Jul 25, 2009 8:55 am Post subject: Re: core dump |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
| Sam Uppu wrote: |
| The script is a binary ... |
"script" is the wrong word to describe a binary application.
| Sam Uppu wrote: |
| When the core was debugged, it was found that the application function printfmqftstringlist has called strcpy() which has suffered a SIGSEGV exception. |
strcpy() takes a caller-supplied source string of unknown length, and copies it into a caller-supplied target array of unknown length. It does not do any bounds-checking, depending entirely on the caller to provide a target array large enough to hold the source string. printfmqftstringlist() is failing to do that; either the source string is larger than expected, or it's not properly null-terminated. This is a classic buffer overflow bug.
It does not matter that the application appeared to work fine for ages, until some apparently unrelated change was introduced. strcpy() will only raise a SIGSEGV signal when a buffer overflow is egregious enough to cause a segmentation violation, (i.e. the target array exceeds the bounds of its memory segment). strcpy() will happily (and often silently) clobber the entire memory segment, until then.
The suggested fix is to debug the application source code and recompile.
| Sam Uppu wrote: |
| I think there is problem with linking program to MQ libraries. I am wondering if there are any Header files / Libraries needed to support MQ C clients are missing? |
Header files are for compiling source code, not for running applications. Missing libraries generate a ld (loader) error, not a segmentation violation.
| Sam Uppu wrote: |
| Do I need to include any references to addressibility modules in CLASSPATH and LD_LIBRARY_PATH variables? |
CLASSPATH is for Java, not for C. strcpy() is included in the standard C library, not the MQ libraries; it does not depend on LD_LIBRARY_PATH.
| Sam Uppu wrote: |
| ... we don't know what is inside this binary. |
That's a problem, in and of itself. Supporting unknown code is ... challenging. A lack of source code would limit the more useful options. Here's what's left:- Use the debugger on the core file, to examine the strcpy() arguments, for something obvious such as improperly terminated strings; this might suggest a workaround, in the interim.
- Use a disassembler on the binary, learn assembly language, and debug the assembler source.
- Rewrite the application from scratch, and save the source code this time.
|
|
| Back to top |
|
|
|
|
