| Author |
Message
|
| tanght |
 Posted: Tue Jan 04, 2011 6:05 pm Post subject: Queries about persistent msgs |
 |
|
Novice
Joined: 04 Jan 2011
Posts: 15
|
I encounter one problem in the testing.
environment is z/os 1.9 and MQ version 6.
the queue attribute was set to persistent yes, and according to the application that they did not set its own persistency on the msgs and they will use the defaulty value configured in the Websphere MQ queue properties.
However when we recycle the QMGR, the msgs in the queue are gone.
questions:
1) if application did not specify the persistency in the msgs, it will follow the default value in the queue attribute, is this correct?
2) when i recycle the qmgr, stop qmgr mode(force) was used as some application is still using the qmgr. will this have any impact on this issue?
Wish you guys can help on this....
thanks |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Tue Jan 04, 2011 7:00 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9405
Location: US: west coast, almost. Otherwise, enroute.
|
Exactly what problem are you having? Are the messages disappearing from the queue(s) after qmgr restart?
| Quote: |
| 1) if application did not specify the persistency in the msgs, it will follow the default value in the queue attribute, is this correct? |
No. (perhaps yes, read on.)
The programmer must specify one of these values in the MQMD for each message:
- this message is persistent
- this message is non-persistent
- this message should take the persistence attribute of the queue
The initial value of the MQMD persistence field in the MQMD is MQPER_PERSISTENCE_AS_Q_DEF.
So, if the programmer did not change the persistence field attribute of the MQMD, the message(s) will take the queue persistence attribute.
Programmers have been known to lie about this, and other programmer thingies.
| Quote: |
| 2) when i recycle the qmgr, stop qmgr mode(force) was used as some application is still using the qmgr. will this have any impact on this issue? |
No. By definition, non-persistent messages do not survive qmgr restart.
The real test here is to have the application put some messages to a queue, then browse the messages with CSQUTIL to verify that persistence is set appropriately.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Tue Jan 04, 2011 10:13 pm Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
In addition, there is queue name resolution. lets assume the programmers use "persistence as queue def", now it depends which queue is found first. if you set the qlocal to defpsist(yes), but the programmers refer to a qalias with defpsist(no), then the messages will be non persistent.
so make sure the defpsist attribute is set properly on the queue that is used as a resulot of the name resolution during mnopen.
i personally do not like defpsist to be used, however, if it is used, i prefer to set it the same value for ALL queues used by the application, no matter if the queue is involved in the naming resolution or not. its then just for documentation, or avoid to use the wrong persistence if e.g. the qlocal is used instead of the qalias......
_________________
Regards, Butcher |
|
| Back to top |
|
|
| tanght |
| Posted: Tue Jan 04, 2011 10:47 pm Post subject: |
|
|
Novice
Joined: 04 Jan 2011
Posts: 15
|
Bruce, butcher.. thx for your reply.
when the qmgr restarts, the msg inside the persistent queue was gone.
I will discuss with various team to find out why..
as although application team mentioned that the persistent msgs were in the queue when the qmgr was recycled. the pageset which includes the persistent msgs kept unchanged after the recycle of the qmgr. That is why I am wondering whether the msgs are persistent or not.
I will try to ask them to send some more msgs and verify at the queue level so that I can see which portion is causing the problems. |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Wed Jan 05, 2011 12:00 am Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
please keep in mind that persistence is a message attribute, not a queue attribute. once persistence or non persistence is assigned to a message it will not change by hitting a queue with a different DEFPSIST - value. or - in other words, if there are non persistent messages in a queue with defpsist(yes), they are still non persistent messages and will be treated as non persistent messages.
so imho the messages where non persistent in that queue, or they have been consumed before / after the restart.
as you already said ... let them create new messages and check the persistence once they are in the queue.
what do you mean by unchanged pageset? how do you verify a pageset is unchanged? imho, every pageset is changed during stop / restart of a queuemanager, as they hold checkpoint information .....
_________________
Regards, Butcher |
|
| Back to top |
|
|
| HubertKleinmanns |
| Posted: Wed Jan 05, 2011 2:55 am Post subject: |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
As Mr. Butcher said, persistency is an attribute of the message. Except for temporary dynamic queues an some types of shared queues you may have persistent and non-persistant messages in the queue at the same time.
To prevent non-persistent messages beeing removed during a normal QMgr restart you may use the attribute NPMCLASS.
NPMCLASS(HIGH) means, non-persistent messages survive a normal QMgr restart. But these messages are removed when the QMgr crashes.
NPMCLASS(NORMAL) means, non-persistent messages will be lost after a QMgr restart.
_________________
Regards
Hubert |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Wed Jan 05, 2011 6:43 am Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
from the manual for NPMCLASS(HIGH)
| Quote: |
| The queue manager attempts to retain non-persistent messages on this queue over a queue manager restart. |
so it is just an attempt. if it is just an attempt, it may fail. i would not rely on it.
_________________
Regards, Butcher |
|
| Back to top |
|
|
| exerk |
| Posted: Wed Jan 05, 2011 6:51 am Post subject: Re: Queries about persistent msgs |
|
|
Jedi Council
Joined: 02 Nov 2006
Posts: 6339
|
| tanght wrote: |
| 2) when i recycle the qmgr, stop qmgr mode(force) was used as some application is still using the qmgr. will this have any impact on this issue? |
Yes for any uncommitted messages.
_________________
It's puzzling, I don't think I've ever seen anything quite like this before...and it's hard to soar like an eagle when you're surrounded by turkeys. |
|
| Back to top |
|
|
| tanght |
| Posted: Wed Jan 05, 2011 11:10 pm Post subject: |
|
|
Novice
Joined: 04 Jan 2011
Posts: 15
|
| Mr Butcher wrote: |
please keep in mind that persistence is a message attribute, not a queue attribute. once persistence or non persistence is assigned to a message it will not change by hitting a queue with a different DEFPSIST - value. or - in other words, if there are non persistent messages in a queue with defpsist(yes), they are still non persistent messages and will be treated as non persistent messages.
so imho the messages where non persistent in that queue, or they have been consumed before / after the restart.
as you already said ... let them create new messages and check the persistence once they are in the queue.
what do you mean by unchanged pageset? how do you verify a pageset is unchanged? imho, every pageset is changed during stop / restart of a queuemanager, as they hold checkpoint information ..... |
I am not sure whether my way of understanding pageset is correct or not..
for example, when we issue dis usage, the results are like this:
---------
CSQI010I +MQB1 Page set usage ... 816
Page Buffer Total Unused Persistent Nonpersistent
set pool pages pages data pages data pages
_ 0 0 10078 7979 2099 0
_ 1 1 10078 9164 914 0
_ 2 2 10078 9593 485 0
_ 3 3 20157 19910 226 21
_ 4 1 20157 20118 2 37
_ 5 2 20157 20151 3 3
_ 6 3 44996 44988 8 0
-------
Normally when we define a queue, it is related to a particular stgclass which will be tagged to a particular pageset right? I thought this is the place where persistent msgs stay.
after the recycle of the qmgr, I notice that the persistent data pages kept unchanged. thus I thought the msgs are not persistent.
Also one more question, when we use mq browse utility to browse a queue, I use "PARM=('MQB1,queue name,1,B,S')"
I can see all the msgs in the queue, however I am not able to tell which portion inside the msg is about the persistent settings.
Is there any key work I can refer to?
Thanks for all the help.. |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Wed Jan 05, 2011 11:35 pm Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
As multiple messages can share a single page in the pageset i would not rely on the page counters to decide wheather a specific message may have been persistent or not.
The mq browse utility (at least its just a sample program) will only show the message payload, not the MQMD header that can tell you if the message is persistent. Either improve that sample program to show more fields, or use any of the existing tools aroud.
As you are on z/OS, you may have the MA10 supportpac around so you can brose the queue from ISPF, switch to hex and check the persistence. or MA17 supportpac. Or the MO71 supportpac, or the MA01 supportpac, or or.
All these tools and others too are able to give you the information if a message is persistent or not by browsing the queue.
You should have more (and more handy) tools in your shop then just the mq sample programs.
If a message is already consumed, but you know the message content so you are able to identify that message, you could use csq1logp and try to find it in the archive logs (if these are still available). You must search at the date / time the message was put into the queuemanager, as this is the time the message content is logged. the message content is not logged again when the message is get.
If you find the message, it was persistent as only persistent messages are logged on z/OS. If you did not find it, and you are sure you searched everything right (e.g. text strings in the proper codepage), then the message was non persistent.
Just a possibility to find out about the past.
_________________
Regards, Butcher |
|
| Back to top |
|
|
| tanght |
| Posted: Thu Jan 06, 2011 7:50 pm Post subject: |
|
|
Novice
Joined: 04 Jan 2011
Posts: 15
|
Butcher,
I tried to use MA10 to browse the msg, and also I saw the MD.
But I am not sure how to read this.. I try to search for some manual and the forum but still have not found any useful information.
---
MD ...........................4MQSTR ..........
DC44000000000000FFFF00000001000FDDEED4440000000033***
4400000100000008FFFF000000310014482390000003000032***
------
Do I need to do some other things, I just want to chck whether the msgs are persistent or not... |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Thu Jan 06, 2011 8:46 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9405
Location: US: west coast, almost. Otherwise, enroute.
|
The layout of the MQMD is documented in the WMQ Application Programming Reference manual. Search here or on Google.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| HubertKleinmanns |
| Posted: Thu Jan 06, 2011 9:51 pm Post subject: |
|
|
Shaman
Joined: 24 Feb 2004
Posts: 732
Location: Germany
|
tanght,
the browse tool of MA10 is quite poor. You need to look at the WMQ Application Programming Reference manual as bruce2359 suggested.
If you could other tools like MA17 or the MQ explorer, to see the attributes of the MQMD directly.
_________________
Regards
Hubert |
|
| Back to top |
|
|
| skoobee |
| Posted: Thu Jan 06, 2011 10:10 pm Post subject: |
|
|
Acolyte
Joined: 26 Nov 2010
Posts: 52
|
| I do not really see the point of inspecting the msg in the queue to see if it is persistent or not. You already know that it is non-persistent, because it is deleted when the qmgr is stopped and started. You need to concentrate on why the app putting the msg is not setting the persistence attribute to the required value. |
|
| Back to top |
|
|
| Mr Butcher |
| Posted: Thu Jan 06, 2011 10:30 pm Post subject: |
|
|
Padawan
Joined: 23 May 2005
Posts: 1716
|
no this is not for sure. what about if - for whatever reason - the messages have not been consumed by the application before restart? and that after the restart a trigger was created automatically and messages have been consumed? or they have been purged by a colleague or application test? or or or?
What i want to say is that just because messages are gone after a restart does not mean for sure they have been non persistent. There can be other reasons the messages have been gone. If you want to know for sure, check it. dont guess it.
MA10 is not poor. It just gives you the data at a raw level, which may not be comfortable for everybody. But it is fair enough to get the information you need.
You have already been pointed to the documentation, the APR guide shows the layout of the header data areas. in this case you need to look at the MQMD storage layout, i prefer the assembler structure .....
| Code: |
---
MD ...........................4MQSTR ..........
DC44000000000000FFFF00000001000FDDEED4440000000033***
4400000100000008FFFF000000310014482390000003000032***
------
MQMD DSECT
MQMD_STRUCID DS CL4 Structure identifier
MQMD_VERSION DS F Structure version number
MQMD_REPORT DS F Options for report messages
MQMD_MSGTYPE DS F Message type
MQMD_EXPIRY DS F Message lifetime
MQMD_FEEDBACK DS F Feedback or reason code
MQMD_ENCODING DS F Numeric encoding of message data MQMD_CODEDCHARSETID DS F Character set identifier of message
* data
MQMD_FORMAT DS CL8 Format name of message data
MQMD_PRIORITY DS F Message priority
MQMD_PERSISTENCE DS F Message persistence
MQMD_MSGID DS XL24 Message identifier
MQMD_CORRELID DS XL24 Correlation identifier
and so on
|
now if you count the bytes you can see this message is non persistent as the storage area for the persistence attribute is x'00000000'
If you do not know what the value of an attribute means check mq constants manual, e.g.
| Code: |
MQPER_NOT_PERSISTENT 0 X’00000000’
MQPER_PERSISTENT 1 X’00000001’
MQPER_PERSISTENCE_AS_Q_DEF 2 X’00000002’ |
_________________
Regards, Butcher |
|
| Back to top |
|
|
|
|
