ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » General IBM MQ Support » High number of msgs and LogBufferPages

Post new topic  Reply to topic
 High number of msgs and LogBufferPages « View previous topic :: View next topic » 
Author Message
mqrules
PostPosted: Tue Sep 06, 2022 1:55 pm    Post subject: High number of msgs and LogBufferPages Reply with quote

Centurion

Joined: 01 Jun 2005
Posts: 100
Location: US

Hi All,

Platform: MQ 9.2.0.5 on Linux

QM1 --> QM2 on server1--> QM3 on server1

Problem: When there is a huge number of msgs being sent from QM1 (belonging to another company), msgs accumulate on the XMITQ of QM2 and the curdepth of the target local queue (with Maxdeph 10K) shows 0-10 msgs only...IPPROCS showing 5.

LogBufferPages= 0|0-4096
The amount of memory allocated to buffer records for writing,
specifying the size of the buffers in units of 4 KB pages.

The minimum number of buffer pages is 18 and the maximum
is 4096. Larger buffers lead to higher throughput, especially for
larger messages.

I think setting LogBufferPages=4096 (or to some big number, like 18 or higher) in the qm.ini of QM3 would enable a faster write. Needless to say, I will have to find out from the LINUX admins the size of the current memory and how much more we would need.

Any comments, please?
TIA
MR
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Tue Sep 06, 2022 4:09 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

Quote:
msgs accumulate on the XMITQ

There are many possible reasons for this. Check the throughput on the channel to QM3. How many msgs/sec and MB/sec is it doing during peak period?
_________________
Glenn
Back to top
View user's profile Send private message
bruce2359
PostPosted: Wed Sep 07, 2022 6:08 am    Post subject: Re: High number of msgs and LogBufferPages Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

mqrules wrote:
Problem: When there is a huge number of msgs being sent from QM1 (belonging to another company),

Numerically, how many is huge?
mqrules wrote:
msgs accumulate on the XMITQ of QM2

How many messages? Are you experiencing channel errors? Display channel status for the channel servicing the xmitq. Display the results here.
mqrules wrote:
and the curdepth of the target local queue (with Maxdeph 10K) shows 0-10 msgs only...IPPROCS showing 5.

So, the 5 message consumers seem to be keeping up with the workload.

What problem are you experiencing? I ask because much of what you tell us sounds like normal behavior, and not a problem.

Are your transactions missing SLA (Service Level Agreement)? Something else?

I usually leave log buffer pages at zero to let the qmgr allocate as needed.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
mqrules
PostPosted: Wed Sep 07, 2022 6:50 am    Post subject: Reply with quote

Centurion

Joined: 01 Jun 2005
Posts: 100
Location: US

We are talking about millions of msgs in a very short period of time (don't remember the exact time) especially at month ends... There is really no SLA issue and no complaints...Yes, the product is working the way it is designed. I know, I know, I almost hear some say "if it ain't broke, don't fix it "...

However, there is also a separate case where the msgs would accumulate QM1 XMITQ (with the channels running) and we would get a midnight call from the external company asking us if we have an issue -- and almost always the response is no... Bandwith of the Network connection is not a problem...

Anyway, my question is general: Why not increase the LogBufferPages from the default 0 to get more throughput as a measure of fine-tunining that would help cases like this?

Thanks,
TIA
MR
Back to top
View user's profile Send private message
bruce2359
PostPosted: Wed Sep 07, 2022 7:30 am    Post subject: Reply with quote

Poobah

Joined: 05 Jan 2008
Posts: 9394
Location: US: west coast, almost. Otherwise, enroute.

mqrules wrote:
Anyway, my question is general: Why not increase the LogBufferPages from the default 0 to get more throughput as a measure of fine-tunining that would help cases like this?

Thanks,
TIA
MR

Why are you target-locked on logbufferpages? Why do you believe that adjusting this will improve throughput? What else have you tried?

What metric would you use to determine if this tuning effort was successful or not? The external folks calling? Or, not calling? Something else?

What hardware platform is at the originating end? What o/s? What MQ VRMF?

To help us help you, please ... display the channel status of the sender/receiver channel that services the xmitq, and post the results here? Best done when throughput appears to be a problem for you.

Any channel error logged in AMQERR01 for this qmgr?
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live.
Back to top
View user's profile Send private message
Andyh
PostPosted: Wed Sep 07, 2022 8:44 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

Almost everyone doing any volume of persistent messaging should set LogBufferPages to the maximum supported value. This parameter is specified in units of log pages, which on the distributed MQ product are currently always 4KB. Setting 4096 will therefore consume only 16MB of memory which might have been a lot when MQ was first written (1990's) but is now a trivial amount of memory. The size of the log buffer (16MB at 4096) is the size of the largest single write the queue manager can make to the log. The queue manager marshals message data into the log buffer to allow writes from multiple puts (either concurrent puts or a batch from a single writer) to be flushed to disk in a single synchonous write. Setting LogBufferPages to 4096 eliminates this as a possible restriction and is a trivial change (requires the QMgr to be recycled).
As you report the messages to be building up on a transmit queue you may also want to check that channel batching is enable and to measure the effective batch size being achieved.
Back to top
View user's profile Send private message
mqrules
PostPosted: Wed Sep 07, 2022 6:13 pm    Post subject: Reply with quote

Centurion

Joined: 01 Jun 2005
Posts: 100
Location: US

Thank you Andyh and others for your valuable comments.
MR
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » General IBM MQ Support » High number of msgs and LogBufferPages
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.