ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » Clustering » MQ Clustering performance

Post new topic  Reply to topic
 MQ Clustering performance « View previous topic :: View next topic » 
Author Message
dextermbmq
PostPosted: Fri Jul 24, 2020 7:57 am    Post subject: MQ Clustering performance Reply with quote

Voyager

Joined: 26 Jul 2014
Posts: 77

Hello,

Writing this post to have suggestions on how can I further tune my design with regards to throughput and performance

1-A cluster setup with 5 Queue Managers. 2 Full Repositories. 3 Partial Repositories. All communication within cluster is TLS1.2 mutual authentication enabled.

2-Among the 3 partial repositories, 1 is acting as a Gateway where all the applications connects and issue MQ PUT calls to Alias Queues.The target Queue of Alias Queues are created on remaining 2 Partial repositories.

3-Consuming Applications connect to the 2 Partial Repositories and perform MQ GET from both.All communication with applications is also TLS1.2 protected.

5-I am looking at quite a huge incoming load in coming months ~8-10 Million records per business day i.e 8-10 Million records in 10 business hrs(approx 9 AM to 7 PM)

6- We are running MQ on Physical servers with 8 cores and 32 GB RAM.All are persistent messages and average message size is ~15KB

7- I am a bit concerned on MQ's processing capability because of load. Since all the inbound transactions land on the Gateway Queue manager and get Load balanced to remaining 2 Partial repositories, is it possible that the CLUSSDR Channels can take more time to process the load to 2 downstream partial repositories for MQ GET and there would be frequent pileups on Gateway Queue Manager's System Cluster Transmit Queue ? Is there any data available that can be referred to see how a Queue Manager running on x core handles y Million transactions over cluster channels.

Also, any suggestions that can improve the throughput (adding more partial repositories for MQ GET is one option I could think of) Or would having separate transmission queues(DEFCLXQ(CHANNEL)) for 2 downstream Queue Managers be of any use instead of common System.Cluster.Transmit.Queue
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Sat Jul 25, 2020 1:26 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Have you created a baseline? What performance does your baseline show? Is it sufficient to cover peak traffic? Do you have a specific SLA?
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
dextermbmq
PostPosted: Sat Jul 25, 2020 4:08 am    Post subject: Reply with quote

Voyager

Joined: 26 Jul 2014
Posts: 77

Most of the messaging is asynchronous. There are a couple of integrations where the SLA comes into picture (SLA of 60 secs with End to end journey. There are other application components involved as well which push the message to MQ or which GET from MQ)

Regarding the baseline - A similar environment running on a different infrastructure has the QTIME details on the SCTQ of G/W Queue Manager as :

QUEUE(SYSTEM.CLUSTER.TRANSMIT.QUEUE) TYPE(QUEUE)
CURDEPTH(0) IPPROCS(2)
LGETDATE(<>) LGETTIME(<>)
LPUTDATE(<>) LPUTTIME(<>)
MEDIALOG( ) MONQ(LOW)
MSGAGE(0) OPPROCS(7)
QTIME(16442, 14809) UNCOM(NO)

This infrastructure processes around 1.5-2 Million transactions / day (business hrs) and we have not received any issues from any of the applications around the SLA of transactions being processed

The infrastructure which I am building now has almost 4 times the load with same design.
Back to top
View user's profile Send private message
gbaddeley
PostPosted: Sun Jul 26, 2020 7:50 pm    Post subject: Reply with quote

Jedi

Joined: 25 Mar 2003
Posts: 2492
Location: Melbourne, Australia

Hi Dexter. You suggest 277 TPS peak, which is well within what MQ can handle, given sufficient resources. Is the consumer app (2 instances) capable of processing this volume of workload? As already mentioned, you should perform a volume test in your equivalent non-production environment. This will reveal the bottlenecks and confirm your expectations as you keep pushing it higher.
_________________
Glenn
Back to top
View user's profile Send private message
dextermbmq
PostPosted: Mon Jul 27, 2020 3:47 am    Post subject: Reply with quote

Voyager

Joined: 26 Jul 2014
Posts: 77

Thanks gbaddeley & fjb_saper.

Yes, the consuming apps are capable enough to handle this load. The incoming load would be diverted to different queues and different application would connect to dedicated queue for consumption of the Load. From the MQ Get perspective I am pretty comfortable. As suggested, I would perform a Load Test as well.

My worry was just about the movement of the messages from Gateway Queue manager to underlying 2 partial repositories for MQ GET. Will that Queue Manager be able to transfer this load across to partial repositories without any delays over TLS enabled clustered channels. gbaddeley's response gives me some confidence though (these 5 queue managers are running across 2 servers each having 8 core - seems reasonable resources). BTW is the QTIME [QTIME(16442, 14809) UNCOM(NO)] normal or is it on the high side . Any insights based on experience ?
Back to top
View user's profile Send private message
fjb_saper
PostPosted: Mon Jul 27, 2020 4:28 am    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

If you are worried about peak throughput, maybe you should build overlapping clusters. One for high speed messages, and one for slower speed messages.
This way, if you have a big message and you send it via the slow speed cluster, it won't affect the high speed part.
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » Clustering » MQ Clustering performance
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.