| Author |
Message
|
| bruce2359 |
 Posted: Fri May 29, 2020 12:01 pm Post subject: |
 |
|
Poobah
Joined: 05 Jan 2008
Posts: 9442
Location: US: west coast, almost. Otherwise, enroute.
|
This is an O/S issue. The UNIX ps command will report on processes. Contact your UNIX support.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
 |
| bruce2359 |
| Posted: Sat May 30, 2020 10:31 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9442
Location: US: west coast, almost. Otherwise, enroute.
|
Each process/thread needs other resources, like RAM/CPU. The error didn't say that the pthread_create() itself failed, but that a resource needed was unavailable - possibly Stack space, file space. Again, get your UNIX tech folks involved.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Sun May 31, 2020 3:17 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2527
Location: Melbourne, Australia
|
Did you try running the 'mqconfig' command from mqm user? Did it report PASS for all requirements? What is the output of 'ulimit -a' from mqm user?
_________________
Glenn |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Sun May 31, 2020 3:42 pm Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7717
|
We ran into this when trying to connect several thousand MQ Clients concurrently.
For us we had to increase the nofiles and nproc settings for the mqm account. Make sure after changing the settings to stop the QM, log out of mqm and back into mqm before restarting the QM.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Sun May 31, 2020 4:01 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9442
Location: US: west coast, almost. Otherwise, enroute.
|
From https://www.ibm.com/support/knowledgecenter/SSFKSJ_8.0.0/com.ibm.mq.ref.adm.doc/q117780_.htm
| Quote: |
The mqconfig command is run to verify the system configuration matches or exceeds that which is required by IBM MQ. The configuration values are minimum values, and large installations might require values greater than those checked by this command.
For further information about configuring your system for IBM MQ, see Operating System configuration and tuning for IBM MQ on UNIX and Linux systems, and IBM i . |
And ...
https://www.ibm.com/support/pages/how-configure-unix-and-linux-systems-ibm-mq
| Quote: |
| Do not assume that an "all clear" report from mqconfig means you cannot run out of resources. Run the mqconfig script when your system is processing its peak volume to ensure that the actual resource usage, shown as a percentage, is not excessively high. |
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| yanaK |
| Posted: Sun May 31, 2020 8:39 pm Post subject: |
|
|
Acolyte
Joined: 28 May 2020
Posts: 69
|
@bruce2359 is there a way to get the exact resource that was needed?
@gbaddeley it did pass (but again this issue is intermittent hence not sure if it was run at the correct time)
@PeterPotkay so it seems the q mgr restart is necessary - will try that - also how did you come up with the numbers for nofiles and nproc?
Thanks all for the help. |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Jun 01, 2020 5:10 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9442
Location: US: west coast, almost. Otherwise, enroute.
|
| yanaK wrote: |
| @bruce2359 is there a way to get the exact resource that was ... |
Yes. The Mqconfig report to STDOUT identifies the values of both maximum and current utilization of each resource. Search google for “mqconfig output” to see examples of mqconfig output.
Mqconfig executes quickly. You could set up a cron job to run it frequently.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| yanaK |
| Posted: Mon Jun 01, 2020 7:40 am Post subject: |
|
|
Acolyte
Joined: 28 May 2020
Posts: 69
|
Thanks.
A couple of other questions:
1. Why does this system behave differently than the other ones in the cluster ? (although this has v7.1 while others have v7.0 and also the OS us RHEL 7 vs RHEL 6)
2. What's the quickest way to load test MQ? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Jun 01, 2020 9:08 am Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9442
Location: US: west coast, almost. Otherwise, enroute.
|
| yanaK wrote: |
Thanks.
A couple of other questions:
1. Why does this system behave differently than the other ones in the cluster ? (although this has v7.1 while others have v7.0 and also the OS us RHEL 7 vs RHEL 6) |
Far too many variables for a simple answer here. If this is a brand new qmgr, or a brand new LPAR, or brand new hardware, or brand new o/s instance, or there are one or more configuration differences, or more/less RAM, or more/less CPU, different NIC cards, RAM vs. SSD, or ...
So, what exactly and precisely is different among the qmgrs n your cluster? Please don't be too quick to say "nothing." We've been mislead before.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| yanaK |
| Posted: Mon Jun 01, 2020 1:43 pm Post subject: |
|
|
Acolyte
Joined: 28 May 2020
Posts: 69
|
Yes this is a brand new qm - goal is to upgrade the OS for each of the members in cluster.
Hardware is RHEL 7
MemTotal: 5925680 kB
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 45
model name : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
stepping : 7
microcode : 0x714
cpu MHz : 2600.000
cache size : 20480 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
The other members in the cluster are RHEL 6 (their CPU has 2 cores and RAM is same)
Also the MQ version is 7.1.0.7 in this one vs 7.1.0.1 in the other ones.
Let me know any other info you need. Thanks |
|
| Back to top |
|
|
| gbaddeley |
| Posted: Mon Jun 01, 2020 3:12 pm Post subject: |
|
|
Jedi Knight
Joined: 25 Mar 2003
Posts: 2527
Location: Melbourne, Australia
|
1 core is a low CPU resource. Its going to spend most of its time swapping between process contexts. MQ has about 20 processes and many more threads. Would you consider increasing the number of cores to say 4 or 8?
_________________
Glenn |
|
| Back to top |
|
|
| yanaK |
| Posted: Mon Jun 01, 2020 3:47 pm Post subject: |
|
|
Acolyte
Joined: 28 May 2020
Posts: 69
|
We can - thanks - can that be the reason?
Can increasing nproc help? |
|
| Back to top |
|
|
| bruce2359 |
| Posted: Mon Jun 01, 2020 5:02 pm Post subject: |
|
|
Poobah
Joined: 05 Jan 2008
Posts: 9442
Location: US: west coast, almost. Otherwise, enroute.
|
| yanaK wrote: |
| We can - thanks - can that be the reason? |
Most definitely. If you expect similar the 4 qmgrs to meet similar SLAs, then they must be configured similarly.
| yanaK wrote: |
| Can increasing nproc help? |
Only if nprocs was an issue in the first place.
Please only change one thing at a time to see what happens. If you change two things, you will not know which one resolved the issue.
_________________
I like deadlines. I like to wave as they pass by.
ב''ה
Lex Orandi, Lex Credendi, Lex Vivendi. As we Worship, So we Believe, So we Live. |
|
| Back to top |
|
|
| yanaK |
| Posted: Mon Jun 01, 2020 5:53 pm Post subject: |
|
|
Acolyte
Joined: 28 May 2020
Posts: 69
|
Agree - so what's your suggestion?
1. up the core
2. increase nproc (and make sure to restart qm)
3. increase nofiles (and make sure to restart qm)
4. anything else? |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Tue Jun 02, 2020 4:21 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20729
Location: LI,NY
|
Up the memory. I would not go below 16 GB 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
|
|
