| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| amqzmuc0 process in sleep |
« View previous topic :: View next topic » |
| Author |
Message
|
| shashivarungupta |
 Posted: Wed Apr 16, 2014 5:31 am Post subject: amqzmuc0 process in sleep |
 |
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
Hi,
While we ran the 'top' command on the server, it was noticed that amqzmuc0 process was in 'sleep'.
Though some of the application Java processes were consuming high cpu on the server.
As far as I understand, sleep is the normal state of the process when its waiting on the timer for a slow device may be network or terminal for I/O or may be CPU.
Please share your opinion about the sleeping mq process !!
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
 |
| tczielke |
| Posted: Wed Apr 16, 2014 6:02 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
I have found that I mostly find MQ processes in a sleeping state when you check their process state with top or ps. If you really want to understand why they are in the sleeping state you can do something like the following. Please note there would be some overhead to doing an strace (or the equivalent command for your Unix environment).
7867 is the pid for amqzmuc0
strace -p 7867
Process 7867 attached - interrupt to quit
futex(0x2b6d5acd09fc, FUTEX_WAIT, 0, {30, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(7857, SIG_0) = 0
futex(0x2b6d5acd09fc, FUTEX_WAIT, 0, {30, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(7857, SIG_0) = 0
futex(0x2b6d5acd09fc, FUTEX_WAIT, 0, {30, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(7857, SIG_0) = 0
futex(0x2b6d5acd09fc, FUTEX_WAIT, 0, {30, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(7857, SIG_0) = 0
futex(0x2b6d5acd09fc, FUTEX_WAIT, 0, {30, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(7857, SIG_0) = 0
For my use case, amqzmuc0 (MQ critical services process) was sleeping on a futex (fast userspace mutex) system call. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Fri Apr 18, 2014 1:56 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
In my case as well, it gives the following out put. But I tried for amqzlaa0 process, which was on FUTEX_WAIT. (when I ran the top command only amqzlaa0 was in sleep mode and was flashing on the screen.)
| Code: |
# strace -p 6831
Process 6831 attached - interrupt to quit
restart_syscall(<... resuming interrupted call ...>) = -1 ETIMEDOUT (Connection timed out)
kill(6773, SIG_0) = 0
kill(6843, SIG_0) = 0
futex(0x7f4b51715ecc, FUTEX_WAIT, 0, {10, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(6773, SIG_0) = 0
kill(6843, SIG_0) = 0
futex(0x7f4b51715ecc, FUTEX_WAIT, 0, {10, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(6773, SIG_0) = 0
kill(6843, SIG_0) = 0
futex(0x7f4b51715ecc, FUTEX_WAIT, 0, {10, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(6773, SIG_0) = 0
kill(6843, SIG_0) = 0
futex(0x7f4b51715ecc, FUTEX_WAIT, 0, {10, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(6773, SIG_0) = 0
kill(6843, SIG_0) = 0
futex(0x7f4b51715ecc, FUTEX_WAIT, 0, {10, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(6773, SIG_0) = 0
kill(6843, SIG_0) = 0
futex(0x7f4b51715ecc, FUTEX_WAIT, 0, {10, 0}q) = -1 ETIMEDOUT (Connection timed out)
kill(6773, SIG_0) = 0
kill(6843, SIG_0) = 0
|
To run the 'strace' it required 'root' access (for me here on my local), unless someone already have necessary privileges to do so (on the prod server). Generally mq admins don't have root level of access. Anyways.. that's different topic.
Here, I believe that if any process of mq goes to sleep mode , that means.. its a normal behaviour of mq process unless there is a scarcity of resources like cpu or memory for the rest of other hundreds of processes on the server.
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| tczielke |
| Posted: Fri Apr 18, 2014 5:22 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
| You can strace an mqm process without being root, if you are able to su to the mqm user. My experience has been that it is very common to see Unix/Linux processes in the sleep state, when you check their process state with ps or top. The operating system scheduler likes to kick you off the cpu when you are waiting for an event to complete (i.e. sleeping). |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Fri Apr 18, 2014 8:45 pm Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| Quote: |
| you can strace an mqm process without being root, if you are able to su to the mqm user |
It depends.. (I'm able to sudo to mqm but it doesn't have privileges to do so.)
| Quote: |
| it is very common to see Unix/Linux processes in the sleep state, when you check their process state with ps or top. The operating system scheduler likes to kick you off the cpu when you are waiting for an event to complete (i.e. sleeping). |
How would you determine if the sleeping process is in hung state or not ?
[/quote]
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| tczielke |
| Posted: Sat Apr 19, 2014 5:03 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
| Quote: |
It depends.. (I'm able to sudo to mqm but it doesn't have privileges to do so.)
|
Which operating system platform are you using here? Also, if you could list the commands you run to do the strace and the results, that would be helpful for me to understand exactly what you meant by your statement.
| Quote: |
How would you determine if the sleeping process is in hung state or not ?
|
The one experience I have had with MQ processes hanging was with a Solaris 10/NFS v3 environment. In that case the Solaris 10 NFS v3 file lock client was having sporadic issues where it would become unstable and hang any processes that asked for a file lock until the non-global zone was rebooted. We were able to detect why the MQ and other processes were hanging by doing a truss (the Solaris 10 equivalent of strace) which showed that the MQ process was waiting indefinitely for a file lock system call to complete. So basically, strace would be helpful with this type of endeavor. |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Sat Apr 19, 2014 5:52 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| tczielke wrote: |
| Quote: |
It depends.. (I'm able to sudo to mqm but it doesn't have privileges to do so.)
|
Which operating system platform are you using here? Also, if you could list the commands you run to do the strace and the results, that would be helpful for me to understand exactly what you meant by your statement. |
It is x86_64 GNU/Linux.
| Quote: |
$ sudo su - mqm
password:
-bash-4.1$ strace -p 6685
attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted
-bash-4.1$ exit
logout
$ sudo su -
[root]# strace -p 6685
Process 6685 attached - interrupt to quit
restart_syscall(<... resuming interrupted call ...>) = -1 ETIMEDOUT (Connection timed out)
kill(6652, SIG_0) = 0
futex(0x7f4c3947142c, FUTEX_WAIT, 0, {30, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(6652, SIG_0) = 0
futex(0x7f4c3947142c, FUTEX_WAIT, 0, {30, 0}) = -1 ETIMEDOUT (Connection timed out)
kill(6652, SIG_0) = 0
futex(0x7f4c3947142c, FUTEX_WAIT, 0, {30, 0}q^C <unfinished ...>
Process 6685 detached
[root]# |
where that process id was of one of the mq processes, i.e. amqzlaa0.
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| tczielke |
| Posted: Sat Apr 19, 2014 6:00 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
After you do the sudo su - mqm, find the pid of your new mqm shell (i.e. 12345) and do the following:
ps -p 12345 -o uid,ruid,suid,fuid
Now do the following for that 6685 pid:
ps -p 6685 -o uid,ruid,suid,fuid
Do you notice any differences in the values? |
|
| Back to top |
|
|
| shashivarungupta |
| Posted: Sat Apr 19, 2014 6:10 am Post subject: |
|
|
Grand Master
Joined: 24 Feb 2009
Posts: 1343
Location: Floating in space on a round rock.
|
| tczielke wrote: |
Do you notice any differences in the values? |
mqm 5559 5558 0 19:32 pts/1 00:00:00 -bash
mqm 6563 6536 0 Apr18 ? 00:00:03 /opt/mqm/bin/amqzmur0 -m QM1
-bash-4.1$
-bash-4.1$ ps -p 5559 -o uid,ruid,suid,fuid
UID RUID SUID FUID
494 494 494 494
-bash-4.1$ ps -p 6563 -o uid,ruid,suid,fuid
UID RUID SUID FUID
494 500 494 494
-bash-4.1$
_________________
*Life will beat you down, you need to decide to fight back or leave it. |
|
| Back to top |
|
|
| tczielke |
| Posted: Sat Apr 19, 2014 6:29 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
That is what I was expecting. It looks like the user whose uid is 500 started that queue manager under their own user id. That is considered bad practice in our shop. You can run into odd security issues when you do this. We require that the admins su to mqm and then start the queue manager.
My guess is that the user whose uid is 500 could probably strace that 6563 pid. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
