| Author |
Message
|
| PeterPotkay |
 Posted: Wed Feb 12, 2014 9:56 am Post subject: |
 |
|
Poobah
Joined: 15 May 2001
Posts: 7723
|
There is no dependency on the cluster refreshing for this change.
There is no such thing as a cluster refresh anyway. No, issuing the CLUSTER REFRESH comand does not refresh the cluster.
If you make a change to a clustered object that is relevent to the cluster, the change is immedietly sent to the Full Repositories, which then immedietly notify all interested Partial Repositories.
sebastia, I know you don't want to hear this, but I think you're heading down a path where things will work fine most of the time. But your setting yourself up for problems.
What happens if a big burst of messages come in and all the clustered instances of the queue back up as designed? Are you going to drop the priority of all of them?
If one gets just a tiny bit deeper than the other, you drop its priority, so more messages go to the other one, so those start rising faster, so you drop those meanwhile the first one emptied so you raise it so its priority and now its depth rises while the others drop etc etc etc.
If you implement this idea across a lot of queues in a big cluster, the cluster is going to be busy fiddling with your settings constantly. To the point where your overall performance degrades.
If you do this for one set of queues only, and you set the threshhold relatively high for the q depth before you take action, and the liklihood that the queue depth going above 1 during normally processing is almost zero, then maybe I would be OK with it. But now you have established a pattern that works, and if you start copying it everywhere it might get applied in the wrong scenarios.
If you were to do this, don't use WMB! The queue is presumably backing up because the message flow can't consume the messages for some reason - why do you assume a flow will still be able to check the q depth in this situation? Use a proper monitoring tool.
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Feb 12, 2014 2:58 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20768
Location: LI,NY
|
 with Peter. This is not a scenario for the broker, but a scenario for a monitoring tool. It can regulate better.
Now if your aim is to favor online vs batch, you should have an overlapping cluster each cluster having a different QOS, and sending its messages across with a different priority.
Have fun 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| sebastia |
| Posted: Wed Feb 12, 2014 11:54 pm Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Thanks a lot, coleagues, for sharing your opinions.
Let me say my opinion too.
*) monitoring tool - I dont think one overall monitoring / "flow control" application is possible. And a "centralized" solution is not appropiate in my opinion. I think each consumer (MB flow) has to indicate its own sources its desire to receive less traffic. So, the flow control is distributed. I like that (a lot).
*) if customer's architecture has no problems, I am happy, and queue priority stays stable. If a flow has design problems, some other people will have to fix it.
*) in order to the cluster parameters not to change too often, the "queue depth monitoring flow" shall get control ... once every 10 seconds only.
*) if all queues get overloaded, all priorities shall go low and cluster algorithm shall choose the one most apropiate in this environment.
*) I am not setting myself up for problems, jajaja. I work as solutions provider, and my customer has asked me what shall we do in such and such situation. And cluster "static" configuration is maybe not enough for a techie customer, so ... I have to provide a dynamic solution.
*) putting more cluster receiver channels is also a "static" configuration, from my point of view.
*) if refresh period is in the "minutes" range, then I shall forget about modifying CLWLPRTY. But we know that if "PUT(DISABLED)" is set for a queue, this setting propagates almost instantly, so I shall run some testing.
Again, thanks for sharing your thoughts. Sebastian. |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Thu Feb 13, 2014 3:08 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| sebastia wrote: |
| I like that |
A complex system of independently acting agents - it's certainly a grateful subject for studies in Chaos Theory  |
|
| Back to top |
|
|
| sebastia |
| Posted: Thu Feb 13, 2014 3:39 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Hey !
What I am sure of ... is that it is not possible to have a single central site having all the info and calculating some kind of flow control parameters and then distributing them across the cluster.
Small independent bodies managing their own resources looks good to me (I am in Catalonia, of course, jejeje) |
|
| Back to top |
|
|
| PeterPotkay |
| Posted: Thu Feb 13, 2014 5:01 am Post subject: |
|
|
Poobah
Joined: 15 May 2001
Posts: 7723
|
| sebastia wrote: |
Hey !
What I am sure of ... is that it is not possible to have a single central site having all the info and calculating some kind of flow control parameters and then distributing them across the cluster.
|
Robust enterprise level MQ monitoring solutions can do this easily, from a central location. And can have different criteria for different queues applied easily to one template that's applied consistently.
I'm glad you are considering how often you will check and that it won't constantly. That proposed 10 second interval shows you have considered this.
As long as you are aware of the risk that using WMB to monitor a queue serviced by WMB exposes you to the potential of not being being able to have WMB tell you WMB has an issue 
_________________
Peter Potkay
Keep Calm and MQ On |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 13, 2014 6:30 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sebastia wrote: |
| it is not possible to have a single central site having all the info and calculating some kind of flow control parameters and then distributing them across the cluster. |
Yes it is. Look at most of not all of the commerical monitoring solutions.
| sebastia wrote: |
| Small independent bodies managing their own resources looks good to me (I am in Catalonia, of course, jejeje) |
And thus I accept the view on independance. But this is MQ not the EU and centralized, organized control with a holistic view of the estate actually works well & should not be compaired with any organizations based in Madrid or Brussels.
Again I repeat - your site, your (independant) choice. You asked for opinions, you've got them, implement your chosen solution having considered these opinions and go forth in peace and happiness with it.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sebastia |
| Posted: Thu Feb 13, 2014 7:48 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
*) yes, I am aware of the chance that a overloaded flow will not be able to monitor the queue, so the flow control wont be acting.
Thanks for the clue.
*) commercial monitoring solutions ... have their own weaknesses, never forget that
*) central and organized control ... is not always possible in a complex multi-app envir
Thanks for your allowance for my independence - I shall recommend my client to go by this path - descentralized and local flow control
Great discussion, thanks. Now all I need id the CLWLPRTY change to propagate as fast as PUT(DISABLED), jejeje.
I am asking Hursley on that item.
Will keep us updated. Tx a lot. Sebastian. |
|
| Back to top |
|
|
| Vitor |
| Posted: Thu Feb 13, 2014 8:09 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| sebastia wrote: |
| *) commercial monitoring solutions ... have their own weaknesses, never forget that |
And vendors that fix them. Tivoli is quite robust.
| sebastia wrote: |
| *) central and organized control ... is not always possible in a complex multi-app envir |
That's the use case for which most of these tools are sold! 
| sebastia wrote: |
| Thanks for your allowance for my independence |
Independance in your right and not my gift  
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| sebastia |
| Posted: Fri Feb 14, 2014 12:36 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
Hi, Vitor. Let me ask you a question.
If you were the owner of a company, medium size lets say, using MQ ... would you buy Tivoli to monitor it ?
I worked on flow control on a previous job (own network, own protocols, own op sys, own hw) with excelent experts. And I learned not to try to centralize the logic of the overall computing.
Thanks for your democratic attitude. ( ) |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Fri Feb 14, 2014 12:39 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| Vitor wrote: |
| Independance in your right and not my gift |
Nonsense. Rights are gifts by those with the power or disinterest to grant them. There's no such thing as an 'inherent' or 'inalienable right'. There's always a sysadmin with his finger hovering over the enter key somewhere. |
|
| Back to top |
|
|
| sebastia |
| Posted: Fri Feb 14, 2014 12:44 am Post subject: |
|
|
Grand Master
Joined: 07 Oct 2004
Posts: 1003
|
not in my network
not in my project
() |
|
| Back to top |
|
|
| zpat |
| Posted: Fri Feb 14, 2014 1:05 am Post subject: |
|
|
Jedi Council
Joined: 19 May 2001
Posts: 5867
Location: UK
|
| Vitor wrote: |
| Independance in your right and not my gift |
Anyone living in the USA who can't spell independence properly should be put on a special rendition flight to camp X-ray!
_________________
Well, I don't think there is any question about it. It can only be attributable to human error. This sort of thing has cropped up before, and it has always been due to human error. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Feb 14, 2014 5:23 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| zpat wrote: |
| Vitor wrote: |
| Independance in your right and not my gift |
Anyone living in the USA who can't spell independence properly should be put on a special rendition flight to camp X-ray! |
Don't trample my First Amendment Rights. It's not my fault you didn't understnad my subtile protest against the American language.  
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vitor |
| Posted: Fri Feb 14, 2014 5:28 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Gralgrathor wrote: |
| There's always a sysadmin with his finger hovering over the enter key somewhere. |
Only if enjoys spending many hours in a root cause analysis meeting with me and 3 levels of his immediate management & the SVP of every business area affected
Or doesn't mind if his finger gets broken in a freak accident.
Or don't mind if I explain change control processes and areas of responsibility over, and over, and over again.
Usually their fingers break.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
