| |
|
RSS Feed - WebSphere MQ Support
|
RSS Feed - Message Broker Support
|
  |
|
| HTTP Reply Identifier (Memory or Queue) |
« View previous topic :: View next topic » |
| Author |
Message
|
| matuwe |
 Posted: Sat Jun 25, 2011 10:48 am Post subject: HTTP Reply Identifier (Memory or Queue) |
 |
|
Master
Joined: 05 Dec 2007
Posts: 296
|
Hi,
I seem to have a high memory usage on my broker. We where doing perfomance testing, and tested 50 000 Http Request hitting the broker. I know Broker will manage the Http reply state for me, but my question is The state information, is it stored in a queue some where, or is it stored in memory..
I suspect memory as you have to use the same execution group for requests and reply flow. Could these state information be using a lot of memory?
Thanks   |
|
| Back to top |
|
 |
| mqjeff |
| Posted: Sat Jun 25, 2011 11:49 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| You really think it's the storage of 50,000 HTTP Reply identifiers, which is about two bytes long, is really going to use that much more memory than about 2 bytes * 50,000? |
|
| Back to top |
|
|
| matuwe |
| Posted: Sat Jun 25, 2011 1:17 pm Post subject: |
|
|
Master
Joined: 05 Dec 2007
Posts: 296
|
Hi , Not really the reply identifiers.. I guess I am trying to figure out if broker only stores the the reply identifier or it also stores the the original request..
I know that my memory problems can also be related to big files that I am reading into the broker. I am also trying to work out if I read a 80MB file in batches, so if that whole files is in the broker , does it mean broker will have it stored in Memory while processing it, Is it possible to read the whole thing at once, using segmentation and process it once and for all and not in batches?
If Broker stores the the message in memory while processing, does it mean as soon as the message leaves the broker , the memory usage should drop? I guess My biggest problem may be processing the big files...  I do some ESQL replace values and XSLT, then write it out to files. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sat Jun 25, 2011 1:23 pm Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
The memory used to represent an 800 mb file depends on many factors, including the method you tell the FileInput to read it in - whole file, delimited chunks, parsed records, etc, etc.
It also depends which parser you use to access the data and whether you use Immediate or Complete parse timing. |
|
| Back to top |
|
|
| matuwe |
| Posted: Sat Jun 25, 2011 1:38 pm Post subject: |
|
|
Master
Joined: 05 Dec 2007
Posts: 296
|
 Oww I am using delimiter so it will read it in small chunks.
But I don't understand
| Quote: |
It also depends which parser you use to access the data and whether you use "Immediate or Complete parse timing".
|
I am parsing into XMLNSC domain. Currently my message is keeping all messages optional.
SO my question is, will it become disaster if i use the whole file option?
Oww another thing to mention , is my message will always hop through 10 message flows.
Thanks for all the assistance  |
|
| Back to top |
|
|
| matuwe |
| Posted: Sat Jun 25, 2011 1:53 pm Post subject: |
|
|
Master
Joined: 05 Dec 2007
Posts: 296
|
Are the any rule/ don't do's when developing flows.
Like the SHARED ROW, I am sure that information stays in memory, I am storing most of my routing information using shared row. Cold that have an impact? |
|
| Back to top |
|
|
| mqjeff |
| Posted: Sun Jun 26, 2011 4:03 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
I'm a bit confused on how you are parsing an xml document in chunks.
Message Broker has always done parsing on demand, which is to say it will leave the contents of the message as a raw stream of bytes and not expend extra memory and cpu creating a logical understanding of the message content unless and until you try and access that logical understanding of the message content.
Or you tell the input node that you want to parse Immediately or Completely. You know, the node property called 'parse timing'.
Yes, your message flows will use memory. Yes, the way you code your message flows will impact the amount of memory they use. Yes, using a shared row will use memory and that memory will persist when your message flow is not running.
No, I am not capable of giving you extensive performance tuning advice or assistance through this venue. A fair bit of it is just a matter of thinking clearly about what's going on. Then the rest of it is gathering and analyzing data. And then making changes and measuring the impact of the changes.
You might consider reading the performance reports. They are there to help you with this.
You might also consider hiring a consultant to walk through a performance exercise with you. |
|
| Back to top |
|
|
|
|
|
| |
|
Page 1 of 1 |
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
|
|
