ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Message Transformation Speed

Post new topic  Reply to topic
 Message Transformation Speed « View previous topic :: View next topic » 
Author Message
pauillac
PostPosted: Thu Aug 09, 2001 3:09 am    Post subject: Reply with quote

Novice

Joined: 13 Jul 2001
Posts: 21

Hi,

Any idea on which transformation would be faster or they are more or less the same in terms of speed.

Transformation from one XML message

1. to another XML message
2. to a linear message format with '|' as delimiter
XITBRDSNN12300121559498010100000000|38DA5415|99999999|3001|11SHT123458

What factors would affect the relative processing speeds of both transformation?
Back to top
View user's profile Send private message
kolban
PostPosted: Thu Aug 09, 2001 4:42 am    Post subject: Reply with quote

Grand Master

Joined: 22 May 2001
Posts: 1072
Location: Fort Worth, TX, USA

The concept of MQSI performance is quite involved due to the vast array of permutations available with nodes and settings. When a message is received by an MQInput node, it is held in its binary form until first reference to an internal field within the message. At this time, the message is parsed into its message tree representation. The cost of this action varies by the data type of the message but is one of the most expensive operations. As the message flows from node to node, it is actually the tree representation that is flowing. When the tree finally reaches an MQOuput node, again the parser (or better termed - constructor) regenerates a binary representation of the data from the abstract tree. This may involve the same parser or a different parser. Either way, the act of parsing and then construction are the two most expensive operations.

To answer your question:

When the XML message arrives, you will pay the price for parsing the message into its tree representation. From then, as the message flows through nodes, the tree fields are modified in memory. Finally, when you hit the MQOutput, the tree is again flatted to binary. Your question seems to boil down to:

Which is cheaper, using the XML parser or, say, the TREEASIS parser or NEON parser. The gut answer I would say is its about a wash. I would also focus on desired function as opposed to bending your desires to meet performance goals.
Back to top
View user's profile Send private message
pauillac
PostPosted: Thu Aug 09, 2001 7:24 am    Post subject: Reply with quote

Novice

Joined: 13 Jul 2001
Posts: 21

Hi kolban,

Thank you very much for your unfailing support. Let me let you in on the actual motivation behind asking this question.

Currently, we are thinking about two design options

1. Front-end system passes XML into MQSI. This XML is based on an in-housed developed standard. MQSI would convert this XML message into an IFX-compliant XML message with additional fields added. This IFX XML message will be passed to the host where there is a XML to PAS (a proprietary message format) converter. So the mainframe in this case is XML-aware. The message flow goes the other way round when the host returns a response, i.e. PAS to XML conversion in mainframe and XML to XML conversion in MQSI.

2. Front-end system passes XML into MQSI. This XML is based on an in-housed developed standard. MQSI would translate this XML into a PAS message with additonal data added taken from DB. This PAS message will be passed directly to the host without further conversion. The message flow goes the other way round when the host returns a response, i.e. PAS to XML conversion in MQSI.

PAS format is a variable length linear message format with '|' as a delimiter:

XITBRDSNN12300121559498010100000000|38DA5415|99999999|3001|11SHT123458

(**PAS would be handled as a custom message in MQSI.)

The arguments for the first design is:

- Offloading MQSI so that we can buy cheaper machines and tap into the resources of the mainframe to handle some conversion work.
(**Here the assumption is XML to XML conversion is less resource intensive than XML to variable length linear message conversion. Please point out if this is wrong.)

The argument for the second design is:

- Why don't we go a step further and do the XML to PAS translation instead of XML to XML conversion? There is just one conversion to do instead of two. (In view of cutting hardware cost, I don't know whether the performance penalty is justifiable to choose the first design.)

The controversial point in my team is XML to XML conversion is just node to node mapping after the incoming message is converted into a tree. Whereas XML to PAS conversion is not so straight forward. The XML tree structure is mapped to a variable length linear structure instead of a comparable tree structure.


Questions
1.How would you go about PAS handling in MQSI given the linear structure? A specailized node or some other means?

2.Would hierrachical structure (XML) parsing be slower than variable length linear structure (PAS) parsing for the incoming message or the other way round?

3.Would a tree-tree structure mapping (XML to XML) be faster than tree-linear structure mapping (XML to PAS) during message conversion or the other way round?

4.Which design would you choose if you were me and why?

Comments from other guys are also welcome.
Back to top
View user's profile Send private message
kolban
PostPosted: Thu Aug 09, 2001 7:36 am    Post subject: Reply with quote

Grand Master

Joined: 22 May 2001
Posts: 1072
Location: Fort Worth, TX, USA

Thanks for the kinds words and well written follow up. Your background makes the questions very clear. Let me try and address:

1. PAS handling in MQSI could be handled by either a custom parser (TREEASIS may already work just fine) or else using the New Era of Networks in-built parser. Essentially what we would need to do here is to map the hierarchical parsed tree and its augmentations that your business logic requires into the '|' delimited fields. This sounds like walking the tree and creating such. Not too dificult.

2. XML parsing gives you named field-value pairs. Each tag implies a name and assoicated value. With the PAS, there is no self-defining field name. This may be a consideration in the design, not from a performance perspective but more of a logistical one. There would be more work in mapping field indexes to XML tag names.

3. No faster or slower. The construction of the final output message is independent of the cost of the input parse message. After input, the message is in a "tree". The cost of serializing the tree should be about the same and is not a function of how the tree was constructed initially.

4. Because I have a lot of experience with both custom parsers and New Era of Networking technologies, both are a wash. I would personally go with a custom parser because I have more control and can grind them out quickly. Once written, it then simply becomes a "component" in the MQSI tool-box for continued re-use.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » WebSphere Message Broker (ACE) Support » Message Transformation Speed
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.