| Author |
Message
|
| Vinayak.Satapute |
 Posted: Mon Jan 10, 2011 2:34 am Post subject: Approach to store the data as xml file&read It using Jav |
 |
|
Acolyte
Joined: 20 Dec 2006
Posts: 70
|
Hi All,
I have scenario where I have mapping table as below.
Table1
--------
CrewType Value
F CR1
C CR2
the above table stored as xml file as below.
<?xml version='1.0'?>
<xml>
<CrewTypes>
<Crew CrewType='F'>CR1</Crew>
<Crew CrewType='C'>CR2</Crew>
</CrewTypes>
</xml>
Now this file can be updated as and when there is a new record. Now this data need to be mapped in the outgoing xml message ex: F-->CR1, but at the same time,can not hard code this mapping under esql considering future additional record updates and esql code change.
So Design approach is to store this table as xml file as shown above and read it using Java compute node and then map the data.
(Other design approach was to keep the table in DB and retrieve DB: considering the the Performance and connectivity to DB this approch was dropped)
Is there anything you can shed on the approach of saving this file on broker local system and reading it using JavaCompute Node. Or is there any other better way you can suggest.
Thanks & Regards,
Vinayak Satapute.[/img] |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Mon Jan 10, 2011 3:04 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
Store the table in a ROW shared variable. If the value is 'default', initiate a Read of the DB. Then periodically refresh it using a TimerNode.
Pretty standard practice these days.
Don't forget to put all the shared variable code inside a BEGIN ATOMIC-> END structure.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| mqjeff |
| Posted: Mon Jan 10, 2011 3:13 am Post subject: |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 17447
|
| Broker v7.0.0.2 has introduced a FileRead node... |
|
| Back to top |
|
|
| Vinayak.Satapute |
| Posted: Tue Jan 11, 2011 12:08 am Post subject: |
|
|
Acolyte
Joined: 20 Dec 2006
Posts: 70
|
Hi,
Thanks for the update. Currently I am running with broker version 6.1.0.2 and yet to consider the broker update to 7.0.0
The table I provided is part of main table. The main table is having many different type of records. And I dont see it will be optimal to store this data as table and read the data from DB every time there is request comes in, which will definitely add in to delay in processing the message.
So I was mentioning to keep this data as XML file in broker file system and read it using JavaCompute Node for further processing in the flow.
Appreciate your input on this method of implementation.
the actual table data presented as XML file as shown below.
********************************************************
<?xml version='1.0'?>
<xml>
<WorkCodes>
<Work WorkCode='OP'>C</Work>
<Work WorkCode='SNY'>C</Work>
<Work WorkCode='OBS'>C</Work>
<Work WorkCode='PS'>X</Work>
</WorkCodes>
<CrewTypes>
<Crew CrewType='F'>CR1</Crew>
<Crew CrewType='C'>CR2</Crew>
</CrewTypes>
<Ranks>
<Rank Id = 'CAPT'>01</RANK>
<Rank Id = 'FO'>02</RANK>
<Rank Id = 'SO'>03</RANK>
<Rank Id = 'CDT'>04</RANK>
<Rank Id = 'IFS'>01</RANK>
<Rank Id = 'CS'>02</RANK>
<Rank Id = 'LS'>03</RANK>
<Rank Id = 'FS'>04</RANK>
</Ranks>
</xml>
*****************************************************
Thanks. |
|
| Back to top |
|
|
| fatherjack |
| Posted: Tue Jan 11, 2011 12:49 am Post subject: |
|
|
Knight
Joined: 14 Apr 2010
Posts: 522
Location: Craggy Island
|
| Vinayak.Satapute wrote: |
| I dont see it will be optimal to store this data as table and read the data from DB every time there is request comes in, which will definitely add in to delay in processing the message. |
You don't need to. Do as smdavies99 suggests and stored it as a row shared variable.
_________________
Never let the facts get in the way of a good theory. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Tue Jan 11, 2011 2:04 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
What is the SLA for this flow to work?
Howm many gazillion messages are you going to process a day?
These questions will influence how you proceed.
If there is no SLA (must respond within 1msec etc) and you are processing hundreds of messages ad day the a DB lookup (provided dozens of other things are reading/updating the table at the same time) will not adversely affect the overall system response.
Also, it is probably worth developing the flow to use a DB lookup every time and then once it is working modify it to give you optimal performance as per the agreed SLA.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Jan 11, 2011 8:46 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Vinayak.Satapute wrote: |
| I dont see it will be optimal to store this data as table and read the data from DB every time there is request comes in, which will definitely add in to delay in processing the message. |
It's unlikely to be a problem unless you have very tight SLA. Most modern databases will quickly realise you keep selecting the same thing over & over and will cache it in memory for you (some will allow you to flag it to be cached). Once this happens, disc I/O will be eliminated and the DB select will become faster than the file read.
But even this is eliminated by the suggestion of smdavies99. Read it from the DB once, store it in the shared variable and it's in memory for the life of the flow.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Vinayak.Satapute |
| Posted: Thu Jan 13, 2011 2:34 am Post subject: |
|
|
Acolyte
Joined: 20 Dec 2006
Posts: 70
|
Hi,
Thanks for the updates given. Yes, don't have any such strict SLA with respect to system performance and also with the message count.
Will be considering the suggestion on storing the Table in shared ROW variable. If anything further I will update you guys.
Thank you All.
Cheers. |
|
| Back to top |
|
|
| Seb |
| Posted: Thu Jan 13, 2011 2:38 am Post subject: |
|
|
Apprentice
Joined: 27 Mar 2009
Posts: 41
|
|
| Back to top |
|
|
|
|
