| Author |
Message
|
| djeripo |
 Posted: Fri Sep 23, 2005 1:02 pm Post subject: JDBC Adapter performance |
 |
|
Master
Joined: 25 Jan 2004
Posts: 225
|
Hello All,
We would like to enhance the performance of a JDBC 2.6.2 adapter running on AIX with Message Broker 5 (CSD 05).
This particular application is migrated from Webmethods to Messagebroker.Our requirement is that ,JDBC adapter should process somewhere around 700 events in 4 minutes.
Initially we had java min heap set to 64m and max heap set to 512m.
I changed min heap to 256 m and things seemed to change a quite a bit.But still its taking 14 minutes.For the same task ,Webmethods used to take 4 minutes.
What I am looking for here is,
Other than changing the java heap (memory) ,Is there any property or some other configuration in the JDBC adapter which allows us to improve the performance.
I tried increasing the value of jms.NumConcurrentRequests to '20'.It did not make any difference. |
|
| Back to top |
|
 |
| RocknRambo |
| Posted: Fri Sep 23, 2005 7:21 pm Post subject: |
|
|
Partisan
Joined: 24 Sep 2003
Posts: 355
|
Did you try increasing the PollQuantity and reducing the PollFrequency
-RR |
|
| Back to top |
|
|
| djeripo |
| Posted: Fri Sep 23, 2005 8:21 pm Post subject: |
|
|
Master
Joined: 25 Jan 2004
Posts: 225
|
| Yes.Tried it.It did not help. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Sat Sep 24, 2005 4:39 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20697
Location: LI,NY
|
| Did you try increasing the number of threads allowed to the DB ? |
|
| Back to top |
|
|
| hopsala |
| Posted: Sat Sep 24, 2005 8:37 am Post subject: |
|
|
Guardian
Joined: 24 Sep 2004
Posts: 960
|
are you sure the bottleneck is the JDBC adapter and not the flow? (if you're using IDL this is relevant)
What about normal db solutions - a better indexing for the table based on the fields in the SELECT? (if I remember correctly connector_id, object_name and event_status; Raise trace leves to 5 and you'll have a debug print of the sql statement)
Lastly, you could create a rather odd workaround - have two JDBC adapters running on the same event table, each with a different ConnectorID, and code your trigger to INSERT once this row, once the other (easiest way is to make it random). I've never tried it, but it might just work. |
|
| Back to top |
|
|
| djeripo |
| Posted: Mon Sep 26, 2005 7:11 am Post subject: |
|
|
Master
Joined: 25 Jan 2004
Posts: 225
|
fjb_saper wrote :
| Quote: |
| Did you try increasing the number of threads allowed to the DB ? |
I suppose the equivalent property in JDBC adapter for WBIMB is jms.NumConcurrentRequests.I increased that to 20.Still no luck.
hopsala wrote :
| Quote: |
| are you sure the bottleneck is the JDBC adapter and not the flow? (if you're using IDL this is relevant) |
Yes it is the JDBC adapter.The reason being,earlier this application was on WebMethods which used to ready from the same DB and same time.During that time for the same work load it used to take 4 mts copmared to that of 14 mts now.
| Quote: |
| Lastly, you could create a rather odd workaround - have two JDBC adapters running on the same event table, each with a different ConnectorID, and code your trigger to INSERT once this row, once the other (easiest way is to make it random). I've never tried it, but it might just work. |
We really are agianst the idea of running multiple instances of the same adapter as a way to incorporate best practices.In any case if there is performance hiccup ,we can always increase the memory on the adapter rather than going for multiple adapters for the best reasons known to you all |
|
| Back to top |
|
|
| Ratan |
| Posted: Mon Sep 26, 2005 8:58 am Post subject: |
|
|
Grand Master
Joined: 18 Jul 2002
Posts: 1245
|
Did you increase the "Maximum Database Connections" property. I might be wrong but if your number of concurrent requests is 20, then may be having MaximumDatabaseConnections also to 20 might help.
_________________
-Ratan |
|
| Back to top |
|
|
| bwilliam45 |
| Posted: Mon Sep 26, 2005 9:15 am Post subject: Try setting "Parallel Process Degree" to "2&q |
|
|
Newbie
Joined: 26 Sep 2005
Posts: 1
|
Try setting - in the "Resources" tab of Connector Configurator - the Parallel Process Degree to 2. It is 1 by default.
See: Implementing concurrent processing of requests by connector agents
Also might want to consider increasing the "Maximum number of concurrent events" on the collaboratin object into which the JDBC Connector's adapter agent is sending the messages / data. |
|
| Back to top |
|
|
| RocknRambo |
| Posted: Mon Sep 26, 2005 7:10 pm Post subject: |
|
|
Partisan
Joined: 24 Sep 2003
Posts: 355
|
I believe... djeripo is working with WBIMB as the integration broker.
and I dont think so the Connector Agent Parallelism (CAP) is available for MB!!!!
-RR |
|
| Back to top |
|
|
| djeripo |
| Posted: Wed Sep 28, 2005 5:15 am Post subject: |
|
|
Master
Joined: 25 Jan 2004
Posts: 225
|
As RR said, CAP(Connector Agent Parellelism ) is not availble for connectiors which work with Intergration Broker as WBIMB.
As suggested by members,I increased ,I incresed the 'Maximum Database Connections' from 5(default) to 20.That increased the performance of the adapter considerably.Now it could process 400 events in 7.5 mts.Thats an acheivement when compared to 14 mts.
Now I can goahead and open a PMR with IBM.I would try to avoid opening PMR's with IBM as you all know thats a deep pain.Since ,now I have a solution,Let IBM take its time. |
|
| Back to top |
|
|
| samkd |
| Posted: Fri Sep 30, 2005 6:41 am Post subject: Adapter agent is not multithreaded |
|
|
Newbie
Joined: 17 Sep 2005
Posts: 1
|
| JDBC adapter agent is not multi threaded. So, the only way to increase its processing speed is to have multiple instances of the agent. You can probably multithread the Controller with the jms parameter you mentioned. I would check the queues, to figure out the bottleneck and change the thread count accordingly. |
|
| Back to top |
|
|
| djeripo |
| Posted: Fri Sep 30, 2005 10:15 am Post subject: |
|
|
Master
Joined: 25 Jan 2004
Posts: 225
|
| Quote: |
| You can probably multithread the Controller with the jms parameter you mentioned |
Since the Integration broker we are using is 'WBIMB',I dont think there is any concept of Controller.I tried to increase the numofconurrent requests,which has no impact on the performance.I also increased the memory on the adapter which also has no impact.The only property which does really impacted the performance was MAX_DATABASE_CONNECTIONS.By default it was '5'.I increased it to '20'.Which increased the performce by 100% .(From 14 mts ,it came down to 7 mts).
I opened a PMR with IBM and am awaiting for their response(It could even be a month).
For the connector cfg properties for Adapter which has Integration broker has WBIMB,there is very limited scope where as for Integration Broker as ICS,there is concept of CAP(Connector Agent Parallelism). |
|
| Back to top |
|
|
|
|
