| Author |
Message
|
| bprasana |
 Posted: Fri Aug 02, 2013 7:23 am Post subject: Roll back strategy |
 |
|
Disciple
Joined: 18 Apr 2005
Posts: 179
|
Hi,
I am using WMB 8.0.0.1.
We receive a request as SOAP over HTTP.
at ESB we have to split the message into multiple parts and send them one by one to MQ.
Requirement is ROLLBACK ONLY failed messages and not the entire batch. We are supposed to continue processing till the last record.
Exception needs to be raised only for failed messages.
If I use a loop to send multiple propagate statements on any failure during the mapping or during the 'PUT to Q', the entire batch gets rolled back.
We thought of having a try catch node before this 'while loop' compute nde and then try to record the exception list in Catch path. And connect the catch path back to same while loop so that while loop processes from where it has left. We are setting suitable Environment variables to ensure while loop processes only from where it has left.
in the catch path we also include a flow order node to first send the message back to Main path and then Throw exception after all records are processed.
Conceptually this should work. But Looks like a complex solution for a simple problem. So Is there a better strategy? What approach did you take? |
|
| Back to top |
|
 |
| McueMart |
| Posted: Fri Aug 02, 2013 7:27 am Post subject: |
|
|
Chevalier
Joined: 29 Nov 2011
Posts: 490
Location: UK...somewhere
|
After splitting the message, put each part as an individual message onto a queue (without doing any mapping/validation etc - this part should never 'fail'). Have a second flow which reads this queue and processes each message individually (does mapping/lookups etc).
The only complication is if the sending system (which sent the original HTTP request) needs a response message informing it of how many passed/failed.
If this is the case then you could look into aggregation. |
|
| Back to top |
|
|
| dogorsy |
| Posted: Fri Aug 02, 2013 8:23 am Post subject: |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
code a continue handler in the compute node to catch the failures and continue processing.
so, as you said
use a loop to send multiple propagate statements
in the event of a failure, the handler will intercept it, do whatever you want, and then continue processing. |
|
| Back to top |
|
|
| bprasana |
| Posted: Fri Aug 02, 2013 6:21 pm Post subject: Interim Qs |
|
|
Disciple
Joined: 18 Apr 2005
Posts: 179
|
Putting them in a Q looks like a more elegant and classic approach.
If I were to use Global Cache as Lancelotinc suggested, I need to keep the logic I had written as is just that the transformed messages are stored in cache which won't roll back.
How scalable is this solution ? As of now I get just 5 requests per batch.
If I were to use continue handler, will that handle MQ put errors? I need to read on this. |
|
| Back to top |
|
|
| kash3338 |
| Posted: Fri Aug 02, 2013 7:35 pm Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
When you say SOAP over HTTP, do you use SOAP nodes or HTTP?
What would be the response to the client?
When you say ROLLBACK, what exactly do you want to do? Send back a error response to the client with the messages that failed or just the info about the messages failed?
When you say validation/transformation errors, what sort of validation do you do? DO you use a Message Set to validate? What format do you transform to?
When you say error related to 'PUT Q', will it not affect the entire batch as all the messages are going to same queue?
Given all these we can come up with different approaches to handle them.
One way you can do is to have a try/catch and in case of any error drop the message onto a queue and continue with your propagate loop. Finally when you are done with all the records, construct a response message by looping through the error queue and send back the response. |
|
| Back to top |
|
|
| bprasana |
| Posted: Fri Aug 02, 2013 8:57 pm Post subject: details |
|
|
Disciple
Joined: 18 Apr 2005
Posts: 179
|
We are using one way SOAP communication. So client is not bothered about the failures. Yes we are using SOAP Input node.
Rollback in this case means 2 things
- Do not send that message to downstream
- alert recovery team. by throwing an exception to Common error handler connected to soap input catch.
We do have a message set for both request (wsdl- content and value validation set to on demand) and response(cobol copybook).
Yes you are right about 'PUT to Q' affecting the entire batch though in most cases.
Using an error Q sounds like a good idea. But will the propagate loop continue if an exception occurs? |
|
| Back to top |
|
|
| kash3338 |
| Posted: Fri Aug 02, 2013 9:08 pm Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
The purpose of Try/catch is to handle the error and continue processing. It will work the way you want in this scenario. Your flow cane be something like below,
Main FLow:
| Code: |
SOAPInput --> Compute(To Propagate loop) --> Try/Catch
Try --> Compute(To transform) --> MQOut
Catch --> Compute (if required to format error message) --> MQOut(Error Q)
|
Error Flow:
| Code: |
MQInput(Error Q) --> Handle the error in the way you want.
|
|
|
| Back to top |
|
|
| dogorsy |
| Posted: Fri Aug 02, 2013 9:42 pm Post subject: |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| kash3338 wrote: |
The purpose of Try/catch is to handle the error and continue processing. It will work the way you want in this scenario. Your flow cane be something like below,
Main FLow:
| Code: |
SOAPInput --> Compute(To Propagate loop) --> Try/Catch
Try --> Compute(To transform) --> MQOut
Catch --> Compute (if required to format error message) --> MQOut(Error Q)
|
Error Flow:
| Code: |
MQInput(Error Q) --> Handle the error in the way you want.
|
|
yes, you can use a try/catch, but a continue handler is far easier, you catch the exception in the compute node, deal with it and continue processing where you left. No need for environment variables and logic to return to the loop.
in the handler you can do anything you want, create an error message, analyze the exception list, write to a queue, etc. and the continue with the loop, so I do not understand the reluctance of using it, but then again, people are always afraid of what they don't know. |
|
| Back to top |
|
|
| kash3338 |
| Posted: Fri Aug 02, 2013 9:47 pm Post subject: |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| dogorsy wrote: |
| yes, you can use a try/catch, but a continue handler is far easier, you catch the exception in the compute node, deal with it and continue processing where you left. No need for environment variables and logic to return to the loop. |
I guess you did not get the point. There is no need for a Environment variable in this case and there is no need to write any logic to return back to my loop. It works the way we want in this scenario.
In what way do you say Continue Handler is easier in this case? Using a Try/Catch you catch the exception and do the required exception processing and put in a MQ Queue. Later the next record is automatically processed.
The purpose of Try/Catch is this kind of a scenario. |
|
| Back to top |
|
|
| dogorsy |
| Posted: Fri Aug 02, 2013 9:50 pm Post subject: Re: Interim Qs |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| bprasana wrote: |
| Putting them in a Q looks like a more elegant and classic approach. |
you can propagate from within the HANDLER
| bprasana wrote: |
If I were to use continue handler, will that handle MQ put errors? I need to read on this. |
the handler catches ALL exceptions, it is up to the user to handle the errors ( and the same goes for try/catch, it only catches the exceptions, it is the user's responsibility to handle them ) |
|
| Back to top |
|
|
| dogorsy |
| Posted: Fri Aug 02, 2013 9:57 pm Post subject: |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| kash3338 wrote: |
In what way do you say Continue Handler is easier in this case? Using a Try/Catch you catch the exception and do the required exception processing and put in a MQ Queue. Later the next record is automatically processed.
The purpose of Try/Catch is this kind of a scenario. |
with the handler you do not need to leave the compute node, which means you have access to all the variables, etc. You also catch the exception and can put to a queue. |
|
| Back to top |
|
|
| kash3338 |
| Posted: Fri Aug 02, 2013 9:59 pm Post subject: Re: Interim Qs |
|
|
Shaman
Joined: 08 Feb 2009
Posts: 709
Location: Chennai, India
|
| dogorsy wrote: |
| you can propagate from within the HANDLER |
PROPAGATE again from a Handler? Already his design has a PROPAGATE in the first leg.
Probably you can try a small flow using Try/Catch and the Handler and find yourself which suits better. I guess you have a different understanding of the way Try/Catch works. |
|
| Back to top |
|
|
| dogorsy |
| Posted: Fri Aug 02, 2013 10:15 pm Post subject: Re: Interim Qs |
|
|
Knight
Joined: 13 Mar 2013
Posts: 553
Location: Home Office
|
| kash3338 wrote: |
| dogorsy wrote: |
| you can propagate from within the HANDLER |
PROPAGATE again from a Handler? Already his design has a PROPAGATE in the first leg. |
so ? what is the problem ?
Do you have an undocumented rule for using PROPAGATES ?!!!
| Quote: |
Probably you can try a small flow using Try/Catch and the Handler and find yourself which suits better. I guess you have a different understanding of the way Try/Catch works. |
I don't need to. I know very well how the try/catch works, but obviously not as well as you, from what you said, you seem to have been part of the design team. However, you do not seem to know much about handlers.
the proposed try/catch is an aberration.
compute --> catch --> ( process the error ) ----|
^ |
|--------------------------------------------------
in the event of an error, the msg goes to the catch terminal, you process it and then back to the compute. If you have enough errrors, that will lead to a stack overflow. |
|
| Back to top |
|
|
|
|
