| Author |
Message
|
| whydieanut |
 Posted: Wed Aug 17, 2011 2:07 am Post subject: Audit Logging using Monitoring Events |
 |
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
Hi all,
I am trying to use the Monitoring Events feature of WMB 7 to generate Audit Logs.
Setting monitoring causes Audit Events to be emitted by the flows and these are published to a Topic. I am subscribing to these Topics and consuming the Event XMLs using an Audit Logging flow.
I am having trouble figuring out how to correlate these audit events in case of a an exception and subsequent failure.
In addition, I would like to know what are some good approaches in this kind of a scenario to model and log the Audit Events. Is anyone else using the Monitoring feature as a means to log audit info? If so how are you guys going about doing this? |
|
| Back to top |
|
 |
| smdavies99 |
| Posted: Wed Aug 17, 2011 3:06 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
If you read the docs you will see that there are three different events.
TransactionStart
TransactionEnd
TransactionRollback
You can match a TransactionStart with EITHER a TransactionEnd or a TransactionRollback.
I pair them up using a collector Node.
MqInputNode-->Compute Node-->(start)--------->CollectorNode Terminal 1
-->(end/rollback)-->CollectorNode Terminal 2
This next bit is VERY important.
Do you understand that the timestamp on the end/rollback event is NOT accurate. When the thread ends broker goes into a tidy up. In this tidy up the end/rollback event will be published. This can be delayed for some time if broker is busy.
If you want accurate end event timestamps you will have to do the following
- Add a Passthru Node after the last valid node in the flow.
- Configure it to emit a monitoring event
- Make your auditing service handle a terminal Event and somehow match up the events(start and terminal).
If that is not possible then you will have to forgo using the normal flow monitoring start/end/rollback and use Terminal Events throuought.
Oh, and be careful with the collector node. On the output side the catch terminal is very fussy. There are some other posts here about its behaviour. Hopehully IBM will have finally sorted it out in 7.0.0.3 ????
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| whydieanut |
| Posted: Thu Aug 18, 2011 3:13 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
Thanks for the reply.
Never really though about the Transaction events. Will give them a try.
Apart from that, I couldn't really find any good resources for using the audit events as a means for logging, so I am currently following my instincts on how to use them 
As of now, my audit event processing flow just picks up ALL event XMLs it receives from ALL flows emitting events, and am planning on logging them as is in the DB, extracting certain fields of interest.
I'd be grateful if you could provide any good resources for following best practices/patterns while implementing logging using audit events. |
|
| Back to top |
|
|
| smdavies99 |
| Posted: Thu Aug 18, 2011 4:37 am Post subject: |
|
|
Jedi Council
Joined: 10 Feb 2003
Posts: 6076
Location: Somewhere over the Rainbow this side of Never-never land.
|
How are you correlating the events you have stored in the Table?
Answering that might give you a hint as to why using the CollectorNode is a good way to go forward.
_________________
WMQ User since 1999
MQSI/WBI/WMB/'Thingy' User since 2002
Linux user since 1995
Every time you reinvent the wheel the more square it gets (anon). If in doubt think and investigate before you ask silly questions. |
|
| Back to top |
|
|
| whydieanut |
| Posted: Thu Aug 18, 2011 10:34 pm Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
I wanted to keep the log collecting flow as simple as possible. Don't really think there's a need for me to wait to collect ALL events before storing them. I plan on storing each event as it comes into my flow and then use the eventCorrelation Id to track events from from a single message.
One issue with this approach was when using an Error handling flow connected to my Catch terminal, which has a throw node at the end, to rollback any changes done. Now this causes the message to go to the failure terminal. When I configure an event on the Failure terminal of the input node, it doesn't retain the eventCorrelation Id.
Also I need the Failure event in case of tracking failures happening in the Input node itself (eg. parsing failures).
So now my Failure terminal events can be either due to the Error handling flow throwing an exception at the end on purpose or due to genuine failures at the input node. In the first case I'd like to be able to correlate it to the rest of the events for that message (start, processed, error etc.). And in the latter case, I am not concerned about it as this would be the first and only event emitted by that message.
Now that you've mentioned using the Transaction Events, this problem should be solved.
Anything wrong in the above approach? |
|
| Back to top |
|
|
| whydieanut |
| Posted: Fri Aug 19, 2011 2:46 am Post subject: |
|
|
Disciple
Joined: 02 Apr 2010
Posts: 186
|
One thing about using transaction events, is that I noticed that when the failure terminal is connected and a rollback happens, it is treated as a new transaction when the message enters the failure flow.
Imagine the following scenario:
Transaction events set on Input node (trans start, trans end, trans rollback)
Input node's Catch terminal wired to Error Handler subflow which processes the exception and then Throws to rollback changes.
Failure node connected to failure flow.
Now in case of an exception in the main flow, I get the following evnets:
trans start (with correlationid_1)
trans rollback (with correlationid_1)
trans start (with correlationid_2)
trans end (with correlationid_2)
This might cause some problems later I feel. Have to think through it again. I am beginning to feel that connect the failure terminal is a BAD idea now...
Any comments on this behaviour? Is this what's happening with your flows too? |
|
| Back to top |
|
|
|
|
