| Author | 
		  Message
		 | 
		
		  | isolomatin | 
		  
		    
			  
				 Posted: Wed Jun 13, 2007 8:24 am    Post subject: Database insertion problem | 
				     | 
			   
			 
		   | 
		
		
		   Newbie
 
 Joined: 13 Jun 2007 Posts: 5
  
  | 
		  
		    
			  
				Hi all,
 
 
one of the requirements for software we are building is to return unprocessed messages to a database. This works by placing a message with a uuid onto a queue, reading it using a flow, getting the message from a storage db by using uuid then inserting it into another db.
 
 
The storage db is DB2 8.1.0.14 on AIX. Its codepage is 1208. The db that the message has to be inserted into is DB2/400. Its codepage is 1025 (data is kept as 1025 but the label on the db says 37, business requirement, can't change this).
 
 
Now the problem. There is a field called MSGBDY on DB2/400. It's a varchar with maximum size 16386 bytes. The error I am getting when inserting into the database is:
 
 
 Text = [IBM][CLI Driver] CLI0109E  String data right truncation. SQLSTATE=22001
 
 
 Here is the code:
 
 
DECLARE msgID CHAR;
 
DECLARE b BLOB;
 
DECLARE hdrLen INT;
 
DECLARE msgHdr CHAR;
 
DECLARE bdyLen INT;
 
DECLARE msgBdy char;
 
						
 
SET b = X'000000000000' || SUBSTRING(InputRoot.BLOB.BLOB FROM 17 FOR 2);
 
SET hdrLen = CAST(b AS INTEGER);
 
SET msgHdr = CAST(SUBSTRING(InputRoot.BLOB.BLOB FROM 19 FOR hdrLen) AS CHAR CCSID 1025);		
 
 
SET b = X'000000000000' || SUBSTRING(InputRoot.BLOB.BLOB FROM 4115 FOR 2);
 
SET bdyLen = CAST(b AS INTEGER);
 
SET msgBdy = CAST(SUBSTRING(InputRoot.BLOB.BLOB FROM 4117 FOR bdyLen) AS CHAR CCSID 1025);	
 
 
SET msgID = cast(SUBSTRING(InputRoot.BLOB.BLOB FROM 1 FOR 16) as char ccsid 1025);
 
 
declare len1 int;
 
set len1 = length(msgBdy);
 
declare len2 int;
 
set len2 = length(msgID);
 
declare len3 int;
 
set len3 = length(msgHdr);
 
 
declare msgTms char;
 
set msgTms = cast(substring(InputRoot.BLOB.BLOB from 20501 for 26) as char ccsid 1025);
 
				
 
INSERT INTO Database.{DB_SCHEMA}.{DB_TABLE} (MSGID, MSGHDR, MSGBDY, MSGSTM) 
 
VALUES (msgID, msgHdr, msgBdy, msgTms);
 
 
Here is the exception list:
 
 
ExceptionList
 
	RecoverableException
 
		File = F:\\build\\S600_P\\src\\DataFlowEngine\\ImbDataFlowNode.cpp
 
		Line = 616
 
		Function = ImbDataFlowNode::createExceptionList
 
		Type = ComIbmMQInputNode
 
		Name = EMRS/Handlers/ReturnToMSC#FCMComposite_1_4
 
		Label = EMRS.Handlers.ReturnToMSC.GetMessageForMSC
 
		Catalog = BIPv600
 
		Severity = 3
 
		Number = 2230
 
		Text = Node throwing exception
 
		RecoverableException
 
			File = F:\\build\\S600_P\\src\\DataFlowEngine\\ImbComputeNode.cpp
 
			Line = 464
 
			Function = ImbComputeNode::evaluate
 
			Type = ComIbmComputeNode
 
			Name = EMRS/Handlers/ReturnToMSC#FCMComposite_1_5
 
			Label = EMRS.Handlers.ReturnToMSC.InsertIntoDB
 
			Catalog = BIPv600
 
			Severity = 3
 
			Number = 2230
 
			Text = Caught exception and rethrowing
 
			RecoverableException
 
				File = F:\\build\\S600_P\\src\\DataFlowEngine\\ImbRdl\\ImbRdlStatementGroup.cpp
 
				Line = 573
 
				Function = SqlStatementGroup::execute
 
				Type = ComIbmComputeNode
 
				Name = EMRS/Handlers/ReturnToMSC#FCMComposite_1_5
 
				Label = EMRS.Handlers.ReturnToMSC.InsertIntoDB
 
				Catalog = BIPv600
 
				Severity = 3
 
				Number = 2488
 
				Text = Error detected, rethrowing
 
				Insert
 
					Type = 5
 
					Text = EMRS.Handlers.ReturnToMSC_InsertIntoDB.main
 
				Insert
 
					Type = 5
 
					Text = 38.3
 
				Insert
 
					Type = 5
 
					Text = INSERT INTO Database.{RUMSCIN_SCHEMA}.{RUMSCIN_TABLE} ( MSGID, MSGHDR, MSGBDY, MSGSTM )  VALUES (msgID, msgHdr, msgBdy, msgTms ) 
 
				DatabaseException
 
					File = F:\\build\\S600_P\\src\\DataFlowEngine\\ImbOdbc.cpp
 
					Line = 227
 
					Function = ImbOdbcHandle::checkRcInner
 
					Type = 
 
					Name = 
 
					Label = 
 
					Catalog = BIPv600
 
					Severity = 3
 
					Number = 2321
 
					Text = Root SQL exception
 
					Insert
 
						Type = 2
 
						Text = -1
 
					DatabaseException
 
						File = F:\\build\\S600_P\\src\\DataFlowEngine\\ImbOdbc.cpp
 
						Line = 355
 
						Function = ImbOdbcHandle::checkRcInner
 
						Type = 
 
						Name = 
 
						Label = 
 
						Catalog = BIPv600
 
						Severity = 3
 
						Number = 2322
 
						Text = Child SQL exception
 
						Insert
 
							Type = 5
 
							Text = 22001
 
						Insert
 
							Type = 2
 
							Text = -99999
 
						Insert
 
							Type = 5
 
							Text = [IBM][CLI Driver] CLI0109E  String data right truncation. SQLSTATE=22001
 
 
Debug info:
 
 
msgID = MI10002980      
 
b = 0000000000003f1a
 
hdrLen = 141
 
msgHdr = :IO:I\n:FROM:NAME1234  \n:TO:PIN=123456\n:PRI:N\n:CRN:UNID(UNID:UNID:UNID:UNID:UNID:UNID:UNID:UN)\n:FORM:Memo\nTYPE: 940\nNKS : 12345678912345678900
 
bdyLen = 16154
 
msgBdy = <16154 bytes, looks like msgHdr>
 
len1 = 16154
 
len2 = 16
 
len3 = 141
 
msgTms = 2007-06-10-04.27.48.448000
 
 
You can see that the length is correct but why wouldn't it insert it into the column? The column size permits the insertion.
 
I tried a lot of things but that field just keeps outgrowing the column size.
 
 
Can anyone help?
 
 
Thanks! | 
			   
			 
		   | 
		
		
		  | Back to top | 
		  
		  	
		   | 
		
		
		    | 
		
		
		  | vk | 
		  
		    
			  
				 Posted: Wed Jun 13, 2007 8:52 am    Post subject:  | 
				     | 
			   
			 
		   | 
		
		
		   Partisan
 
 Joined: 20 Sep 2005 Posts: 302 Location: Houston 
  | 
		  
		    
			  
				Hi isolomatin,
 
 
Are you sure that the failure is for MSGBDY field? The error message in the exception tree will not say which column had an issue during the insert. Did you check the length of other columns to see if data for any of those columns are exceeding the defined length?
 
 
Do an mqsichangetrace and get the execution group trace. That would clearly show the SQL query and the values which the broker tried to insert.
 
 
Regards,
 
VK. | 
			   
			 
		   | 
		
		
		  | Back to top | 
		  
		  	
		   | 
		
		
		    | 
		
		
		  | isolomatin | 
		  
		    
			  
				 Posted: Wed Jun 13, 2007 7:26 pm    Post subject:  | 
				     | 
			   
			 
		   | 
		
		
		   Newbie
 
 Joined: 13 Jun 2007 Posts: 5
  
  | 
		  
		    
			  
				Hi vk,
 
 
I actually broke up the insert into four queries for testing.
 
First I did an insert with msgid, then an update with msghdr, then an update with msgbdy and then an update with msgtsm.
 
 
The exception is thrown on update msgbdy query.
 
 
Nevertheless I will try mqsitrace command and report the results.
 
 
Thanks. | 
			   
			 
		   | 
		
		
		  | Back to top | 
		  
		  	
		   | 
		
		
		    | 
		
		
		  | isolomatin | 
		  
		    
			  
				 Posted: Thu Jun 14, 2007 12:44 am    Post subject:  | 
				     | 
			   
			 
		   | 
		
		
		   Newbie
 
 Joined: 13 Jun 2007 Posts: 5
  
  | 
		  
		    
			  
				Here is the error message from AS/400.
 
 
Message ID . . . . . . :   SQL0404       Severity . . . . . . . :   30         
 
Message type . . . . . :   Diagnostic                                          
 
Date sent  . . . . . . :   14/06/07      Time sent  . . . . . . :   11:38:39   
 
                                                                               
 
Message . . . . :   Value for column or variable MSGBDY too long.              
 
Cause . . . . . :   An INSERT or UPDATE statement or a SET or VALUES INTO      
 
  statement or a GET DIAGNOSTICS statement specifies a value that is longer    
 
  than the maximum length string that can be stored in MSGBDY. The length of   
 
  MSGBDY is 16384 and the length of the string is 20275.                       
 
Recovery  . . . :   Reduce the length of the string from 20275 to a maximum of 
 
  16384 and try the request again.                                             
 
 
So the problem boils down to at what point 16154 bytes sent by broker become 20275 bytes received by AS/400? Any ideas?
 
 
I also ran the trace but the output file produced is in utf8 and the formatted is in ascii. no help about sizes of data being inserted there. | 
			   
			 
		   | 
		
		
		  | Back to top | 
		  
		  	
		   | 
		
		
		    | 
		
		
		  | isolomatin | 
		  
		    
			  
				 Posted: Fri Jun 15, 2007 4:41 am    Post subject:  | 
				     | 
			   
			 
		   | 
		
		
		   Newbie
 
 Joined: 13 Jun 2007 Posts: 5
  
  | 
		  
		    
			  
				Guys,
 
 
help me out here. Somebody must know something about this problem!
 
Thanks. | 
			   
			 
		   | 
		
		
		  | Back to top | 
		  
		  	
		   | 
		
		
		    | 
		
		
		  | jefflowrey | 
		  
		    
			  
				 Posted: Fri Jun 15, 2007 4:49 am    Post subject:  | 
				     | 
			   
			 
		   | 
		
		
		   Grand Poobah
 
 Joined: 16 Oct 2002 Posts: 19981
  
  | 
		  
		    
			  
				I'd say it's probably when you cast msgbdy to a character... 
 
 
the data might not actually be in the right codepage... so when you cast it to that CCSID, then you might be getting back the default character representation of the data - which is a hex string and thus about twice as long... _________________ I am *not* the model of the modern major general. | 
			   
			 
		   | 
		
		
		  | Back to top | 
		  
		  	
		   | 
		
		
		    | 
		
		
		  | kspranava | 
		  
		    
			  
				 Posted: Fri Jun 15, 2007 7:23 am    Post subject:  | 
				     | 
			   
			 
		   | 
		
		
		    Centurion
 
 Joined: 27 Apr 2003 Posts: 124
  
  | 
		  
		    
			  
				Hi,
 
 
If am not wrong, bdylen cannot go beyond 255, because
 
 
SET b = X'000000000000' || SUBSTRING(InputRoot.BLOB.BLOB FROM 4115 FOR 2); 
 
SET bdyLen = CAST(b AS INTEGER); 
 
SET msgBdy = CAST(SUBSTRING(InputRoot.BLOB.BLOB FROM 4117 FOR bdyLen) AS CHAR CCSID 1025); 
 
 
 
Above line retrieves 2 chars from body, which can hold a max value of FF (dec equiv 255). But, your trace shows bdylen = 16154 
 
 
Can you double check again?
 
 
Thanks,
 
kspranava. | 
			   
			 
		   | 
		
		
		  | Back to top | 
		  
		  	
		   | 
		
		
		    | 
		
		
		  | isolomatin | 
		  
		    
			  
				 Posted: Fri Jun 15, 2007 10:57 am    Post subject:  | 
				     | 
			   
			 
		   | 
		
		
		   Newbie
 
 Joined: 13 Jun 2007 Posts: 5
  
  | 
		  
		    
			  
				jefflowry,
 
 
 The data originally is a blob as stored in DB2/400.
 
 Not sure how DB2/400 works but a blob is a blob, it's just a stream of bytes.
 
 I'll double check about the code page though. It might not be 1025.
 
 
 kspranava,
 
 
 in this message format there is a two byte length field preceding the actual data. Two bytes can represent numbers up to 65 535. | 
			   
			 
		   | 
		
		
		  | Back to top | 
		  
		  	
		   | 
		
		
		    | 
		
		
		  | 
		    
		   |