| Author |
Message
|
| Jenum |
 Posted: Tue Sep 25, 2018 5:11 am Post subject: It is impossible to set correct CCSID |
 |
|
Novice
Joined: 13 Nov 2012
Posts: 24
|
Hi all.
I want to send message with CCSID=1251 with some properties:
| Code: |
queueManager = ...;
int openOptions = MQConstants.MQOO_INPUT_SHARED | MQConstants.MQOO_OUTPUT | MQConstants.MQOO_BROWSE;
MQQueue queue = queueManager.accessQueue(..., openOptions);
MQPutMessageOptions pmo = new MQPutMessageOptions();
pmo.options = CMQC.MQPMO_NO_SYNCPOINT;
MQMessage message = new MQMessage();
message.characterSet = 1251;
message.write(Charsets.convert("Some text", message.characterSet));
message.setStringProperty("first", "1");
message.setStringProperty("second", "2");
message.setStringProperty("third", "3");
queue.put(message, pmo);
|
But message in queue has CCSID=1208 (in MQMD and MQRFH2 headers).
After some research, i found, that problem in methods MQMessage#writePropertiesRfh2 and MQHeaderList#write (which call in MQDestination#put): first set CCSID=1208 in MQMD and second set similar characterSet in all headers.
I use com.ibm.mq.allclient.jar with version 8.0.0.8.
Is it correct behaviour or some mistake in IBM libraries? |
|
| Back to top |
|
 |
| Vitor |
| Posted: Tue Sep 25, 2018 5:50 am Post subject: Re: It is impossible to set correct CCSID |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Jenum wrote: |
| Is it correct behaviour or some mistake in IBM libraries? |
Sounds correct to me (given the caveat my Java isn't first class).
The CCSID in the MQMD defines the code page for the payload of the message, which from the MQMD's point of view includes the RFH2. The RFH2 CCSID contains additional descriptive information for the actual payload including the CCSID of the data.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Jenum |
| Posted: Tue Sep 25, 2018 5:53 am Post subject: |
|
|
Novice
Joined: 13 Nov 2012
Posts: 24
|
Please, find this:
| Quote: |
| (in MQMD and MQRFH2 headers) |
There are no problem if just MQMD contains characterSet = 1208, but MQRFH2.characterSet = 1208 too, so original value (1251) is missing. |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Sep 25, 2018 7:51 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Jenum wrote: |
Please, find this:
| Quote: |
| (in MQMD and MQRFH2 headers) |
There are no problem if just MQMD contains characterSet = 1208, but MQRFH2.characterSet = 1208 too, so original value (1251) is missing. |
I don't see anywhere in the code you specific indicating that the RFH CCSID should be 1251. You set that for the message, but not in the header.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Jenum |
| Posted: Tue Sep 25, 2018 7:54 am Post subject: |
|
|
Novice
Joined: 13 Nov 2012
Posts: 24
|
| I don't create MQRFH2-header manually, i just set some properties. Is there any chance to set neccesary characterSet in this way? |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Sep 25, 2018 8:12 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Urgh.
How does your queue manager handle properties? Are you sure you're getting an RFH header in the message or are the properties native on the message?
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Jenum |
| Posted: Tue Sep 25, 2018 8:25 am Post subject: |
|
|
Novice
Joined: 13 Nov 2012
Posts: 24
|
| Quote: |
| How does your queue manager handle properties? |
Which property of queue manager you mean?
I try to send message to queue (MQPUT) with non-1208 ccsid body and some properties. For some reason, i can't set message properties by MQRFH2 object, only by setProperty-methods of MQMessage object. But how to set necessary body ccsid in this case? |
|
| Back to top |
|
|
| Vitor |
| Posted: Tue Sep 25, 2018 9:22 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Jenum wrote: |
| Quote: |
| How does your queue manager handle properties? |
Which property of queue manager you mean? |
I mean how is your queue manager set to handle properties?
If it's not in a compatibility setting, you won't get an RFH2.
Failing that, you need someone who knows more Java than I do.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| tczielke |
| Posted: Tue Sep 25, 2018 10:37 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
I see similar behavior to what Jenum is reporting.
If I use one of my MQ base Java samples to set the MQMessage.characterSet=1251 and then do a PUT without setting any message properties, the message is PUT on the queue with CCSID 1251.
However, if I then add in a MQMessage.setStringProperty call before the PUT and run the sample again, I now get a message with CCSID 1208, even with the MQMessage.characterSet being set to 1251.
To me this looks like a defect. I don't see this behavior documented anywhere in the MQ manual that calling a MQMessage.setStringProperty would alter the CCSID of the message that is PUT to the queue.
Jenum - You may want to open a PMR to have IBM investigate why this is happening.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
| rekarm01 |
| Posted: Wed Sep 26, 2018 7:10 pm Post subject: Re: It is impossible to set correct CCSID |
|
|
Grand Master
Joined: 25 Jun 2008
Posts: 1415
|
Is there any particular reason for using the IBM MQ classes for Java, rather than the IBM MQ classes for JMS?
| Jenum wrote: |
| I want to send message with CCSID=1251 |
The MQMD class supports the following values for characterSet: CMQC.MQCCSI_Q_MGR, 850, 819, 37, 1200, and 1208. That's not to say that 1251 wouldn't work, only that it's not documented, and might not be supported. Does the same problem occur with one of these other, documented characterSet values?
| Jenum wrote: |
| Code: |
MQMessage message = new MQMessage();
message.characterSet = 1251;
message.write(Charsets.convert("Some text", message.characterSet)); |
|
Setting message.format could help; that way the other classes know that characterSet is important, and should be preserved when chaining headers:
| Code: |
MQMessage message = new MQMessage();
message.characterSet = 1251;
message.format = MQConstants.MQFMT_STRING;
message.writeString("Some text"); |
... and writeString() might be easier to use here; it has conversion built in. |
|
| Back to top |
|
|
| Jenum |
| Posted: Wed Sep 26, 2018 9:31 pm Post subject: |
|
|
Novice
Joined: 13 Nov 2012
Posts: 24
|
| Quote: |
Is there any particular reason for using the IBM MQ classes for Java, rather than the IBM MQ classes for JMS? |
Our framework to work with MQ already use classes for Java. Another reason is JMS doesn't support some features, like segmentation.
| Quote: |
| Does the same problem occur with one of these other, documented characterSet values? |
Yes, problem occurs even with documented 850 CCSID.
| Quote: |
| Setting message.format could help |
Sad, but no.
| Quote: |
| ... and writeString() might be easier to use here; it has conversion built in. |
No, resulting CCSID will be 1208.
After decompiling MQDestination and MQMessage classes, the reason for such problem in method MQMessage#writePropertiesRfh2(int), which forcely set characterSet of message to 1208. So, tczielke is right: it should be PMR to IBM. |
|
| Back to top |
|
|
| Jenum |
| Posted: Wed Sep 26, 2018 9:44 pm Post subject: |
|
|
Novice
Joined: 13 Nov 2012
Posts: 24
|
May be it would be helpfull for someone:
my work around for this problem is using reflection (it is very, very bad solution, but i can't found anything else):
| Code: |
MQMessage msg = new MQMessage();
msg.characterSet = 1251;
msg.setStringProperty(...);
msg.write(Charsets.convert("...", msg.characterSet));
int ccsid = msg.characterSet;
if (ccsid != 1208) {
try {
/* Converting message properties in MQRFH2 header */
Method m = msg.getClass().getDeclaredMethod("writePropertiesRfh2", int.class);
m.setAccessible(true);
m.invoke(msg, 1208);
/* Deleting all message properties, otherwise it would be converted again in put method */
Enumeration<String> props = msg.getPropertyNames("%");
while (props.hasMoreElements()) {
String prop = props.nextElement();
msg.deleteProperty(getCanonicalPropertyName(prop));
}
/* Setting necessary encoding in last (and single: MQRFH2) header */
msg.seek(0);
MQHeaderList hList = new MQHeaderList(msg);
MQRFH2 rfh2 = (MQRFH2) hList.get(0);
rfh2.setCodedCharSetId(ccsid);
} catch (NoSuchMethodException | InvocationTargetException | MQDataException e) {
throw new RuntimeException(e);
}
}
queue.put(msg, pmo);
if (ccsid != 1208) {
msg.seek(0);
try {
/* i need to return MQMessage with properties. Good solution would be copy of MQMessage before previously steps, but i don't know, how to do it :( */
Method m = msg.getClass().getDeclaredMethod("performProcessingAfterGet");
m.setAccessible(true);
m.invoke(msg);
} catch (NoSuchMethodException | InvocationTargetException e) {
throw new RuntimeException(e);
}
}
return msg;
|
|
|
| Back to top |
|
|
| tczielke |
| Posted: Thu Sep 27, 2018 5:37 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
To help me understand, why is it an issue if the message is PUT on the queue with a CCSID of 1208? 1208 would be able to encode all the character symbols that would be in 1251.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
| Jenum |
| Posted: Thu Sep 27, 2018 5:43 am Post subject: |
|
|
Novice
Joined: 13 Nov 2012
Posts: 24
|
| tczielke, 1251 - just example, we often work with CCSID 866 which doesn't compatible with 1208. |
|
| Back to top |
|
|
| tczielke |
| Posted: Thu Sep 27, 2018 5:52 am Post subject: |
|
|
Guardian
Joined: 08 Jul 2010
Posts: 941
Location: Illinois, USA
|
I am not familiar with working with CCSID 866, but from a character symbol stand point, I would think any character symbol encoded in 866 would also be encoded in 1208 ( UTF-8 ). If you are at MQ v9, MQ supports full unicode, so 1208 would be the full 1-4 byte encoding suite of Unicode. Unicode pretty much covers everything you would probably want to work with from a character symbol stand point.
This is what I am observing with the IBM MQ base Java. If you set the MQMessage.characterSet to a supported CCSID, it will PUT the message on the queue with that CCSID. However, if you start using these message property methods, then it switches to putting the message on the queue in 1208 ( UTF-8 ). However, UTF-8 can pretty much encode any character symbol that another CCSID could, as far as I know.
_________________
Working with MQ since 2010. |
|
| Back to top |
|
|
|
|
