ASG
IBM
Zystems
Cressida
Icon
Netflexity
 
  MQSeries.net
Search  Search       Tech Exchange      Education      Certifications      Library      Info Center      SupportPacs      LinkedIn  Search  Search                                                                   FAQ  FAQ   Usergroups  Usergroups
 
Register  ::  Log in Log in to check your private messages
 
RSS Feed - WebSphere MQ Support RSS Feed - Message Broker Support

MQSeries.net Forum Index » User Exits » Exits and MQ installed in a non-default location

Post new topic  Reply to topic
 Exits and MQ installed in a non-default location « View previous topic :: View next topic » 
Author Message
RogerLacroix
PostPosted: Thu Apr 11, 2013 12:55 pm    Post subject: Exits and MQ installed in a non-default location Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3252
Location: London, ON Canada

All,

I wrote a blog item today about the following and I was curious if anyone else knew about this.

Did you know if you use any kind of MQ exit (API, channel, data conversion, cluster, publish, etc..) with an install of MQ in a non-default location then there is a high probability that exit will not work.
http://publib.boulder.ibm.com/infocenter/wmqv7/v7r1/index.jsp?topic=%2Fcom.ibm.mq.doc%2Fzd00140_.htm

If your exit uses any MQ API calls or is linked with mqm, mqic, mqmvx, mqmvxd, mqutl or mqmzf libraries then it will fail. Period - no exceptions.

I don't who came up with the "Procedure" section of that web page but whatever they are on, they should be sharing it because it is "one wild and crazy" solution (Steve Martin reference). It is really not a solution but rather a rewrite of your exit!! I'm not kidding.

I have no clue how you are supposed support an exit for a migration between 2 versions of MQ on the same server with multi-install.

I won't even talk about my stuff. Lets use BlockIP2 as an example. It will work with any MQ version less than v7.0.1.6 and any MQ version greater than or equal to v7.0.1.6 IF AND ONLY IF MQ is installed in the default location. If you have 2 versions of MQ installed on your server for migration purposes, v7.0.1.9 (installed in a default location) and v7.1.0.2 (installed in a non-default location). When you go to use BlockIP2 with WMQ v7.1.0.2 (installed in a non-default location) , you will get an error. Something like:
Code:
08/04/13 17:00:10 - Process(18418.4) User(mqm) Program(amqrmppa)
                    Host(servername) Installation(Installation2)
                    VRMF(7.5.0.1) QMgr(QMGRNAME)


AMQ6175: The system could not dynamically load the shared library
'/var/mqm/exits64/exitname'. The system returned error message
'/opt/mqm/lib64/libmqz_r.so: undefined symbol: xcsAllocateMemBlock'.

EXPLANATION:
This message applies to UNIX systems. The shared library
'/var/mqm/exits64/exitname' failed to load correctly due to a problem
with the library.

ACTION:
Check the file access permissions and that the file has not been corrupted.

The WORST part of that web page is what it does not say; if you update your exit as the "Procedure" states then it will NOT work with any release before WMQ v7.1. i.e. v6.0.*.* or v7.0.0.* or v7.0.1.*

I think a lot of people are going to be feeling a lot of pain in the near future.

Food for thought.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter


Last edited by RogerLacroix on Thu Apr 11, 2013 4:20 pm; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website
PeterPotkay
PostPosted: Thu Apr 11, 2013 4:14 pm    Post subject: Reply with quote

Poobah

Joined: 15 May 2001
Posts: 7717

We will be setting up some new MQ servers soon, with MQ 7.5. We use MQAUSX. While we don't have any definite plans to have multiple instances of MQ installed on these servers going forward, you never know, we may want to. So naturally the discussion turned to where should we install the first install - in the default location, or come up with some directory naming pattern / standard assuming multiple installations are coming, and install even the first version in the non standard location, in keeping with the directory naming pattern.

But after reading this, I think we should, actually, apparently we have to, install 7.5.0.1 in the default location, and hope this gets sorted out before we ever need to install a second parallel copy on the server if we want our exits to work. Correct?
_________________
Peter Potkay
Keep Calm and MQ On
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Thu Apr 11, 2013 4:37 pm    Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3252
Location: London, ON Canada

PeterPotkay wrote:
But after reading this, I think we should, actually, apparently we have to, install 7.5.0.1 in the default location, and hope this gets sorted out before we ever need to install a second parallel copy on the server if we want our exits to work. Correct?

For right now, true. I'm reading & thinking and reading & thinking, etc... I'm still scratching my head but it appears I need 2 versions of the same security exit. One for pre v7.1 and one for v7.1 or higher (non-default install location). I just can't see how to make one exit that works for both situations.

For multi-installation, IBM is already setting environment variables with crtmqenv/setmqenv commands. A normal solution would be to say remove the RPATH from the linking of the exit then you need to explicitly set the local path for the OS. i.e. PATH for Windows, SHLIB_PATH for HP-UX, LIBPATH for AIX & IBM i (OS/400) and LD_LIBRARY_PATH for Solaris & Linux.

I just don't understand why IBM decided on this extremely complex and not backwards compatible solution.

Basically, if you are using a security exit (or any exit) and you do not update as per the "Procedure", then you either
- stop using the exit so that you can use multi-install or
- only use MQ in a default install location and the security exit

I'll keep you posted.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
fjb_saper
PostPosted: Thu Apr 11, 2013 7:07 pm    Post subject: Reply with quote

Grand High Poobah

Joined: 18 Nov 2003
Posts: 20696
Location: LI,NY

Roger,
When trying this did you make sure to start each qmgr in the right environment? I.E. before each qmgr start run
Code:
setmqenv -m <qmgrname>


Thanks for letting us know.
_________________
MQ & Broker admin
Back to top
View user's profile Send private message Send e-mail
Michael Dag
PostPosted: Fri Apr 12, 2013 2:04 am    Post subject: Reply with quote

Jedi Knight

Joined: 13 Jun 2002
Posts: 2602
Location: The Netherlands (Amsterdam)

am i too simple to understand?
there can be multiple versions of mq on the system.
from 7.0.1.6 (min) they can co-exist in there own location.
from 7.0.1.6 the libraries are 'intelligent' in finding which version you are running hence the need for not linking to specific libraries?
obviously pre- 7.0.1.6 programs and libraries will not be able to deal with multi-version and only 'see' the default one.

so i guess you need to supply 2 versions of your exit, the regular one and the multi-version 'aware' one, awkward maybe but not impossible. Over time you will only need one version again obviously...

if you need help testing some stuff let me know and i can lend a hand when availability lets me.
_________________
Michael



MQSystems Facebook page
Back to top
View user's profile Send private message Visit poster's website MSN Messenger
mqjeff
PostPosted: Fri Apr 12, 2013 3:13 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

Michael Dag wrote:
so i guess you need to supply 2 versions of your exit, the regular one and the multi-version 'aware' one, awkward maybe but not impossible.

Generally straight forward.

And clearly documented as a decision that needs to be made by an administrator when they install more than one instance or do not install in the default location. "If you install in non-default locations, your exits might need to be recompiled".

It's always the kind of tradeoff one has to keep to the forefront when using exits of any kind.
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Fri Apr 12, 2013 8:56 am    Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3252
Location: London, ON Canada

mqjeff wrote:
It's always the kind of tradeoff one has to keep to the forefront when using exits of any kind.

That's BS.

On distribute platforms, most people are aware of "local path for the OS" and so should IBM. i.e. PATH for Windows, SHLIB_PATH for HP-UX, LIBPATH for AIX and LD_LIBRARY_PATH for Solaris & Linux.

For example, I have several client LDAPs libraries (OpenLDAP, Mozilla/Netscape, etc..) installed on Windows, Unix & Linux. When I want to use a particular client LDAP library with my regular application, I simply set the appropriate local path and do my test with my application. I even do this type of testing (switching client LDAP libraries) for security exit testing (i.e. MCA (amqrmppa) -> mqausx -> mqausxldap64.so -> client LDAP SSL libraries)

There is nothing complicated about it. For security exit testing, I simply stop the queue manager, set the local path for the particular client LDAP library I want to use then restart the queue manager and do more testing. As I said, nothing complicated about it

IBM had 2 choices: use/set local path for the OS with setmqenv/crtmqenv commands or create a new complex solution (which causes existing exits to not work with WMQ v7.1 or higher that are installed in a non-default location).

I'm still scratching my head over this - I've modified one exit to use MQIEP, tested it under WMQ v7.5 and it does work.

Here's a fun fact, WMQ v7.0.1.6 to v7.0.1.9 does NOT support MQIEP, even though those versions support multi-install. Hence, the dividing line is WMQ v7.1. I don't think that most people will realize that their multi-install with v7.0.1.9 will not support their updated exit that is for mutli-install.

I only have about 100 customers using exits from various products - think about the hundreds or thousands of people uses BlockIP2 or other home-grown exits that will not work with WMQ v7.1 or higher installed into a non-default location.

Now if I was into conspiracy theories, I would say that IBM did this on purpose, since there was an obvious simple solution, to get customer to stop using security exits and use the new CHLAUTH feature that was coincidentally released in WMQ v7.1. Hummm. Makes you wonder!!

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
mqjeff
PostPosted: Fri Apr 12, 2013 10:09 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

RogerLacroix wrote:
mqjeff wrote:
It's always the kind of tradeoff one has to keep to the forefront when using exits of any kind.

That's BS.


Do you disagree that platform specific exits that are compiled at a specific software level may need recompilation when one makes any major changes to the platform or the product that uses exits?

Do you disagree that an informed MQ Administrator will keep this in mind when making the decision to use or not use an exit to provide certain specific functionality?

Which, exact, part of my statement that the use of exits requires tradeoffs and that one must keep those tradeoffs in mind do you consider to be BS?

RogerLacroix wrote:
On distribute platforms, most people are aware of "local path for the OS" and so should IBM. i.e. PATH for Windows, SHLIB_PATH for HP-UX, LIBPATH for AIX and LD_LIBRARY_PATH for Solaris & Linux.

For example, I have several client LDAPs libraries (OpenLDAP, Mozilla/Netscape, etc..) installed on Windows, Unix & Linux. When I want to use a particular client LDAP library with my regular application, I simply set the appropriate local path and do my test with my application. I even do this type of testing (switching client LDAP libraries) for security exit testing (i.e. MCA (amqrmppa) -> mqausx -> mqausxldap64.so -> client LDAP SSL libraries)

There is nothing complicated about it. For security exit testing, I simply stop the queue manager, set the local path for the particular client LDAP library I want to use then restart the queue manager and do more testing. As I said, nothing complicated about it


Except that, as noted elsewhere, your "regular application" likely doesn't run as a setuid program.

Which it is reasonably clear, and well documented, that MQ does.

And, as noted previously, this does indeed make things more complicated.

RogerLacroix wrote:
IBM had 2 choices: use/set local path for the OS with setmqenv/crtmqenv commands or create a new complex solution (which causes existing exits to not work with WMQ v7.1 or higher that are installed in a non-default location).

Or, perhaps, they only had one choice because other choices that were made for very good reasons made it impossible to do anything else.

RogerLacroix wrote:
I'm still scratching my head over this - I've modified one exit to use MQIEP, tested it under WMQ v7.5 and it does work.

So the documentation is correct, and helpful. Good to know.

RogerLacroix wrote:
Here's a fun fact, WMQ v7.0.1.6 to v7.0.1.9 does NOT support MQIEP, even though those versions support multi-install. Hence, the dividing line is WMQ v7.1. I don't think that most people will realize that their multi-install with v7.0.1.9 will not support their updated exit that is for mutli-install.

Perhaps this distinction is made clear in the part of the documentation that says "your exit MAY need to be recompiled".

RogerLacroix wrote:
I only have about 100 customers using exits from various products - think about the hundreds or thousands of people uses BlockIP2 or other home-grown exits that will not work with WMQ v7.1 or higher installed into a non-default location.

Given that exits have always been clearly documented as a choice-of-last-resort, I sincerely hope there aren't that many people using homegrown exits.

And given that anyone using 7.1 or later has absolutely no reason to continue using BlockIPv2 or any other security exit except in very specialized circumstances, I would hope that most people would sigh relief when they were able to stop using an adhoc solution and start using bespoke functionality.

RogerLacroix wrote:
Now if I was into conspiracy theories, I would say that IBM did this on purpose, since there was an obvious simple solution, to get customer to stop using security exits and use the new CHLAUTH feature that was coincidentally released in WMQ v7.1. Hummm. Makes you wonder!!


I think it's deadly obvious that CHLAUTH rules were added simply and only because everyone was very upset at having to use Exits in the first place, when it should have been built into the product.
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Fri Apr 12, 2013 10:47 am    Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3252
Location: London, ON Canada

mqjeff wrote:
Do you disagree that platform specific exits that are compiled at a specific software level may need recompilation when one makes any major changes to the platform or the product that uses exits?

You are making an assumption that the change was truly required.
mqjeff wrote:
Which, exact, part of my statement that the use of exits requires tradeoffs and that one must keep those tradeoffs in mind do you consider to be BS?

Maybe BS was too strong but I was referring to the "tradeoff" part. Just because IBM did a Microsoft stunt, you can't turn around and say "you snooze, you lose". IBM has ALWAYS said that exit points are for users to extend and/or enhance the functionality of MQ.

mqjeff wrote:
Except that, as noted elsewhere, your "regular application" likely doesn't run as a setuid program.

Smoke and mirrors. I just set the UID of several test applications and then did a couple of tests. It made absolutely no difference - I can pick and chose whatever version product shared library that I want.

mqjeff wrote:
Or, perhaps, they only had one choice because other choices that were made for very good reasons made it impossible to do anything else.

Well, given that other products in the Windows, Linux and Unix world can have multi-installs and you use the local path to do the selection of the particular product, I don't see anything special about MQ.

mqjeff wrote:
Given that exits have always been clearly documented as a choice-of-last-resort, I sincerely hope there aren't that many people using homegrown exits.

IBM has ALWAYS said that exit points are for user to extend and/or enhance the functionality of MQ and nothing about "choice-of-last-resort".

mqjeff wrote:
And given that anyone using 7.1 or later has absolutely no reason to continue using BlockIPv2 or any other security exit except in very specialized circumstances, I would hope that most people would sigh relief when they were able to stop using an adhoc solution and start using bespoke functionality.

BlockIP2 provided a filtering mechanism and CHLAUTH provides a filtering mechanism, but there are many companies who want more security than what a filtering mechanism provides.

Also, there are many, many companies in the financial sector that give a security exit to the client company to provide for mutual verification/authentication (nothing to do with my stuff). When companies can't interact with 3rd party companies because they are using WMQ in a non-default location - lots of $hit will hit the fan.

Jeff, I know you work for IBM but you used to be far more objective before you worked at IBM.

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
mqjeff
PostPosted: Fri Apr 12, 2013 11:24 am    Post subject: Reply with quote

Grand Master

Joined: 25 Jun 2008
Posts: 17447

RogerLacroix wrote:
mqjeff wrote:
Do you disagree that platform specific exits that are compiled at a specific software level may need recompilation when one makes any major changes to the platform or the product that uses exits?

You are making an assumption that the change was truly required.
mqjeff wrote:
Which, exact, part of my statement that the use of exits requires tradeoffs and that one must keep those tradeoffs in mind do you consider to be BS?

Maybe BS was too strong but I was referring to the "tradeoff" part. Just because IBM did a Microsoft stunt, you can't turn around and say "you snooze, you lose". IBM has ALWAYS said that exit points are for users to extend and/or enhance the functionality of MQ.


And IBM has also always said that they are complicated, difficult things and require advanced programming knowledge and advanced knowledge of MQ.


RogerLacroix wrote:
mqjeff wrote:
Or, perhaps, they only had one choice because other choices that were made for very good reasons made it impossible to do anything else.

Well, given that other products in the Windows, Linux and Unix world can have multi-installs and you use the local path to do the selection of the particular product, I don't see anything special about MQ.


I'm not aware of any specific reasons why it works this way either. But I also trust that the people who write this software make the decisions they make for sound technical reasons or to meet specific customer requirements, and not for whim.

RogerLacroix wrote:
mqjeff wrote:
Given that exits have always been clearly documented as a choice-of-last-resort, I sincerely hope there aren't that many people using homegrown exits.

IBM has ALWAYS said that exit points are for user to extend and/or enhance the functionality of MQ and nothing about "choice-of-last-resort".


And, again, always said that they are complicated, require advanced skills and knowledge, can break the queue manager if written poorly, and that they won't provide support for them.

You don't read that to mean "find a better way to do this, if you can"?

RogerLacroix wrote:
mqjeff wrote:
And given that anyone using 7.1 or later has absolutely no reason to continue using BlockIPv2 or any other security exit except in very specialized circumstances, I would hope that most people would sigh relief when they were able to stop using an adhoc solution and start using bespoke functionality.

BlockIP2 provided a filtering mechanism and CHLAUTH provides a filtering mechanism, but there are many companies who want more security than what a filtering mechanism provides.

Also, there are many, many companies in the financial sector that give a security exit to the client company to provide for mutual verification/authentication (nothing to do with my stuff). When companies can't interact with 3rd party companies because they are using WMQ in a non-default location - lots of $hit will hit the fan.


Presumably those companies made that choice understanding the risks they took. Presumably those companies are prepared to deal with changes to things that they were warned could indeed be changed in ways that would require them to make changes to their exit code.

And I don't know of any financial company that would implement a migration without extensive testing, including with their own customers.

RogerLacroix wrote:
Jeff, I know you work for IBM but you used to be far more objective before you worked at IBM.

It's true, I get paid by IBM to assist customers with being successful using the features of MQ and Broker, and if necessary using other means, including exits or user-defined plugins or etc.

And you get paid to sell exits and support exits.

I don't think anyone is unclear on how objective either of us are.

What's the complaint here?
Back to top
View user's profile Send private message
RogerLacroix
PostPosted: Fri Apr 12, 2013 11:47 am    Post subject: Reply with quote

Jedi Knight

Joined: 15 May 2001
Posts: 3252
Location: London, ON Canada

mqjeff wrote:
You don't read that to mean "find a better way to do this, if you can"?

No, I don't and never said anything that implied that.

mqjeff wrote:
And I don't know of any financial company that would implement a migration without extensive testing, including with their own customers.

Sure. And I have some very nice land in Florida for sale too.

mqjeff wrote:
And you get paid to sell exits and support exits.

I don't think anyone is unclear on how objective either of us are.

Do you really think I would post issues here or on the MQ ListServer as a marketing stunt? That would be the worst marketing strategy in the world.

The better marketing strategy would be to be selfish and not say anything at all but simply update my exits. Then when other people stuff breaks, point out how wonder my stuff is.

I post problems/issues for other people to be aware of and what to do to resolve the issues (and to point out really stupid decisions).

In your pre-IBM honor, I'm going to spend a bunch of time trying to see if "force-feeding" MQ with the normal local path stuff and removing RPATH from the exit will actually work (of course it won't be supported by IBM).

Regards,
Roger Lacroix
Capitalware Inc.
_________________
Capitalware: Transforming tomorrow into today.
Connected to MQ!
Twitter
Back to top
View user's profile Send private message Visit poster's website
Andyh
PostPosted: Sat Apr 13, 2013 6:47 am    Post subject: Reply with quote

Master

Joined: 29 Jul 2010
Posts: 237

If you could affect which version of an exit a setuid program were to invoke by simply setting an environment variable inherited by that program then there would be a huge integrity hole in your system.
Back to top
View user's profile Send private message
zpat
PostPosted: Sat Apr 13, 2013 7:37 am    Post subject: Reply with quote

Jedi Council

Joined: 19 May 2001
Posts: 5849
Location: UK

I disagree that the 7.1 clauth feature can replace BlockIP2 - that exit can do more things, and do them more flexibility than clauth records can at present (Block IP being a misnomer).

Let's hope IBM continue to enhance clauth so that it can one day replace BlockIP2 - and meantime let's hope the author of the exit will keep it working - including from non-default install locations.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic  Reply to topic Page 1 of 1

MQSeries.net Forum Index » User Exits » Exits and MQ installed in a non-default location
Jump to:  



You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
Protected by Anti-Spam ACP
 
 


Theme by Dustin Baccetti
Powered by phpBB © 2001, 2002 phpBB Group

Copyright © MQSeries.net. All rights reserved.