| Author |
Message
|
| Gralgrathor |
 Posted: Wed Aug 14, 2019 2:49 am Post subject: Overriding properties promoted in subflows |
 |
|
Master
Joined: 23 Jul 2009
Posts: 297
|
Good morning,
I'm having a little problem with properties in subflows, and I can't determine whether I just didn't RTFM well enough, or something is actually broken here.
To describe the situation:
Running IIB 10.0.0.17 for Win64.
1. I create a static library with a subflow containing a File Output node, and promote some of its Remote properties to the subflow:
https://imgshare.io/images/2019/08/14/TransferFileNodeProps.png
https://imgshare.io/images/2019/08/14/TransferFile.png
2. I drag the subflow into a message flow in the main application and override its promoted properties there.
https://imgshare.io/images/2019/08/14/TestApp.png
In this case, when I try to deploy, the deploy fails because the identity 'SFTP_IDENTITY_SUBFLOW' doesn't exist:
| Code: |
| BIP3386W: File node 'SFTP' in message flow 'TransferFile'. The remote user identifier supplied as 'SFTP_IDENTITY_SUBFLOW' is invalid. |
But I would have expected it to go with the override value: 'SFTP_IDENTITY_MAINFLOW'.
It seems too basic a thing for it to be broken, and I can't find anything in the documentation that contradicts my expectations in this.
So my question: is this working as designed, and did I just not spend long enough Googling? Or is it a known bug, with a fix to come in some later fixpack? Or is it something else?
Thanks in advance for your answers!
Gr,
Gr.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
 |
| fjb_saper |
| Posted: Wed Aug 14, 2019 4:18 am Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
Did you promote the subflows properties to the mainflow? 
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Wed Aug 14, 2019 4:46 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| fjb_saper wrote: |
| Did you promote the subflows properties to the mainflow? |
In my original application I did, and then tried to override them on the main flow, and through properties files applied to the bar. It still stuck with the values set at the subflow level. In this demo, I was simply trying to see if the subflow picked up on the overrides, so I didn't re-promote the properties to the main flow.
It appears to me as if the subflow simply isn't picking up on overrides from outside the subflow/library.
UPDATE: same occurs when I move the subflow into the application, and put everything in the default broker schema.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine.
Last edited by Gralgrathor on Wed Aug 14, 2019 5:00 am; edited 1 time in total |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Aug 14, 2019 5:00 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Understand I can't see the images in your post, as the corporate network security is deliberately set to "paranoid". So you may have proved some of this.
So I would check that SFTP_IDENTITY_SUBFLOW is properly promoted as a message flow property SFTP_IDENTITY_MAINFLOW, using mqsireadbar. Also try to override the property with <subflow>.SFTP_IDENTITY_SUBFLOW.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Wed Aug 14, 2019 5:13 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| Vitor wrote: |
| Understand I can't see the images in your post |
They're not all that interesting. Some nodes, coupla lines. Just to prove that I wasn't simply dreaming this up 
| Vitor wrote: |
| So I would check that SFTP_IDENTITY_SUBFLOW is properly promoted |
Right. So I promoted the Security identity on the File Output node as SFTP_IDENTITY on the subflow.
On the subflow, I entered the value 'SFTP_IDENTITY_SUBFLOW'.
I then dragged the subflow into the main flow, and in the main flow, entered 'SFTP_IDENTITY_MAINFLOW' into SFTP_IDENTITY.
Read bar produces this (I cut out the less relevant bits):
| Code: |
BIP1052I: Reading Bar file using runtime mqsireadbar...
TestApp.bar:
TestApp.appzip (8/14/19 2:57 PM):
application.descriptor (8/14/19 2:57 PM):
TransferFile.subflow (8/14/19 2:57 PM):
TestFlow.cmf (8/14/19 2:57 PM):
Deployment descriptor:
...
TransferFile#SFTP_IDENTITY = SFTP_IDENTITY_SUBFLOW
TransferFile#SFTP_HOST = SFTP_HOST_SUBFLOW
TransferFile#SFTP_DIRECTORY = SFTP_DIRECTORY_SUBFLOW
...
TestFlow#TransferFile.SFTP_DIRECTORY = SFTP_DIRECTORY_MAINFLOW
TestFlow#TransferFile.SFTP_HOST = SFTP_HOST_MAINFLOW
TestFlow#TransferFile.SFTP_IDENTITY = SFTP_IDENTITY_MAINFLOW
...
BIP8071I: Successful command completion. |
| Vitor wrote: |
| Also try to override the property with <subflow>.SFTP_IDENTITY_SUBFLOW. |
I've tried every possible override using mqsiapplybaroverrides. Overriding anything on the level of the mainflow results in the same error. I wasn't able to figure out how to address properties on the FileOutput node directly using properties files.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Aug 14, 2019 5:26 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
| Gralgrathor wrote: |
| I then dragged the subflow into the main flow, and in the main flow, entered 'SFTP_IDENTITY_MAINFLOW' into SFTP_IDENTITY. |
Did you explicitly promote that property from the sub flow to the main flow?
| Gralgrathor wrote: |
| I've tried every possible override using mqsiapplybaroverrides. Overriding anything on the level of the mainflow results in the same error. |
Run mqsiapplybaroverrides then run mqsireadbar again. Check what exactly has changed.
| Gralgrathor wrote: |
| I wasn't able to figure out how to address properties on the FileOutput node directly using properties files. |
Include the node name in the override. So TransferFile#SFTP_IDENTITY becomes TransferFile#<what you called the node>.<File Identity or whatever the node property is called because I'm too lazy to look it up>
Use of # and . may vary because I can't remember which goes where.
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Wed Aug 14, 2019 6:39 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| Vitor wrote: |
| Did you explicitly promote that property from the sub flow to the main flow? |
In the TestApp version, I promoted only from the Node to the Subflow. In the original application, I promoted from the Subflow Node to the main flow as well, same result.
| Vitor wrote: |
| Check what exactly has changed. |
I've overridden the properties promoted to the subflow, and the properties on the subflow node in the main flow. So the output of readbar now reads:
| Code: |
...
TransferFile#SFTP_IDENTITY = SFTP_IDENTITY_SUB_OVERRIDE
TransferFile#SFTP_HOST = SFTP_HOST_SUB_OVERRIDE
TransferFile#SFTP_DIRECTORY = SFTP_DIRECTORY_SUB_OVERRIDE
...
TestFlow#TransferFile.SFTP_DIRECTORY = SFTP_DIRECTORY_MAIN_OVERRIDE
TestFlow#TransferFile.SFTP_HOST = SFTP_HOST_MAIN_OVERRIDE
TestFlow#TransferFile.SFTP_IDENTITY = SFTP_IDENTITY_MAIN_OVERRIDE
... |
When deploying, the broker picks up the overrides set on the subflow, but not those on the main flow. So the behaviour is pretty much as before, only now with overrides in stead of the hardcoded values:
| Code: |
| File node ''SFTP'' in message flow ''TransferFile''. The remote user identifier supplied as ''SFTP_IDENTITY_SUB_OVERRIDE'' is invalid. |
| Vitor wrote: |
| Include the node name in the override. |
I've seen the examples in the docs, but haven't put much time in playing with them yet; setting the properties on the file node directly would be suboptimal anyway, since it would require manual actions in the delivery pipeline.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Wed Aug 14, 2019 12:45 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
To know exactly what properties to set, run mqsireadbar.
This will show you all the properties that can be changed via mqsiapplybaroverride....
Normally you would:
- promote the properties from the node to the (subflow) level
- promote the properties from the subflow node to the flow level
Very interesting if you have multiple occurrences of the subflow and all occurrences have the same value. You'd set it only once on the main flow.
- set the properties at the flow level.
Alternatively you can try and keep them at the subflow level and set them via:
flow#subflownodename#propertyname = value
Have fun
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Thu Aug 15, 2019 12:22 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| fjb_saper wrote: |
Normally you would:
- promote the properties from the node to the (subflow) level
- promote the properties from the subflow node to the flow level
- set the properties at the flow level.
|
Yep, this is what I'd expect. I'm quite miffed by the fact that the broker doesn't pick up on the properties at flow level, or flow subflow-node level at deployment.
| fjb_saper wrote: |
| Very interesting if you have multiple occurrences of the subflow and all occurrences have the same value. |
The actual application doesn't. The intention was to create a subflow for use in interface applications, where the guideline is: one source and one target per application. The TestApp that I wrote to test this phenomenon has just the mainflow and the subflow; it doesn't even use a library (although I tested it with a library as well). Colleagues of mine tried the same thing, and couldn't get it to work.
With a promoted property on a Compute node in a subflow, it works as expected: I promote the Datasource to the subflow, drag the subflow into the main flow, set the override on the subflow-node in the mainflow, and it's picked up by the compute node, no problem. Do the same thing with the Security Identity on a FileOutput node, and it doesn't pick up anything beyond the subflow (haven't gotten around testing other properties yet).
I still haven't found any documentation on this behaviour that tells me I should expect anything other than this to work. I could not find any APARs that exactly match this. I'm thinking it's time for a PMR.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| fjb_saper |
| Posted: Thu Aug 15, 2019 5:32 pm Post subject: |
|
|
Grand High Poobah
Joined: 18 Nov 2003
Posts: 20756
Location: LI,NY
|
| Gralgrathor wrote: |
| fjb_saper wrote: |
Normally you would:
- promote the properties from the node to the (subflow) level
- promote the properties from the subflow node to the flow level
- set the properties at the flow level.
|
Yep, this is what I'd expect. I'm quite miffed by the fact that the broker doesn't pick up on the properties at flow level, or flow subflow-node level at deployment.
| fjb_saper wrote: |
| Very interesting if you have multiple occurrences of the subflow and all occurrences have the same value. |
The actual application doesn't. The intention was to create a subflow for use in interface applications, where the guideline is: one source and one target per application. The TestApp that I wrote to test this phenomenon has just the mainflow and the subflow; it doesn't even use a library (although I tested it with a library as well). Colleagues of mine tried the same thing, and couldn't get it to work.
With a promoted property on a Compute node in a subflow, it works as expected: I promote the Datasource to the subflow, drag the subflow into the main flow, set the override on the subflow-node in the mainflow, and it's picked up by the compute node, no problem. Do the same thing with the Security Identity on a FileOutput node, and it doesn't pick up anything beyond the subflow (haven't gotten around testing other properties yet).
I still haven't found any documentation on this behaviour that tells me I should expect anything other than this to work. I could not find any APARs that exactly match this. I'm thinking it's time for a PMR. |
Wouldn't the security idendity be part of the 'configurable service' and as such be set either by the configurable service (IIB) or a policy (ACE)???
_________________
MQ & Broker admin |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Thu Aug 15, 2019 11:06 pm Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| fjb_saper wrote: |
| Wouldn't the security idendity be part of the 'configurable service' and as such be set either by the configurable service (IIB) or a policy (ACE)??? |
That would be one way of configuring the node. But I don't see how I could have a generic subflow for dealing with the target connections if the node itself is configured by a configurable service.
It turns out, though, that the subflow does pick up on the overrides on the main flow.
If I create the identity SFTP_IDENTITY_SUB, and then deploy, the broker starts complaining about the identity SFTP_IDENTITY_MAIN missing. If I run the application with the identity overridden by the main flow, it uses the identity configured in the main flow, not the subflow.
So this may be related to another APAR:
http://www-01.ibm.com/support/docview.wss?uid=swg1IT18907
The broker apparently checks the identity configured on the node/subflow during roll-out, before it does anything with the main flow and its overrides. If I have time I'll see if I can run a trace during deploy, see what's happening.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| abhi_thri |
| Posted: Thu Aug 15, 2019 11:57 pm Post subject: |
|
|
Knight
Joined: 17 Jul 2017
Posts: 516
Location: UK
|
| Gralgrathor wrote: |
| fjb_saper wrote: |
| Wouldn't the security idendity be part of the 'configurable service' and as such be set either by the configurable service (IIB) or a policy (ACE)??? |
That would be one way of configuring the node. But I don't see how I could have a generic subflow for dealing with the target connections if the node itself is configured by a configurable service. |
hi...you can achieve this by overriding the 'Server and port' parm to different config service names, i.e the file handling subflow should promote 'Server and port' among other params and in the main flow you can set unique config service name per implementation which should then pick up the security identify configured for that config service. |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Fri Aug 16, 2019 1:51 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
| abhi_thri wrote: |
| hi...you can achieve this by overriding the 'Server and port' parm to different config service names, i.e the file handling subflow should promote 'Server and port' among other params and in the main flow you can set unique config service name per implementation which should then pick up the security identify configured for that config service. |
Thanks - but I'm guessing that would result in the same error during deploy: the broker checks the value set on the subflow first, and if that cfg service/identity doesn't exist, the deploy fails. I'll give it a try, though.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| Gralgrathor |
| Posted: Wed Jun 17, 2020 1:55 am Post subject: |
|
|
Master
Joined: 23 Jul 2009
Posts: 297
|
It was a matter of RTFM after all:
When applying overrides on subflows contained in libraries, you need to provide the -y parameter with mqsiapplybaroverrides. If you don't, the command will create overrides in the BAR that are seemingly valid, but don't get picked up.
Since I was planning on using a single properties file for the overrides in my automated deployment process, I modified my script to split up the overrides file based on property keys and use one mqsiapply per library affected.
_________________
A measure of wheat for a penny, and three measures of barley for a penny; and see thou hurt not the oil and the wine. |
|
| Back to top |
|
|
| Vitor |
| Posted: Wed Jun 17, 2020 4:44 am Post subject: |
|
|
Grand High Poobah
Joined: 11 Nov 2005
Posts: 26093
Location: Texas, USA
|
Thanks for posting the final outcome for the benefit of future readers
_________________
Honesty is the best policy.
Insanity is the best defence. |
|
| Back to top |
|
|
|
|
