Details
-
Type:
Sub-task
-
Status:
Closed
-
Priority:
Medium
-
Resolution: Fixed
-
Affects Version/s: 3.1.0 GA
-
Fix Version/s: 3.2.0 GA
-
Component/s: Job, Spoon / User Interface, Transformation
-
Labels:None
-
Customer Case:
Description
This is also one that comes back from time to time.
As proposal:
- Named parameters of key/value pairs. That you can supply to pan, kitchen, spoon without having to change scripts. the type of the key/value would be String/String. In the command line tools the arguments would like:
pan.bat .... /param dbname=ADS_P1 /param direction=OUT
- A parameter in a job is described as
key: name of the parameter
description: description of the parameter
the type of a parameter is always string (current variables are also only string)
- The parameters would be injected as variables right before the startup and would overwrite variables if they already existed (parameters have priority over existing variables).
- When you execute a job or transformation via spoon a check should be done whether they have named arguments. If they do, the dialog should be changed to allow people to enter values for them (in place of the current positional arguments).
- If you start a sub transformation without named parameters you can still used the variables defined in the top execution. If the subtransformation has named parameters, the job entry dialog should "ask" for them. If they are not set parameters are assumed to be empty and will overwrite the existing variable in sub transformations.
For an example of use in subtransformations:
Suppose the top job has as named parameter "dbname", when a subtransformations wants to take over the dbname it can either:
a) define its own named parameter dbname (or another name) and define it as "key dbname, value ${dbname}" in the sub transformation job entry
b) don't define named parameters and use ${dbname} as the top execution will set the variable defined as parameter.
If however you use method a) and ${dbname} would not have been set dbname will be set to "" a.k.a. null
- On the command line no check will be done whether the names of the parameters match the names of the job being executed.... unless an extract parameter is specified called "strictparam". With strictparam on a check will be done whether all parameters defined on the job/transformation exist on the command line (and vica versa). Without strictparam it allows people to run transformations and supply more arguments than required for some transformation (think of it as optional parameters): as use-case e.g. 1 script running transformations with some username and password as parameters,. but 1 or 2 jobs don't need those username and password (but you would still like to run them using the same script)
Only the parameters which are define on job/transformation will be set up as variables. I don't use strictparam, I have a job with dbname as parameter, but I supply -param dbname=ADS -user=sipsch, ${user} will not have been setup as variable afterwards.
The advantages:
- Most people are used to named arguments from other ETL tools
- If you want to use parameters as variables it avoids 1 extra layer of setting up the variables via a get system info/set variable step.
- Named parameters can coexist with the current positional parameters
Other things to change
- Get System info to be able to do the same for named parameters as positional parameters (getting them in a row)
- transformation and job job entry which would then also need to be able to pass named parameters.
- Maybe the start dialogs in spoon and the arguments dialog.
As proposal:
- Named parameters of key/value pairs. That you can supply to pan, kitchen, spoon without having to change scripts. the type of the key/value would be String/String. In the command line tools the arguments would like:
pan.bat .... /param dbname=ADS_P1 /param direction=OUT
- A parameter in a job is described as
key: name of the parameter
description: description of the parameter
the type of a parameter is always string (current variables are also only string)
- The parameters would be injected as variables right before the startup and would overwrite variables if they already existed (parameters have priority over existing variables).
- When you execute a job or transformation via spoon a check should be done whether they have named arguments. If they do, the dialog should be changed to allow people to enter values for them (in place of the current positional arguments).
- If you start a sub transformation without named parameters you can still used the variables defined in the top execution. If the subtransformation has named parameters, the job entry dialog should "ask" for them. If they are not set parameters are assumed to be empty and will overwrite the existing variable in sub transformations.
For an example of use in subtransformations:
Suppose the top job has as named parameter "dbname", when a subtransformations wants to take over the dbname it can either:
a) define its own named parameter dbname (or another name) and define it as "key dbname, value ${dbname}" in the sub transformation job entry
b) don't define named parameters and use ${dbname} as the top execution will set the variable defined as parameter.
If however you use method a) and ${dbname} would not have been set dbname will be set to "" a.k.a. null
- On the command line no check will be done whether the names of the parameters match the names of the job being executed.... unless an extract parameter is specified called "strictparam". With strictparam on a check will be done whether all parameters defined on the job/transformation exist on the command line (and vica versa). Without strictparam it allows people to run transformations and supply more arguments than required for some transformation (think of it as optional parameters): as use-case e.g. 1 script running transformations with some username and password as parameters,. but 1 or 2 jobs don't need those username and password (but you would still like to run them using the same script)
Only the parameters which are define on job/transformation will be set up as variables. I don't use strictparam, I have a job with dbname as parameter, but I supply -param dbname=ADS -user=sipsch, ${user} will not have been setup as variable afterwards.
The advantages:
- Most people are used to named arguments from other ETL tools
- If you want to use parameters as variables it avoids 1 extra layer of setting up the variables via a get system info/set variable step.
- Named parameters can coexist with the current positional parameters
Other things to change
- Get System info to be able to do the same for named parameters as positional parameters (getting them in a row)
- transformation and job job entry which would then also need to be able to pass named parameters.
- Maybe the start dialogs in spoon and the arguments dialog.
If you want to run transformations save in XML using variable to define the path of the files, it will be possible with named parameters ! Actually we must create a transformation to define path variable, so it's impossible to use variable to define the path of this transformation :(
Another solution would be to have an access to command line arguments in a job and not only in a transformation.