SMA – Calling child runbooks from other child runbooks launched from a foreach -parallel loop may not return reliable results

Consider the following 3 runbooks: A:

workflow DEV_MDTestArea
{    
    $i = 0
    Do{
        $Tests = @("vm-md-srv2008r2","sma-imagetest1")
        ForEach -Parallel ($Test in $Tests){
           DEV_MDTestArea2 -Computer $Test
        }
        $i++
    }While($i -lt 20)
}


B:

workflow DEV_MDTestArea2
{
    param(
        [parameter(Mandatory=$false)]
        [String]$Computer
    )
    
    $hostname = Dev_MDTestAreaHostname -Computer $Computer
    $strOutput = "${Computer}: ${hostname}"
    write-verbose $strOutput
    
}

C:

workflow Dev_MDTestAreaHostname
{
    param(
        [parameter(Mandatory=$false)]
        [String]$Computer
    )
    Start-Sleep 14
    $Hostname = (gwmi -class Win32_ComputerSystem -Computer $Computer).Name
    $Hostname
}

It appears that if two child runbooks (b) with the same parent call another child runbook (c) and both instances of the child runbook (c) run at the same time only one (b) will receive the return data. As a result, when runbook a is run it returns incorrect data:

Verbose	9/11/2014 7:44	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:SMA-imagetest1: SMA-IMAGETEST1 VM-MD-SRV2008R2
Verbose	9/11/2014 7:44	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:vm-md-srv2008r2:
Verbose	9/11/2014 7:47	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:SMA-imagetest1:
Verbose	9/11/2014 7:48	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:vm-md-srv2008r2: SMA-IMAGETEST1 VM-MD-SRV2008R2
Verbose	9/11/2014 7:44	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:vm-md-srv2008r2: VM-MD-SRV2008R2 SMA-IMAGETEST1
Verbose	9/11/2014 7:44	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:SMA-imagetest1:
Verbose	9/11/2014 7:48	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:SMA-imagetest1:
Verbose	9/11/2014 7:48	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:SMA-imagetest1:
Verbose	9/11/2014 7:44	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:vm-md-srv2008r2: VM-MD-SRV2008R2 SMA-IMAGETEST1
Verbose	9/11/2014 7:44	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:SMA-imagetest1:
Verbose	9/11/2014 7:48	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:vm-md-srv2008r2: SMA-IMAGETEST1 VM-MD-SRV2008R2
Verbose	9/11/2014 7:48	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:SMA-imagetest1:
Verbose	9/11/2014 7:45	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:SMA-imagetest1:
Verbose	9/11/2014 7:45	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:vm-md-srv2008r2: SMA-IMAGETEST1 VM-MD-SRV2008R2
Verbose	9/11/2014 7:45	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:vm-md-srv2008r2: VM-MD-SRV2008R2 SMA-IMAGETEST1
Verbose	9/11/2014 7:45	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:SMA-imagetest1:
Verbose	9/11/2014 7:48	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:vm-md-srv2008r2: SMA-IMAGETEST1 VM-MD-SRV2008R2
Verbose	9/11/2014 7:48	8d2d3588-5d57-4c8d-98e6-0da5ac15b571:[localhost]:vm-md-srv2008r2: VM-MD-SRV2008R2 SMA-IMAGETEST1

Note that the bug does not appear if I remove the Start-Sleep line. By adding the Start-Sleep line, I am ensuring that both instances of runbook C run simultaneously. I will be submitting this bug to Microsoft this morning and will try keep you posted.

Update after testing with Microsoft:

We have reproduced the issue using strictly the Powershell ISE which means that the bug is either contained in the Powershell Workflow engine (which translates the powershell to XML for the Windows Workflow Foundation) or in the Windows Workflow Foundation itself. I am attempting to get my case moved to the Powershell team so that we can take this further.

Here is a more simple example:

workflow Dev_MDTestAreaHostname
{
    param(
        [parameter(Mandatory=$false)]
        [String]$Number
    )
    Start-Sleep 14
    If($Number -eq "One"){
        return 1
    }
    If($Number -eq "Two"){
        return 2
    }
}
workflow DEV_MDTestArea2
{
    param(
        [parameter(Mandatory=$false)]
        [String]$Number
    )
     
    $returnvalues = Dev_MDTestAreaHostname -Number $Number
    $strOutput = "${Number}: ${returnvalues}"
    write-warning $strOutput
}
workflow DEV_MDTestArea
{    
    $i = 0
    Do{
        $Tests = @("One","Two")
        ForEach -Parallel ($Test in $Tests){
           DEV_MDTestArea2 -Number $Test
        }
        $i++
    }While($i -lt 5)
    
}
 
 
DEV_MDTestArea

This returns the unexpected results:

WARNING: [localhost]:One:
WARNING: [localhost]:Two: 1 2
WARNING: [localhost]:One:
WARNING: [localhost]:Two: 1 2
WARNING: [localhost]:One:
WARNING: [localhost]:Two: 1 2
WARNING: [localhost]:Two:
WARNING: [localhost]:One: 2 1
WARNING: [localhost]:Two:
WARNING: [localhost]:One: 2

Slightly modifying the code removing the “Parallel” switch as a sanity check:

workflow Dev_MDTestAreaHostname
{
    param(
        [parameter(Mandatory=$false)]
        [String]$Number
    )
    Start-Sleep 14
    If($Number -eq "One"){
        return 1
    }
    If($Number -eq "Two"){
        return 2
    }
}
workflow DEV_MDTestArea2
{
    param(
        [parameter(Mandatory=$false)]
        [String]$Number
    )
     
    $returnvalues = Dev_MDTestAreaHostname -Number $Number
    $strOutput = "${Number}: ${returnvalues}"
    write-warning $strOutput
}
workflow DEV_MDTestArea
{    
    $i = 0
    Do{
        $Tests = @("One","Two")
        ForEach ($Test in $Tests){
           DEV_MDTestArea2 -Number $Test
        }
        $i++
    }While($i -lt 5)
    
}

DEV_MDTestArea 
 

This returns the expected results:

WARNING: [localhost]:One: 1
WARNING: [localhost]:Two: 2
WARNING: [localhost]:One: 1
WARNING: [localhost]:Two: 2
WARNING: [localhost]:One: 1
WARNING: [localhost]:Two: 2
WARNING: [localhost]:One: 1
WARNING: [localhost]:Two: 2
WARNING: [localhost]:One: 1
WARNING: [localhost]:Two: 2

Update from Microsoft
This has been confirmed as a bug with an expected fix in the next version of Powershell (5.0)

Update with a work around
So, if you are in a situation where you need to call multiple instances of child workbooks and wait for the group of them to complete (basically a foreach -parellal), you are stuck with using the Start-SMARunbook option which will start the runbook in a seperate instance of powershell. The downside is that you are starting another workflow engine as well, which taxes your server….It is also much slower, but gets the job done:

workflow DEV_MDTestArea
{
    $arrJobList = @()
    $strWebServiceEndPoint = 'https://YourSMAWebServer'
    $Tests = @("one","two","three")
  
    ForEach ($Test in $Tests){
           $strJob = Start-SMARunbook -Name DEV_MDTestArea2 -Parameters @{"Number"=$Test} -WebServiceEndPoint $strWebServiceEndPoint 
           $arrJobList += $strJob
    }    

    $ShouldLoop = $true
    $arrFinishedJobs = @()

    While($shouldLoop){
        $shouldLoop=$false
        ForEach($strPSJob in $arrJobList){
            If($strPSJob -notin $arrFinishedJobs){
                $Job = Get-SmaJob -ID $strPSJob -WebServiceEndPoint $strWebServiceEndPoint
                $status = $job.JobStatus
                $boolJobRunning = (($status -ne "Completed") -and ($status -ne "Failed") -and ($status -ne "Suspended") -and ($status -ne "Stopped"))

                If(!($boolJobRunning)){
                    $arrFinishedJobs += $strPSJob
                    $strOutPut = (Get-SmaJobOutput -ID $strPSJob -Stream Any -WebServiceEndPoint $strWebServiceEndPoint).StreamText
                    $seperator = ":[localhost]:"
                    ($strOutPut -split $seperator, 0, "simplematch")[1]
                }else{
                    $shouldLoop = $true
                }
            }
        }
        Start-Sleep 2
    }
}

For more information on the pros/cons of calling the runbook with the Start-SMARunbook command vs implicitly referencing the runbook, see here:
http://blogs.technet.com/b/orchestrator/archive/2014/01/10/sma-capabilities-in-depth-runbook-input-output-and-nested-runbooks.aspx

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s