SMA – Time to Execution Performance

The following TechNet blog post from the Orchestrator team can be used for measuring stress and load performance of your SMA environment: Configure SMA for Optimum Performance

This got me wondering what other ways I could improve performance. I noticed one key area in my scripts was the time while the script was in the “starting” phase. We know that SMA compiles the workflows at run time…so that was my target area.

First I needed a way to measure how long a workflow takes to queue, compile and execute…this is messy but it works:

$i = 0
Do{
    $i++
    write-host $i
    Measure-Command{
        $strJobID = start-smarunbook -webserviceendpoint "https://SMASERVERHERE" -name "dev_mdtestarea"		
        do{
            $objJob = get-smajob -webserviceendpoint "https://SMASERVERHERE" -id $strJobID -erroraction 'stop'
        }While($objJob.JobStatus -ne "Completed")
    }
    start-sleep 10
    write-host "-----------------"
}While($i -lt 11)

so….why 11 times? Really, its just a number not too little and not too much.

Then, I need a baseline workflow ….this one is about as basic as it gets:
Test 1:

dev_mdtestarea{
    $test="1"
    $test="1"
}

In the 11 iterations, this script took an average of 3.64 Seconds to execute in my environment. Definitely not quick, but ok considering we are using a 2 servers to get the job done (web service end point and runbook server).

Lets add in some nested workflows:
Test 2:

dev_mdtestarea{
    dev_mdtestarea2
    dev_mdtestarea3
}
dev_mdtestarea2{
    $test="1"
}
dev_mdtestarea3{
    $test="1"
}

This took an average of 13.898 seconds to execute in my environment. This is an increase of 5.129 seconds PER nested workflow.

Lets see what happens when we add inline scripts:
Test 3:

dev_mdtestarea{
    dev_mdtestarea2
    dev_mdtestarea3
}
dev_mdtestarea2{
    InlineScript{$test="1"}
}
dev_mdtestarea3{
    InlineScript{$test="1"}
}

This took an average of 16.85 seconds to execute in my environment. This is an increase of 1.476 seconds PER nested inline script.

I am assuming the increase in runtime is from the compiling engine. Lets try Measure-Command from within SMA:

Within SMA Test 1: (run from within SMA…no start-smarunbook command)

workflow DEV_MDTestArea
{   
    Measure-Command{
        $test = "1"
        $test = "2"
    }
}

TotalSeconds : 0.0032354
Within SMA Test 2: (run from within SMA…no start-smarunbook command)

workflow DEV_MDTestArea
{   
    Measure-Command{
       Inline-Script{
         $test = "1"
       }
       Inline-Script{
         $test = "3"
       }
    }
}

TotalSeconds : 0.3801307

You can see that the inline-script took a little longer, but no where close 1.47 seconds per inline script.
I tried testing nesting nested workflows however this resulted in SMA throwing an error “Root element is missing”. Likely because it is converting Measure-Command as an inline-script. Although these tests are not perfect, it has convinced me that with SMA you need to keep in mind the performance impact not just when running, but also compiling scripts.

So what can we take from this? Best practice would dictate to avoid long scripts that aren’t reusable…I personally like to break my workflows into reusable functions and then use a control script to tie it all together. Think for example of a simple patching workflow (something commonly automated in Orchestrator….probably less common in SMA). I would want: 1. Workflow to get the missing updates 2. Workflow to compare each update against a blacklist 3. Workflow to install an update 4. Workflow to Reboot a computer 4. Workflow Email a report 5. Controller workflow. In a simple scenario such as this, I can expect to add close to 25 seconds of execution time for compiling just based on the nested workflows alone. In the real world each line of code contributes to the overall compile time. In a situation where you are patching clusters or groups of servers the compiling time can quickly reach more than 1 minute. Keep in mind then that the length and content will affect the time it takes to start your script.

Of note, here are the results for running these in Powershell:

Native Powershell Test 1:

Workflow DEV_MDTEstArea1{
    $test = "1"
    $test = "2"
}
$sum = 0
Foreach($i in 0..10){$i++; $sum += (Measure-Command{DEV_MDtestArea1}).TotalSeconds}
$sum / 11

Output: 0.135524481818182 (seconds)

Native Powershell Test 2:

Workflow DEV_MDTEstArea1{
DEV_MDTEstArea2
DEV_MDTEstArea3
}
Workflow DEV_MDTEstArea2{
    $test = "1"
}
Workflow DEV_MDTEstArea3{
    $test = "2"
}
$sum = 0
Foreach($i in 0..10){$i++; $sum += (Measure-Command{DEV_MDtestArea1}).TotalSeconds}
$sum / 11

Output: 0.354800945454545 (seconds)

Native Powershell Test 3:

Workflow DEV_MDTEstArea1{
    DEV_MDTEstArea2
    DEV_MDTEstArea3
}
Workflow DEV_MDTEstArea2{
    InlineScript{$test = "1"}
}
Workflow DEV_MDTEstArea3{
    InlineScript{$test = "2"}
}
$sum = 0
Foreach($i in 0..10){$i++; $sum += (Measure-Command{DEV_MDtestArea1}).TotalSeconds}
$sum / 11

Output: 0.506956245454545 (seconds)

Obviously, I don’t understand something of how SMA works….
I would expect the difference in timing to be the same regardless of where the script is executed. However it is evident that compiling process in SMA isn’t as efficient as the compiling process in powershell. The end result is somewhat obvious: Running workflows in Powershell is much more efficient than running workflows in SMA.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s