All virtual clients do not run for entire duration of test

dgibb

Hi,

I've observed some strange behaviour when running a single-actor test with 5 virtual clients. The test has 5 rows of state information with a single column "Username", containing the data "alan", "bob", "charlie", "dave", "eddie".

I set the Ratio to 1 for the single script, specify "simple test with a maximum of 5 virtual clients", and run it for 1 minute. I expect to see 5 virtual clients running for 1 minute and each row of state information should get used roughly the same number of times. Here is my script:

Public Class BrowserClient2
Inherits Control.VirtualClient

Protected Overrides Sub Run()

Dim username As String = Me.StateInformation.Get("Username")

Me.LogToFile("Username: " & username)

If username.StartsWith("a") Then
Me.BeginPage("USER A")
ElseIf username.StartsWith("b") Then
Me.BeginPage("USER B")
ElseIf username.StartsWith("c") Then
Me.BeginPage("USER C")
ElseIf username.StartsWith("d") Then
Me.BeginPage("USER D")
ElseIf username.StartsWith("e") Then
Me.BeginPage("USER E")
End If

WebClient.Get("http://localhost/myapplication/styles.css")

If username.StartsWith("a") Then
Me.EndPage("USER A")
ElseIf username.StartsWith("b") Then
Me.EndPage("USER B")
ElseIf username.StartsWith("c") Then
Me.EndPage("USER C")
ElseIf username.StartsWith("d") Then
Me.EndPage("USER D")
ElseIf username.StartsWith("e") Then
Me.EndPage("USER E")
End If

End Sub

End Class

As you can see I have crafted this so that I'm making an extremely trivial web request (eliminating any effect caused by my application), and so that the results clearly show the distribution of state information usage.

Here are the results in "Summary Information by Page":

# Page Time to connect Time to first byte Time to last byte Count
1 USER A 10 14 14 181
2 USER B 11 14 16 166
3 USER C 8 11 11 374
4 USER D 5 7 7 1186
5 USER E 3 3 3 12899

As you can see, the state information row containing the username "eddie" has been used vastly more times than any other, and the overall distribution is very uneven. Examination of the log file reveals why this is so - after about 10 seconds into the 1 minute test, the only virtual client that is still running is Virtual Client 4. All of the others have simply stopped.

I've been seeing this behaviour in many of the tests I've been running, which has spurred me to create this example in the hope that you'll be able to reproduce it.

If you wish I can supply the trace log file by email. I'm using ANTS Load 1.60 Build 451, the edition which allows only 25 virtual clients.

Regards
David

dgibb

I discovered the solution to this problem myself.

The problem was that my tests have been using up too much of the resources on the test machine. ANTS Load was dropping off the virtual clients because it detected this, and marked the tests as *FAILED*.

When I introduced a call to RandomSleep(100, 200) in the script, the CPU usage was kept down and all virtual clients completed the test, with a completely even distribution of state data usage.

Hope this helps anyone who's been having similar problems.

Brian Donahue

Hi dbibb,

Thanks, I was about to suggest monitoring the situation by adding the ANTS:Host/Virtual Clients/_Total performance counter to the test. That would show you if Virtual Clients were dropping away due to program error. Switching SSL tests back to plain HTTP may help alleviate this, at secure requests require a tremendous amount of overhead compared to non-secure ones.