State Information - not all users used

Ian1971

I have set up a load test with 4 actors each with identical state information consisting of 5 states.
I was expecting that each actor would cycle through the states in the state information as the test progresses so at the end I would have had each actor run under the 5 different states (assuming the test was long enough).
What I actually see is that some states are called overly many times and some not at all.
I have tried running a long test to see if all the states are used but they aren't.
After my test, each script (actor) had been run at least 50 times but only 2 or 3 of the states were ever used.

Can anyone explain this behaviour?

Brian Donahue

Hi Ian,

Please have a look at the 'summary information by script duration' section of the ANTS Load report? This lists how many times each script had run during a 'multiple actor' test. If the script had only run twice, then it would have only used the first two rows of state information.

Ian1971

Here is the results in that section
- Summary information by script duration
This section shows you the average script duration for this test.

Note that all timings are in milliseconds

# Script Duration Count
1 PartTrackingTester 11356 50
2 ProductTrackingTester 18519 48

So, that seems to indicate the each script was run about 50 times

Any ideas?

Brian Donahue

Hi Ian,

Maybe the requests with the state information that you're supplying are being rejected by the server. Can you use the LogToFile() method to log the state information's value to the ANTSTraceLog?

For instance:

Dim username As String=StateInformation.Get("username")
Dim password As String=StateInformation.Get("password")
Me.LogToFile("User Name: " &username &" Password: " &password)

Ian1971

In order to rule out our application I have created a test with 4 actors each calling scripts that only calls the code you posted to output the username and password to the logfile.
What I have found is that ANTS is still behaving strangely.
With 1 actor and 4 users of state info I found that all 4 users were used with a good spread between each user.
With 2 actors and 4 users of state info I found that only 3 users were ever used, two of those were used a lot and 1 rarely
With 3 & 4 actors and 4 users of state info I found that only 2 users were ever used.

I ran my tests for about 1/2 min each time which resulted in each script being called about 30 times so I would have expected to see all the users used.

As well as this I noticed that some actors are called more than others e.g.
# Script Duration Count
1 NewClient1 999 27
2 NewClient2 999 27
3 NewClient3 1000 23
4 NewClient4 1000 46 <--much higher than the others

I can send you my ANTS project file if you think it will help

Regards
Ian

Brian Donahue

Hi Ian,

There is also (coincidentally?) a similar thread which has just opened up:

http://www.red-gate.com/MessageBoard/vi ... .php?t=906

There must be some issue with the 'Load runner' that is killing all but the last virtual client. I've forwarded this to the programmers to have a look into.

Thanks for your patience.

Ian1971

Thanks Brian. I'm not sure if that other post is the same issue exactly as he seems to have solved it by adding randomsleep at the beginning of his script, which I already have. Also he was reporting an issue with 1 actor whereas my problem is with >1 actor. But please keep us posted

Cheers
Ian

Brian Donahue

Hi Ian,

I was going to ask, as I had in the other topic, for you to add the ANTS:Host performance counters to the load test and see if the discrepancy was happening because the virtual clients were exiting due to a lack of resources on the computer. Adding a randomsleep probably slowed down the number of requests so that the test could run without any virtual clients failing.

Ian1971

Just to confirm, I have re-tested and logged various performance counters and there is no indication of any resource issue on the machine. In fact resource use is very low.

Cheers
Ian

Brian Donahue

Hi Ian,

Thanks. Basically, you've worked around the problem by putting the script to sleep in a few places, and everything's pretty much working okay for you?

Ian1971

No. Not at all. I was just saying that the issue isn't caused by resource problem on my machine. I still cannot run multiple actor tests with several users of state info and get all the users used. My current workaround is to run each actor with the state info separately, which isn't really ideal as I wanted to pull all the tests into one set of results, and aso takes me a lot longer to do.

Brian Donahue

Hi Ian,

Ah, I see. I don't think your first idea is workable as each script (actor) is required to have its' own individual state information.

It may make things a bit easier if you use a spreadsheet as an authoritative source of data, then when you want to run a load test, delete the currently attached state information associated with each actor and re-load it by importing it from the spreadsheet (which you'd need to export to a .csv format first...).

Ian1971

I'm not sure I understand what you are saying.
I have a spread sheet with rows like this
username, password

It contains 5 rows for 5 unique users.
All I want to do is apply that set of 5 users to each actor, which I do via the state info dialog for each actor. So I click on that state info button for each actor and upload the spreadsheet.
I then find that if I have more than 1 actor not all users are used.

Are you saying that this won't work and that each actor has to have 5 completly different rows of state info? if so that doesn't make much sense to me.
Do you want me to send you my project?

Brian Donahue

Hi Ian,

Yep, that may be helpful. Can you please send the project to support at red-gate.com?

Ian1971

I've sent the file

Cheers
Ian

Brian Donahue

I've got it!

I'll see if it reveals anything.

Brian Donahue

Hi Ian,

From what I've seen from the behavior of ANTS Load using your test scripts, I can conclude that state information is being assigned at the start of each virtual client.

Try a simple test: one actor and one virtual client. The trace log looks like this:

VirtualClient 0 04/05/2005 16:45:11
User Name: user1 Actor: 1

VirtualClient 0 04/05/2005 16:45:12
User Name: user1 Actor: 1
<repeats>

Now try two virtual users in your test:

VirtualClient 0 04/05/2005 16:45:11
User Name: user1 Actor: 1

VirtualClient 1 04/05/2005 16:45:12
User Name: user2 Actor: 1
<repeats>

So if you have a database of state information for an actor, the virtual client will read a line from it and assign that line to the virtual client. Each time a new virtual client starts, it uses the same line of state information throughout the whole test until the virtual client's thread exits.

Hopefully this doesn't pose too much of a problem.

Ian1971

If I understand correctly, you are saying that each virtual client gets assigned a single row of state info for the entire duration of the test.

So with 4 rows of state info and 4 actors and 4 virtual clients the following would happen.
Virtual Client 1 starts. Is assigned to Actor 1. Gets first row of state info (user1). Executes script. Restarts with same row of state info
Virtual Client 2 starts. Is assigned to Actor 2. gets first row of state info (user1). Executes script. Restarts with same row of state info
Virutal Client 3 starts Is assigned to Actor 3. gets first row of state info (user1). Executes script. Restarts with same row of state info
Virutal Client 4 starts Is assigned to Actor 4. gets first row of state info (user1). Executes script. Restarts with same row of state info


This would explain what I am seeing. I am not convinced this is desirable behaviour though (not for me in any case). It doesn't seem intuitive to me. With this behaviour the only way to achieve what I need would be to have 16 virtual clients (each actor needs a virtual client for each line of state info) (or multiples of 16) so that each Actor gets assigned 4 virtual clients and each getting a row of state info.
If I go back to my original project where I had 6 actors and 5 rows of state info I would need to have 30 virtual clients (or multiples of 30) to see all my state info used in all actors. You can see how this multiplies up pretty rapidly.

I was expecting the following logic:
Virtual Client 1 starts. Is assigned to Actor 1. Gets first row of state info (user1). Executes script. Restarts. Gets next row of state info
Virtual Client 2 starts. Is assigned to Actor 2. gets first row of state info (user1). Executes script. Restarts. Gets next row of state info
Virutal Client 3 starts Is assigned to Actor 3. gets first row of state info (user1). Executes script. Restarts. Gets next row of state info
Virutal Client 4 starts Is assigned to Actor 4. gets first row of state info (user1). Executes script. Restarts. Gets next row of state info

This way in this example I only need 4 virtual clients to use all my rows of state info against every actor. And in the example of my original project with 6 actors I would only need 6 virtual clients.

Even better would be this logic:
Virtual Client 1 starts. Is assigned to first available actor. Gets first row of unused state info for that actor. Executes script. Restarts. Is assigned to next available actor and with that actors next row of state info
etc...

With this logic even with 1 virtual client it would use all actors and all state info (given enough time).

Does any of this make sense?

Ian

Brian Donahue

Hi Ian,

If we could figure out a way to kill the VirtualClient, it should start a new one in another thread and grab some different state information again. I don't know of a way to do that except to get the virtual client to throw an unhandled exception.