Waiting 30 seconds for package list...

mwoffenden

I know that a recent release added the pseudo-drop-down to list the themes.

However this is not really any better, as there is a delay of 30 seconds or more before the package list shows up. And this is on a fast development machine.

So the initial issue, which was waiting a really long time for the list to be generated, is not at all solved. We're still waiting a long time.

Please consider taking a look at the code, since there's no valid reason it should take more than 1/2 second to display the list of packages.

Thank you in advance.

csmith

Dear Mr Woffenden

Our testing of this feature, on a range of hardware, found that the list could sometimes take 2-3 seconds to populate. We certainly did not see a delay of 30 seconds at any time.

In terms of "what is happening behind the scenes" - the reason for a delay of any kind is that, following a click on the drop-down button, the client browser is making an Ajax request back to the server, which in turn is polling the nuget feed for the latest package names.

So, your delay may be down to the speed of the connection between the Deployment Manager server and Nuget server (if you are using an stand-alone Nuget server) or the number of packages in DM's embedded nuget server if you are using that (the embedded feed is not performant when it contains a very high number of packages).

Regarding your comment on the ability of the programmers working on the product, let me assure you that in my 15 years experience of working in a technical role in the software industry, I have never worked with a more talented bunch of software engineers than I do now in the Deployment Manager Team.

Regards
Chris Smith

mwoffenden

csmith wrote:

...your delay may be down to the speed of the connection between the Deployment Manager server and Nuget server

Thanks for your note. I did not mean to say that your team is not talented, I am sure they are extremely so.

However, in my case, network latency is not part of the equation, as the DM user interface and the packages are both on the same machine. And I am loading less than 20 packages.

What else could be causing this?

lee5i3

I occasionally get the same problem, I just try again and it works..

I'm not too sure what could cause this either, especially since I've had timeout even using a localhost nuget server

mwoffenden

lee5i3 wrote:

I occasionally get the same problem, I just try again and it works

I do see that after the first "30 second" wait, the package list loads instantly. So there is some sort of caching going on. But it still makes no sense that it would take so long to load a list of 20 items from my local C drive. I'm hoping the deployment team can shed some light on this.

lee5i3

This is what I get when it times out..
This is using TeamCity and the DeploymentManager.exe


[11:04:05][Step 5/5] Finding project: MyProject
[11:04:05][Step 5/5] Handshaking with Deployment Manager server: http://localhost:7070/api
[11:04:25][Step 5/5] Handshake successful. Deployment Manager version: 2.1.8.8; API version: 1.0.0
[11:04:25][Step 5/5] Finding steps for project...
[11:04:26][Step 5/5] Getting package versions for each step...
[11:04:32][Step 5/5] Using latest version (13.08.14.335117) of package MyProject.Database.
[11:05:02][Step 5/5] The operation has timed out
[11:05:02][Step 5/5] You can change the timeout period using the --httpConnectTimeout argument.
[11:05:02][Step 5/5] Process exited with code 1

My project only has 2 steps, website and database, yet still times out

second time I run it.. works fine, so I'm had to add a retry to TeamCity so it'll deploy

csmith

Hi

We've discussed this in the team and we think the pause if probably because the embedded nuget server has been 'unloaded' by IIS and needs to be reloaded before we receive the package list. This reload causes the delay.

More background
The embedded nuget server is hosted in IIS as a sub-site of the main "Deployment Manager Portal" site. Both the portal and the nuget server share an IIS Application Pool also called "Deployment Manager Portal". By default, this application pool has an idle time-out setting of 20 minutes. So, if both the nuget server and portal are not accessed for 20 minutes, the pool is unloaded from memory. Accessing either the nuget server or the portal will then mean that the sites have to be reloaded (causing a delay).

What can be done?
Firstly, we should confirm this is the cause! Michael, Lee - when you see this behaviour, what happens if you try to repeat the process within 20 minutes? Does the drop-down populate more quickly?

If it does, extending the timeout on the application pool is likely to impove things (in terms of how often you see the behaviour). To do this:

*Access IIS Manager on the machine running DM * Select the Application Pools node * Right-click on the "Deployment Manager Portal" node * Select Advanced Settings * Change the "Idle Time-out" setting under Process Model to something like 10 hours

If this is still a significant issue for users we could look at forcing the embedded nuget server to reload whenever the DM Portal reloads. Another approach is for users to install/use an enterprise scale nuget repository, like MyGet or NuGet Gallery.

Best regards,
Chris

lee5i3

For me, it work the instant I try again..

I'll try it 20 mins later to see it works

.. I might just set it to 0 so it never idles, especially since it deploys to my development box which has a nightly server restart

lee5i3

I don't mind if it just takes longer since its starting up.., but a timeout failure sucks because it fails the deploy.. anyway to extend the timeout?

commandline mentions using --httpConnectTimeout but can't seem to get that to work

lee5i3

It deploy successful approx 25 mins later...




I'll try again in an hour or 2 to see if it'll fail then

lee5i3

2 hours later..

It deploys fine, so it seems like this is only happening on REALLY long wait, and I am not touch deployment manager in between.

This is also on the default settings, did not touch the app pool

justin.caldicott

Just to add my 2ct to this.

I saw very similar build results in TeamCity. I was calling DeploymentManager.exe to create the release and deploy. The process was returning error code 400, after two minutes of trying to find the latest version number for packages. Lee, what do your TeamCity logs look like? Similar?

I fixed it by manually removing older packages from the built-in feed. I'd been creating a lot of them from the automatic commit trigger. @Michael, you mentioned having 20 packages in the feed. Are there more than that if you include all of the different versions of the packages?

Deployment Manager's built-in feed is using NuGet.Server, which doesn't have any indexing. If you need a faster feed, then you might look at using either ProGet, MyGet or the NuGet Gallery codebase.

Personally I really like using the built-in feed, as I like to keep deployment and dependency packages separate. (I'm using NuGet packages for internal dependencies too.) Also, the Deployment Manager server feels like the right place to keep these packages, as it's the only consumer. So my vote would be for adding indexing to the Deployment Manager built-in feed, so that it "just works", even with larger numbers of packages.

Thoughts?

mwoffenden

I guess I'm confused about all the different ***get packages and options. Anyway, here is my experience to date, from what I can remember.

When I first installed DM I could not get the built in feed to work. So, I installed ProGet and got that to work. ProGet was easy enough to install but still has an IIS dependency, I think, so I am not sure if IIS is contributing to the issue.

I will say that opening the list of packages from the NuGet Package Explorer takes the same amount of time to open (about 30 seconds.) Again, after the first "caching" of the feeds, the list subsequently opens in an instant.

So, ProGet seems not be be a solution to the "wait" issue.

If anyone has a solution I'd love to hear about it.

lee5i3

justin.caldicott wrote:

Just to add my 2ct to this.

I saw very similar build results in TeamCity. I was calling DeploymentManager.exe to create the release and deploy. The process was returning error code 400, after two minutes of trying to find the latest version number for packages. Lee, what do your TeamCity logs look like? Similar?

I fixed it by manually removing older packages from the built-in feed. I'd been creating a lot of them from the automatic commit trigger. @Michael, you mentioned having 20 packages in the feed. Are there more than that if you include all of the different versions of the packages?

Deployment Manager's built-in feed is using NuGet.Server, which doesn't have any indexing. If you need a faster feed, then you might look at using either ProGet, MyGet or the NuGet Gallery codebase.

Personally I really like using the built-in feed, as I like to keep deployment and dependency packages separate. (I'm using NuGet packages for internal dependencies too.) Also, the Deployment Manager server feels like the right place to keep these packages, as it's the only consumer. So my vote would be for adding indexing to the Deployment Manager built-in feed, so that it "just works", even with larger numbers of packages.

Thoughts?

That would explain a lot, I think for my situation, I have no problem removing older packages, as this is happening on a commit, and I'm sure my directory is getting pretty big.

I will just add an extra step to remove anything older than a week