Options
[OT?] How to compress files on remote server?
swinghouse
Posts: 120 Bronze 2
Pardon me if this is a bit off topic, but I've run into the following problem:
I run NuGet Pack on my TeamCity server to create NuGet packages that Deployment Manager can consume. This generally works very well, but in one instance I need to package about 2 GB's worth of files (don't ask!) and now the going get tough...
The first error, reported by NuGet Pack in TeamCity, was quickly remedied:
I was prepared for this thanks to your description at the end of this document: http://www.red-gate.com/supportcenter/C ... t_packages
But the second error has stumped me so far:
I realize that this is more of a TeamCity rather than Red Gate issue, but I wonder if you have any tips on how to compress files on the source server before they are packed by the build server? This would certainly reduce the risk of the above type of errors and also quicken the build process.
Thanks!
/Mattias
I run NuGet Pack on my TeamCity server to create NuGet packages that Deployment Manager can consume. This generally works very well, but in one instance I need to package about 2 GB's worth of files (don't ask!) and now the going get tough...
The first error, reported by NuGet Pack in TeamCity, was quickly remedied:
Failed to publish artifacts. Artifact size exceeds maximum allowed size of 300000000 bytes
I was prepared for this thanks to your description at the end of this document: http://www.red-gate.com/supportcenter/C ... t_packages
But the second error has stumped me so far:
[pack] Exception of type 'System.OutOfMemoryException' was thrown.
I realize that this is more of a TeamCity rather than Red Gate issue, but I wonder if you have any tips on how to compress files on the source server before they are packed by the build server? This would certainly reduce the risk of the above type of errors and also quicken the build process.
Thanks!
/Mattias
Comments
The process of creating the NuGet package will compress the contents and even if you could compress the contents before packaging it would not reduce the size to a significant enough degree.
The pack and unpack process of the NuGet packages occurs in memory so even if you could package it, you may hit a problem on the agent when you attempt to unpack it.
The easiest option to solve this is to reduce the size of the package contents - but this doesn't sound possible from what you wrote. The other option is to take advantage of using multiple package steps in a Deployment Manager project to split up the deployment into more manageable chunks.
Best regards,
Andy
Thanks for the answer!
Actually, zipping the folder on the source server resultated in a zip file that TeamCity and Deployment Manager could manage! (The folder in question contains image files.)
Our solution is a bit convoluted but it does the trick:
1. A scheduled task on the source server runs a ps1 script that creates a zip file.
For this we use the Write-Zip cmdlet from PowerShell Community Extensions
2. A scheduled build on the TeamCity server creates a NuGet package.
3. Deployment Manager pulls this NuGet package from TeamCity.
4. Deployment Manager executes a PostDeploy.ps1 script that extracts the files on the target server.
Here we use the Expand-Archive cmdlet from PowerShell Community Extensions.