Build server pipeline failing after auto-update of SQL Change Automation (v 4.4.22194.30982)

Our build servers are failing after the auto-update of SQL Change Automation (v 4.4.22194.30982) this morning. In our Azure Dev Ops pipelines, we have a specific step for SCA (that is NOT failing). However, when the Visual Studio Build "Build solution" step is executed, the SCA build fails because te DBClean task cannot locate a specific version of Newtonsoft.Json. (See error from build below).

I know that I could rework my VS solution configuration to bypass the SCA build in the "Build solution" step, however, this feels like a bug in SCA and I really don't want to have to edit many, many, many solutions and pipelines to get this fixed. 

Anyone else out there have this issue? Any ideas for resolution (without reworking the solution Build Configuration)?

Thanks,
Pam


##[error]C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\SqlChangeAutomation\SqlChangeAutomation.Data.Schema.Ssdt.targets(455,5): Error MSB4018: The "DBCleanTask" task failed unexpectedly.
System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified.
File name: 'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'
at RedGate.Versioning.Engine.Shared.Logging.Api.Configuration.LoggerConfigurationManager.GetLogLevel(IServiceIdentifier serviceIdentifier)
at RedGate.Versioning.Engine.Shared.Logging.Serilog.SerilogLoggerConfiguration..ctor(IServiceIdentifier serviceId, ILoggerConfigurationManager configurationManager)
at RedGate.Versioning.Engine.Shared.Logging.Serilog.SerilogLoggerConfiguration..ctor(IServiceIdentifier serviceId)
at RedGate.Versioning.Client.MsBuild.MsBuildModule.Load(ContainerBuilder builder)
at Autofac.Module.Configure(IComponentRegistry componentRegistry)
at Autofac.ContainerBuilder.Build(IComponentRegistry componentRegistry, Boolean excludeDefaultModules)
at Autofac.ContainerBuilder.UpdateRegistry(IComponentRegistry componentRegistry)
at Autofac.Core.Lifetime.LifetimeScope.CreateScopeRestrictedRegistry(Object tag, Action`1 configurationAction)
at Autofac.Core.Lifetime.LifetimeScope.BeginLifetimeScope(Object tag, Action`1 configurationAction)
at Autofac.Core.Lifetime.LifetimeScope.BeginLifetimeScope(Action`1 configurationAction)
at Autofac.Core.Container.BeginLifetimeScope(Action`1 configurationAction)
at RedGate.Versioning.Client.MsBuild.SqlChangeAutomationTask`2.Execute()
at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()
Tagged:

Answers

  • alovhaugalovhaug Posts: 7 Bronze 2
    You are not alone. I just ran a build myself and am seeing this problem also. In my case, it is with Visual Studio 2019 not 2017, but yes otherwise the same error message. I really don't want to have to rework the solution build configuration, so I hope there will be a solution. :-(

    --Avonelle
  • clopezclopez Posts: 5 Bronze 1
    I am experiencing exactly the same issue after the update last night.
    The "DBCleanTask" task failed unexpectedly.
    System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies.
    I am invoking MSBuild of VS2017 on a SQLChangeAutomation solution.
    I am in the process to rollback to a previous version.
    https://download.red-gate.com/checkforupdates/SQLChangeAutomation/SQLChangeAutomation_4.4.22184.30843.exe [download.red-gate.com]
  • alovhaugalovhaug Posts: 7 Bronze 2
    clopez said:
    I'll be interested if this works, but also surprised. I was already at an older version.

    For what it is worth, I find it strange that the version of assembly Newtonsoft.Json in the MSBuild\SqlChangeAutomation\ folder is the same on my development computer as on my build computer, but while it builds fine on my development computer, I get this error on the build server. (It is a substantially newer version than 6.0.0.0, but it doesn't seem to be an issue locally. Hmmmm....)

    --Avonelle
  • clopezclopez Posts: 5 Bronze 1
    Yes. It did work. We tested in another computer where we did not have automatic updates and MSBuild was working fine. After installing the latest version of SQL Change Automation it started reporting the error mentioned. Uninstalling and reinstalling the older version solved the issue.
    Please note that if you have multiple versions of VS in the same computer (let's say you have VS2017 and VS2019), the installation of the older version did not properly override both extensions, just the latest VS. It was a painful trial and error to guarantee that both VS had the older extension.
  • Hi, 

    Sorry to hear you're experiencing this issue!
     
    The long term fix would be to move to using the latest SQL Change Automation PowerShell tasks as ReadyRoll is no longer supported
     
    https://documentation.red-gate.com/sca/deploying-database-changes/add-ons/azure-devops
     
    If you wish to workaround this until you can make the transition, you will need to lockdown to the following version:


    Kind regards

    Dan Calver | Redgate Software
    Have you visited our Help Center?

  • clinkeclinke Posts: 11 New member
    edited July 18, 2022 11:19AM
    We have this issue locally also (when we deploy the project in VS2022), so going back is difficult. I didn't find an older vsix-component for download.
  • PDQPDQ Posts: 8 Bronze 1
    Hi DanC,

    Sorry...my original post must have been misleading. We are using on-prem Azure Dev Ops server (or what used to be called TFS). We have not used ReadyRoll for quite some time (years?).

    The only Redgate component that is accessed from my pipeline is "Redgate SQL Change Automation Build", with the version setting of "Latest". According to the VS Marketplace, this plugin has not be updated since 5/11/2022, so I do not perceive that the version of SCA Build is the issue.

    Where would you expect that I might see the "ReadyRoll" reference that you're referring to above?

    Thanks,
    Pam
  • PDQPDQ Posts: 8 Bronze 1
    Hi clinke,

    The only way I could get a previous version working on my build server was:
    • I uninstalled SQL Change Automation completely from the server
    • Since we have multiple versions of VS on the server, I confirmed that the extension is not still being referenced in any of the versions of VS. (If it is, remove the extension from VS)
    • I Installed the specific version of SCA that most recently worked in a successful build. Thanks to clopez above, I used the link that he added to his post (minus the exe) and found the Redgate downloads directory that allowed me to locate the version I was looking for (https://download.red-gate.com/checkforupdates/SQLChangeAutomation)
    • Installed this version. When prompted, take one of the options that installed the component into VS. After installation, all versions of VS were using the specific version that I wanted.
    Other things to note:
    • You'll likely want to turn off the "Automatic-update" option in VS for this component, or it will be overwritten the next time that Redgate auto-updater runs
    • The uninstall / reinstall of SCA required reboots of the build server
    Hope this helps,
    Pam
  • clopezclopez Posts: 5 Bronze 1
    I am experiencing exactly the same issue after the update last night.
    The "DBCleanTask" task failed unexpectedly.
    System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies.
    I am invoking MSBuild of VS2017 on a SQLChangeAutomation solution.
    I am in the process to rollback to a previous version.
    https://download.red-gate.com/checkforupdates/SQLChangeAutomation/SQLChangeAutomation_4.4.22184.30843.exe [download.red-gate.com]
  • clopezclopez Posts: 5 Bronze 1
    I am experiencing exactly the same issue after the update last night.
    The "DBCleanTask" task failed unexpectedly.
    System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies.
    I am invoking MSBuild of VS2017 on a SQLChangeAutomation solution.
    I am in the process to rollback to a previous version.
    https://download.red-gate.com/checkforupdates/SQLChangeAutomation/SQLChangeAutomation_4.4.22184.30843.exe [download.red-gate.com]
  • Hi @PDQ

    I'm not sure what your pipelines look like, you may have a task inside which uses the ReadyRoll.

    I've done some testing and using purely the PowerShell addons it's working as expecting with the latest versions

    Kind regards

    Dan Calver | Redgate Software
    Have you visited our Help Center?

  • PDQPDQ Posts: 8 Bronze 1
    Hi DanC,

    I hate to belabor the point, but my original post describes the pipeline issues I am having. And, per posts from other users, they also are experiencing the same issue.

    "...we have a specific step for SCA (that is NOT failing). However, when the Visual Studio Build "Build solution" step is executed, the SCA build fails because te DBClean task cannot locate a specific version of Newtonsoft.Json..."

    So, the task that is failing is the Microsoft "Build solution" task. It worked prior to the latest update of Redgate tools and now is failing. I do not have the ability to control how the "Build solution" task interfaces with Redgate tools. 

    Would it be possible for Redgate to weigh in on why the "Build solution" task is failing since this is the error that was introduced with the most recent update of SCA?

    Best regards,
    Pam
  • clinkeclinke Posts: 11 New member
    Dito. I don't bother product names. If something broke and it was working before, this is clearly an error for me. And a serious one, because it breaks the workflow.
  • Hi @PDQ

    I now understand what's happening here, the Visual Studio MSBuild "Build Solution" step is no longer supported and is a legacy method, however, there is a bug report for investigating this and I'm waiting on further information

    Instead, the currently supported method of build and deployments is using the Redgate PowerShell add-ons and will avoid this issue, these are defined in the following documentation. 

    https://documentation.red-gate.com/sca/deploying-database-changes/add-ons/azure-devops/use-the-azure-devops-build-extension-with-sql-change-automation-project


    Kind regards

    Dan Calver | Redgate Software
    Have you visited our Help Center?

  • clinkeclinke Posts: 11 New member
    Yea that's fine, but our developers want to deploy their changes directly using Visual Studio. I didn't get any message or information that this way is deprecated. How is it possible in the future to deploy changes out of Visual Studio? We don't want to run manually created PowerShell-Scripts, because it only breaks our workflow. So which GUI is intended to deploy changes locally in the future? Flyway?
  • PDQPDQ Posts: 8 Bronze 1
    Hi DanC,

    The link to the documentation refers to AzureDevOps -- not Azure DevOPS Server (formally known as TFS). I believe that all of us having this issue are using on-prem build machines.

    I read that Redgate will not longer support the MS Build task. However, since this step is a required step for building other projects in my solution, I'm not at liberty to simply exclude this task from my pipeline. I also know that there is a very painful workaround that would require creating VS configurations to specifically exclude building the SCA project during the MS Build task. However, the impact of making this change to many, many, many solutions is untenable.

    I would respectfully request that Redgate consider the impact of this change on its users and perhaps, with the end of your support of the MS Build task, at least provide code that does not otherwise break pipelines.

    Thanks,
    Pam

  • alovhaugalovhaug Posts: 7 Bronze 2
    @DanC - from my standpoint, the Visual Studio MSBuild "Build Solution" step doesn't have to "work" for a build and deployment, it just shouldn't generate an error as it is doing now. As @PDQ is doing, we are using the supported Powershell add-on steps for the actual build and deployment. But my solution has both the SCA project AND other projects associated with the build altogether. I use the Visual Studio MSBuild "Build Solution" step in my pipeline to build all the other projects. I don't want to have to manage multiple solutions, and having extra build configurations is also a pain, especially if you suddenly have to make changes for a lot of solutions as @PDQ has to do.

    --Avonelle
  • clinkeclinke Posts: 11 New member
    There is a new version released, 4.4.22200.31053, but the error still persists
  • Hi @PDQ

    I'm sorry this isn't ideal and is a frustrating issue, however, the Visual Studio MSBuild has been legacy for a few years now and was replaced by the Invoke-DatabaseBuild and the Azure DevOps SCA Build Task which is now the supported method of building your projects. 

    I would argue the new method of using the SCA Build Tasks is easy to use and we have a lot of documentation and support on this to help you transition towards it. Please see the documentation here: https://documentation.red-gate.com/sca/deploying-database-changes/add-ons/azure-devops/use-the-azure-devops-build-extension-with-sql-change-automation-project

    Alternatively, you can use a PowerShell step in your pipeline and use the Invoke-DatabaseBuild which achieves the same objective: 
    https://documentation.red-gate.com/sca/reference/powershell-cmdlets/invoke-databasebuild

    Kind regards

    Dan Calver | Redgate Software
    Have you visited our Help Center?

  • PDQPDQ Posts: 8 Bronze 1
    Hi @DanC,

    I still do not think that you're completely understanding the issue. My pipelines are using the SCA Build Task as is described in your previous post above--we have been for years.

    The issue remains this:
    • The SCA project is part of the VS solution. 
    • The VS Build task started failing with the release of SCA 4.4.22194.30982
    • The VS Build task does not allow for exclusion of the SCA project from the solution--instead would require me to rework/redefine the VS configurations for approximately 50 solutions, then updating the 150 pipelines (50 each of WIP, QA, and Prod).
    • The fact that Redgate chooses to remove the support for VS Build is totally fine with me--but it is NOT fine that this release of SCA has broken 150 pipelines.
    We can agree to disagree on this topic--I just won't do it on a public forum any longer. Instead, will work through our support agreement with Redgate so that I can talk directly to someone to perhaps better explain the issue.

    Thanks much for your responses,
    Pam
  • Hi,

    Thank you for reporting this issue.  We are taking a closer look at this in the development team, but need a bit of time to understand exactly what's going on.  In the meantime, please consider installing a previous version - https://download.red-gate.com/checkforupdates/SQLChangeAutomation. Pam has provided great instructions above if you have multiple Visual Studios installed and it seems like users have had success with v 4.4.22184.30843.

    Pam - Thank you for working directly with support on this.  I think seeing exactly what you have in your pipeline will help us work through this.

    Thanks you!  
    Thank you!
    Stephanie M. Herr    :-)
    Product Manager Database DevOps
  • xmissilexmissile Posts: 1 New member
    edited July 22, 2022 6:17PM
    This is a temporary workaround for those who are in a serious bind and thus cannot roll back to a previous version.

    1.  Create/open an existing sln in VS.
    2.  Open Manage NuGet packages and browse for "Newtonsoft.Json".
    3.  Use the version dropdown to select and then install "6.0.6".  Note that this is version 6.0.6.17820.
    4.  Locate the dll under "..packages\Newtonsoft.Json.6.0.6\lib\netcore45".
    5.  Copy the dll to a safe folder (one that you will not delete!), e.g. "C:\Program Files (x86)\Red Gate".
    6.  Uninstall the 6.0.6 version from your sln, if applicable.
    7.  Open a developer CLI running as admin.
    8.  Verify that Newtonsoft.Json is not already installed by running "gacutil -l Newtonsoft.Json" (that's an L for list).
    9.  If no existing version, then cd to the safe folder.
    10.  Run "gacutil -i Newtonsoft.Json.dll".

    Important Note - run "gacutil -uf Newtonsoft.Json" from the safe folder prior to upgrading SCA.  Please post your experience on this thread.
  • mailbnade03mailbnade03 Posts: 1 New member
    I've done some testing and using purely the PowerShell addons it's working as expecting with the latest versions

  • Hi All,

    We have a new version "4.5.22292" which contains the fix for DbCleanTask failing due to "Could not load file or assembly Newtonsoft.Json"

    Kind regards

    Dan Calver | Redgate Software
    Have you visited our Help Center?

  • clinkeclinke Posts: 11 New member
    Thanks, for us it's now working again
  • InkZInkZ Posts: 4 Bronze 1
    Hi, has this been broken again in the latest build?  Getting this error after an autoupdate to 4.5.22321.
  • Hi @InkZ

    Would you be able to share the log output for this error with the version?

    Kind regards

    Dan Calver | Redgate Software
    Have you visited our Help Center?

  • InkZInkZ Posts: 4 Bronze 1
    edited November 22, 2022 11:39AM
    Version is : 4.5.22321.

    Error output is below:

    C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\SqlChangeAutomation\SqlChangeAutomation.Data.Schema.Ssdt.targets(455, 5): error MSB4018: The "DBCleanTask" task failed unexpectedly.
    System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The system cannot find the file specified.
    File name: 'Newtonsoft.Json, Version=6.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'
       at RedGate.Versioning.Engine.Shared.Logging.Api.Configuration.LoggerConfigurationManager.GetLogLevel(IServiceIdentifier serviceIdentifier)
       at RedGate.Versioning.Engine.Shared.Logging.Serilog.SerilogLoggerConfiguration..ctor(IServiceIdentifier serviceId, ILoggerConfigurationManager configurationManager)
       at RedGate.Versioning.Engine.Shared.Logging.Serilog.SerilogLoggerConfiguration..ctor(IServiceIdentifier serviceId)
       at RedGate.Versioning.Client.MsBuild.MsBuildModule.Load(ContainerBuilder builder)
       at Autofac.Module.Configure(IComponentRegistry componentRegistry)
       at Autofac.ContainerBuilder.Build(IComponentRegistry componentRegistry, Boolean excludeDefaultModules)
       at Autofac.ContainerBuilder.UpdateRegistry(IComponentRegistry componentRegistry)
       at Autofac.Core.Lifetime.LifetimeScope.CreateScopeRestrictedRegistry(Object tag, Action`1 configurationAction)
       at Autofac.Core.Lifetime.LifetimeScope.BeginLifetimeScope(Object tag, Action`1 configurationAction)
       at Autofac.Core.Lifetime.LifetimeScope.BeginLifetimeScope(Action`1 configurationAction)
       at Autofac.Core.Container.BeginLifetimeScope(Action`1 configurationAction)
       at RedGate.Versioning.Client.MsBuild.SqlChangeAutomationTask`2.Execute()
       at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
       at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()

    WRN: Assembly binding logging is turned OFF.
    To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
    Note: There is some performance penalty associated with assembly bind failure logging.
    To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
  • @InkZ

    Thank you, going to forward this onto the team now!

    Kind regards

    Dan Calver | Redgate Software
    Have you visited our Help Center?

Sign In or Register to comment.