Warehouse v Data Factory Version Compatibility
BrightM
Posts: 1 New member
Hello. I'm currently building out an Azure data warehouse and using Azure Data Factory for the ETL. We have Redgate SQL Prompt for querying which continues to be a great tool now it works in the cloud as well.
We are using Azure DevOps with VisualStudio 2022 integrated with Github to manage the structure history and build of the data warehouse. We've also got a separate git repo to manage the Azure Data Factory version control. What I would like to know is whether there is anything in Redgate's kit that allows us to keep the two releases in sync, i.e. v 1.1.4 of the warehouse must only be released with versions 1.1.3 to 1.1.7 of data factory because 1.1.5 introduces two new tables that are covered in 1.1.8 of our ADF project.
I suppose it's something similar to a product compatibility list or similar. Would we have to maintain this ourselves or is there Microsoft or Redgate tooling that facilitates it?
We are using Azure DevOps with VisualStudio 2022 integrated with Github to manage the structure history and build of the data warehouse. We've also got a separate git repo to manage the Azure Data Factory version control. What I would like to know is whether there is anything in Redgate's kit that allows us to keep the two releases in sync, i.e. v 1.1.4 of the warehouse must only be released with versions 1.1.3 to 1.1.7 of data factory because 1.1.5 introduces two new tables that are covered in 1.1.8 of our ADF project.
I suppose it's something similar to a product compatibility list or similar. Would we have to maintain this ourselves or is there Microsoft or Redgate tooling that facilitates it?
Tagged:
Answers
Thank you for reaching out on the Redgate forums regarding your compatibility question.
It may be possible using Flyway to implement a solution.
If you were storing metadata in your databases that outlined the version and then use Flyway as a version-control for your migrations. You could use the metadata in migration scripts to deploy up to a certain version or use callbacks to enforce compatibility checks at deployment.
You would potentially implement a compatibility table in your warehouse. Then use flyway to check this table when running deployment scripts. It would run migration scripts up to the permitted versions.
If you are new to Flyway, we have our documentation page here:
https://documentation.red-gate.com/fd/
and you can download a trial here:
https://www.red-gate.com/products/flyway/
Otherwise, I can assist in putting you in contact with our solution engineers / sales teams to discuss Flyway and how it may be used to assist your business process.