Data Profiling Tools
ben_b
Posts: 84 Silver 2
I feel like this comes up for me every couple of years for me.
We want to get data owners / stewards to look at specific "critical data elements" and use their industry expertise to feedback data quality issues.
For example, an insurance company may have an 'insured' table. This table may be in a system which has had other systems migrated into it. It also has been through yearly upgrades by the vendor over the course of 20 years.
As such, the rows in the "gender" column may be something like
M - 20%
F - 30%
Male - 10%
Female - 35%
Null - 5%
we want a tool that profiles the data and lets people navigate it - giving a very simple visual summary, i.e. a radar chart showing the distribution of values etc.
I've researched this topic a few times over the last 8 years and nothing seems to have changed.
The option seem to be
Free - but terrible
Enterprise Level - massively over priced and bundled in with tonnes of other products we dont need. and would take years to setup.
every time I seem to settle on Idera SQL Data Profiler. It is basic but miles ahead of other products which haven't changed in years and I dont have to have loads of meetings with enterprise sales staff to use it. It is also free.
My question: Has anyone else got any recommendations?
My long-shot request: Can red-gate create their own version of "SQL Data Profiler" please. Probably not much of a market but I would pay!
Cheers
Ben
We want to get data owners / stewards to look at specific "critical data elements" and use their industry expertise to feedback data quality issues.
For example, an insurance company may have an 'insured' table. This table may be in a system which has had other systems migrated into it. It also has been through yearly upgrades by the vendor over the course of 20 years.
As such, the rows in the "gender" column may be something like
M - 20%
F - 30%
Male - 10%
Female - 35%
Null - 5%
we want a tool that profiles the data and lets people navigate it - giving a very simple visual summary, i.e. a radar chart showing the distribution of values etc.
I've researched this topic a few times over the last 8 years and nothing seems to have changed.
The option seem to be
Free - but terrible
Enterprise Level - massively over priced and bundled in with tonnes of other products we dont need. and would take years to setup.
every time I seem to settle on Idera SQL Data Profiler. It is basic but miles ahead of other products which haven't changed in years and I dont have to have loads of meetings with enterprise sales staff to use it. It is also free.
My question: Has anyone else got any recommendations?
My long-shot request: Can red-gate create their own version of "SQL Data Profiler" please. Probably not much of a market but I would pay!
Cheers
Ben
Tagged:
Comments
1. 'analyse data' feature in excel
2. 'column profiling' feature in power bi.
struggling to get the Power BI to work but will do some more testing.
more information below:
https://support.microsoft.com/en-us/office/analyze-data-in-excel-3223aab8-f543-4fda-85ed-76bb0295ffc4
https://sqlskull.com/2021/02/28/profile-data-in-power-bi/