# Data Quality on columns \[DONE\]

**URL:** <https://community.easymorph.com/t/data-quality-on-columns-done/533>\
**Category:** Feature requests\
**Created:** [April 26, 2018, 9:07am UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533 "2018-04-26T09:07:34Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![RJO](https://community.easymorph.com/user_avatar/community.easymorph.com/rjo/32/4084_2.png) [@RJO](https://community.easymorph.com/u/RJO)\
**Post date:** [April 26, 2018, 9:07am UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/1 "2018-04-26T09:07:34Z")

</div>

There is a feature many of today's data wranglers have : data quality on columns. You can see the different values, the distribution of values at a glance. You can also filter dynamically using the same screen and see the distribution of other columns changing at the same time.

You could think of a new panel at the bottom of easymorph that could detail the content of columns in the table the user has selected.

Also still looking forward to have the possibility to create cross table and export them, like you can do with chart.

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [April 26, 2018, 1:50pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/2 "2018-04-26T13:50:39Z")

</div>

We've added data profiling in version 3.8 which has just been released.

It shows various counts for a selected column, with the ability to instantly create a filter action for each count.

Also, the profiler shows a distribution histogram for numerical values. The histogram has two rulers for filtering a range of values visually.

The profiler window is floating which means you can click and profile other columns without closing the profiler window.

 ![image](https://community.easymorph.com/uploads/default/original/1X/86649978b2f52891f8fa07e5515f7533cb3958bd.png)

---

<div class="post-metadata">

**Author:** ![RJO](https://community.easymorph.com/user_avatar/community.easymorph.com/rjo/32/4084_2.png) [@RJO](https://community.easymorph.com/u/RJO)\
**Post date:** [April 26, 2018, 3:31pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/3 "2018-04-26T15:31:38Z")

</div>

That's good but quite limited.  
Example : if you have a string column containing names, you expect to have the count number by name in a histogram.

Here i feel that only number can be profiled, is that correct ?

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [April 26, 2018, 3:45pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/4 "2018-04-26T15:45:42Z")

</div>

Yes, the histogram shows distribution for numeric values. Counts for values not available in the profiler yet. If you need to see counts for distinct values, right-click the column and pick Aggregate/Count.

---

<div class="post-metadata">

**Author:** ![RJO](https://community.easymorph.com/user_avatar/community.easymorph.com/rjo/32/4084_2.png) [@RJO](https://community.easymorph.com/u/RJO)\
**Post date:** [April 26, 2018, 4:03pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/5 "2018-04-26T16:03:56Z")

</div>

Okay, that's a good start. Just waiting for values count and if possible interactions with bars to auto filter.  
That could be a good thing. Also a dedicated panel with these informations on all columns (not one by one) could be nice !

---

<div class="post-metadata">

**Author:** ![reynsnivea](https://community.easymorph.com/letter_avatar_proxy/v4/letter/r/9f8e36/32.png) [@reynsnivea](https://community.easymorph.com/u/reynsnivea)\
**Post date:** [May 4, 2018, 6:57am UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/6 "2018-05-04T06:57:12Z")

</div>

Good point ! Is there a workaround to achieve the profiling on all the columns with the current profiling options ? It does not seem possible yet because I cannot see an option to write the output of the profiling to a table ?

Does easyMorph forsee to elaborate the profiling options in the near future ?

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [May 4, 2018, 12:10pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/7 "2018-05-04T12:10:54Z")

</div>

> [@reynsnivea](#):
>
> Is there a workaround to achieve the profiling on all the columns with the current profiling options ? It does not seem possible yet because I cannot see an option to write the output of the profiling to a table ?

Not possible at this point.

> [@reynsnivea](#):
>
> Does easyMorph forsee to elaborate the profiling options in the near future ?

We will be adding [automated data quality suggestions](https://twitter.com/easy_morph/status/986661965782241280) in 3.8.1 and full-table profiling in 3.9.

---

<div class="post-metadata">

**Author:** ![reynsnivea](https://community.easymorph.com/letter_avatar_proxy/v4/letter/r/9f8e36/32.png) [@reynsnivea](https://community.easymorph.com/u/reynsnivea)\
**Post date:** [September 6, 2018, 9:25am UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/8 "2018-09-06T09:25:34Z")

</div>

Hi Dmitry,

I have installed the version 3.9 but it appears that the full-table profiling is not implemented yet ?  
Is this still on the roadmap ? When will it be released ?

Kind regards

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [September 6, 2018, 12:55pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/9 "2018-09-06T12:55:55Z")

</div>

It's still on the roadmap but not assigned to a specific release at this point.

---

<div class="post-metadata">

**Author:** ![reynsnivea](https://community.easymorph.com/letter_avatar_proxy/v4/letter/r/9f8e36/32.png) [@reynsnivea](https://community.easymorph.com/u/reynsnivea)\
**Post date:** [December 20, 2018, 3:27pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/10 "2018-12-20T15:27:01Z")

</div>

Any news on this topic? We would realy appreciate to be able to do automatic profiling of entire datasets so that we do not have to incorporate a set of tests in each project.

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [December 20, 2018, 4:23pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/11 "2018-12-20T16:23:34Z")

</div>

No release assigned for this feature yet.

---

<div class="post-metadata">

**Author:** ![reynsnivea](https://community.easymorph.com/letter_avatar_proxy/v4/letter/r/9f8e36/32.png) [@reynsnivea](https://community.easymorph.com/u/reynsnivea)\
**Post date:** [March 15, 2020, 11:18am UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/12 "2020-03-15T11:18:36Z")

</div>

Hi,

Is the full table profiling feature still on the roadmap ?

Thanks

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [March 15, 2020, 9:01pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/13 "2020-03-15T21:01:48Z")

</div>

Yes, it is. Time to production is about 3 months.

---

<div class="post-metadata">

**Author:** ![reynsnivea](https://community.easymorph.com/letter_avatar_proxy/v4/letter/r/9f8e36/32.png) [@reynsnivea](https://community.easymorph.com/u/reynsnivea)\
**Post date:** [March 16, 2020, 7:30pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/14 "2020-03-16T19:30:48Z")

</div>

Nice, thanks !

---

<div class="post-metadata">

**Author:** ![reynsnivea](https://community.easymorph.com/letter_avatar_proxy/v4/letter/r/9f8e36/32.png) [@reynsnivea](https://community.easymorph.com/u/reynsnivea)\
**Post date:** [June 8, 2023, 6:29am UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/15 "2023-06-08T06:29:03Z")

</div>

@dgudkov

Is a feature for profiling an entire dataset in 1 action still on the roadmap ?

Thanks !

---

<div class="post-metadata">

**Author:** ![dgudkov](https://community.easymorph.com/user_avatar/community.easymorph.com/dgudkov/32/5_2.png) [@dgudkov](https://community.easymorph.com/u/dgudkov)\
**Post date:** [June 8, 2023, 5:47pm UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/16 "2023-06-08T17:47:11Z")

</div>

[The "Table metadata" action](https://help.easymorph.com/doku.php?id=transformations:metadata) has already been released.

Also, full-table profiling is available if you right-click an action and select "Table metadata".

 ![image](https://community.easymorph.com/uploads/default/original/2X/7/7aaefc5276ce808251bf93d28dde467a9d101a0c.png)

 ![image](https://community.easymorph.com/uploads/default/original/2X/a/a89dad38750ca7161aaa67a2d858ea852e108387.png)

---

<div class="post-metadata">

**Author:** ![reynsnivea](https://community.easymorph.com/letter_avatar_proxy/v4/letter/r/9f8e36/32.png) [@reynsnivea](https://community.easymorph.com/u/reynsnivea)\
**Post date:** [June 12, 2023, 7:08am UTC](https://community.easymorph.com/t/data-quality-on-columns-done/533/17 "2023-06-12T07:08:43Z")

</div>

ok thanks !
