{{ transformations:DeduplicateAction.png}}
====== DEDUPLICATE ROWS ======
Category: Transform / Filters\\

\\ 
=====Description=====
This action removes all duplicate rows in the entire table. Deduplication can be performed based on all columns, or only on specific columns.\\

\\
=====Use cases=====
Use this action to remove records that may have been duplicated in the source dataset or during previous actions, and keep unique records only. 

\\ 
=====Action settings=====
^Setting ^Description ^
|Apply to|Select whether to base deduplication on all columns in the dataset, or only selected columns.\\  Options:  //All columns// or //Selected columns// (and select the columns to use from the list).|

\\
=====Remarks=====
When deduplicating based on specific columns:
  * The uniqueness of values in non-selected columns will be ignored.
  * Duplicate rows are removed from the dataset from the bottom, up.

\\ 
=====Examples=====

====Example #1====
>Find and remove duplicate rows based on the values in all columns.\\

===Before (source table)===
^ River  ^ Length (km)  ^ Continent  ^
| Nile  |  6650| Africa  |
| Amazon  |  6400| South America  |
| Nile  |  6650| Africa  |

===After (result table)===
^River  ^Length (km)  ^Continent  ^
| **Nile**  |  **6650**| **Africa**  |
| **Amazon**  |  **6400**| **South America**  |

===Action parameters===
> Apply to: All columns

\\
====Example #2====
>Remove duplicate rows based on First and Last Names.

===Before (source table)===
^FirstName ^LastName ^MiddleInit ^
|Leah  |Boswell  |A  |
|José  |Silva  |D  |
|Gino  |Basso  |E  |
|Leah  |Boswell  |F  |
|Gino  |Basso  |R  |
|Santa |Alegio |G |

===After (result table)===
^FirstName ^LastName ^MiddleInit ^
|**Leah**  |**Boswell**  |**A**  |
|José  |Silva  |D  |
|**Gino**  |**Basso**  |**E**  |
|Santa |Alegio |G |

===Action parameters===
>Apply to: Selected columns (FirstName, LastName)

\\ 
=====Community examples=====
  * [[https://community.easymorph.com/t//1269/1|Example: use of Web Request and Iterate Web Request with the Community forum API]] ([[https://community.easymorph.com/uploads/short-url/bGsC1xZtCgMa3rBWiPELwBTZLgX.morph|Project]]; Module: //Main//; Group: //Tab 1//;\\ Table: //Recently active users//; Action position: //9//)
  * [[https://community.easymorph.com/t//1625/4|Create a new table with columns based on row values]] ([[https://community.easymorph.com/uploads/short-url/4oGLUNDHrUfB971H80ciuOAtOTG.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Table 1//; Action position: //3//)
  * [[https://community.easymorph.com/t//1865/4|Dynamical genereting of path based on previous step]] ([[https://community.easymorph.com/uploads/short-url/70xVAnShgK4EiXuSa4HgMJtFAM2.morph|Project]]; Module: //Main//; Group: //Tab 1//; Table: //Calculate folder paths//;\\ Action position: //4//)
  * [[https://community.easymorph.com/t//2156/1|How to process parent-child data structures]] ([[https://community.easymorph.com/uploads/short-url/zmylfn4Qr0ovzu3e5rybl0LVMNG.morph|Project]]; Module: //Module 1//; Group: //Tab 1//; Table: //Longest folder path//; Action position: //2//)

\\ 
=====See also=====
  * [[transformations:keepduplicates|Keep duplicates]]