Revision history [back]

Pandas groupby can be utilized to eliminate duplicate values by grouping the data based on a specific column or multiple columns and then aggregating the data. The aggregate function can be used to apply a function to eliminate the duplicates.

For example, if we have a dataframe with duplicates in the 'name' column, we can eliminate them by grouping the data by the 'name' column and using the aggregate function to select the first value of each group.

import pandas as pd

# create a dataframe with duplicate values
df = pd.DataFrame({'name': ['John', 'Jane', 'John', 'Adam', 'Jane'], 'age': [30, 25, 28, 40, 27]})

# group the data by 'name' column and select the first value of the group
df_grouped = df.groupby('name').first()

print(df_grouped)

Output:

      age
name     
Adam   40
Jane   25
John   30

In this example, the duplicate values in the 'name' column have been eliminated by grouping the data by 'name' and selecting the first value of each group.