How to Remove Duplicate Items in Python List?

In Python, list data structures are used to store multiple collections of data in an ordered sequence of items. Since we know that duplicate items are allowed to be stored in the list data structure, it becomes a problem when the order of the list elements is critical.

This Python guide provides multiple examples for removing duplicate items from the Python list. The contents discussed in this Python blog post are shown below:

Let’s begin!

How to Remove Duplicate Items in Python List?

Different methods are used in Python to delete duplicate items from the list. The inbuilt functions like “set()” and “dict.fromkeys()” can be used to convert the list into any other data type that does not allow any duplicate elements.

Take a look at the method given below:

Method 1: Using set() Function

In the example given below, the “set()” function is used to remove the duplicate element in the Python list.

Code:

list_value = [10, 21, 13, 14, 13, 21]

set_value = set(list_value)
duplicate_removed = list(set_value)
print(duplicate_removed)

In the above code:

  • The list containing duplicate items are initialized in the program.
  • Firstly, the list is converted into the set by using the “set()” function. The “set()” function is used because it stores the element without duplicates.
  • After converting it into a set, the “list()” function is utilized to convert it back into a list.
  • The final created list will be without any duplicate items.

Output:

The above output verified that the duplicate items were removed from the list.

Method 2: Using an Empty List

In the code below, a new empty list is created, and the old list items are appended to that list. Further detail explanation is provided in the below example:

Code:

list_value = [1, 2, 1, 3, 4, 2]

empty_list = []

for i in list_value:
    if i not in empty_list:
        empty_list.append(i)

print(empty_list)

In the above code:

  • The list containing duplicate items is initialized in the program.
  • A new list without any element is initialized in the program.
  • The “for” loop iterates on the list and all the elements that are not present in the empty list are appended using the “append()” function.
  • To check the items present in the list or not, the “if” statement is used along with the “not” operator.

Output:

The above output verified that the duplicate items were removed from the list.

Method 3: Using List Comprehension

In the code example given below, the list comprehension method can be used to remove the duplicate items in the Python list.

Code:

list_value = [11, 22, 3, 4, 3, 22]
empty_list = []
[empty_list.append(i) for i in list_value if i not in empty_list
print(empty_list)

In the above code:

  • There is a great similarity between the list comprehension method and the previous empty list method. The only difference is that we can replace multiple lines of code with single-line code.
  • The “append()” function and “for” loop is initialized inside the square bracket of list comprehension.

Output:

The above output verified that the duplicate items were removed from the list.

Method 4: Using dict.fromkeys() Function

In the example shown below, the “dict.fromkeys()” function is used to convert the items of a list into keys of the dictionary. Dictionary keys do not contain duplicate keys, so this method is very helpful in removing duplicate items from the Python list.

Code:

#create list
list_value = [2, 2, 3, 4, 3, 1]
#create dictionary
dict_value = dict.fromkeys(list_value)
#convert back to list
newlist = list(dict_value)

print(newlist)

In the above code:

  • The list values with duplicate items are initialized.
  • The “dict.fromkeys()” takes the list variable as an argument and returns the dictionary without duplicates.
  • The dictionary is converted back into the list using the “list()” function.

Output:

The above output verified that the duplicate items were removed from the list.

That’s it from this guide!

Conclusion

To remove duplicate items from the list, the “set()”, “dict.fromkeys()”, “list comprehension”, and “empty list” along with “if” statements are used in Python. The “set()” and “dict.fromkeys()” convert the list into a set and dictionary data structure, and these data structures do not store any duplicate items. When the duplicate item is removed, the values can be converted back into the list using the “list()” function. The “list comprehension” and “empty list” are also used to remove the duplicate item from the list concisely. This article covered various methods to delete duplicate items in a Python list.