How to find unique strings or objects using Java stream API

Introduction

In this tutorial we will see an example on how to find unique strings or objects using Java stream API. Stream API was introduced in Java 8 and we are going to see how to use this stream API to remove duplicates from strings or objects.

Java stream API has a method distinct() that removes duplicates and returns unique strings from a list of strings. But for objects we cannot directly apply this distinct() method directly to find out unique elements from a list of objects. Therefore we will use Predicate to remove duplicates and find the unique objects by a particular attribute of the object.

Prerequisites

At least Java 8

Remove Duplicates from Strings

Here is the below example we will see how to remove duplicate strings from a list of strings and return the unique strings.

Let’s say we have the below list of strings:

List<String> strings = Arrays.asList("Ani", "Sou", "Lit", "Sum", "Ani", "Sou", "Pab");

Obviously we have few duplicate strings, such as Ani and Sou. Now we need to remove these duplicate strings from the resultant strings. The final strings become:

Ani, Sou, Lit, Sum, Pab

So the below code returns the unique strings:

List<String> uniqueStrings = strings.stream().distinct().collect(Collectors.toList());

Remove Duplicates from Objects

Now we will see how to remove duplicates from objects. We need to make the object unique by a particular attribute of the object. So let’s create a class User that will have below attributes or properties:

protected static class User {

	private String name;
	private String email;
	private int age;
	
	//...
	
}

First we will create few dummy User objects:

private static List<User> createUserList() {
	List<User> userList = new ArrayList<>();

	User e1 = new User("John", "john@email.com", 21);
	User e2 = new User("Martin", "martin@email.com", 19);
	User e3 = new User("Marry", "marry@email.com", 31);
	User e4 = new User("Stephan", "stephen@email.com", 18);
	User e5 = new User("Garry", "garry@email.com", 26);
	User e6 = new User("Marry", "marry2@email.com", 30);
	User e7 = new User("Garry", "garry2@email.com", 25);

	userList.add(e1);
	userList.add(e2);
	userList.add(e3);
	userList.add(e4);
	userList.add(e5);
	userList.add(e6);
	userList.add(e7);

	return userList;
}

Here we will use two approaches to find out the unique users from a list of users – first approach is to put all the user objects into a map by one of its attributes and second approach is to use Predicate to find out the unique user objects.

Below code put users into map by its name attribute:

List<User> userList = createUserList();
Collection<User> uniqueUsers = userList.stream().collect(Collectors.toMap(User::getName, p -> p, (p, q) -> p)).values();

Below code use Predicate to remove duplicates:

private static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
	Set<Object> seen = ConcurrentHashMap.newKeySet();
	return t -> seen.add(keyExtractor.apply(t));
}

Finally call distintByKey() method to remove duplicates:

List<User> distinctUsers = userList.stream().filter(distinctByKey(User::getName)).collect(Collectors.toList());

The whole source code can be written as:

package com.roytuts.java8.streams;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.List;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;
import java.util.function.Predicate;
import java.util.stream.Collectors;

public final class UniqueStringsJavaStreams {

	public static void main(String[] args) {
		List<String> strings = Arrays.asList("Ani", "Sou", "Lit", "Sum", "Ani", "Sou", "Pab");
		System.out.println("Unique strings");
		List<String> uniqueStrings = strings.stream().distinct().collect(Collectors.toList());
		uniqueStrings.forEach(s -> System.out.println(s));

		System.out.println();

		List<User> userList = createUserList();

		System.out.println("Unique users by name");
		Collection<User> uniqueUsers = userList.stream().collect(Collectors.toMap(User::getName, p -> p, (p, q) -> p))
				.values();
		uniqueUsers.forEach(usr -> System.out.println(usr));

		System.out.println();

		List<User> distinctUsers = userList.stream().filter(distinctByKey(User::getName)).collect(Collectors.toList());
		distinctUsers.forEach(usr -> System.out.println(usr));
	}

	private static <T> Predicate<T> distinctByKey(Function<? super T, ?> keyExtractor) {
		Set<Object> seen = ConcurrentHashMap.newKeySet();
		return t -> seen.add(keyExtractor.apply(t));
	}

	private static List<User> createUserList() {
		List<User> userList = new ArrayList<>();

		User e1 = new User("John", "john@email.com", 21);
		User e2 = new User("Martin", "martin@email.com", 19);
		User e3 = new User("Marry", "marry@email.com", 31);
		User e4 = new User("Stephan", "stephen@email.com", 18);
		User e5 = new User("Garry", "garry@email.com", 26);
		User e6 = new User("Marry", "marry2@email.com", 30);
		User e7 = new User("Garry", "garry2@email.com", 25);

		userList.add(e1);
		userList.add(e2);
		userList.add(e3);
		userList.add(e4);
		userList.add(e5);
		userList.add(e6);
		userList.add(e7);

		return userList;
	}

	protected static class User {

		private String name;
		private String email;
		private int age;

		public User() {
		}

		public User(String name, String email, int age) {
			this.name = name;
			this.email = email;
			this.age = age;
		}

		public String getName() {
			return name;
		}

		public void setName(String name) {
			this.name = name;
		}

		public String getEmail() {
			return email;
		}

		public void setEmail(String email) {
			this.email = email;
		}

		public int getAge() {
			return age;
		}

		public void setAge(int age) {
			this.age = age;
		}

		@Override
		public String toString() {
			return "User [name=" + name + ", email=" + email + ", age=" + age + "]";
		}

	}
}

Testing the Program

Executing the above class will give you below output:

Unique strings
Ani
Sou
Lit
Sum
Pab

Unique users by name
User [name=Stephan, email=stephen@email.com, age=18]
User [name=John, email=john@email.com, age=21]
User [name=Garry, email=garry@email.com, age=26]
User [name=Martin, email=martin@email.com, age=19]
User [name=Marry, email=marry@email.com, age=31]

User [name=John, email=john@email.com, age=21]
User [name=Martin, email=martin@email.com, age=19]
User [name=Marry, email=marry@email.com, age=31]
User [name=Stephan, email=stephen@email.com, age=18]
User [name=Garry, email=garry@email.com, age=26]

Thanks for reading.