Find Duplicate Elements Using Streams
Java coding interview problem for Java 8 Streams: Find Duplicate Elements Using Streams.
Finding duplicate elements is one of the most common Java Collections interview problems.
The problem teaches important concepts:
- Stream API
- Lambda expressions
- Set operations
- HashMap frequency counting
- Collection processing
What are Duplicate Elements?
A duplicate element is a value that appears more than once in a collection.
Example:
Input:
[1,2,3,2,4,1,5]
Frequency:
1 → 2 times
2 → 2 times
3 → 1 time
4 → 1 time
5 → 1 time
Duplicate elements:
1
2
Understanding Duplicate Detection
The basic idea:
Read Element
↓
Check Previous Occurrence
↓
Already Exists?
↓
Duplicate Found
Example
Input:
[10,20,30,20,40,10]
Traversal:
First:
10
Store:
{10}
Next:
20
Store:
{10,20}
Next:
30
Store:
{10,20,30}
Next:
20
Already exists.
Duplicate:
20
Why Use Java Streams?
Traditional Java:
for loop
requires:
- Manual iteration
- Temporary variables
- Extra code
Streams provide:
Collection
↓
Stream Pipeline
↓
Filter / Transform / Collect
↓
Result
Stream Pipeline Concept
A Stream pipeline contains:
Source
|
Intermediate Operations
|
Terminal Operation
Example:
list.stream()
.filter()
.collect()
Flow:
List
↓
stream()
↓
filter()
↓
collect()
↓
Result
Real-World Applications
Log Analysis
Find duplicate:
- Error messages
- Request IDs
- Events
User Management
Detect:
- Duplicate emails
- Duplicate usernames
- Duplicate accounts
Data Processing
Find duplicate:
- Transactions
- Records
- Customer data
E-Commerce
Detect:
- Duplicate products
- Duplicate orders
Problem Statement
Given a list of integers, find all duplicate elements using Java Streams.
Example 1
Input:
[1,2,3,2,4,1,5]
Output:
[1,2]
Example 2
Input:
[10,20,30]
Output:
[]
because no duplicates exist.
Duplicate Detection Approaches
There are multiple ways:
Approach 1
Using:
Set + Stream filter()
Approach 2
Using:
Collectors.groupingBy()
Approach 3
Using:
Frequency Map
Approach 4
Using:
Custom Objects + equals/hashCode
Approach 1 — Using Set With Streams
The most common approach:
Use a Set to remember visited elements.
Algorithm
- Create empty Set.
- Stream through list.
- Try adding each element.
- If add fails, element already exists.
- Collect duplicates.
Visualization
Input:
[1,2,3,2,4,1]
Set:
{}
Read:
1
Add:
{1}
Read:
2
Add:
{1,2}
Read:
3
Add:
{1,2,3}
Read:
2
Already exists.
Duplicate:
2
Java Program — Find Duplicates Using Streams
import java.util.*;
import java.util.stream.Collectors;
public class FindDuplicatesUsingStreams {
public static List<Integer> findDuplicates(
List<Integer> numbers) {
Set<Integer> seen =
new HashSet<>();
return numbers.stream()
.filter(
number ->
!seen.add(number)
)
.distinct()
.collect(
Collectors.toList()
);
}
public static void main(String[] args) {
List<Integer> numbers =
Arrays.asList(
1,2,3,2,4,1,5
);
System.out.println(
findDuplicates(numbers)
);
}
}
Output
[2,1]
Step-by-Step Explanation
Input:
[1,2,3,2,4,1,5]
Create:
seen = {}
Process:
First 1
seen.add(1)
Returns:
true
Not duplicate.
Set:
{1}
Second 2
Returns:
true
Set:
{1,2}
Third 3
Returns:
true
Set:
{1,2,3}
Fourth 2
Already exists.
seen.add(2)
Returns:
false
Duplicate found.
Sixth 1
Already exists.
Duplicate found.
Result:
[2,1]
Why !seen.add() Works?
The Set method:
add()
returns:
true
if element is new.
Returns:
false
if element already exists.
Example:
Set<Integer> set =
new HashSet<>();
set.add(10);
First time:
true
Again:
set.add(10);
Returns:
false
Therefore:
!set.add(value)
means:
Element already exists
Alternative Using ForEach Stream
Another readable approach:
Set<Integer> seen =
new HashSet<>();
Set<Integer> duplicates =
new HashSet<>();
numbers.stream()
.forEach(
number -> {
if(!seen.add(number)) {
duplicates.add(number);
}
}
);
Output
[1,2]
Handling Duplicate Objects
The same approach works with objects.
Example:
Employee
Employee duplicates based on:
employeeId
Need:
equals()
hashCode()
Employee Class
class Employee {
private int id;
private String name;
public Employee(
int id,
String name) {
this.id = id;
this.name = name;
}
public int getId(){
return id;
}
@Override
public boolean equals(
Object obj) {
Employee employee =
(Employee)obj;
return this.id ==
employee.id;
}
@Override
public int hashCode(){
return id;
}
}
Duplicate Employee Detection
Set<Employee> seen =
new HashSet<>();
List<Employee> duplicates =
employees.stream()
.filter(
employee ->
!seen.add(employee)
)
.toList();
Complexity Analysis
Let:
n = number of elements
Stream traversal:
O(n)
HashSet lookup:
Average:
O(1)
Total Time:
O(n)
Space:
O(n)
because Set stores elements.
Advantages
- Clean Java 8+ solution.
- Efficient lookup.
- Less code.
- Uses functional programming style.
Drawbacks
- Requires understanding Streams.
- Uses extra memory.
- Order may not be guaranteed.
Approach 2 — Using Collectors.groupingBy()
The Set approach detects duplicates efficiently.
Another powerful Stream API approach is:
Grouping + Counting
Concept
Convert:
Element
↓
Frequency Count
Example:
Input:
[1,2,3,2,4,1,5]
Frequency Map:
1 → 2
2 → 2
3 → 1
4 → 1
5 → 1
Duplicate condition:
count > 1
Result:
[1,2]
Stream Pipeline
List
↓
stream()
↓
groupingBy()
↓
counting()
↓
filter count > 1
↓
duplicates
Java Program — GroupingBy Approach
import java.util.*;
import java.util.stream.Collectors;
public class DuplicateUsingGroupingBy {
public static List<Integer> findDuplicates(
List<Integer> numbers) {
return numbers.stream()
.collect(
Collectors.groupingBy(
number -> number,
Collectors.counting()
)
)
.entrySet()
.stream()
.filter(
entry ->
entry.getValue() > 1
)
.map(
Map.Entry::getKey
)
.collect(
Collectors.toList()
);
}
}
Output
Input:
[1,2,3,2,4,1,5]
Output:
[1,2]
Step-by-Step Explanation
Input:
[1,2,3,2,4,1,5]
After grouping:
{
1=2,
2=2,
3=1,
4=1,
5=1
}
Filter:
entry.getValue() > 1
Remaining:
1=2
2=2
Extract keys:
[1,2]
Finding Duplicate Frequency
Sometimes the requirement is not only duplicate values but their count.
Example:
Input:
[1,2,2,3,3,3]
Output:
1 → 1
2 → 2
3 → 3
Java Program
Map<Integer,Long> frequency =
numbers.stream()
.collect(
Collectors.groupingBy(
number -> number,
Collectors.counting()
)
);
Output
{
1=1,
2=2,
3=3
}
Find Elements Appearing More Than Twice
Example:
Input:
[1,2,2,3,3,3]
Condition:
count > 2
Code:
frequency.entrySet()
.stream()
.filter(
entry ->
entry.getValue() > 2
)
.map(
Map.Entry::getKey
)
.toList();
Output:
[3]
Finding First Duplicate Using Streams
Problem:
Find the first element that appears more than once.
Example:
Input:
[5,3,4,3,5]
Output:
3
Using Set
public static Integer firstDuplicate(
List<Integer> numbers) {
Set<Integer> seen =
new HashSet<>();
return numbers.stream()
.filter(
number ->
!seen.add(number)
)
.findFirst()
.orElse(null);
}
Dry Run
Input:
[5,3,4,3,5]
Read:
5
Set:
{5}
Read:
3
Set:
{5,3}
Read:
4
Set:
{5,3,4}
Read:
3
Already exists.
Return:
3
Finding Unique Elements Using Streams
Sometimes interviewers ask:
Find elements that appear only once.
Example:
Input:
[1,2,2,3,4,4]
Frequency:
1 → 1
2 → 2
3 → 1
4 → 2
Output:
[1,3]
Java Program
List<Integer> unique =
numbers.stream()
.collect(
Collectors.groupingBy(
number -> number,
Collectors.counting()
)
)
.entrySet()
.stream()
.filter(
entry ->
entry.getValue() == 1
)
.map(
Map.Entry::getKey
)
.toList();
Duplicate Objects With Streams
Example:
Employee list:
Employee(101,John)
Employee(102,Alice)
Employee(101,Bob)
Duplicate criteria:
Employee ID
Using Key Extraction
Set<Integer> ids =
new HashSet<>();
List<Employee> duplicates =
employees.stream()
.filter(
employee ->
!ids.add(
employee.getId()
)
)
.toList();
Result
Employee(101,Bob)
Parallel Streams Consideration
Java provides:
parallelStream()
for parallel processing.
Example:
numbers.parallelStream()
However:
This code is unsafe:
Set<Integer> seen =
new HashSet<>();
because multiple threads modify it.
Problem:
Thread 1
Thread 2
↓
Same Set
Thread-Safe Solution
Use:
ConcurrentHashMap
or:
ConcurrentSkipListSet
Example:
Set<Integer> seen =
ConcurrentHashMap
.newKeySet();
Stream vs Traditional Loop
| Feature | Loop | Stream |
|---|---|---|
| Readability | More code | Compact |
| Performance | Usually faster | Comparable |
| Debugging | Easy | Requires knowledge |
| Parallel Processing | Manual | Built-in support |
| Functional Style | No | Yes |
HashSet Internal Working
HashSet uses:
HashMap internally
When adding:
set.add(value)
Internally:
Value
↓
hashCode()
↓
Bucket
↓
Store Entry
Duplicate detection:
Same hash
+
equals()
↓
Duplicate
Common Interview Mistakes
Mistake 1
Using:
distinct()
alone.
Example:
numbers.stream()
.distinct()
returns:
Unique elements
not duplicates.
Mistake 2
Ignoring frequency requirement.
Question:
Find duplicates
may mean:
- Unique duplicate values
- Duplicate occurrences
- Duplicate count
Mistake 3
Using parallel streams with HashSet.
Can create:
Race conditions
Mistake 4
Forgetting equals/hashCode for objects.
Objects will not be detected correctly.
Edge Cases
| Case | Result |
|---|---|
| Empty list | Empty result |
| No duplicates | [] |
| All same values | Single duplicate |
| Large data | Use HashSet |
| Objects | Implement equality |
Interview Follow-up Questions
Q1. Find duplicate elements using Streams.
Q2. Count frequency of duplicates.
Q3. Find first duplicate element.
Q4. Find unique elements.
Q5. Difference between distinct() and duplicate detection.
Q6. How does HashSet detect duplicates?
Q7. Can parallel streams be used?
Related Java Collection Problems
- Find Duplicate Elements Using Set
- Count Word Frequency Using HashMap
- Remove Duplicate Objects
- Find Intersection of Two Lists
- Group Employees by Department
- Top K Frequent Elements
Key Takeaways
Duplicate detection pattern:
Collection
↓
Track Occurrences
↓
Count / Check Existing
↓
Return Duplicates
Recommended approaches:
Fast duplicate detection
Use:
HashSet + Stream filter()
Need frequency
Use:
Collectors.groupingBy()
+
counting()
Custom objects
Use:
equals()
+
hashCode()
Complexity:
HashSet approach:
Time: O(n)
Space: O(n)
Grouping approach:
Time: O(n)
Space: O(n)
Frequently Asked Interview Questions
Q1. Why use Set for duplicate detection?
Because Set provides constant-time lookup.
Q2. What does groupingBy do?
It groups elements based on a classifier function.
Q3. Difference between distinct() and finding duplicates?
distinct() removes duplicates.
Duplicate detection finds repeated values.
Q4. How are object duplicates detected?
Using:
equals()
and
hashCode()
Interview Tip
When asked:
"Find duplicate elements using Java Streams."
Explain:
- For performance, use HashSet with stream filter.
- For frequency analysis, use groupingBy + counting.
- For objects, define equality correctly.
- Consider duplicate requirements:
- unique duplicates
- frequency
- first duplicate
- Discuss time and space complexity.
For senior Java interviews, discuss:
- HashSet internals.
- Stream pipeline design.
- Collector operations.
- Thread safety with parallel streams.
- Production data processing patterns.
This demonstrates strong understanding of Java Streams, Collections, and efficient duplicate detection techniques.