Remove all Duplicates

Pro Members Only

This lesson is available if you have an active subscription.

Alternatively, some member might be able to gift it to you.

If you already have a subscription, you can sign in.

Remove all Duplicates

Subscription Required

You must have an active subscription to access this content.
If you already have a subscription, you can sign in.

Problem

/**
* Remove all duplicate items from an array
*/
export function removeDuplicates<T>(items: T[]): T[] {

}

Solution

/**
* Remove all duplicate items from an array
*/
export function removeDuplicates<T>(items: T[]): T[] {
return [...new Set<T>(items)];
}

Transcript

00:00 In our previous lesson, we looked at the problem

00:02 of finding a repeated item within a given input array.

00:05 A closely related task

00:06 and something that you will actually use in your real day

00:08 job is the task

00:09 of removing all the duplicates from a given input array.

00:12 And in this lesson, we will look at an I solution along

00:14 with an efficient solution that uses any J script syntax.

00:18 So let's go. Permission should you choose to accept it is

00:22 to take an input array of any type type T

00:25 and return an output array.

00:26 With all of the duplicated items removed,

00:29 here's just an example of what we want this function to do.

00:32 Given this input array of numbers,

00:33 it should remove all duplicates

00:35 and there only duplicate over here is the number one,

00:38 and you can see that it is present only once

00:40 within the output array.

00:41 Right now we just have the scaffold

00:43 for this particular function,

00:44 and you can think for a solution if you want to.

00:52 Now, a quick, simple solution

00:54 and a naive one at that is to look

00:56 through the items in the input array one by one,

00:58 and then for each item, first of all,

01:00 check if you've already added it to the array.

01:03 And if we have, then continue.

01:05 Otherwise, add this item to the array

01:07 as this is the first time we've seen it

01:08 and we haven't found any

01:09 duplicates for this particular item.

01:11 Now, what makes this a naive solution is the fact

01:14 that we are looping through all

01:15 of the items in the input array,

01:16 and then we are essentially looping again within the

01:20 function that is result includes, this will actually go

01:23 through all of the items that we currently have in the

01:25 result till it finds one that matches

01:28 this results in an overall time complexity of ON square,

01:32 we can definitely do better than that.

01:34 And the answer lies in the set data structure.

01:37 The set data structure is essentially designed to make sure

01:40 that the items are unique.

01:42 So we replace our array based implementation with one

01:45 that uses the set data structure.

01:47 In this implementation, we still loop through all

01:49 of the input items, however, we simply add them to the set.

01:53 And if it's a duplicate set,

01:54 data structure will automatically ignore the duplicate.

01:57 Finally, now that we have a set, we have

01:59 to convert it back to an array.

02:01 And the simplest way to do that is

02:02 to use the built-in radar form function, which can be used

02:05 to convert any array like structure

02:07 into a true JavaScript array.

02:10 Now, as for the time complexity

02:11 of this particular implementation, we are looping

02:13 through all of the items still.

02:15 So that's all FN, but within the loop there is OF one

02:19 because the result add is an OF one operation resulting in

02:22 overall OFN.

02:24 Now, a neat thing about the JavaScript set data structure is

02:27 that you can pass in an input array to the set constructor

02:30 and the constructor will essentially loop through

02:33 that input array and add all

02:34 of those items uniquely to the set.

02:37 Now there is no point in creating a variable if

02:39 you're only going to use it once.

02:41 So we are going to inline the set constructor.

02:43 Now, one final tweak that we are going to make over here is

02:46 to utilize the fact that in addition to array form,

02:49 JavaScript provides an alternative way

02:51 for creating an array from any given Iterable,

02:55 and that syntax is known

02:56 as a JavaScript spread syntax. We create a new

02:58 Array and then we spread in our set Iterable into this

03:02 array, and this is the final array that we will return.

03:05 The set will take the input items,

03:07 convert them into a unique collection of items,

03:10 and then we will iterate over this unique collection

03:12 and we will add them into a new array,

03:14 which we will finally return according interview questions.

03:18 Go. Removing duplicates from a given input array is a pretty

03:21 easy one to solve, but then again,

03:22 many programming challenges are a few seen a decent

03:25 solution.