Why distance?

• Many problems that involve thinking about how similar or dissimilar two observations are. For example:
  • May use the same marketing strategy for similar demographic groups.
  • May lend money to applicants who are similar to those who pay debts back.
• Arguably the most important concept in data analysis is distance

Simple example

• Consider 3 individuals:
  • Mr Orange: 37 years of age earns $75k a year
  • Mr Red: 31 years of age earns $67k a year
  • Mr Blue: 30 years of age earns $68k a year
• Which two are the most similar?

On a scatterplot

Distance as a number

• It is easy to think about three individuals but what if there are thousands of individuals?
  • In this case it will be useful to attach some number to the distance between pairs of individuals
  • We will do it with a simple application of Pythagoras' theorem.

Finding the Distance

Finding the Distance

Finding the Distance

Finding the Distance

Finding the Distance

Finding the Distance

Finding the Distance

Finding the Distance

Finding the Distance

Euclidean distance

• In general there are more than two variables.
• Is there a way to apply our intuition in 2 dimensions to higher dimensions?
  • Pythagoras' theorem can be generalised to higher dimensions.
  • This results in a concept of distance called Euclidean distance.

Euclidean distance

We measure $p$ variables for two observations: $x_j$ is the measurement of variable $j$ for observation $x$, $y_j$ is the measurement of variable $j$ for observation $y$. Euclidean distance between $x$ and $y$ is:

$$D(x,y)=\sqrt{\sum _{j=1}^p{(x_j-y_j)}^2}$$

Vectors

• Notice that $x$ and $y$ are examples of vectors.
• For example $x=\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)$ where $x_1$ is age and $x_2$ is income.
• We can think of a data point as
  • A vector of attributes or measurements
  • A point in space
• These are the same thing.

Other kinds of distance

• We will nearly always use Euclidean Distance in this unit, however there are other ways of understanding distance
• One example is the Manhattan Distance also known as block distance.

$$D(x,y)=\sum _{j=1}^p|x_j-y_j|$$

Manhattan Distance

Distance and Standardising data

• We must be careful about the units of measurement.
• Euclidean (and Manhattan) distance change for variables measured in different units.
• For this reason, it is common to calculate distance after the standardising data.
• If the variables are all measured in the same units, then this standardisation is unecessary.
• Some distances are not sensitive to units of measurement (e.g. Mahalanobis Distance)

Distance in R

• R has its own special object for distances known as a dist object
• It can be obtained using the dist() function
• We are going to find Euclidean distances between the beers in the beers dataset. Use:
  • Only beers with price greater than $4.50
  • Only numeric variables.
  • Standardised data
  • Use the function dist to get the distances.

Load packages and data

library(dplyr)
Beer<-readRDS('Beer.rds')

Find Distances

Beer%>%filter(price>4.5)%>% #Only expensive Beers
  select_if(is.numeric)%>% #Only numeric variables
  scale%>%
  dist->d

Labels

• Only numeric variables were used to compute distances.
• The names of the beers are not attached to the dist object.
• This can be achived by assigning the beer names to attributes(d)$Labels
• Here d is the dist object.

Use Beer Names

Beer%>%filter(price>4.5)%>% #Only expensive Beers
  pull(beer)-> #Get beer names
  attributes(d)$Labels #"Attach" them to dist object

Your Turn

• Compute the distance without standardising the data.
• Compute the Manhattan distance for standardised data.
• Compute the Manhattan distance for unstandardised data.

Non-metric Data

• Can we define distance when the variables are non metric?
• The answer is yes!
• We will discuss two approaches:
  • Jaccard Similarity/ Distance
  • Dummy Variables

First a motivation

• Many people use music streaming services like Spotify.
• One of the attractions of these services is they they recommend artists based on the favourite artists of other users who have similar taste in music.
• The data in this case is in the form of a list of favourite artists.

Distance in musical taste

• Suppose there are three customers with the following favourite artists
  • Customer A: Post Malone, Drake, Lil Peep, Billie Eilish
  • Customer B: Post Malone, Lil Peep, Juice Wrld
  • Customer C: Billie Eilish, Ed Sheeran, Ariana Grande
• How do we measure which customers have similar taste and which have different taste?

Jaccard Similarity and Distance

• Jaccard similarity gives us a measure of how close two sets are, in this case the set of each customers favourite musician. The formula is $$J(A,B)=\frac{|A\cap B|}{|A\cup B|}$$
• Where $|A\cap B|$ is the number of elements in both set A and set B and $|A\cup B|$ is the number of elements in either set A or set B.

Jaccard Similarity

• In our example
  • $A\cap B=\left\{\text{Post Malone, Lil Peep}\right\}$
  • $|A\cap B|=2$
  • $\begin{array}{cc}A\cup B=& \{\text{Post Malone, Lil Peep,}\\ & \text{Drake, Billie Eilish, Juice Wrld}\}\end{array}$
  • $|A\cup B|=5$
• The Jaccard similarity will be $J=2/5=0.4$. The Jaccard distance is $d_J=1-J=1-0.4=0.6$

Using dummy variables

• Alternatively the same data can be coded using dummy variables:
  • $X_j=1$ if artist $j$ is a favourite of customer $x$
  • $X_j=0$ otherwise
• The usual distance measures such as Euclidean or Manhattan distance can then be used.

Collaborative Filtering

Figure by Mohamed Ben Ellefi

Recommender Systems

• Famous recommender systems are used by Amazon, Netflix, Alibaba amongst others.
• These systems are usually a hybrid of
  • Collaborative Filtering
  • Content-based Filtering
• The method we discussed is more specifically called memory-based collaborative filtering.

Axioms of Distance

• Care should be taken when using the word distance.
• In formal mathematics a distance is a function with two inputs that has to satisfy four properties.
• These four properties are called axioms.
• The distance measures that we have discussed satisfy the axioms

Axioms of Distance

1. Non-negative: $d(x,y)\ge 0$
   • There cannot be negative distance.
2. Symmetry: $d(x,y)=d(y,x)$
   • It cannot be a different distance from Melbourne to Brisbane that from Brisbane to Melbourne.

Axioms of Distance

1. Identity of indiscernables: if $d(x,y)=0$ then $x=y$ and vice versa
   • The distance from Melbourne to some place is zero then that place is Melbourne. Similarly the distance from Melbourne to itself is zero.
2. Triangle inequality: $d(x,z)\le d(x,y)+d(y,z)$ It cannot be closer to go from Melbourne to Brisbane via Sydney than it is to go from Melbourne to Brisbane directly.

Conclusions

• That concludes the topic on distance.
• This is relevant to the following topics
  • Cluster Analysis
  • Multidimensional Scaling (MDS)
• Now an exercise

Distances between tweets

• Find someone on Twitter or a similar social media site
  • Find the first two tweets
  • Think of a way to compute a Jaccard distance between their tweets
• Hint: Think of the words used in the tweet as a set