PA-1 / Lesson 1

For easier and less intimidating viewing, better use a bigger display. Unless you're watching a video or recalling something

ClassworkProblem Set

1. Introduction and Motivation

1.1 General Overview & Motivation

Probability theory is a branch of mathematics that can make you really rich nowadays. Besides the money aspect, it is a popular subject among experts in fields such as mathematics, computer science, physics, and producing scientific studies at the speed-of-light rate. Also, numerous research projects that rely on probability theory are significantly contributing to global advancements: you can become a part of those! Especially in Artificial Intelligence (AI) and Machine Learning (ML). Furthermore, probability theory is a fundamental tool for the field of statistics, which is widely used to make informed decisions or ... to lie to people.

Below are some of the fields of where one can (and should!) make use of probability theory. Basically any field, where a framework for reasoning under uncertainty is needed, the probability theory comes into play:

At this point, we don’t expect you to fully understand how it helps. But what you should know is that there are many ways to use probability theory. Knowing it well opens the door to lots of opportunities, especially now.

1.2 First steps into the Maths

So let’s take a very first step into the maths behind these. We will start with a seemingly simple question: ”what is probability?”. As you might already know, the probability of an event is a number between 0 and 1 that tells you how likely this event is: the bigger the number is, the more likely it is. But how exactly do we assign these numbers to some events? Well, clearly, number 1 corresponds to “will definitely happen” while number 0 corresponds to “will not happen for sure” (to be honest, even this is not exactly true... but more about this in another course). But what happens between 0 and 1? Also, if it is more convenient for you, you can think in terms of percentages rather than fractions: e.g $1 = 100 \%$ is ”will definitely happen”, while $0 = 0 \%$ is ”will not happen”. In general, we can think of any $x$ as $100x \%$, where $\%$ is basically multiplication by $1/100$.

Let’s move on to a classic example: suppose you have a die that has six sides with numbers 1, 2, . . . , 6. If you roll it, how likely will you see a number 1, 2, 3, 4, 5 or 6? Well, you will definitely see one of those, so it makes perfect sense to answer: “1” (or 100%). Okay, how likely are you to see number 1? In more mathematical words: “What is the probability of seeing 1?”. It seems reasonable to argue that ”Well, there are 6 very similar possibilities, and seeing $1$ is just one of them. So it should be 6 times less likely, and therefore, the answer should be $1/6$.” And it looks like a good explanation. At first ...

Exercise 1.

Does the explanation above mean that if you roll any die, then the chance of seeing a 1 is about $16 \% (\pm 1 \%)$?

Explanation and comments

click on blur to reveal

The problem with the argument above about $1/6$ is the following question: ”Why is it that seeing 1 is exactly 6 times less likely than seeing one of the numbers 1, 2, 3, 4, 5, or 6?” And there is a "No" answer to this question. Who said it should be the case that, say, 1 and 4 are equally likely to be seen? It can be false, actually!
Believe it or not, but the answer to ”why $1/6$?” is pretty much ”because we said so”. You might have certain motivations for why you assigned certain numbers to certain events, e.g., you honestly rolled your own die 1200 times and got 200 ones. But by the end of the day you say ”this is the way of assigning numbers I am considering, let’s work with this”. This might be confusing at first, so we will talk about it in more detail next time.

So even a seemingly easy question, ”What is probability?”, turned out to be not that straightforward. In fact, we have only shown you the tip of the iceberg, the proper answer to this question is complicated. But do not get scared. Not yet, at least ;) We will talk about everything in as many details as necessary.

However, even without any deep theories, it feels right that the chance of blindly picking Bob among 10 people (among which there is one Bob) is $1/10 = 10 \%$. By applying nothing more, but this kind of logic, try doing the quite well-known problem below:

Exercise 2.

Imagine that in a city, $0.1 \%$ of all the people have some kwacha-virus. So, it is a rather rare virus, but of course, no one wants to have it. And therefore Alla from the city decides to go a hospital and do a test to check if she has it. The test seems very good: it will correctly identify $99 \%$ of the people who have the disease and only incorrectly identify $1 \%$ of the people who don’t have it. Unfortunately for Alla, the test returned positive, meaning that she will likely have the kwacha-virus. But what is the actual chance that Alla has the virus?

Explanation and comments

click on blur to reveal

Suppose there are 100000 people living in the city (the exact number doesn’t matter; it will be easier to understand what is happening). Then we know that 0.001 · 100000 = 100 people have the virus. Now, what does the information about the test tell us? It basically says that if you test all 100000 people, then you will get around

  1. $0.99 \cdot 100 = 99$ correct positives for the virus (so miss just one person with the virus);
  2. $0.01 \cdot (100000 − 100) = 999$ false positives, i.e people who don’t have the virus but still got a positive test back;

And all we know is that Alla is among those $99 + 999 = 1098$ people who are positive. But only $99$ of them actually have the virus. Thus the chance that she actually has the virus is $99/1098 \approx 9 \%$. That is it, just $9 \%$! Have you expected it when reading the question?

Using the tools of the next course we will be able to answer such questions more easily using Bayes’ Theorem (that we will prove as well). But the explanation above, which is practically using just logic, is good enough already.

Conclusions and next steps

We have looked at various fields where probabilistic theories are important. Even without perfect understanding of how exactly probability helps, you should be able to see that it is not just useful, it is kind of everywhere. Hopefully this provides motivation to keep on doing probability-related maths. We have even taken our first steps into the maths world of all this, particularly solving a problem with a rather counter-intuitive answer. There are gonna be many problems like these in probability, actually... That problem is also an example of how to deceive with statistics, by the way.

There is quite a journey ahead for mastering the concepts and the tools that can make you a rich finance guy, a great critical thinker and data analyst, an expert in Machine Learning, and so on... This course's main objective is to see a number of probability applications and give you a taste of each. The next lesson will be somewhat philosophical and mathematical in this course, but first, we recommend you attempt the problems in the Problem Set. And read the o_O-story at the end of it.