SP-1 / Lesson 5
For easier and less intimidating viewing, better use a bigger display. Unless you're watching a video or recalling something
Up until today, we have been mostly working with the discrete probability models. Working in a so-called "discrete set up" means that there are finitely many or countably many events, each has some wight, known as probability, assigned to it.
However, as you may know, there are Gaussians, uniform random variables, etc.. that take real values, and assigning its own probability to each $X = c $ for all real $ c \in [0,1] $ (for example) leads to a contradiction. In general, when dealing with continuous random variables, one needs to use a conceptually different set up.
Now, this course is not created to teach you at least a month worth of university-level probability theory, our goal is "Explore and Prepare for the Trader/Quant Jobs". We have already covered a decent part of the "Prepare" part as well as we have explored the job itself (well, mostly you did, if you tried to do the previous problem set well :)). The next steps in preparing for the interviews and assignments is much more theoretical, in particular for this part one must know how to work with integrals, what are standard continuous distributions (e.g the Normal one), have a decent understand of Central Limit Theorem and so on. There is no point in trying to compress all that into one less worth of material, you will not fully understand it if you have not done it before. Thus, the most optimal way of moving forwards for us is assume that you know all that and dive almost straight into problem-solving. This means that:
Moving on to the maths. We will not be diving into such formalities as sigma-algebra or measure theory. Those things are important, but they do not help as much with actually applying probability for real-world problems nor they significantly help passing the interviews. When it comes to the continuous set up, the standard and most common set up is the following:
Definition [Continuous set up; p.d.f]
Let our state space $\Omega$ be a subset of $\mathbb{R}$ that is either the entire $\mathbb{R}$, an interval or a ray. We define a probability density function (short: p.d.f) $f(x)$ as a non-negative function such that the area under it is 1. Formally speaking, we should have $\int_{\Omega} f(x) = 1$s. Note that we need function $f(x)$ to be integrable for all this of make any sense. Moreover, almost always assume even more: this function is continuous or continuous everywhere but finitely many points (and it is clear which ones).
Once we have the p.d.f, we can only be asked questions of the form "what is the probability of a point landing in the interval $[a,b]$". The answer to this question is then "this probability is equal to the area under $f(x)$ between lines $y=a$ and $y=b$". Note that technically speaking, the probability of landing in any segment $[a,a]$ is zero, however we do not bother seeing it as an issue. This is motivated by the fact that in the continuous world we are never really interested in the probability of landing at a specific point. Instead we are interested in probabilities of landing in a specific region, and this is what this model helps to answer.
This is it, the main building block of the continuous world. Next we can define random variables (same definition as before), talk about their p.d.fs (i.e functions that define probabilities of $X \in [a,b]$ for some $a,b$), then define their cumulative distributions (simply $F(t) = \mathbb{P}(X \leq t) = \int_{-\infty}^t f(u) \, du $), then define their expected values, variances, etc... The variance, covariance, correlation have literally the same definition as mentioned in the lesson 3, the only one that differs is this one:
Definition [Mathematical Expectation]
For a random variable $X: \Omega \to \mathbb{R}$, with probability density function $f(x)$, if both $\int_{-\infty}^0 xf(x) \, dx$ as well $\int_{0}^{\infty} xf(x) \, dx$ exist and are finite, then we can define the mathematical expectation of the random variable $X$, denoted as \mathbb{E}[X], as follows: \[\mathbb{E}[X] = \int_{-\infty}^{\infty} x f(x) \, dx\]
By the way, note that the integral limits often have $\infty$ floating around. However, we do not have to have random variables that are defined for all the real values or can take each of the real values. E.g a random variable taking values in $[0,1]$ only is perfectly fine one. The limits in the definitions above should then be technically adjusted, or we can simply assume all the functions to be zero everywhere where it is not defined.
Continuous world is large. Well, probability theory is massive :) But below you can find the main topics that are likely to come up during an interview, that we want you to recall.
For the reasons written at the beginning, todays classwork is not very informative. It merely reminds about the very basics of the continuous set up as well as mentions the basic things you need to remember or google more about. For a fundamental approach on this topic, please see the PA-5 course (or wait until your university, hopefully nicely, covers those things). The next step is a problem, which is not massive this time.