Category neuroscience

Exploration vs Exploitation

One of the inherent problems an agent faces in some arbitrary environment is how to decide whether to explore and discover more of the world around it, or to rather...

Free Energy of Expected Future

The active inference framework proposes agents act to maximise the evidence for a biased generative model, whereas in reinforcement learning the agent seeks to maximise the expected discounted cumulative reward....

The Mathematics of Predictive Processing

Hello! Today we’ll be discussing the mathematics of predictive processing - a modern theory for how much of the processing of information is done in the brain. This is also...

Category tutorial

A Productive Study and Research Workflow with Notion

As a student - or any busy professional - can attest, keeping track of tasks can be a nightmare. Besides simply tracking when tasks are due, it is often helpful...

So Many Options!

Introduction Fokker-Planck equations, and stochastic differential equations in general, have many powerful applications in a wide ranging set of fields. These include modelling of Brownian motion in physical systems, electronic...

The Mathematics of Predictive Processing

Hello! Today we’ll be discussing the mathematics of predictive processing - a modern theory for how much of the processing of information is done in the brain. This is also...

Category reinforcement-learning

Uncertainty in Model Based RL

Perhaps we could use uncertainty estimation to detect where the model may be wrong and then correct for these potential errors without having to collect much more data. This uncertainty...

Exploration vs Exploitation

One of the inherent problems an agent faces in some arbitrary environment is how to decide whether to explore and discover more of the world around it, or to rather...

Free Energy of Expected Future

The active inference framework proposes agents act to maximise the evidence for a biased generative model, whereas in reinforcement learning the agent seeks to maximise the expected discounted cumulative reward....

Introduction to Reinforcement Learning

But what is reinforcement learning? The field of reinforcement learning is at the crossroads between optimal control, animal psychology, artificial intelligence and game theory and has seen a surge of...

Learning with a Model

Where we are Up to this point we have discussed methods primarily relying on the learning of value functions, usually approximating these with some neural network. That is, our focus...

World Models: Learning by Imagination

The World Models (Ha et al., 2018) paper presented at NIPS in 2018 exploits the idea of having an agent train entirely within its latent representation of the world it...

Learning with a Policy

Model-free reinforcement learning algorithms are a class of algorithms which do not use the transition probability information to train and make decisions. In a sense, they are a class of...

Category artificial-intelligence

CRL Task 6: Causal Imitation Learning

And as quickly as that, we’re at task 6 of our quest. Causal imitation learning is perhaps the most fanciful-sounding, but at it’s core it remains as simple a goal...

CRL Task 5: Learning Causal Models

We’ve now come to one of the most vital aspects of this theory - how can we learn causal models? Learning models is often an exceptionally computationally intensive process, so...

CRL Task 4: Generalisability and Robustness

At this point we’ve developed a good sense of the technical theory of causal reinforcement learning. This next section brings together many important ideas and generalises notions of data transfer...

CRL Task 3: Counterfactual Decision Making

In the previous blog post we discussed some theory of how to select optimal and possibly optimal interventions in a causal framework. For those interested in the decision science, this...

CRL Task 2: Interventions - When and Where?

In the previous blog post we discussed the gorey details of generalised policy learning - the first task of CRL. We went into some very detailed mathematical description of dynamic...

CRL Task 1: Generalised Policy Learning

In the previous blog post we developed some ideas and theory needed to discuss a causal approach to reinforcement learning. We formalised notions of multi-armed bandits (MABs), Markov Decision Processes...

Preliminaries for CRL

In the previous blog post we discussed and motivated the need for a causal approach to reinforcement learning. We argued that reinforcement learning naturally falls on the interventional rung of...

Causal Reinforcement Learning: A Primer

As part of any honours degree at the University of Cape Town, one is obliged to write a thesis droning on about some topic. Luckily for me, applied mathematics can...

The Do-Calculus

This Series

Faithfulness

In the last dicussion we sought to rigorously define counterfactual statements and distributions in terms of our DAG formalism of causal inference. This appeared fruitful but the theory is certainly...

Reaching Rung 3: Counterfactual Reasoning

In our last discussion we discussed the so-called ‘rung two’ of the ladder of causation, discussing interventions and randomisation in control trials. This is an incredibly important field in the...

Interventions and Multivariate SCMs

Last time we discussed how we can learn causal structure from data and thought about how this relates to machine learning. Specifically, we noticed that having more data in a...

Causality and Machine Learning

Last time we briefly discussed the theory needed to start thinking about how we can learn, in the statistical sense, causal information from ‘dumb’ data. Some key points were that...

Learning Causal Models

In the last episode we developed the first tools we need to develop the theory needed to formalise interventions and counterfactual reasoning. In this article we’ll discuss how we can...

Causal Models

Last time we discussed and motivated the need for a modern theory of causal inference. We developed some of the basic principles necessary to develop this theory, but we have...

A Causal Perspective

What’s the first thing a statistician will say when you dare say the word cause? If you’ve ever taken a statistics class, I have little doubt it was the classic...

Uncertainty in Model Based RL

Perhaps we could use uncertainty estimation to detect where the model may be wrong and then correct for these potential errors without having to collect much more data. This uncertainty...

Exploration vs Exploitation

One of the inherent problems an agent faces in some arbitrary environment is how to decide whether to explore and discover more of the world around it, or to rather...

Free Energy of Expected Future

The active inference framework proposes agents act to maximise the evidence for a biased generative model, whereas in reinforcement learning the agent seeks to maximise the expected discounted cumulative reward....

Introduction to Reinforcement Learning

But what is reinforcement learning? The field of reinforcement learning is at the crossroads between optimal control, animal psychology, artificial intelligence and game theory and has seen a surge of...

Learning with a Model

Where we are Up to this point we have discussed methods primarily relying on the learning of value functions, usually approximating these with some neural network. That is, our focus...

World Models: Learning by Imagination

The World Models (Ha et al., 2018) paper presented at NIPS in 2018 exploits the idea of having an agent train entirely within its latent representation of the world it...

Learning with a Policy

Model-free reinforcement learning algorithms are a class of algorithms which do not use the transition probability information to train and make decisions. In a sense, they are a class of...

Category machine-learning

Predicting Protein Function with DeepChain

One of the most exciting applications of advancements in artificial intelligence has been the modelling of protein structures and protein folding. Last year DeepMind announced their AlphaFold project, which claims...

CRL Task 6: Causal Imitation Learning

And as quickly as that, we’re at task 6 of our quest. Causal imitation learning is perhaps the most fanciful-sounding, but at it’s core it remains as simple a goal...

CRL Task 5: Learning Causal Models

We’ve now come to one of the most vital aspects of this theory - how can we learn causal models? Learning models is often an exceptionally computationally intensive process, so...

CRL Task 4: Generalisability and Robustness

At this point we’ve developed a good sense of the technical theory of causal reinforcement learning. This next section brings together many important ideas and generalises notions of data transfer...

CRL Task 3: Counterfactual Decision Making

In the previous blog post we discussed some theory of how to select optimal and possibly optimal interventions in a causal framework. For those interested in the decision science, this...

CRL Task 2: Interventions - When and Where?

In the previous blog post we discussed the gorey details of generalised policy learning - the first task of CRL. We went into some very detailed mathematical description of dynamic...

CRL Task 1: Generalised Policy Learning

In the previous blog post we developed some ideas and theory needed to discuss a causal approach to reinforcement learning. We formalised notions of multi-armed bandits (MABs), Markov Decision Processes...

Preliminaries for CRL

In the previous blog post we discussed and motivated the need for a causal approach to reinforcement learning. We argued that reinforcement learning naturally falls on the interventional rung of...

Causal Reinforcement Learning: A Primer

As part of any honours degree at the University of Cape Town, one is obliged to write a thesis droning on about some topic. Luckily for me, applied mathematics can...

The Do-Calculus

This Series

Faithfulness

In the last dicussion we sought to rigorously define counterfactual statements and distributions in terms of our DAG formalism of causal inference. This appeared fruitful but the theory is certainly...

Reaching Rung 3: Counterfactual Reasoning

In our last discussion we discussed the so-called ‘rung two’ of the ladder of causation, discussing interventions and randomisation in control trials. This is an incredibly important field in the...

Interventions and Multivariate SCMs

Last time we discussed how we can learn causal structure from data and thought about how this relates to machine learning. Specifically, we noticed that having more data in a...

Causality and Machine Learning

Last time we briefly discussed the theory needed to start thinking about how we can learn, in the statistical sense, causal information from ‘dumb’ data. Some key points were that...

Learning with a Model

Where we are Up to this point we have discussed methods primarily relying on the learning of value functions, usually approximating these with some neural network. That is, our focus...

Learning with a Policy

Model-free reinforcement learning algorithms are a class of algorithms which do not use the transition probability information to train and make decisions. In a sense, they are a class of...

Category code

So Many Options!

Introduction Fokker-Planck equations, and stochastic differential equations in general, have many powerful applications in a wide ranging set of fields. These include modelling of Brownian motion in physical systems, electronic...

Learning with a Model

Where we are Up to this point we have discussed methods primarily relying on the learning of value functions, usually approximating these with some neural network. That is, our focus...

Learning with a Policy

Model-free reinforcement learning algorithms are a class of algorithms which do not use the transition probability information to train and make decisions. In a sense, they are a class of...

Category causality

CRL Task 6: Causal Imitation Learning

And as quickly as that, we’re at task 6 of our quest. Causal imitation learning is perhaps the most fanciful-sounding, but at it’s core it remains as simple a goal...

CRL Task 5: Learning Causal Models

We’ve now come to one of the most vital aspects of this theory - how can we learn causal models? Learning models is often an exceptionally computationally intensive process, so...

CRL Task 4: Generalisability and Robustness

At this point we’ve developed a good sense of the technical theory of causal reinforcement learning. This next section brings together many important ideas and generalises notions of data transfer...

CRL Task 3: Counterfactual Decision Making

In the previous blog post we discussed some theory of how to select optimal and possibly optimal interventions in a causal framework. For those interested in the decision science, this...

CRL Task 2: Interventions - When and Where?

In the previous blog post we discussed the gorey details of generalised policy learning - the first task of CRL. We went into some very detailed mathematical description of dynamic...

CRL Task 1: Generalised Policy Learning

In the previous blog post we developed some ideas and theory needed to discuss a causal approach to reinforcement learning. We formalised notions of multi-armed bandits (MABs), Markov Decision Processes...

Preliminaries for CRL

In the previous blog post we discussed and motivated the need for a causal approach to reinforcement learning. We argued that reinforcement learning naturally falls on the interventional rung of...

Causal Reinforcement Learning: A Primer

As part of any honours degree at the University of Cape Town, one is obliged to write a thesis droning on about some topic. Luckily for me, applied mathematics can...

The Do-Calculus

This Series

Faithfulness

In the last dicussion we sought to rigorously define counterfactual statements and distributions in terms of our DAG formalism of causal inference. This appeared fruitful but the theory is certainly...

Reaching Rung 3: Counterfactual Reasoning

In our last discussion we discussed the so-called ‘rung two’ of the ladder of causation, discussing interventions and randomisation in control trials. This is an incredibly important field in the...

Interventions and Multivariate SCMs

Last time we discussed how we can learn causal structure from data and thought about how this relates to machine learning. Specifically, we noticed that having more data in a...

Causality and Machine Learning

Last time we briefly discussed the theory needed to start thinking about how we can learn, in the statistical sense, causal information from ‘dumb’ data. Some key points were that...

Learning Causal Models

In the last episode we developed the first tools we need to develop the theory needed to formalise interventions and counterfactual reasoning. In this article we’ll discuss how we can...

Causal Models

Last time we discussed and motivated the need for a modern theory of causal inference. We developed some of the basic principles necessary to develop this theory, but we have...

A Causal Perspective

What’s the first thing a statistician will say when you dare say the word cause? If you’ve ever taken a statistics class, I have little doubt it was the classic...

Category statistics

CRL Task 6: Causal Imitation Learning

And as quickly as that, we’re at task 6 of our quest. Causal imitation learning is perhaps the most fanciful-sounding, but at it’s core it remains as simple a goal...

CRL Task 5: Learning Causal Models

We’ve now come to one of the most vital aspects of this theory - how can we learn causal models? Learning models is often an exceptionally computationally intensive process, so...

CRL Task 4: Generalisability and Robustness

At this point we’ve developed a good sense of the technical theory of causal reinforcement learning. This next section brings together many important ideas and generalises notions of data transfer...

CRL Task 3: Counterfactual Decision Making

In the previous blog post we discussed some theory of how to select optimal and possibly optimal interventions in a causal framework. For those interested in the decision science, this...

CRL Task 2: Interventions - When and Where?

In the previous blog post we discussed the gorey details of generalised policy learning - the first task of CRL. We went into some very detailed mathematical description of dynamic...

CRL Task 1: Generalised Policy Learning

In the previous blog post we developed some ideas and theory needed to discuss a causal approach to reinforcement learning. We formalised notions of multi-armed bandits (MABs), Markov Decision Processes...

Preliminaries for CRL

In the previous blog post we discussed and motivated the need for a causal approach to reinforcement learning. We argued that reinforcement learning naturally falls on the interventional rung of...

Causal Reinforcement Learning: A Primer

As part of any honours degree at the University of Cape Town, one is obliged to write a thesis droning on about some topic. Luckily for me, applied mathematics can...

The Do-Calculus

This Series

Faithfulness

In the last dicussion we sought to rigorously define counterfactual statements and distributions in terms of our DAG formalism of causal inference. This appeared fruitful but the theory is certainly...

Reaching Rung 3: Counterfactual Reasoning

In our last discussion we discussed the so-called ‘rung two’ of the ladder of causation, discussing interventions and randomisation in control trials. This is an incredibly important field in the...

Interventions and Multivariate SCMs

Last time we discussed how we can learn causal structure from data and thought about how this relates to machine learning. Specifically, we noticed that having more data in a...

Causality and Machine Learning

Last time we briefly discussed the theory needed to start thinking about how we can learn, in the statistical sense, causal information from ‘dumb’ data. Some key points were that...

Learning Causal Models

In the last episode we developed the first tools we need to develop the theory needed to formalise interventions and counterfactual reasoning. In this article we’ll discuss how we can...

Causal Models

Last time we discussed and motivated the need for a modern theory of causal inference. We developed some of the basic principles necessary to develop this theory, but we have...

A Causal Perspective

What’s the first thing a statistician will say when you dare say the word cause? If you’ve ever taken a statistics class, I have little doubt it was the classic...

Category finance

So Many Options!

Introduction Fokker-Planck equations, and stochastic differential equations in general, have many powerful applications in a wide ranging set of fields. These include modelling of Brownian motion in physical systems, electronic...

Category productivity

A Productive Study and Research Workflow with Notion

As a student - or any busy professional - can attest, keeping track of tasks can be a nightmare. Besides simply tracking when tasks are due, it is often helpful...

Category notion

A Productive Study and Research Workflow with Notion

As a student - or any busy professional - can attest, keeping track of tasks can be a nightmare. Besides simply tracking when tasks are due, it is often helpful...

Category research

A Productive Study and Research Workflow with Notion

As a student - or any busy professional - can attest, keeping track of tasks can be a nightmare. Besides simply tracking when tasks are due, it is often helpful...

Category natural-language-processing

Predicting Protein Function with DeepChain

One of the most exciting applications of advancements in artificial intelligence has been the modelling of protein structures and protein folding. Last year DeepMind announced their AlphaFold project, which claims...

Category biology

Predicting Protein Function with DeepChain

One of the most exciting applications of advancements in artificial intelligence has been the modelling of protein structures and protein folding. Last year DeepMind announced their AlphaFold project, which claims...

Category privacy

Practical Digital Privacy and Security

The layman view of privacy and security is that they are, for the most part, synonymous. Nothing could be further from the truth. Though seemingly harmless, the adage “if you’ve...