Is Reinforcement Learning Effective in Treating Mental Illness | Health and Technology

Is Reinforcement Learning Effective in Treating Mental Illness | Health and Technology
Is Reinforcement Learning Effective in Treating Mental Illness 

Can Reinforcement Learning Benefit in the Treatment of Mental Health Issues?

Stefano Goria is the co-founder and chief technical officer (CTO) of Themia, a company that aims to make mental health assessments faster and more accurate through an approach that combines the analysis of subtle facial expressions and speech patterns.  Combines video games based on neuropsychology.  Established in 2020, Thymia's arrival comes at a time when it is much needed in the wake of the growing mental health crisis globally in the wake of COVID-19.  As the technical co-founder of Themia, Goria is in charge of developing the AI ​​systems that underpin the company's end-to-end solutions to empower physicians.  With a PhD in theoretical physics and nearly a decade of building machine learning (ML) models for Citi and JPMorgan, he brings a unique perspective and array of skills to the task.

What is your career background and what is the inspiration behind Thymia?

I have a background in theoretical physics.  I did my PhD researching the theoretical aspects of the Higgs boson discovery some time before the particle was discovered.  After graduating, I worked as a quant for eight years – first at a software company and then at large US banks including Citibank and then JPMorgan.

In these roles, I did a wide range of modeling tasks, from classical statistical learning to reinforcement learning (which I love).  I also built on my machine learning and modeling expertise, sharpening my skills to ensure I could bring cutting-edge research into production.

Although these eight years of my career were very satisfying from a learning and technical point of view, I found myself yearning for a higher purpose and calling.  One day I looked around the trading floor at JP Morgan and saw a lot of super smart people incredibly focused on better shuffling of money and wondered: Why am I doing this?

It was then that I decided to channel my energy and drive into something else to join Entrepreneur First in March 2020.  Their approach is to invest in people before a company even exists, bringing together a group of diverse backgrounds and placing them in a structured environment to prepare co-founders for venture capital investments.  It's a difficult proposition to work with, but it was wonderful for me as I got to meet my co-founder, Emilia Molympakis, who had the initial idea and spark for Themia.  Seeing her pitch, I immediately realized that it would be the right way for me to use what I know about complex technology and modeling in a purpose-rich space to help people.

My co-founders have long been exposed to this domain as an academic, which inspired them to bridge the gap between what is essentially known and actually used in clinical practice in today's research.  does.  There is a huge tension between what can be done and what can actually be done, which is why thymia exists.

Today, we have more significant funding and are hiring a team of 12 people to expand.  Things are moving fast and changing a lot - in the beginning, I was coding everything and now leading a small team - and in a few months it will probably become even bigger.


Can you tell me more about the problems Thymia is trying to solve in terms of harmful biases in mental health care or data quality?

Mental health care is in crisis today, and it is reaching a furious pitch as the demand for help far exceeds the supply.  Part of the problem is that mental health care is massive and very complex.  Thymia is attacking a specific angle of mental health care: helping clinicians are objective measures of things that have so far been very subjective in two ways: what the patient is reporting about themselves and the underlying cause.  As what is assumed.

This is a big problem because if you don't have concrete objective metrics to identify symptoms, you have very weak tools for treatment.  The net result is that even if you can see a psychiatrist and the first day they tell you you are depressed, for example, it still takes a long and painful time to get the right treatment.  A big reason is that it's so hard to do something simple like see if something is working.

This is the core problem we are dealing with.  There are a lot of things that can be objective that are going to inform the diagnostic path and clinical decisions about what is the best treatment for a specific individual, but unfortunately these things are not always used.  has been done.  There are things that are known in research, but are not yet making it into clinical reality.  This is the big difference between a doctor who works in physical health and someone who works in mental health;  Today you can't ask for a blood test for mental health - it doesn't exist - and it's so important because it can give you a firmer control over how things are going.  It is not removing the human element, it is actually expanding the time for a physician to devote to the human aspects, rather than trying to infer things that can be measured.

So the main problem we are dealing with is to improve the quality of assessment and measurement of core symptoms relevant to mental health diagnosis, starting with depression.

How does AI fit in and what kind of models are you training and deploying in production?

What we are doing is very exciting from a scientific point of view.  From a data perspective, we are looking at three main modalities of data.  The first is the video, where we examine snippets of recordings and focus on micro-expressions - which can be encoded in a certain way called action units - and then examine things the way you do with your head.  How much you shake, whether you're looking at a part of the screen, whether your eyebrow is raised, whether you're smiling, and more.  We also look at speech and how someone speaks, hence the energy and speed and many other characteristics.  Some of them are intuitively related to depression (such as slow movement and low energy), while other connections are less intuitive but still there.

Then we look at the material.  What someone has to say is very informative - when you have a free speech-stimulating task like describing something that appears on the screen, the way you speak is very clear.  Are you, for example, using a lot of personal pronouns, are there an excess of "I" in relation to the wider population, are you more concerned about future events than the past—all of them  There are very important things that we can capture.

Then we look at behavioral data, which is everything that happens on screen that can be translated into an action with a timestamp.  So you can imagine a continuous stream of data as you do something on the screen.  We do this when we ask people to play simple video games, so we know how they react to specific inputs when they click.  The speed in creating the symptom profile, the number of errors in the game, the reaction time and other features are very telling.

These three modalities of data are very different, which pushes the model handling it to the limits of today's mainstream AI.  So if you look under the hood, we're using a huge collection of techniques - some of them in the deep neural nets area, while others are much more explicit feature engineering relevant to handling a large set of multimodal  Huh.  Timed events - so it is a combination of different techniques.  Sometimes, it's more relevant to look at unsupervised learning - so we look at clustering some features and mapping them to a set of traits - and other times we rely on supervised learning, where we look at that architecture  Trust that we use to change and heal.  Tune in to our model.

This is an interesting area where we can have a large amount of facilities, but not a large amount of data - so we need to be smart as it is not a matter of running the machine long enough to get the right answer.  It's a lot to understand the domain and know what works and what doesn't, which in itself is something very fascinating that brings me very close to how I work as a quant in finance.  Where a model was built - not just answered correctly - was one of the key requirements.

On top of that, we're now introducing reinforcement learning, which for me personally is fantastic because it's something I enjoy doing - including in areas like biodiversity, where I recently wrote a paper  Wrote that outlines an approach to deciding how to best protect geographic areas.  To maximize biodiversity over time.

We also use reinforcement learning at Thymia.  Specifically, we're targeting anhedonia because losing pleasure in doing things is one of the major symptoms of depression.  One neat way to deal with this is to play games.  These can be very simple games where a player chooses between a variety of options and does not know in advance which is good or which is less good.  It provides a very nice mathematical representation where the game can be solved as a reinforcement learning problem.  So what you do is compare the way the human is playing the game with the way the machine plays the game to maximize the reward.  Comparing these is very informative and essentially measures how much a person is enjoying exercise.

Established in 2020, you should have an interesting perspective on mental health in the wake of the pandemic - have you noticed any underlying changes or data drift since starting Thymia?

We were set up in the middle of the pandemic, so this has been our only reality.  We've seen companies in the mental health field really change the way they provide care and navigate a massive increase in demand.  Because of the need for remote care, there is little friction in trying out new equipment.  It will be interesting to see what the actual underlying changes will be in two years' time.

How do you navigate labeling and ground truth?

In terms of labeling, the data we are preparing generally falls under three categories.  The first category to understand symptoms is metrics - for example, we can measure speech rate or energy level in your voice.  These are objective things that can only be measured as part of an AI-powered mental state test, and there are usually no labeling problems with this category.

Other categories are at the symptom and diagnosis label level.  First, let me emphasize that what we want to produce is symptom measurement as opposed to looking at diagnosis because that's what is relevant to clinicians - they really want to have a firmer control over symptoms,  Because that is what opposes healing.  A plain diagnostic label.  So we can use the diagnostics label, but that's to help the model and is not really our main focus.

When we treat symptoms as a supervised problem, we necessarily have to rely on someone labeling the data – either the patient and the physician.  It may seem like a circular loop to try to get rid of subjectivity by asking the patient subjective questions to train the model.  However, the reality is that when you have thousands of these data points, it is not a barrier for the model to work.  On the other hand, we also ask physicians for their views.  Although this is a subjective measure, it is a high quality subjective measure given the domain expertise of the physician.

In building Thymia and thinking about how to improve this data over time, we decided to incorporate the core activities that we administer and the core model into a larger platform where clinicians interact with patients for all their purposes.  Will talk, so that there is a wealth of additional data that can be used to understand the whole picture.

We are also introducing unhelpful teaching practices.  Although this is a very early stage, it is definitely the way to go when the training data grows.  It would be much more interesting than having a bunch of features that speak direction without explicitly labeling a group of traits.


How do you monitor model performance?

Basic Infrastructure: Since Thymia was founded in 2020, we are beneficiary of the fact that MLOPS technology and knowledge is developed and well understood.  We started with a well-structured workflow - from data to modern prototyping to production quality models, deployment to production, and ongoing monitoring infrastructure - from the beginning.  This certainly cannot be taken with an older system;  Companies starting today are in an advantageous position because they already know what's important to track.

We think of monitoring in two axes: auditing and monitoring.  Along with auditability, it is important to demonstrate that the model is built on data collected in an ethical manner with explicit consent.  Along with monitoring, we want to see what the model's quality and performance is over time.

What do you think are some of the unique ethical concerns of AI in mental health care?

Broadly speaking, this is a broad topic.  We at Thymia are embedding a few things as core which include ethical use of technology, informed data use, and more.

One of the key things to emphasize is that we've done from the beginning - and we're making sure it's clear and implemented going forward - guaranteeing patients that the data generated by the model of thymia can be used  Will only be done in the clinical path and will never be shared with third parties, which is unfortunately common with popular mental health apps today.  We make sure that patients have a clear idea from the very beginning that Thymia is here to help their physicians and there is no other hidden aspect.

Ultimately, we think it's important to build a technology that can help in a variety of modalities.  If someone isn't comfortable going to a video, they might be comfortable using their voice or playing a video game.  No matter what, each is a great data stream for us and we will always provide communication, choice and transparency.

Broadly speaking, we are following the guidance of the UK Government and the Alan Turing Institute's Principles for Responsible Design of AI Systems, which is a great starting point for organizations to do the job right from the start.  We also follow the healthcare framework and are exploring possible certified parts of the platform as a medical device.

What is the common thread between mental health, biodiversity and financial services?

Applicability of reinforcement learning.  It is always a source of satisfaction that what I have learned is applied from one place to another and learns something new.  Every time I apply it it's so exciting, because when you frame a problem a little differently you learn a little more about the domain and more about the way the model works.

Furthermore, working on complex modalities with non-premium dependencies is another commonality.  And if you like finance and health, the flip side is that both are very regulated, so it pushes you to do things in terms of observability, interpretability and auditability from the start.  The general element is that this is a regulated space and you should be able to answer questions on what you are doing.

Source: Forbes.com, Directnews99.site