Foundation Models for Reinforcement Learning, The GPT Hype, and the calls for alternative AI- Thoughts #1

This week has seen some overlooked developments in the field

Mar 02, 2023

Hey, it’s Devansh 👋👋

Thoughts is a series on AI Made Simple. In issues of Thoughts, I will share interesting developments/debates in AI, go over my first impressions, and their implications. These will not be as technically deep as my usual ML research/concept breakdowns. Ideas that stand out a lot will then be expanded into further breakdowns. These posts are meant to invite discussions and ideas, so don’t be shy about sharing what you think.

If you’d like to support my writing, please consider buying and rating my 1 Dollar Ebook on Amazon or becoming a premium subscriber to my sister publication Tech Made Simple using the button below.

Help me buy chocolate milk

p.s. you can learn more about the paid plan here.

This is going to be an underrated week for AI.

One particular publication will have potentially interesting consequences for the future of AI Research.

Deepmind’s Foundation Model for Reinforcement Learning

Deepmind released a foundation model for Reinforcement Learning (that’s my LinkedIn post about it, in case any of you want to discuss it there). Their publication is titled Human-Timescale Adaptation in an Open-Ended Task Space and it caught my eye for a few reasons. Firstly, are the claims-

we demonstrate that training an RL agent at scale leads to a general in-context learning algorithm that can adapt to open-ended novel embodied 3D problems as quickly as humans

The adaptability is particularly interesting. A big problem holding RL back has been the cost of exploring the rules of the system. The madlads at Deepmind countered this by implementing a Meta-Learning protocol to make the agents more adaptable-

An early reading of their publication does add to the hype around it. It seems like there are certain tasks, where our AI adapts quicker than humans-

As with all such papers, the devil is in the details. So a more detailed analysis will be needed before making any conclusions. However, this might have some implications going forward.

Is the Time for Alternative AI

What makes this recent development very interesting is that AI Legend Yann LeCun has posted a few months back about how the impact of RL had been as small as he had expected.

Yann LeCun @ylecun

The impact of RL has been exactly as small as I had predicted. The impact of Self-Supervised Learning has been even bigger that I had predicted.

Dan Becker @dan_s_becker

5 years ago (after Alphago beat Lee Sedol), I and many others thought RL would soon change the world. It's impact has been smaller than we anticipated (mainly due to the Sim2Real problem) If LLM's have less impact by 2027 than many now expect, what will be the reason?

While this tweet was accurate for the time, this Deepmind team might have taken it personally. Even if this publication does not go on to have a world-changing impact like finally getting Tottenham Hotspurs to win a cup, this should definitely make one contribution- boosting attention given to alternative forms of AI.

I know this is going to shock some of you, but there are streams of AI that have nothing to do with Large Language Models and GPT. The attention given to GPT has hijacked attention from these streams. Over this weekend, my university held a hackathon. The winner- an app that produced notes for students using GPT. Now given the delusion problem exhibited by these models, one would imagine that a note-producing app would have a few issues. But what do I know

Artificial Intelligence Made Simple

Serious Problems With ChatGPT And Other LLMs That Internet Influencers Won’t Tell You About

My last post on the free Alternatives to ChatGPT did really well. So now it’s time for me to rain on your parade and cover something extremely important. ChatGPT has been all the rage recently. People have been using it for all kinds of tasks- from writing sales emails and finishing college assignments, to even as a possible alternative to Google Search…

2 years ago · 11 likes · 2 comments · Devansh

The hype around GPT is a double whammy of bad- it causes GPT to be used in places where it has no business, and it takes away from genuinely amazing research being conducted in other types of AI. Take a look at the video below, which is about self-assembling AI. How many people even know about this, despite all the potential? How much compute/attention is dedicated to fields like Evolution and other paradigms? Not enough. Fortunately, there are some prominent voices in AI, such as

Gary Marcus

calling for more attention to alternate forms of AI.

This Deepmind Publication might just end up being a key reason that some of the hype is dispersed onto other fields in Machine Learning and AI.

Are there any fields that you think deserve more love? Let me know.

That is it for this piece. I appreciate your time. As always, if you’re interested in working with me or checking out my other work, links will be at the end of this email/post. If you like my writing, I would really appreciate an anonymous testimonial. You can drop it here. And if you found value in this write-up, I would appreciate you sharing it with more people.

Upgrade your tech career with my newsletter ‘Tech Made Simple’! Stay ahead of the curve in AI, software engineering, and tech industry with expert insights, tips, and resources. 20% off for new subscribers by clicking this link. Subscribe now and simplify your tech journey!