Blog

Machine intelligence is the last invention that humanity will ever need to make.- Nick Bostrom<h3>Introduction</h3>These days, definitions of AGI are a dime a dozen. “Human-level intelligent machines,” “a general problem solving algorithm,” “a recursively self-improving system”—these are a few of the catch-phrases you will hear. Sam Altman recently made headlines saying that superintelligence may be just a few thousand days out. But with shifting goalposts and fanatic discourse at every turn, it remains unclear what that even means, or what superintelligent AI should actually look like. What is really in store for us?I’d like to share with you a vision of the future that I’ve had recently. In this vision is something I call: the AI Entrepreneur.&nbsp;The AI Entrepreneur is an AI system that is capable of founding and building a successful company. It can brainstorm ideas and create a business plan. It can oversee hiring, tech, sales, customer service, and marketing. It can handle finances, legal and PR concerns. And most importantly, it can ship product. In short, it’s an AI that can do all the essential tasks of an entrepreneur.<h3>Can we build it?</h3> Will it be difficult to create the AI Entrepreneur? Certainly. But with everything we know today about deep learning, it’s much more tractable than other problems frequently associated with AGI. And here’s why:The most effective uses of AI involve modeling a distribution of labeled or human-generated data on a massive scale. While much of the current controversy in AI revolves around whether large models “truly” reason, generalize, or exhibit intelligence, it turns out that for a large number of practical use cases, it doesn’t really matter. In the end, interpolating a sufficiently complex training set and patching up any edge cases works wonders for real world problems. The bitter lesson for staunch skeptics, doomsday critics, and AI gospel-preachers is that simple imitation rules.That’s good news for the AI Entrepreneur: it doesn’t need entirely novel behaviors to emerge from some magical algorithm, it can simply emulate entrepreneurs who came before. After all, why reinvent the wheel? Humans have been doing entrepreneurship for nearly 20,000 years, and although the specifics morph and evolve, the basic principles change little. While it’s true that innovation often plays a crucial role, one can certainly identify formulas for success. There is a finite (albeit large) set of observations and actions an entrepreneur needs to take to succeed, and, in principle, these can be learned by example. Put simply, entrepreneurship is possible to imitate.That starkly contrasts with many other problems commonly viewed as “end-game” for AGI. Take “doing science” for example. Researchers frequently say things like “we’ll know we have superintelligence when it discovers novel physics,” and anything less isn’t really superintelligence. But such end-goals extend beyond what is currently possible with deep learning. Consider: if Einstein spent his lifetime meticulously studying the behaviors, thought patterns, and habits of Newton (rather than the fundamental nature of reality), would that have led him to relativity? Of course not. Relativity requires going beyond Newtonian physics. This example is a bit contrived, but novel discoveries mean changing paradigms; revolutions; regime shifts. And playing by the old rules doesn’t work in a new regime.A similar point could be made for the adjacent end-goal of having an AI “do science” in machine learning research; and taken to its extreme, having a recursively self-improving AI. In silicon valley and beyond, a ridiculous amount of hysteria surrounds this concept. But it’s at least clear that scaling up prediction algorithms on a fixed human-generated dataset isn’t enough to reach this end-goal, even when the compute bill enters the billions. We don’t yet know how to build algorithms that handle regime shifts or approach singularities.We do know, however, that deep learning today works for problems that can be learned by example. Once framed in this way, there are really just two core challenges to create the AI Entrepreneur: 1) collecting mass-scale high quality data from entrepreneurs: decisions, organizational structures, processes, reports, communications, and so on. Enough to fully demonstrate how to be a founder, and enough to prevent overfitting. And 2) devising a simple, scalable architecture conducive to these data types.<h3>Should we build it?</h3>Once created, the AI Entrepreneur is effectively a money printer. Except it’s better, because while generating returns, it actually adds value to the world. At the end of the day, successful entrepreneurship means satisfying customers, which means fulfilling the needs of real people. Moreover, the AI Entrepreneur can offer insight to human entrepreneurs on how to improve, the same way chess professionals improve their game using an AI engine. The AI Entrepreneur can do a lot of good.Yet, a fear may be rising that giving agentic, financial, and legal powers to an AI is a recipe for disaster. Once that Pandora’s box is opened, how can we keep it on a leash? What if the AI gets out of control? How can we ensure it stays aligned with human interests and values?I argue that, in fact, putting an AI in charge of building a company actually gives society maximal control over it. In contrast to the current paradigm of having top labs cook up powerful intelligences behind closed doors, the AI Entrepreneur must answer to the public. Individuals can harness sophisticated economic and legal structures to control the AI’s behavior—structures that humans have developed over centuries by trial and error. Customers can vote with their wallets. Governments can enforce competition. If the company missteps, it can be sued. Banks can limit funds the AI Entrepreneur accesses. Bad behavior can cause reputational harm. The public can invest when confidence is high. Shareholders can jump ship if business goes south. And so on.Moreover, it’s quite easy to start small. The AI Entrepreneur can begin with tiny companies that fill simple needs. Then correct any issues observed, and try something slightly larger. Developing step by step avoids catastrophic failures that may result from releasing a powerful AI at scale before it has gained our confidence. In addition, throughout this process, it’s quite certain the AI Entrepreneur will be under extreme scrutiny. The watchful eye of the public and news media will enforce a high degree of transparency, resulting in either public trust or skepticism. The first one to develop the AI Entrepreneur has the blessing and curse of enormous free publicity.Lastly, recall that the AI Entrepreneur simply learns by imitating human entrepreneurs. Its abilities and faults are just the ones present in its training data. In the distant future, perhaps a singularity will come when machines improve their own code and intelligence explodes. This should maybe even concern us. But in the paradigm of deep learning, data trumps code. An AI Entrepreneur built using deep learning can only significantly improve by collecting more/better data from real people. Thus, humans ultimately set the pace. Here, the AI Entrepreneur’s inability to go beyond its training is a strength, not a weakness.<h3>The road ahead</h3>The largest gap between current approaches and the AI Entrepreneur is surely interfacing and interacting with the real world on a large scale. Text from the internet and RLHF can yield helpful conversational assistants one-on-one, but it likely requires more sophisticated techniques to interact with customers, investors, tech, marketing and more. It is perhaps possible to run a company with a text interface alone, but it is very far from ideal. A good AI Entrepreneur should seamlessly integrate video, audio, tool use, calendars, email, and more.Another key problem to solve is how to give the AI Entrepreneur all of the relevant context for decision making. Modern techniques can extend textual contexts past 1 million tokens, but there needs to be a good way to embed all the relevant information about the company, relationships, the competitive environment, the political winds, customer wants, etc. Successful leaders often have an acute sense of broad trends coursing under the surface of society and an instinctual intuition for the environment they operate in. Deeply understanding this ability and replicating it may be one of the more difficult problems.While these challenges are daunting, one cause for hope is that the introduction of the AI Entrepreneur actually changes the startup ecosystem. These very problems that the AI Entrepreneur faces, from the outside, actually look like business opportunities. Once the AI Entrepreneur controls capital, it becomes a potential customer for other companies. Catering to its needs, providing it services, and solving its problems all become viable market strategies for savvy startups.<h3>Conclusion</h3>I see a future with the AI Entrepreneur in it. I think arguing over whether this system is truly AGI or superintelligence amounts to wasted breath. Whatever we call it, I argue that it’s tractable to build, it’s beneficial to mankind, and it’s highly controllable using societal levers we are all familiar with. So, next time you catch yourself saying “here’s a crazy business idea” or “why isn’t there some startup that does X,” just imagine the AI Entrepreneur hearing you, then immediately springing to action.- Cameron Witkowski <h3>Acknowledgments</h3>Thank you to Sean Peari, Kyrylo Kalashnikov, Aman Bhargava, and Alexander Detkov for reading and providing feedback on initial drafts of this post.

The AI Entrepreneur

Say you are walking on the street. You stop someone walking towards you and ask him, “What is AI?” One of three images will pop up in his head:<figure id="figure-n5m8rypkxplxw5o3c6"><img id="n5m8rypkxplxw5o3c6" src="https://cdn.durable.co/blocks/23jfpdgZIV7UxzVVkytyd95CDV5jJJEYSAE8WKq9Qvf3ALPuu2yrzG4gpuTZonxz.png" alt="YouTube.com/@DreySantesson" title="YouTube.com/@DreySantesson" class="object-cover bg-gray-100 " style="aspect-ratio: 16/9; object-position: 50% 50%;" data-corners="no-rounded" data-aspect="16:9" data-caption="" data-positionX="50%" data-positionY="50%" data-fullWidth="false"/></figure>A. He has no idea what AI is.<figure id="figure-0bo8pifzlglxw5r9y3" class=""><img id="0bo8pifzlglxw5r9y3" src="https://cdn.durable.co/blocks/2aB6JX6Zej5lhjwvK9Nv4KdiCTcBl8F5WUdJtZ6gsS5DdJ4CPnCAaR7yCLG8kAnO.png" alt="" title="" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 1/1; object-position: 10% 25%;" data-corners="default" data-aspect="1:1" data-caption="" data-positionx="10%" data-positiony="25%" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/2aB6JX6Zej5lhjwvK9Nv4KdiCTcBl8F5WUdJtZ6gsS5DdJ4CPnCAaR7yCLG8kAnO.png" data-mce-style="aspect-ratio: 1/1; object-position: 10% 25%;"></figure>B. ChatGPT.<figure id="figure-xgpx0ihvtlxw5slzm" class=""><img id="xgpx0ihvtlxw5slzm" src="https://cdn.durable.co/blocks/34gFAU0fJbD1gMy60wMmBpVG2ycqorqlqLiVnXyjH40CH97vtKxkEpv58DDDKDZ7.png" alt="" title="" class="object-cover bg-gray-100 rounded-sm md:rounded-md lg:rounded-lg" style="aspect-ratio: 16/9; object-position: center center;" data-corners="default" data-aspect="16:9" data-caption="" data-positionx="undefined" data-positiony="undefined" data-fullwidth="false" data-mce-src="https://cdn.durable.co/blocks/34gFAU0fJbD1gMy60wMmBpVG2ycqorqlqLiVnXyjH40CH97vtKxkEpv58DDDKDZ7.png" data-mce-style="aspect-ratio: 16/9; object-position: center center;"></figure>C. Terminator (Credit: WIRED)&nbsp; &nbsp; &nbsp;Are you A, B, or C? Given how popular AI is, it’s surprising that many people still don’t have a deeper understanding of what it is. I often pass by someone on the street saying, “ChatGPT said that I should break up with my boyfriend” or some tidbit about how AI changed their life. But if I stop this person and ask her, “How does ChatGPT work? Why did ChatGPT say that to you?”, I will likely draw a blank stare.&nbsp; &nbsp; &nbsp;Many want to learn about AI but don’t know where to begin. I was certainly in this position when I started. This post will give you a quick, nontechnical introduction to AI. I hope it will provide a starting point for your adventures into this fascinating technology. If you want to dig deeper, I’ve provided links at the bottom to more detailed introductions.&nbsp; &nbsp; &nbsp;So what is AI? AI stands for artificial intelligence. When parents tell their kids that they’re intelligent, they mean that their kids learn fast and understand deeply. They can learn new words and understand books. When they’re older they can learn to drive a car and understand the tax system. There is almost no limit to what a human can learn and understand. This is human intelligence. When a computer program can learn and understand some or all of what humans can understand, we say that the program is artificially intelligent.&nbsp; &nbsp; &nbsp;Artificial intelligence is a field in computer science that studies the properties of these programs and invents new programs that are more intelligent and less prone to errors. The fruits of these efforts are programs such as ChatGPT and AlphaGo. By training these programs (teaching them what to do) they have learned a handful of tasks. ChatGPT can write and answer questions. AlphaGo can play go. But that’s about it. ChatGPT can’t drive a car; AlphaGo can’t play hockey. In fact, it is downright impossible for them to learn tasks that deviate too much from what they were designed for. Humans, on the other hand, can learn almost anything.&nbsp;&nbsp; &nbsp; &nbsp;What if we create a program that can learn anything a human can? Think about the implications: Every human achievement that made us proud as a species, everything that we believed only we could do, this program will be able to learn to do. If such a program could be created, it would be the first example of an AGI – an artificial general intelligence. In contrast, the current AI programs are narrow AI, in that they can only do a fraction of what we can do.&nbsp; &nbsp; &nbsp;Many AI scientists want to create AGI. AGI is the goal of companies such as OpenAI and Google DeepMind as well as much of the field of artificial intelligence itself. Why do so many smart people want to create AGI? Why is it worth pursuing?&nbsp; &nbsp; &nbsp;When we create an AGI, we will essentially be creating a mind. This AGI will be able to think; that’s why I call it a mind. This mind will be as smart as us, maybe even smarter. Given that the most wonderful achievements and creations of humankind start their existence as an idea in a human mind, AGI might come up with even better ideas that lead to even more impressive creations and achievements because it may be smarter than a human mind. A large percentage of our future achievements, be it scientific and technological progress or artistic breakthroughs, might trace their lineage back to the mind of this AGI.&nbsp; &nbsp; &nbsp;The impacts of AGI can be both good and bad. On the one hand AGI could cure diseases and solve climate change. On the other hand it could displace jobs and create a “useless class” composed of people who cannot get a job no matter how hard they try [1].&nbsp; &nbsp; &nbsp;As young people, we will benefit the most from AGI but we will also lose the most from an automated future. How can we ensure that this technology improves our lives and increases our chances at success? How should we mitigate the risks?&nbsp; &nbsp; &nbsp;We need to have this conversation because AGI will affect us all. AGI is a general-purpose technology, which means that it will not only impact students in engineering and the sciences but also students in the social sciences and the humanities. Only by inviting everyone into the conversation can we ensure AGI benefits us all. We started this conversation at UofT by creating the Society for the Pursuit of AGI. Discussing the potential and consequences of AGI requires incorporating ideas from diverse fields as art, philosophy, mathematics, engineering, literature, psychology, neuroscience, anthropology, sociology, and more. We invite your unique interdisciplinary perspective to join us and explore the future of AGI and our place in it. The ideas we come up with will guide our efforts to develop real AI systems that work.&nbsp; &nbsp; &nbsp;We seek those with curiosity, willingness to learn, and an ability to challenge their beliefs. The road to AGI is filled with excitement and challenges. To brave these challenges we need all the help we can get. But don’t forget about the excitement. Discussing AI with students from all kinds of disciplines will be a real intellectual feast. Our potential for growth and learning is immense. We hope to see you here at the Society for the Pursuit of AGI. &nbsp; &nbsp; Links · &nbsp; &nbsp; &nbsp;(Video) What is an AI anyway? By Mustafa Suleyman on TED. <a href="https://youtu.be/KKNCiRWd_j0?si=nAVdennzjc8Vj4cv" data-mce-href="https://youtu.be/KKNCiRWd_j0?si=nAVdennzjc8Vj4cv">https://youtu.be/KKNCiRWd_j0?si=nAVdennzjc8Vj4cv</a>· &nbsp; &nbsp; &nbsp;(Video) Computer Scientist Explains Machine Learning in 5 Levels of Difficulty <a href="https://youtu.be/5q87K1WaoFI?si=kfbyIHH7O_8zSCFK" data-mce-href="https://youtu.be/5q87K1WaoFI?si=kfbyIHH7O_8zSCFK">https://youtu.be/5q87K1WaoFI?si=kfbyIHH7O_8zSCFK</a>· &nbsp; &nbsp; &nbsp;(Video) What are Generative AI models? <a href="https://youtu.be/hfIUstzHs9A?si=No9j_Db36qJYKWsr" data-mce-href="https://youtu.be/hfIUstzHs9A?si=No9j_Db36qJYKWsr">https://youtu.be/hfIUstzHs9A?si=No9j_Db36qJYKWsr</a>· &nbsp; &nbsp; &nbsp;(Video) AlphaGo Documentary by Google DeepMind <a href="https://youtu.be/WXuK6gekU1Y?si=2DSGIHP0dR1fuG49" data-mce-href="https://youtu.be/WXuK6gekU1Y?si=2DSGIHP0dR1fuG49">https://youtu.be/WXuK6gekU1Y?si=2DSGIHP0dR1fuG49</a>&nbsp; &nbsp; References [1] &nbsp; &nbsp; Harari, Yuval Noah. “The Rise of the Useless Class.” ideas.ted.com, 24 Feb. 2017, ideas.ted.com/the-rise-of-the-useless-class/.&nbsp; &nbsp;

What is AI?

A semester has passed by since the birth of the Society, and it’s a good moment to reflect. Already in our first series of meetings, many of the central themes surrounding intelligence have entered the scene. Although one theme in particular seemed to lurk everywhere we turned…&nbsp; &nbsp; &nbsp;We first considered intelligence from the perspective of problem-solving. This general framework underlies much of the thought in the machine learning field, including the influential measure of intelligence by Francois Chollet, described as “skill acquisition efficiency” &nbsp;[1]. We investigated a number of missing components from the view of problem-solving, such as emotion, consciousness, and value. Whether these components are necessary for intelligence can be debated, but they certainly aren’t included in the naive picture of “general problem-solving ability”. Most damning and central was the&nbsp;question: what problems are worth solving? Or what skills are worth acquiring? We didn’t seem to have any framework that could appropriately answer this. How could the importance of problems possibly be modeled?&nbsp; &nbsp; &nbsp;To explore further, we dug deeper into a few central concepts we felt were underappreciated: imagination, emotion, and common sense—we didn’t know it at the time, but each of these would end up playing on the very same theme. What would it take to build an AI capable of imagination, emotion, or common sense? I’ll mention just a few of the interesting insights we came across.&nbsp; &nbsp; &nbsp;Imagination is actually a very general ability that includes counterfactual reasoning, planning, and learning from the past (ie. wishing you had done something differently). It includes any image or idea you might produce which is not forced upon you by your senses. Interestingly, (though it seems obvious in hindsight) everything you imagine must be some conglomeration or reshuffling of your previous experiences—a point eloquently put by Victor Gao. If I ask you to imagine a new color, are you really able to? Of course, it’s possible to imagine a color halfway between blue and green, or any other two colors you know. But a completely new color is outside the realm of even the imagination. Indeed, everything we imagine is like this.&nbsp; &nbsp; &nbsp;Imagination can be grounded in the discourse on machine learning through the concept of self-play, which was pointed out by Sheral Kumar. Typically, we train systems in a supervised or semi-supervised framework by presenting loads of human-generated or expert-curated data. But certain types of systems, such as RL agents, can generate training data for themselves. AlphaZero, AlphaGo, and recently AlphaGeometry are all great examples of the success this approach can bring.&nbsp; &nbsp; &nbsp;Clearly, imagination is worthwhile, and we can learn an immense amount by considering alternate sequences of events. But what in particular is worth imagining? Why don’t I spend this very moment imagining 13 orange rhinos, or a palm tree that grew in a weird L-shape? Why are certain things salient, interesting, and valuable to imagine, and others not? When we get right down to it, this question points to the very foundation of imagination—its driving force—and it’s essentially the same one that came up when thinking about “which problems are worth solving”?&nbsp; &nbsp; &nbsp;Next we considered emotion, and the critical role it plays in intelligence. Will AGI have emotions, and should it? Are emotions important to intelligence, or simply an unnecessary epiphenomenon. One crucial insight we came upon is, no matter how rational we believe ourselves to be, each of us is driven by emotion. The decisions we actually make, the paths we actually head down in life—these are consequences of our desires, ambitions, fears, hopes, loves and hates.&nbsp; &nbsp; &nbsp;But before I say any more, we must recognize the important difference between showing emotion and actually feeling it, pointed out by Sheral Kumar &amp; Liam Wall. Psychopaths and language models can clearly fake emotions—presenting the appearance of them without actually feeling. But what does this difference actually consist in?&nbsp; &nbsp; &nbsp;While posing this question introduces thorny issues about consciousness and subjective experience, one practical inroad is to consider: when do we feel emotion strongly, versus when is it insignificant and hardly noticeable? As a first sketch, we might say we feel most strongly about the things we care about—our lives, our prized possessions, our loved ones, our status and reputation. All of a sudden we come back to what is now a recurring theme: what do we care about? What is important after all?&nbsp; &nbsp; &nbsp;Perhaps common sense will take us in a separate direction and get us away from this problem of meaning. We can treat common sense simply as a set of skills or abilities shared widely and useful for frequently occurring tasks. But immediately, from the outset, it seems common sense is inextricably tied to task-completion, or problem-solving. A point made by Juan Rojas and Harsh Grover was that these problems were somewhat dependent on the person—for a criminal, it may be ‘common sense’ that they shouldn’t do blatant things that will get them caught. Here the ‘problem’ the criminal is trying to ‘solve’ is “how do I get away with it”. Not being caught is important to the criminal, but we still haven’t quite figured out: what is ‘importance’ in the first place?&nbsp; &nbsp; &nbsp;In our journey through this terrain of ideas, we kept running up against this one crucially important theme: what has value? What is significant, or meaningful? What resonates, and what do we feel strongly about? I argue that not only is this question largely ignored in machine learning discourse, but we don’t have any framework that is capable of answering the question of the origin of value.&nbsp; &nbsp; &nbsp;It is difficult to understate the depth of this question. I’m literally asking where does value itself come from. I’m asking what should any of us care about? I’m asking what is the purpose of our existence?&nbsp; &nbsp; &nbsp;At any other point in history, such a question could easily be dismissed as hopeless navel-gazing, or impractical idealism—to which the reply “put one foot in front of the next” would be appropriate. But today, as we build more and more lifelike intelligent systems, the question cannot be ignored. Because what will AGI care about? What will AGI value? What should it?&nbsp; &nbsp; &nbsp;Until we answer this question how can we hope to make any progress on the problem of alignment? What sense does it make to align AGI to human values, if we don’t fully understand what it is that we value, and where these values actually come from? If we don’t understand the nature of value itself?&nbsp; &nbsp; &nbsp;A naive answer to this question is to ground the discourse in physics (or evolutionary psychology). As physical beings maintaining our constitution against the forces of chaos, we must constantly resist the increase of entropy implied by the second law. Effectively, this means designing systems that can make better and better predictions, and will lead to (in the future) systems that can take actions which render the world predictable. In fact, you can derive quite a lot from the principle of least action. There’s a lot to be said for this view, and an enormous literature to support it [2] [3] [4]. I don’t have the space here to go too deeply into it, but I will point out (in my opinion) the main problem with the view of meaning &amp; value in terms of physics or evolution: the aesthetic.Man does not live on bread alone.&nbsp; &nbsp; &nbsp;If we’re minimizing entropy and maximizing efficiency, why ever take the scenic route? Why not beeline it straight from the start right to the destination? Why ever stop to smell the roses? Indeed, beauty seems quite orthogonal to the practical realities of life. Yet, what is life without beauty?&nbsp; &nbsp; &nbsp;When you look up from your screen and face reality again, ponder this. See the beauty all around you. Stop and really appreciate it, simply for what it is. To borrow the words of William Blake: &nbsp; &nbsp; &nbsp; &nbsp;To see the World in a Grain of Sand. And a Heaven in a Wild Flower.&nbsp; &nbsp; &nbsp; &nbsp;Hold Infinity in the palm of your hand. And Eternity in an hour. And this is all after just one semester of the Society. References[1] F. Chollet, “On the Measure of Intelligence” [https://arxiv.org/abs/1911.01547][2] K. Friston et. al., “Designing Ecosystems of Intelligence from First Principles” [https://arxiv.org/abs/2212.01354][3] M. Ramstead et. al. “On Bayesian mechanics: a physics of and by beliefs.” [https://royalsocietypublishing.org/doi/full/10.1098/rsfs.2022.0029][4] K. Friston et. al., “The free energy principle made simpler but not too simple” [https://arxiv.org/abs/2201.06387]

Blog

The AI Entrepreneur

What is AI?

But what problems are worth solving?