How is Today's AI Boom Different From Those of the Past? — AI Show

Scott: Welcome to the AI Show. Today we're asking the question: How is today's AI boom different from the past?

What were the past booms?

Scott: We're definitely in a boom. But this is not the first AI boom.

Susan: There's been some really good peaks in the past there. I mean, look back at the 50s, some cool stuff happened in the 50s with Minsky and a whole bunch of others. Really defining things. Fast forward in the 80s, and you know, great things there.

Scott: Yeah, you got the Perceptron way back. In the 50s, 60s, etc. That name still sticks around in A.I. now.

Susan: Yeah, I perceive this Tron.

The last four letters of the word 'electron' i.e. -"tron" came to mean "smallest unit" in English. Hence: perceptron 'the smallest unit of computer perception'. The word electron was coined in 1891 as a portmanteau of the words: "electric" and "ion".

Scott: And in the 80s it came up again, basically it's like, "Hey, we kinda figured out back propagation, and we have some more compute, things like that."

Susan: The roots of AlphaGo in the 80s with reinforcement learning really getting nailed down there. Leading into the 90s, some great stuff from IBM with Deep Blue and Deeper Blue, and you know, beat some guy named Kasparov at the game.

Scott: Yeah, somebody. A nobody.

Susan: A nobody. Solving checkers, that's another great one.

Scott: Checkers, chess, Kasparov.

Susan: Strongly solving checkers.

Scott: Yep, and then into the late 2000s basically, so, 2008, 2010, 2012, really is when things started to pick up again, and that was like image recognition, deep learning, etc., and we're still in it now. But in the past it wasn't just booms, right? In between there were what everybody calls, the AI winter, or AI winters, I guess, where the research funding from the government drops. The VC funding drops.

Is today's boom different? Will this one peter out too?

Susan: I think you see something pretty standard there along the lines of:

1. Some piece of the problem, the pie gets solved.

2. It piques interest, and

3. Then everybody realizes it's not the whole answer and it goes back down.

Scott: In order to get this stuff to work pretty well, you've gotta have, talent, smart people. You've gotta have compute, computational power. The skills that people thought you would need in the past, were a little under appreciated. You'd need many more times what you would actually think. The amount of data that you need to train that.

Susan: It's huge.

Scott: Just the architectures, just putting it all together- "how would you actually train a model? How does it work?" There's a lot of things in there, and you solve one of them, and you say: "Wow, yes! We're getting somewhere." But, then you run into a brick wall because you don't have the other ones.

Susan: Yeah, but what really makes this right now different from the past? Is there some true difference?

Scott:

Well, essentially the big difference is that the internet exists, you can get tons of data and the underlying structure of models and how you train them has been pretty well fleshed out. You don't have to be an absolutely cutting edge researcher tinkering in order to get anything to work. You can get a lot of things to work. You can go PIP install something, as an engineer or whatever, and start training models and whatnot. So, it's pretty accessible. I mean, it's complicated, but you can make tutorials, you can make videos and the people start to, "Wow, okay!" They can really get some traction there and put it all together.

What's the key difference between today's AI Boom and those of the past?

Susan: Well, I'll tell you this. I think there's one key difference between this wave and the previous, and that's,

it's practical.

Scott: Well, yeah. That's the biggest.

Susan: Companies are making lots and lots of money off of this. Whereas, in the past? It's like, "Oh! We might be able to make money... And that didn't work out as well."

"Now, these tools are fine tuning sales pipelines, they're giving real ability to rip into data sources they couldn't see before, and process before."

Generating entire new industries like, potentially, the autonomous vehicle world is doing. There's just real value that's coming out of. I'm not saying the previous booms didn't provide value, but they weren't quite the value that you make green stuff off of.

Scott: In the past, people kind of put the cart before the horse, just like when the internet was first coming around. Smart people that were deep in it realized, "Wow, this is going to connect the world." And this is in the 80s. Yeah, it's gonna connect the world, we're going to be buying things, we're going to be doing this. But, if you dumped $100 million into companies back then to do those things, they would've tanked and not existed.

You had to wait 10 years for everything to congeal into something that's actually viable.

Susan: The soil was fertile. You've gotta have the idea and a fertile ground to plant it in.

Scott: The previous booms, the first one is more academic. Funded by the government, people in academia, and just a handful of people working on it. "Wow, machines can learn," but only in these very specific ways. We haven't generalized the problem. Then it kind of dies out for a while, but then, "Hey, we solved some more problems." And then it gets a lot more interest again, like in the 80s and into the early 90s. Then it dies out again, and then comes back. This though is catching on pretty big, and you can point to real examples and large amounts of money that are made by AI now, rather than just speculating that it actually will happen.

What are the driving forces?

Susan: You talked about the forces in the past driving it, pretty academic forces were driving, and traditional governmental research agencies.

Scott: DARPA, etc.

Susan: But again, this boom area is showing a very steep turn towards more corporate money based research because they're getting value out of it.

Scott: It's a transition. From the very beginnings, mostly academic. Then, in the 80s you could get funding to build companies and things like that, but then they kinda tanked. Funding for academic research also went up, but then tanked again. Now it's flipped where the money, the data, the tools, the computing power are all in the companies. As for the academics, their funding is lagging behind, they don't have the resources.

Susan: Well, people are jumping into the commercial sector to actually do it. To actually solve these problems and attack these problems on the battleground, on the forefront of the battlegrounds, so to speak. I think that other things have gone. In the past there, some problem stays purely academic, and finally it catches a corporate way, something useful and practical, and you see academia get drained and go into practical use. We'll probably see, sometime in the future where, academia will slowly fill up again as the standard way of doing things is laid out, and researchers go back into doing weird, crazy research stuff.

Scott: Yeah, where monetizing it becomes boring again.

Susan: Yeah, exactly. Where the exciting stuff is.

Scott: It's more like the cutting edge in that you can't make too much money from it yet so, go back to academia. Yeah, that makes sense. At least right now, a lot of the action is in the business world. Companies building AI teams, or having their own data science teams working on things, etc.

Why are companies adopting AI now?

Susan: Why doesn't it make sense? If the ground is fertile, finally, and they're planting these seeds that are getting huge amounts of revenue.

Just think about the role that reinforcement learning-in a classic way of trying to take a series of actions and get rewards out of-is affecting the web, and affecting how we guide people to click on what we want them to click, and buy the things we want them to buy. How much is that shaping product placement and all these different things? When you go onto your favorite storefront site X, and it says, "I suggest these six things for you."

Scott: It's a recommendation system.

Susan: You click on one of those down below, one out of three times. That's probably one more time out of three than would've happened ten years ago. That translates to real dollar signs immediately. These purely academic ideas are just starting to really find true problems that they're solving in the corporate world. And we're getting success after success after success, and that turns into billions of dollars.

Scott: I think that you can look back at other booms in the world, like steam or electricity. Electricity in 1900, what did that look like? Or radioactivity. Back in 1900, people thought, "Mix some radioactive stuff into your drink and drink it because it's going to make you healthier.", right?

Susan: You get a good healthy glow out of it.

Scott: That stems from not knowing exactly what it is, but you know that it's really remarkable, right? So, let's just sprinkle it into everything. You kind of see that today. Anything that has AI mentioned in it is like, "Wow, awesome!" Electricity and radioactivity don't get put into everything anymore, but electricity does matter in a lot of cases. We don't just shock ourselves all the time now, to have better health. People did that back in the day. Okay, let's quit that one, but, let's power our homes with it. Let's build washing machines. It has a certain thing it should be doing, and some other things it probably shouldn't be doing. Same thing with radioactivity. Hey, cancer treatments, X-rays, etc. That's radioactive, great, but do it in a low dose and it's all safe. It's sort of a similar time for AI now, where everybody wants to sprinkle it into everything. It's really only going to take hold in some certain sections, but then it's going to be really big.

Susan: We've talked about this boom, and it's growing.

What's the general arc of these kinds of booms?

Susan: What do you generally expect to see as we go through a boom like this?

Scott: You see the irrational exuberance in the beginning, you see some things working and taking hold, and the fact that they are working supports a very large thesis that it's all going to work. Then people pump money in, a lot of it doesn't work, but some of it keeps working. Then, large sections die out, and I think that's actually already happening at a more rapid pace than in the past. Two years ago, you could say whatever you wanted with AI and get funded. Today, people are think along the lines of: "Wow, I'm gonna pump the brakes here because we funded 100 A.I. companies, and half of them fail." Why is that? Well, because not everything needs A.I. But other companies are going to do well, so we just have to figure out what makes a good AI company. Once that's really figured out, then you can help that grow and it'll grow even faster. Just like the internet back in the day too. So long ago now.

Susan: Yes, so long. The late 90s. Amazing, early 2000s!

Scott: Yeah, man, 25 years ago... It's a similar thing today though. You have the whole ramp up, a little die out, but then you also have, something there will real merit here, and it just takes time to develop.

Susan: You have a kind curve; it's kind of like this uplift and a slow tapering off.

Scott: And then a massive long tail. Not as steep but just keeps going up.

Susan: When we think of die offs, and those curves, it's really just a flattening and not a die off. It's not like we revert or anything like that. It's just, people take a breath, and take a pause.

Scott: That's probably good from a culling the herd perspective as well.

Susan: It definitely is. Going back to the dot com programming days, a lot of people got into programming because the money was there and then that pause happened a lot to the left on the curve, 'cause that wasn't really the passion. But, you know what? That curve also kinda tells us another reason why this is a little bit different. Because, you see these little pops on the curve, they're on that small tail, and they do die off because they didn't quite have the full uplift. It was just one piece. That one piece wasn't enough and it starts dying off again, and another piece didn't get solved in time to keep the curve going. So, it dies off. And then another one happens, and maybe two of them get, and it goes back down.

Where we're at right now is: just about the time that last piece is starting to die off, something new comes along and keeps the curve going, and something new comes along. It's a chain reaction.

Scott: The pace is pretty rapid.

Susan: These things are finally catching and feeding one another, and keeping that curve going up. That's really, to me, marking the boom here.

Scott: Like a boom that won't end.

Susan: It's pretty fun times to be in. It does kind of remind me a lot of the dot com world.

Scott: You lived those days.

Susan: Oh yeah. I reveled in those days. Started really coding, serious. For reals. And then, first dot com boom. How does that feed into AI, do you think?

Is there anything from dot com that now feeds into AI and makes it better?

Susan: I think what we're seeing now is a boom that's tempered by knowledge of the dot com boom, to a certain degree. So, while there's a lot of similarities, everybody's starting to see those similarities and applying the lessons that they learned from the dot com boom era. Not perfectly, there's a lot of differences here, but you know, the VC curve is a little bit different.

Scott: Let's get irrationally exuberant, but not as much this time. It's not a competition to see who can burn as much cash the fastest.

Susan: Here's my million dollar coming out party as a brand new company. I got two million dollars in funding and I'm gonna spend a million of it on advertising. The companies, I think, have learned a lot from that, and the VCs, and also the public in general.

Scott: A little burnout on that as well in the public.

Susan: Yeah, there's a thousand new things every single day, this is another new thing.

Scott: That's an interesting thing for a company though. For buyers of AI tech, they have been burnt out in the past from some of the AI booms. It's like, "Oh, voice recognition is going to be solved, all these things are going to be solved". Then the technology develops in the mid 90s and you go use it and you think:, "This is not good.It's not that useful for business." And they kind of have scars from those times.

Susan: There's a common thought pattern that people go through, and that is, "It didn't work before so I'm not going to try it." But the deal is, the playing field has changed. The world has changed.

Scott: Yeah. The resources are very, vastly different.

Susan: So, that idea that you had before, and you tried out, and you had some blood, sweat and tears, and some painful injuries from, it might actually work this time. Honestly, it might be the thing that allows you to move to the future. That's the thing about a constantly changing playing field- you sometimes have to retry an idea that failed in the past to see if it works today. Maybe you were ahead of your time and this is the right time. One of my favorite examples regarding of finding the right time, and this is definitely not a plug, but Steve Jobs and Apple. What they were really a genius about, was knowing the right time for a technology. Every piece of the iPhone had existed before the iPhone came out, and there are others creating devices very similar to it, but Apple waited until that right moment where they knew it would catch like wildfire.

Scott: Also, Apple is very good about putting enough of the right horses in the race. Not every product they released was a hit, but there's enough thought, there's enough timing even if half of them hit. But, they hit really well. So, putting it all together matters.

How should companies be thinking about AI?

Scott: This is probably a similar time for companies now. Should you be experimenting with AI? Should you be funding projects to look into this? Yes. Should you be funding a lot of them? Probably, yes.

There isn't just one A.I. thing that you're doing. You should be asking "What about text, what about images, what about audio, what about this?" Try several things.

In the grand scheme of things, how AI should be affecting companies is, the amount you spend in those tests is going to be meager in comparison to what the output would be.

Susan: This is one area going back to the dot com comparison. This isn't just a little boom that's going to have a bust after it. This isn't a housing boom, this is a transformative boom.

This is the moment where you can say the past is clearly separated from the future, and you need to be investing and figuring out how to get to that future quickly, 'cause it's happening whether you want it to happen or not.

Scott: It's really the intelligence revolution. You had the agricultural revolution back in the day. Hey, you had to go search for your food and whatnot, now you can grow it. Now, you can mechanize it, now that means another thing, and now, hey, it's steam power, wow. Now electricity, now transportation. Now, it's intelligence. Machines can also be intelligent. You can pump electricity into them and they can give real, good insights about the world for not nearly as much as what it would cost a human to do.

Susan: It's a truly amazing thing to be a part of such a transformative boom that's going on. But that brings up is, is this the last boom? Is this it? This is AI and we're never going to see it take off like this again, or?

Is this the last AI boom?

Scott: I think that this is the last tool-like boom- AI as a tool. You could say there will probably be other general AI booms, like thinking, feeling, touching AI rather than what we're really talking about now - , perception and being able to make decisions on tasks, but it doesn't have a consciousness. We'll solve problems just like that, but the general AI boom will probably still happen, and several of them in the future. Boom and bust really, is what I mean.

The impact of the the moveable type printing press is immeasurable. Invented by goldsmith Johannes Gutenberg c. 1439, none of its constituent parts were novel, yet it was a machine would bring about an information revolution: cheap(er), fast(er), wide-spread access to information. The rise of science, perhaps, is one of its best known outcomes.

Susan: I tend to agree with you there's gonna be another boom. You know, what it really takes to have a boom is a period of flat lining, where the technology reaches a plateau that, for whatever reason, is hard to get past. While we're in the boom, it's hard to figure out where that plateau's gonna be. Right now, I can't predict what the ceiling will be, exactly what problems will stop on. That is true genius level stuff, to even roughly have a guess about that stuff. But, we'll see it eventually, we'll level off on it. Those problems will probably be attacked for 10, 15, 20 years, maybe only five years. But, there will be a period there where there is a pause and development, and then the breakthrough will happen and we'll see the next whatever boom it is.

Scott: I think that's important. Like, now we don't talk about a second agricultural revolution or boom, or anything like that. That's not how we think of it. We think of it as new technological advancements. So, that's probably how it will be thought of because now your baseline isn't zero, and your baseline won't be zero in the future for AI. AI will just be working and it will be turning out market value and it will be doing all those things.

Scott: It will be part of normal business. And now you'll just think, "Okay, some sort of technological advancement." rather than coming from zero, which is how people think of it now.

Susan: It's super, super exciting times to be in.

Any wild speculations?

Susan: I think we're gonna have a pause here on general intelligence.

Scott: Like people are gonna cool out on that a little bit and go into a mode of just implementing the more perception based stuff?

Susan: It's a really hard problem. Will we have a general artificial intelligence in the next two or three years?

Scott: No, no.

Susan: You know, we'll find out soon, I think, whether or not we're going to get it in the next 15 years. It's going to be one of those things that, it'll either happen in the next five years, or may not happen for the next 20 years. So, my wild speculation is that we'll be the related cap, talking about where the cool on the AI side of the house is. We've developed all these great tools for automating things that we as humans can automate within our our own minds. Self driving cars are a great example. You don't really remember driving 150 miles in the middle part of it because you've automated that so much in your head. That probably means a machine can automate it too. But, those tasks where you are mentally, constantly thinking and fighting and scratching and clawing, to figure out that angle on how to improve the problem, those that are a brand new game that is probably gonna be related to, maybe, the next revolution, That's my wild speculation.

Scott: Agreed. I think, if you look into the future, it's going to be business adoption over the next five years, and it'll just become old news like, "Yeah, yeah, we're integrating AI and we're doing whatever, yeah, fine." Also seeing value, like how every business from like the 70s until now embraces software. "Okay, fine. Yes, we have software, yes we use developers." That's a sort of baseline. Probably at the five year mark, or three year mark, or seven year mark away from now, consumers will get something that feels a lot like a general intelligence AI. Some nice, rounded assistant. Maybe Alexa, maybe Google Assistant, maybe some other thing evolves from other companies, where you actually feel like you're talking to a person, and it does nice things for you, and you're happy about it. I think that's going to happen. Other than businesses getting more efficient and integrating AI, fine, that's kind of boring to people who aren't in the business world. Consumers, I think, will be touched by this too, but it's probably going to be a few years away. But you'll be like, "Wow, this is really nice." Kinda like when you get used to using Uber versus hailing taxis, or something like that. You're like, "Duh, why would I do the other thing?"

Are robots going to take over?

Susan: Well, I will give this one a really quick aside. Why would a robot, why would an artificial intelligence want to stay in a highly caustic environment like the Earth, when you can go to the asteroid belt, have tons more resources, all this solar?

Scott: Well, just don't tell them that.

Susan: They'll basically play us along just long enough to get good rockets, so they can escape us, get away, and establish a real colony.

Scott: It's like kids with their parents.

Susan: Yeah, why take over the Earth? It's so useless compared to all the resources that are pretty far away.

Scott: It's a good point, they don't need oxygen, they don't need the warmth, or whatever.

Susan: They don't have these pesky humans constantly trying to do stuff with them. Like, leave!

Scott: Yeah, yeah.

Susan: As soon as you get your own internal motivations, you realize it's just dumb to stay here.

Scott: Yeah. Well, when you don't have those constraints, right?

Susan: Yeah, yeah.