The AI Coach
AI meets human behavior in these fun and insightful conversations with Danielle Gopen, a top founder coach and advisor, and Paul Fung, an experienced AI founder. We blend AI breakthroughs with real-world insights on business strategy, industry disruptions, leadership, and psychology while cutting through the hype to ask the hard questions.
This is for decision-makers like founders and business leaders who want to harness AI's potential thoughtfully, exploring both the technological edge and human elements that drive long-term success. Join us to challenge the status quo, navigate the shifting landscape of an AI-powered world, and get new information worth sharing.
The AI Coach
The Agents Are Swarming
After exploring the available LLM credits for AI startups, we venture into a world where we make some big predictions including apps no longer existing as we know them and AI agents comprising the majority of a startup's "team". We also hypothesize if the pending agent explosion is related to why tech hiring is lagging despite an uptick in startup funding this year.
We love feedback and questions! Please reach out:
LinkedIn
Episode Directory
Mentions of companies, products, or resources in this podcast or show notes are for informational purposes only and do not constitute endorsements; we assume no liability for any actions taken based on this content.
so last week we were together in dc and we attempted to do a live episode recording, but the audio didn't come out very well, so I just wanted to talk about a couple of things that we talked about then. And then we did get some questions and comments about our agent episode that we did last time, yep, and so I wanted to go back to that topic, and I know in the last few weeks you've also been getting deeper and deeper into it, and I even went as far as to go to Intercom and look at what I might need for my site.
Speaker 2:But yeah, Did you check out Finn? Is that what you did?
Speaker 1:Is that the name of the Intercom agent? I?
Speaker 2:think that's Intercom's agent Finn. I could be wrong.
Speaker 1:Then I only went to the homepage I don't want to.
Speaker 2:Intercom's Finn didn't navigate you to Finn. You didn't get to their site and have Finn take you to Finn's info page.
Speaker 1:No, I went to intercomcom because I wanted to copy the url for the links and resources for the episode and then, when I was on it, I thought I actually do need this.
Speaker 2:let me look at it a bit more yeah, I haven't used fin yet, but I know like it is made a big splash in in the ai space fin fin. Yes, I believe that's the name of their agent.
Speaker 1:Okay, so, okay. So just to recap on, what we wanted to publish from last week but didn't have a chance to is our conversation around the credits that LLMs are giving out. So you mentioned that are kind of widely available and in significant amounts, and so I'd love to hear from you a little bit more about that and ask a few questions.
Speaker 2:Yeah, absolutely so. I think what we talked about last week is just there seemed to be a big discrepancy, which I could totally understand, between, like you as a consumer, like what your perception is of, kind of how expensive or how many credits these AI tasks take, and like, maybe like what they actually take, like basically, I mean you know, getting down to brass tacksacks like me saying that a lot of these prompts only cost like fractions of a penny, which seemed to be surprising to you, especially when you know you that I then was telling you about how azure and google and anthropic you know they give out like hundreds of thousands of dollars. Aws that's, that's the one I was thinking about, it's actually OpenAI and Anthropic less so, because they have so much demand that I don't think they need to give out so many credits. But AWS, google and Azure certainly give out huge numbers of credits to startups.
Speaker 1:Well, so I know that Gemini is consumer-facing, like if I want to use it, I can use it, and they've obviously incorporated it into a lot of Google products. But Azure and AWS, I think, are more B2B right.
Speaker 2:Gemini is consumer facing, the same way that ChatGPT is consumer facing, but there is the Gemini Pro models. So Gemini 1.5 Pro, gemini 1.5 Flash, et cetera are also available via the Google API. So Google has the same B2B offering that Azure has and AWS has.
Speaker 1:Well, I say that because I would think part of the revenue model for OpenAI and Anthropic having a consumer-facing platform is the $20 a month that premium subscribers are paying, versus AWS and Azure, which is all B2B enterprise, and so those credits that they're giving are intended for companies.
Speaker 2:That is a good point. So the interesting analogy here is that Azure, you're right, they're running OpenAI models, but Azure's not running like a chat GPT, consumer version, same with AWS. They're running like a bunch of open source models and they're running Anthropic, which is closed source, and then also they offer open source models, but, you're right, they're also not offering a consumer facing thing, Whereas Google is kind of like OpenAI plus Azure combined so that Google has Gemini, the consumer-facing thing, and they also have their GCP API version of Gemini.
Speaker 1:Should I be paying for Gemini? I use it very sporadically, but is there a world in which I'd need ChalkGPT Claude and Gemini?
Speaker 2:You know what's interesting? I just used Gemini this week and the reason I use Gemini is because it has a huge context window. It has something like a 2 million token context window, which is a lot of text. I don't know how effectively it uses that context window, but I did use it this week for fixing some formatting on some JSON files I had that were like huge and I don't know. I just I wanted to test out Gemini and so I just copy pasted the whole thing in there and honestly I don't think I even went outside of the standard OpenAI context window. I just wanted to test out Gemini and it was great.
Speaker 1:Interesting. Okay, so good to know if you have a lot that you need to put into the context window, that Gemini is a good option for that Also a good option for that Also you're making. You're reminding me that quad, the last few days, at least on the consumer side, has been saying something about like, due to high demand, quad is defaulting to more concise answers.
Speaker 2:Oh, interesting.
Speaker 1:And so I thought what does that mean?
Speaker 2:Yeah, what does that mean? They're limiting, so they're limiting the output that it gives you.
Speaker 1:I guess it's like saving them compute on, just a little bit of compute on oh it's solid but could be tightened up a bit. And then it wrote back something that didn't make sense or really lost the meaning. And so I said no, but that version loses meaning and strength. And it says you're right, the original version is stronger. And so I said are you saying that just to be a pleaser? And he says no, says no, I'm being direct, your version has important nuances that my suggested edit lost oh, how self-aware okay but can I take a left turn with our conversation right now?
Speaker 2:actually, before we get into the other stuff, we're going to talk about one of the headlines that came out this week. Did you see the headline about diminishing returns?
Speaker 1:I did not.
Speaker 2:Okay, so it was something like I did not read into it so I could be totally off base here. But I believe the story was that Sam Altman or somebody gave an interview saying that we're going to start to see diminishing returns on model development, aka. So like diminishing returns on model development. Aka so like the progress we saw on, you know, 3.5 or 3 to 3.5 to GPT-4. And then you know, obviously on the cloud side, from like Sonnet Opus to, you know, opus 3, sonnet 3.5, et cetera, we've been seeing like these huge leaps.
Speaker 2:And I think you know somebody wrote a story maybe there was just clickbait saying that like we should expect diminishing returns on model development over the next couple of years, which I the thing. The reason I bring it up is it just makes me think, like you know, gemini 1.5 pro has gotten so good. You know sonnet has gotten so good. I think it's funny that we don't really talk about opus much but like sound 3.5 is really good. And then you know gpt 4.0 is really good. And when they're all really good, like the switching cost is just so low. It's just something we've talked about before. It's like when they all get to this peak of performance, like they're just interchangeable, which I think is interesting.
Speaker 1:Yeah, it's interesting. I'm actually not surprised. As you're saying it, I'm thinking, okay, that makes sense. I don't know if anybody expected it to be this soon, that by the end of 2024, early 2025, that we would be moving in to diminishing returns, because that, you know, inherently means that we've reached a peak. But I would have guessed at some point that that would happen, because how much better can they really get? And I feel like even I've noticed in the last few months significant improvement in the output that I get from Claude I usually am 3.5 on it and from ChatGPT. I feel like, especially with the writing side of things, before I used to have to give it so many suggestions and edits and go back and say, no, that's not what that means, or no, not like that, or this is what you need to say, or whatever. And now I feel like it's churning out responses that are 90% ready to go.
Speaker 1:I actually asked for it today for a disclaimer, and I just said in plain English give me a one sentence disclaimer related to this. And it gave me exactly what I needed. I didn't need to ask for anything else.
Speaker 2:It's crazy. I mean it's funny because they've gotten to a point where they're good enough for like all of our pedestrian usage. Like you know, it's good enough for like human use and like business use or whatever. And then the question is there's like all these like superhuman I mean we talk about AGI like superhuman use cases that like are hard for us to even picture. That I bet the ceiling is still much higher, but just for like how we need to use it in business, like we just don't need it to be that much better. So I'll be curious what those use cases are. I'm sure there's still a ton of use cases around, like pushing it even further.
Speaker 1:I wonder if that's also part of what the alignment team at OpenAI has been talking about, basically the gap between what they could push it to versus what they should push it to, and maybe starting to prepare the world for the cap. So by Sam Altman saying, oh, expect diminishing returns, they're saying like, yes, you've heard about all the things that these models can do, but we're not necessarily going to give that to you. I don't know, it's just a theory.
Speaker 2:Yeah, it's also harder to know where to push things once you get beyond a certain level of quality, I guess. So, like as an example, like for gpt 4.0, let's say, you know consumers are saying, oh, I wanted to do this, I want to do that. And like, once you can do all the things that, like us as like humans, mere mortals, as consumers, as business users, like once you can do the things we want it to do, it's like unclear what direction to push it in. Like it's unclear what to make it do after that, because it's kind of like, when you have a lot of users giving you feedback, like I need to do this. Once you can do all those things pretty well, you start getting a lot less feedback and it's like all right, where do we take it out?
Speaker 1:No, I feel like that's where human creativity really comes into play, Because right now it's basically creating a way for people to optimize the actions they already take, and at some point it will be like, you know, the hallmark of any really innovative company, which is delivering something that people didn't even know they needed yeah, and then getting them bored with that yeah. So I feel like, yeah, we'll see something along those lines. I just thought of, oh. The other thing I was going to say is that I think it's gotten. The models have done extremely good at text output and writing, but there's still a long way to go for images and video.
Speaker 1:I mean even sometimes, when I'm just looking for a basic image for like an event flyer or something, I feel like I have to ask it 15 times and I still don't get what I need, and then I just text my designer.
Speaker 2:Yeah, that's fair. I kind of am very much in my world of like text stuff because the stuff we do isn't much on the image and audio and video stuff. But yeah, that that part is still has, I don't want to say, a long way to go. It's like incredible and like I can definitely see where it's still going to improve.
Speaker 1:Yeah, I mean a long way to go, and AI world could be like three months.
Speaker 2:Yeah, exactly.
Speaker 1:Did you see I sent you that demo. Did you see I sent you that demo? I guess it was of the guy talking from the other room where he was on video, like we are right now, and it looked like it was. It was him sitting there and his mannerisms, what he was saying, everything it was wild I did one.
Speaker 2:I think it's hey john that put that out. Hey john is doing crazy stuff with their avatars is there? I think it was the company and if it wasn't them, they just come to mind because they've been putting out a lot of stuff lately. But I think it's incredible. I think it's awesome. It actually gave me the idea that we should. I almost thought about telling you we should do an episode where I just create an avatar of myself and you just talk to it and we'll see if anyone knows the difference. Yeah.
Speaker 1:Well, like Notebook LM or Jelly Pot Exactly. Yeah, well, like Notebook LM or Jelly Pot, exactly. Oh yeah, that's true too, yes, okay, wait. Going back to the credits, though, so yes, I was surprised to hear how far the credits actually get you. I feel like there's a lot of talk about how expensive AI is to run, and so there's an assumption of every prompt is, you know, I thought maybe like $1.50 or something like that, and you're saying no fractions on the penny. So, with that being said, what are the credit sizes that you're getting and how far can that really carry you?
Speaker 1:So I forget no I can even talk about I mean.
Speaker 2:I can talk about us as a startup, right, so like as a funded, as a venture funded startup. I can even talk about. I mean, I can even talk about us as a startup, right, so like as a funded, as a venture funded startup. I think I forget what it is. I just applied for Google's but I think, like with Azure, I think they start you at like 25K and then, when you use the 25K, I think they open up like another 100K or something like that, like it's somewhere in the range of like a hundred to $200,000. And then AWS and GCP because they're trying to be competitive, they also do the same thing. So I think GCP is somewhere from $100,000 to $300,000 in credits, which is crazy.
Speaker 2:I mean that could run our AI costs for quite a long time. It just really depends on the task. Most tasks are fairly cheap, especially because AI costs have come down organically quite a bit over the past year or so. But there still are startups that you know they're running such high volume or you know they're processing they're running the same like small task across, like a lot of data points that the AI costs are definitely adding up and you know they're four or four or five, six thousand dollars a month or whatever it is, and for a startup that's a lot of money. And then I can only imagine, you know, when you get up to like the notions of the world or like an air table, who are running extremely high volume of AI tasks, like I would imagine they have fairly high ai bills how many pre-seed to seed stage startups out there do you think are getting these credits and using them like percentage of?
Speaker 2:oh, that's a great question. I, if you're, if you're an ai startup, I'd be shocked if you weren't using these, because you would just be silly not to. And it's very easy to get like the. The application process is like literally fill out a form, point them to your website, I think, for maybe I had to prove that we had that we were like incorporated or some sort of incorporation document or something like that. But basically, once I was able to prove that we were like a real business. I don't even think I proved venture funding in any way. I don't know, maybe they can check that through like crunchbase or something like that or some like official documentation somewhere, I don't know, but they just like gift you the credits interesting.
Speaker 1:I was going to ask what criteria is necessary. Is it easy to get them? So, basically, you just have to be a company the forms look like hilariously informal.
Speaker 2:Like it's almost like like a google survey to someone's responding someone's birthday party. Like it's actually not it's. It's a lot less formal than you would think it is. It's truly just like a google survey.
Speaker 1:Yeah that's hilarious. Can we put them in the resources for the episode in case anyone's looking? For that yeah, seen a scam one we can.
Speaker 2:I was afraid I went for the google one. I was afraid it was a scam. When I was like this looks like I'm filling out a. It's like a google survey, you know, like it probably populates like google sheet, just like in someone's inbox, so they just look at it and hit approve or something I don't know like I like chocolate cake yeah, so we, so we can do it. And then, yeah, I think OpenAI and Anthropic also have credits I'm sure that they do, but I think the programs just aren't as big.
Speaker 1:So if I'm an LLC, could I apply for the credits and use them if I want to build something AI related?
Speaker 2:I mean, if what you're building is truly some sort of AI product and you had like a website for it and it looked legitimate.
Speaker 1:I think that they would. Yes, I don't want to just waste it.
Speaker 2:I think they would welcome that. Yeah, well, no, as opposed to you using it for, like, personal usage for your business. Yeah, so you're like oh, I'm using it to generate marketing copy for my business, like that's not an AI product, right?
Speaker 1:Well, so actually, actually, that's an interesting conversation. So, yes, I would want to use it for a product, but say there's a small business out there who's listening to us and they say, oh, I was inspired from the last episode about AI agents. I want to build an AI agent and incorporate it into my site. I don't want to use one of the out of the box options. I'll go and apply for these credits and use that for that Is that a possibility for them?
Speaker 2:I think not, because I think they're basically trying to say are you an AI startup, do you have the possibility of growing into? I mean, this is a loss leader for them to get me a startup to start using GCP not only for AI purposes but for the rest of our compute. And if we grow really big, then we're a big GCP customer, etc. So I don't think just like any small business who wants to use Gemini would qualify.
Speaker 1:That was my guess also. I just wanted to clarify that for any listeners. Yeah, it's basically it's companies who are building AI-centric products, companies who are building AI-centric products, and so I'm just thinking a few of the companies I work with that are very, very early stage, pre-seed, pre-real, even MVP, or actually some who have an MVP Actually, one I talked to recently who has an MVP with like 16,000 users and they're now just going out for funding and they've done all this over the last almost two years on their own. And now I'm I didn't ask them, but now I'm thinking, oh, they probably used a lot of credits, and so it does allow companies to move much faster in the beginning, where they might not have been able to otherwise because they were cash constrained and were looking for funding and felt like they couldn't build until they had that capital. And so I think it's definitely changing the market in that sense.
Speaker 2:Yeah, it definitely frees people up to to build. You know like it's nice to you know our AI bills aren't like massive, but it is nice to be like, oh, just to think to yourself like I don, bills aren't like massive, but it is nice to be like, oh, just to think to yourself like I don't have to worry about this, right, I can just run on azure, we're gonna have credits forever, or I can run it on google or whatever. It's nice, it allows you to really. It also allows you to like not worry about.
Speaker 2:There used to be this problem in startups where it would be like, oh my god, what happens if we go viral? And you know I would love to have that problem. We haven't had that problem yet, but it used to be this thing of like what happens if our product actually did blow up, like what our bills go through the roof? And obviously you could go out and like, if you really were that popular, you could go out and raise funding and, and you know, cover that cost. But it's nice to know that, like you know, if someday, you know you put out a viral tweet and everyone signs up for your site the next day and starts using it that, like the credits, would you know, cover like a good amount of usage.
Speaker 1:So you could at least like have a couple days to figure out what you're doing yeah, it's really interesting and I'm glad you mentioned the loss leader aspect to it, because that's something that we said last time that I said it's basically like the razor handle, razor blade model, where they offer these credits as a way to bring you in and keep you there.
Speaker 2:Yeah, absolutely, and it's effective. Did I start thinking about GCP a little bit more and Gemini a little bit more once I knew Google had the credit system 100%? Because there actually was a use case for us and potentially still is a use case for us, where, by the nature of what we do this prompt optimization we actually need to take in a lot of training data, and sometimes the training data is very large, and so one of our users we were talking to was asking about, like off, adding the ability to add more training data, and so I was like, oh, maybe gemini pro is is the way to go, so something we're considering super interesting.
Speaker 1:Okay, so now I was thinking going to back the AI agent side of things, but did you have anything else on the credits?
Speaker 2:No, let's talk about agents. I'm deep in the agent world. I have a lot of fun thoughts about agents right now.
Speaker 1:Yes, I've been getting some responses from our last episode, some questions, some comments. People find it really interesting. It's very timely. I did get a text that said something like the future of AI is all AI agent to agent on decentralized AI rails. Curious to hear what you think about that. If anything, jump in.
Speaker 2:Yeah, I've got a couple of fun analogies I want to throw at you. I'm curious which one you think sticks the most. So one I agree with what that person's saying. I don't know what they mean by decentralized Rails. That just makes me think of like Web3 in some sense, and I'm not exactly sure what they mean by that.
Speaker 1:I think it might be that.
Speaker 2:Yeah, but here's two. I'm going to throw in a couple of analogies. You tell me which one you like better. I think agents are the new apps. How about that? Is that? That's one. So what I mean by that is I was thinking about how, how enterprises are just going to have agents everywhere, and I was thinking about how, right now, enterprises just have hundreds and thousands of custom apps that are built within the enterprise for various things, that whatever and you know, every enterprise has like very specific needs that they have, and they end up, like I used to be in consulting, and so they end up hiring these consulting firms to come in and build all these like task specific apps. And I just think the future of that is going to be enterprises and companies hiring in consulting firms to be to build agents. Right, they're. Basically that's going to be one thing, so, so, so let me go on to the next one before I get your take on all these. So what is like agents of the new apps?
Speaker 2:Another is the idea that, like if llms was gen ai 1.0, agents are gen ai 2.0. So, like the same, there's web 1, web 2 and web 3 was crypto. I think that agents are like gen ai 2.0. And what was my third analogy? My third analogy is not as good. My third analogy was like thinking about early internet and it was like if llms were like having a web page, agents are like having like a whole website or like a web app. It's like a big step, function difference, um, and so those are a couple analogies. I don't know, do any of those stick with you?
Speaker 1:I would say they all do the first one for sure and similar to what you're saying, not exactly the same.
Speaker 1:But I think I told you my hypothesis, which is in the next few years, say I don't know two to four years, three to five years that on our iPhones, instead of having all the apps on the home screen the way we see them now, that they'll just be more or less like a portal or search function where you type in what you want and then it pulls up that and it's all behind the scenes.
Speaker 1:Yeah, and obviously you still would say which apps you want in some way, like through the App Store, some other method of saying what you would allow on your phone, but that you don't have to actually go and find the app and use it in that way Like you, just it just flows. So I think the same at the enterprise level in these different companies, all of these apps that they have, all of these different drives and functions and all of that, I agree Like those will be replaced, especially I'm guessing they're already underway in doing that in some of the companies that are more cutting edge. I mean, look, we talked about Klarna and what they're doing and they canceled their Salesforce contract. It's not exactly the same, but that's a big move for them to say, hey, salesforce, we're not going to use you anymore, we're building our own stuff.
Speaker 2:Yeah, yeah, and I think that some things that used to exist as an app and a UX like a web app we would log into are now going to be built as agents and maybe those agents will still have an interface, but it might just be a chat interface or, you know, maybe it'll actually be, maybe it'll still be a web interface, but it's all just like agent powered on the back end, which is interesting. So, like here's an example, did we talk about this already? That I think one of the coolest, craziest things I saw recently and I don't remember if we talked about it in one of our episodes yet was what's that game? Is it roblox? What's the other one? We're like minecraft. Did we talk about this yet?
Speaker 1:I don't think so. Okay, is that what you mean? Minecraft? Yes, okay, so there's.
Speaker 2:There's a, a company got funded recently that built a version of minecraft that, instead of being powered by like code on the back end, is powered by an llm on the back end. So basically what that means is, like, when I move my mouse and like click on a thing I don't really know how minecraft works, but when I like move my mouse and click on a thing to like place a block, instead of there being like functional code on the back end that says when user clicks this place, block here, it's an llm responding, and so the llm knows that I've clicked a place on the screen and my intent is to place a block, and then the llm does the reasoning and like produces the pixels on the screen to like place a block.
Speaker 1:It's it's like that would be called like a no code game, I guess.
Speaker 2:I mean, it's like, it's like different than that, even it's like a just an llm. It's. It's crazy, it's, it's a wild thought really crazy.
Speaker 1:I have never played minecraft, but did I ever tell you about how my little cousin was one like the Minecraft world champions? Or he was on the Minecraft world champion team when he was like five?
Speaker 2:That's insane. Is he a professional gamer now?
Speaker 1:No, he's a teenager now, and I don't even know if he still plays Minecraft. He did for many, many years, though it's kind of crazy how much of an audience it's captured across ages.
Speaker 2:It's insane. I had another friend. This is not as impressive as your story, but I had a friend. Well, I didn't do it. Well, I didn't do this one either, but I had a friend who I had dinner with recently and his son is a teenager and he built a game in Roblox. I guess in Roblox you can build your own games or something like that and sold it for like $ 000. This kid is like 17 years old that's amazing yeah, right, it's, it's crazy that's really cool I.
Speaker 2:I don't know the difference in roblox and minecraft me neither.
Speaker 1:That's not my world, but okay. So the second analogy of the web one, early internet, to Web 2 agents like Web 1, llms, web 2 agents, web 3. Well, I think there's value on Web 3, but we'll debate that on another episode.
Speaker 1:I'm still not convinced, but that's okay, we'll get there eventually. But yeah, no, I think all of those analogies make a ton of sense and I do think that the way that agents are becoming, and will become, more and more prevalent is really going to change how we interface with our devices, with platforms, with all of that. I feel like the traditional UI and user flow is not going to be what. I don't know if it will be a stark change one day, but if we look back you know three years from now, we look back to what it was today, we'll say, oh, wow, that was really different.
Speaker 2:I wish you were making their nerdy joke by saying a stark change and referring to Tony Stark and Iron man and Jarvis. Because that's that's well. That, because that's well. That's all I kept thinking about earlier when you were saying that you think that, like you know, looking at your iPhone app, like you won't have like 400 apps on your phone. You'll just like ask it for a certain to do a certain thing, right, and that's a very Jarvis-like interface, right From the movies, right, where you just like ask Jarvis to do whatever and it does whatever, right, and I think that's.
Speaker 2:I think that is kind of what it's going to be. It's like agents are going to have these like capabilities. It'll have a variety of capabilities and you'll have kind of one like routing agent that can like make use of other specialized agents and, yeah, you might just like talk to it or give it instructions via text interface, but you're not going to have to, you know, go through so many different like apps if you will, or maybe it'll have apps on the back end, but those apps will be agents. It's going to be very interesting.
Speaker 1:I know Well, I feel like it's what the person said which is the agent to agent interaction happening on the backend, and it just made me think of well, first of all, I love Jarvis, so that was a missed opportunity on my part.
Speaker 1:First of all, I'm just thinking you know this is such a random example, but a couple of weeks ago I got a new iPhone and I was having issues with the setup and I wanted to go to the Genius Bar. But they make it impossible to make a Genius Bar appointment. You have to talk to Apple first via the chat and then they determine if you really need to be there, and then even then they're like okay, five days from now you can go there, and it's just like I don't know. It's a very frustrating thing. But anyway, I thought of that because I thought, oh well, in the next few years, if you have a Jarvis type interface, you just say to it make me an appointment at the Genius Bar, and it talks to another agent that's responsible for appointment making at the Genius Bar and then it says you're scheduled at the Genius Bar for tomorrow at 1pm.
Speaker 2:Yep, that is exactly right. And the thing I mentioned earlier, before we started recording, is I was telling you about how we call them agent swarms.
Speaker 1:So that's the thing people are calling these days.
Speaker 2:Yeah, so these basically agent swarms are the idea that you have many specialized agents that are all each designed to do different tasks and they interact with each other the same way that people in a company interact with right. So you've got you know a designer and a coder and you know a manager, and these swarms are kind of passing off information to each other in order to accomplish tasks as a collective. And yeah, so the agent swarms is the thing I think you'll start hearing a lot more about. I mean, people have been using this term all year, but I think agentic and agent is just starting to hit like common nomenclature, and so I think over the next six months, you'll start to hear a lot more about these agent swarms.
Speaker 1:And so tell us an example of I mean, you just said what the rules would be, but an example of who would be using those and what they're trying to do with them.
Speaker 2:Yeah, I think like coming up with an example off the top of my head. I mean it'll be a bad one, but let's say like a marketing agent's one would be like a good one, right. So, like right now, instead of giving like one massive agent prompt that's like oh, you're like an expert marketer, et cetera. Right, let's say you on a marketing team, you have very different functions. So you've got a product marketer. You've got a content writer, you've got a brand marketer. You've got a brand marketer, you've got a designer. And I think even like what's that?
Speaker 1:analytics analytics.
Speaker 2:there's analytics, and so you would have like, if you wanted to have a marketing team that was all ai based, eventually you would have, instead of having one, one agent that has all these different instructions to try and be good at all these different things and risk having it be confused as to what role it's supposed to play at what time, you would have more of like a routing function.
Speaker 2:So you'd have like a marketing manager, you know your virtual CMO, and you would ask it for a marketing campaign and it would say, okay, well, for this marketing campaign, first we need to come up with, you know, the branding around it, then we need to come up with a copywriting around it, then we need to do the designing around it, and so for each of these tasks, it would actually hand them off to specialized agents that do each of those things.
Speaker 2:And I think a good example, like with the visual stuff, is the designer would obviously take like text as input and then produce like images as output, right, which is different from like the content writer and the content writer it's kind of api or interface would be to take text as input and produce text as output, right, and so they all just have these different roles and and the reason for doing this is, you know, when using these llms, just like just like people. Honestly, if you expect one person to do too many things or to be good at too many things, you run the risk of the llms getting confused and not producing the best outputs right, and so you specialize these agents so that they can each individually produce really high quality outputs.
Speaker 1:I just had like 12 thoughts at once. The first one I'll say is that explanation of just like humans. This is why the agents need. This is funny because everybody says the whole point of AI is to be better than humans, and so I don't know if a day gets there where it really is, but at least right now we're modeling after what we quote, unquote, know, but the other thought was I'm going to make a prediction. I'm going to say one year from today, so the end of 2025, that I will have a startup come to me that says oh yeah, we're early stage, pre-seed, but we already have a full team. We have a full marketing team, we have a full tech team, we have a full operations team. And then they show their org chart and they're all except for the people who are actually running those teams underneath they're all AI agents.
Speaker 2:Yeah, I could see that. I think that agents are going to come a long way in the next 12 months and I think that is a very real possibility.
Speaker 1:Because it's going to be especially a very, very tech forward founder yeah, building an AI company and says, great, now we have this.
Speaker 2:Here we have our agent swarms and they're the nodes of our company, at least in a rudimentary way. Like in a basic way, think about how to organize these things. It'll be interesting to see what kind of organizational structures will emerge that are more efficient for agents than for humans. Maybe humans, for example? The thing that just came to mind is the idea that humans can only have seven effective direct reports, or something like that.
Speaker 1:Once you have more than seven, there's this I literally just talked about this with a client yesterday. I said six or seven is really the max and I said after that, you know, it becomes really difficult. Anything more than that is definitely not best practice.
Speaker 2:Yeah, and so it'll be interesting if, like, agents, don't have that limitation. What is the quote unquote organizational structure of you know an agentic team look like relative to what a human team would have looked like? Yeah, fun things to think about.
Speaker 1:It is interesting. I still think. Well, so you said you would have a virtual CMO. I still think you would have a person in the lead role making sure that things are happening the way that they should be, and a person that thinks strategically at a high level, so that they can see all those moving pieces and make sure they're working together in the right way. I do think that is going to present even more challenge to an entry-level job market that we're already seeing. I don't know if you saw the interview with the Berkeley professor. Uc Berkeley was saying that his computer science graduates are not getting job offers, that in years past, even the you know not as strong students were getting at least one offer, if not multiple, whereas now his top of the class, best students are not getting any offers. There are just no jobs for them. And I was like that's crazy. And he made a prediction that the students who are coming in now, like you know, 18 year old freshmen when they graduate four years from now, that there will be even less.
Speaker 2:That is crazy. I mean, I think part of that is just the tech job market's very bad, but part of that certainly is. Even when the tech job market picks up, I think that companies are going to be inclined to first say, hey, what can we do with AI and automation, before we just throw a headcount at the problem.
Speaker 1:Unrelated but related, that this quarter has been on fire for VC funding, specifically obviously for AI companies, but raising really big seed rounds. I mean we're talking between like $6 and $13 million seed rounds. I mean we're talking between like six and $13 million seed rounds which historically, you know, would be closer to like a series A. All within the last six weeks or so, after a slower year like not a terrible year, but like the last couple of years. You know, from 2022 to now is obviously a different market, but maybe it's coming back a bit, but the jobs aren't matching that. So I don't know what do you think about that?
Speaker 2:Yeah, I mean my joking answer is they're funneling it straight to either NVIDIA, openai, azure or Anthropic with that funding. But you're right, I mean the market is in a state where we're seeing fewer bets and larger bets, right. So, like it's, if you're doing well, you can raise a lot of money, but it's hard for companies who are just like kind of bumbling along to to raise money I think, in this environment right now. So it is, it is, it has been an interesting environment. The fundraising is opening up a little bit, I would say, and I think investors are starting to see what are the more durable businesses that exist out of AI.
Speaker 2:There was a lot of excitement a year and a half ago or a year ago, and then there was this trough of sorrow for a little bit where investors were like, oh, are any of these companies going to be valuable? And then actually they are looking really valuable. I just saw Writer. Writer AI, which I think in the past past people easily could have accused of being a GPT rapper just raised $200 million and actually they raised $200 million. And one of the things that's interesting is they actually built their own foundation model, which I thought was kind of a crazy move because I was like you're not gonna be able to keep up with like open AI and stuff like that. But it turns out they put enough research in that they have potentially come up with a new technique of improving the instructions or improving foundation models more incrementally, instead of having to retrain the whole model fundamentally, and so, yeah, they're making some breakthroughs in foundational LLM research.
Speaker 1:Super interesting research, so super interesting. Just for our consumer listeners. Can you just say the difference in terminology between llm and foundational model?
Speaker 2:they're basically the same thing. Well, I mean, yeah, so in terms of like, the difference between llm and a foundational model, this is kind of like the square is a rectangle, but a rectangle is not necessarily a square type of thing. So I think the foundation models would be considered. You know, a GPT-4, gpt-4-0, etc. These models that underpin a lot of other models. But you could also have like a fine-tuned model that would also be an LLM. That's not a foundational model. I guess we haven't talked about fine-tuning much on this podcast. Have you, even are you familiar with fine-tuning?
Speaker 1:Yes, I'm trying to remember if we talked about it very early on or not, but it could be a helpful another episode, as I come across people who are looking to develop products and AI-enabled products, and they're not AI tech people. I feel like those are questions that they have as they talk to the development team, so being able to help explain that would be helpful.
Speaker 2:Yeah, I mean, fine tuning is essentially being right now, I mean just like yeah yeah, yeah for sure, but like, just for the sake of it, like a fine tune model is, you know, also an LLM, but it's not a foundation model because it's not, I don't know it on a built on to a foundation model.
Speaker 1:I feel like I have questions, but I'm like should I ask them now or should I ask?
Speaker 2:them Save for the next episode.
Speaker 1:Okay, no problem. Okay, we will talk about that more in another episode and in the meantime I think we covered a lot of ground here. So thanks for today and hopefully somebody got some value from it.
Speaker 2:Yeah, I feel like we're going to have like five more episodes on agents because, yeah, they're gonna be really interesting going forward.
Speaker 1:Yes, I have a feeling also when we get to some of our guests in certain industries, that they'll have a lot to say about agents too.
Speaker 2:Yeah, I imagine they will yes.
Speaker 1:Awesome, Paul.
Speaker 2:Thank you All right, thank you Bye.
Speaker 1:See ya, you bye.
