The AI Age of Reasoning

Craig: you dropped an article about sequoia and generative ai's act o one, which obviously is a play on strawberry. yeah. tell me what's going on and what your thoughts are because i've got different thoughts, and that's great.

Jon: yeah. no. that's that's what i like. so yeah. so, the article kind of talks about the evolution of generative ai from rapid pretrained responses to more deliberate reasoning responses. so the foundation layer of the gen ai market is stabilizing, i would say, maybe, specifically with key players. so you got microsoft openai, aws anthropic, meta, google deepmind, or gemini, whatever you wanna call it. so openai's o one model, it's highlighted as quite a significant event advancement, and i think you would agree with that because it demonstrates true general reasoning capabilities achieved through inference time compute. right? so it draws parallels between alphago's approach and the emerging reasoning capabilities in language models, but also emphasizes the importance of what we'll call it system two thinking of in ai, which i think system two is a huge leap. so in this article, they're they propose a new scaling law. so the more inference time given to a model, the better it reasons. i would say that makes sense, but i'm happy to discuss that. so the shift towards inference time compute is expected to move the industry towards inference clouds that can scale dynamically based on task complexity, which is great. so we're thinking we're thinking scalability, spinning up things. so the article also discusses the potential impact on saas industry, suggesting as a transition to service as a software business models, which i think makes sense. and the last one that i think makes sense would, it explores the investment landscape in ai, and what they're suggesting is that the application layer may be the most interesting for venture capitals going forward. but thoughts?

Craig: well, i definitely think the application layer is the most interesting for investors going forward. it's funny that you mentioned this because i was having a discussion with, the investor, john ruffalo, on linkedin because he was talking about how, investors are gonna be focused on the application layer on very similar things, and i was asking him some questions about that. so it'll be interesting to see if there's specific, areas and markets that they're going after when they're talking about where they're gonna invest in ai at the application layer.

Jon: yeah. but it makes sense. and for those that don't know, there's essentially seven layers. the application layer is the topmost layer, and it directly interacts with end user applications and provides network services to applications and defines protocols and applications that use to exchange data. there's other layers. there's the presentation layer, the session layer, transport layer, network layer, data link layer, and intrinsical layer. but we won't get into those. so that's the osi model. i know. jesus. that brought me back to

Craig: my yeah. ages. ages ago. i wonder if i'm just checking to see if there's an osi map here on the page. keep talking.

Jon: yeah. but yeah. so, like, obviously, ai is not going anywhere, and we all know that it's we're getting closer to true ai, but it's still more machine learning. right? it's still following a set of instructions. and, although i think i did read that one of the models did pass the turing test, which i don't know how relevant the turing test is nowadays. but, but that being said, it does make sense that the application layer for integration with ai to get investors makes the most sense, because it's the one that's the most closely related to, software development, as far as, like, if someone's building a app. right?

Craig: so it's the part of the app we use.

Jon: yes. exactly. so it makes sense that when you're so for those of you who are integrating ai into your pitch decks, that makes the most sense at the application layer because then it just integrates easily. it's less onerous on you to explain it and how it integrates, if that makes sense.

Craig: yeah. absolutely makes sense. so when i say the application layer is what you use, like, the model in the back end may switch. re the reasoning may switch, but the application layer is the one that talks to you and what talks to the other apps. yeah. so it's what makes, saas work.

Jon: yes. exactly.

Craig: this is really go ahead.

Jon: yeah. go ahead. no. what are your thoughts, though? like, you said you have some thoughts on this.

Craig: well, the price

Jon: i covered them.

Craig: no. you didn't cover them because i was talking about i like where that was going. but when you posted this article, i took a completely different approach where we're standing on the new era of, ai age of reasoning. right? we're saying, i read this article. it's okay. great. we're strawberry and, is a leap forward in reasoning. well, yes and no. we find that some other models beat it on several things. we find that, open, chachi g b four beats it on certain things. so lately, chachi g b four has been under what i would say, underperforming for me. so i wonder if they, taken it down as a result of putting more processing power into other things or whether it's just, you know, feeling under the weather or whatnot. but my point being is that there was a lot of white papers that have went out, that, recent one from apple that says, you know what? these things really aren't thinking. they're just doing some pattern matching. they don't understand what's happening. large language models lack true understanding and reasoning capabilities despite their advantage pad advanced pattern recognitions. incorporating casual reasoning is insufficient to overcome fundamental fundamental limitations in ai. things like they change one word in something, and seventy five percent of the time, it would take, it off in a completely different direction.

Jon: yeah. well, that's also the hallucination problem. right?

Craig: right.

Jon: because they lose the context.

Craig: yeah.

Jon: so here's here's what i think where we're at right now. we're in it till you make it stage.

Craig: yeah.

Jon: so we're in it till you make it because you are essentially making it seem like it's ai without it being ai because of all the rules that you've written. the strawberry thing, i would say, is a programming error as far as as how they handle the tokenization the, of the input and output. and so they just fixed that with o four and added that reasoning just to kind of give you a little bit more transparency, but also i think to help them in the long run to develop better models. because now they're the output is you're seeing how it's coming up with the answer, checking itself, but that also helps the the people who are building these llms as well. right? so, yeah, i think we're in it till you make it stage as far as when it comes to ai.

Craig: absolutely. and when i was talking at the beginning about that, new nvidia tweaked model, it uses, i think, a lam a llama three point one seventy b, and it outperforms the four zero five b and, open, chat c d b four and o one on some things and claude on some things. but one of the tests they were talking about is they ask a straightforward question at the beginning and then put in a bunch of different information that's irrelevant to, the question. meaning, like, the example give me example. steve, is in line and is about to purchase the pencil, stapler, and paper. the pencil cost five dollars, the stapler cost ten dollars, and the paper is four dollars and fifty cents. how much money does he need to buy this stuff? last week, the items were on sale for ten percent off for pencils, twenty percent off for paper. it goes on and adds all this in for different information and puts it in. and most of the time, these reasoning models will get it wrong. and even, even o one, strawberry. so we take a step back and then they said, well, one of the things they found is if you added a line at the end that says, please read this prompt again. a lot of times, it would catch that. really? and it would realize that the rest of the information was irrelevant because it would reread the question. and so, oh, yeah. okay. and then catch it rather than trying to figure that out. so, again, it's not reasoning. it's using skip logic. and it's funny. as i play with a lot of these imaging tools and stuff, you gotta be very specific. and i use ai to help me with some of these prompts, but it's not very good at prompt in yeah.

Jon: it's gonna happen. how does this translate to the because you use a lot of, image generating models. right? yep. so how did how does this translate to what you do as far as like, do you have to get it to redo it? do you have to work like, have you worked on your do you have a prompt that works every time? do you change it? like, what's

Craig: you you definitely have to change it every time. there's a lot of randomness, like, a lot of randomness. i'm working within a system, and, for example, we're doing some stuff. i'm on this whole ai robot kick as you saw that graphic, so i'm creating a bunch of those images, which is great. and i get a pair of robots that i like, and i keep telling it to use the same robots. and sometimes it would come up with similar robots, and sometimes there'd be completely different robots. so, you know, as an example. there are ways to do it. there are things, models you can use where you can say image to image. so based on this image, create another image, but those are kind of finicky too. but they're coming along. they're all coming along. and when you tell it to do stuff, a lot of times, like, this is a great example. the prompts was when i was creating robots, create two identical robots. one is this color green and one is this color blue. but the problem is once you say identical robots, most of the time, the robots will look identical. it will ignore one is blue and one is green. they'll just be identical. so they'll make them both blue or make them both green. it'll just you know? so you and those are prompts generated by ai, and then i go back to it and say, okay. let's not use this word because then it ignores these other words, and let's change this and change this, and it comes back with a better prompt. and i'm ideally trying to train it at least in those conversations to help me better. but it actually doesn't it helps you better for a couple of prompts, but that's it. i wanna give you another example. when i was setting up some of the local llms and some of the models and stuff, i was using, chatgtp to help me do the configuration. and i'm saying, okay. we're setting this up on my mac, and, you know, we went through all the wrong commands. we got the right commands and got what ports everything's on. and i got it to create a configuration document for me. so, okay, let's create this configuration document so i can have it so i will remember ips, ports, containers, buckets, api calls, api endpoints, all that stuff. got it. and then i ask it later, like, later, got the document, and then i say, okay. i wanna do this with it. and it comes back with ten different options. only one of those options apply to my configuration. if you're using docker, do this. if you're using this, do this. if you're using it's like, wait a minute. we just did my configuration. you just created my configuration document. you have all this information, yet, you know, you're you're gonna give completely out of context stuff again.

Jon: yeah.

Craig: and we're in that conversation. so like you say, it loses context all the time yeah. and can't keep that conversation. now that's, you need longer token lengths. i chat g t p four was better, i would say, a couple of weeks ago than it is now. i think whether they've shortened the context length or whatever, it loses that context much quicker than it used to. anyway, this yeah. go ahead.

Jon: the two i and i i the so ranking these three from from favorite to least favorite, perplexity, one, clawed, two, and chat gbt, i would even say four. i always just check chat gbt. chat gbt does have a larger context window when i'm trying to do, like, a lot of big things, but, the other two, when you have a shorter context windows, those are phenomenal. did you also know that you can ask how much context it has left?

Craig: i did not. i'm gonna have to try that.

Jon: yeah. you can ask how much context it has left in that chat. yeah.

Craig: i've asked her why it keeps losing context, and it just gives me apologies. and i'm like yeah. i like we talked about briefly. i yell at you it a lot because i'm like, why are you giving me this information? you know we're talking about this. i just told you to do this. and, usually, i'm not, like, getting mad. i'm saying, like, why did you do that? how do i make sure you don't do that again? how do i

Jon: oh, that's smart. what does it say?

Craig: well, sometimes it tells me, oh, great. like, because i do is, like, let's work together, so you understand me more, and i can prompt you better so we got a better understanding. and it usually comes back, oh, that's great. we you know, next time, do this or that. and, because the first time i said, you know, you got this wrong. do better. so i'm sorry. no. i don't do better like somebody's lecturing you. i mean, how do we, you know i meant this. how do we make that sure you get that? and, like, things like, i would tell it to give me a doc, blah blah blah blah blah. but if i want the output in a doc file, pretty much the last thing i have to do is give me the output in a doc file. yeah. you can ask it at the beginning. you can ask it in the middle, but it doesn't really do it unless you put it at the end as an example. one of those things you start working with, it's okay. but you build your better prompts that way. are we at the age of reasoning? i don't know. there's a lot of flaws in this, and the more i use it, i love it in a lot of ways. ai, it helps me. blank page syndrome helps me fill in blanks. it helps me in a lot of ways

Jon: yeah.

Craig: as a i'm mostly solopreneur, but it's very frustrating in a lot of other ways. so

Jon: for sure. it's it's

Craig: got a long way to go. i keep seeing, you know, by twenty twenty five, we're gonna hit a certain, level of intelligence. i'm not sure that's actually gonna happen.

Jon: yeah. and one last thing before we move on. i just, since we're on the topic of ai, and i know you and i have both done this, but for those of you that don't know, or haven't heard of this, i would suggest going to whatever ai you're using and asking, from all of our interactions, what is one thing that you can tell me about myself that i may not know about myself?

Craig: yeah. that apparently came from sam ultimate and stuff, so it was pretty interesting to do.

Jon: yeah. i know you posted about it on linkedin. so, guys, go check out craig's linkedin post. it's phenomenal. it's actually great, and people are responding. so i feel free to chime in on on what you guys think. and we can cover it more in-depth, on another time, but i

Craig: just

Jon: wanted to bring that.