How to learn programming and CS in the AI hype era – interview with prof Mark Mahoney [Podcast #215]

[00:00] Welcome back to the Free Code Camp [00:02] podcast. I'm Quincy Larson, teacher and [00:03] founder of Free Code Camp. And today I'm [00:05] interviewing a computer science [00:07] professor who built his own free [00:09] platform for learning programming. [00:12] Before we hear from him, I want to give [00:13] you some quick community news. Free [00:15] Coamp just published a new course on AI [00:17] assisted development taught by Bo KS uh [00:21] where you'll learn common terminal [00:23] workflows and tips for pair programming [00:25] with your LLM tools. You'll also get [00:28] exposure to tools like GitHub C-Pilot, [00:30] Claude Code, Gemini CLI, OpenClaw, and [00:34] at the end of the day, the entire goal [00:36] of using these tools is to build out [00:37] more features without compromising the [00:40] maintainability of your codebase, and [00:42] you're going to learn how to do just [00:43] that in this course. Check it out after [00:46] you finish listening to this podcast. [00:48] That's free co camp also published a [00:49] beginner level course on AI literacy for [00:52] everybody and you can share it with your [00:54] family if they don't know anything about [00:55] how these LLM tools work, how computer [00:57] vision models work, anything like that. [00:59] This is a great first step in them [01:02] understanding how it works. Uh you're [01:04] going to learn about the two traits that [01:06] define artificial intelligence, [01:09] autonomy and adaptability. Uh and then [01:12] you're going to build your own image [01:13] classifier right on your phone or on [01:15] your laptop. Uh, this course also delves [01:17] into things like algorithmic bias, [01:19] environmental costs of training and [01:21] running LLM systems, a ton of stuff. [01:24] It's about a 1-hour course right here on [01:26] the free Code Camp YouTube channel. [01:27] You're also going to learn in this third [01:30] resource how to build your own QR code [01:32] generator using JavaScript. This [01:34] tutorial will walk you through [01:35] generating QR codes entirely in your [01:37] browser without any need for a backend. [01:40] So, it's kind of like a client side app. [01:42] And you'll learn how to validate input, [01:45] clear previous output, and use [01:46] JavaScript libraries to render the code [01:49] instantly on the client side. And then [01:51] you'll learn how to extend the project [01:53] further with some stretch goals like [01:54] custom styling, Wi-Fi support, and more. [01:56] It's a pretty quick tutorial on the free [01:59] code camp publication. We are also hyped [02:02] to announce that the bad website club is [02:04] back for another responsive web design [02:07] boot camp based on the free code camp [02:08] responsive web design curriculum. It [02:10] starts April 24th and runs for 10 weeks. [02:13] You can join our Discord community and [02:15] tune in for the live streams. It's led [02:17] by volunteer devs who are passionate [02:19] about helping folks learn CSS and [02:21] JavaScript fundamentals. The [02:23] announcement is linked in the [02:24] description. And song of the week, [02:27] 2008's [02:29] Strange Overtones. The Talking Head [02:31] singer David Burn blends his voice with [02:33] that of legendary synth producer Brian [02:36] Eno. And there's a lot of organ, there's [02:38] a lot of synth. The entire affair plays [02:40] out over an infectious palm muted guitar [02:43] line and some really driving bass. This [02:45] is a perfect midweek jam. Put it on [02:48] during your commute. [02:51] If you want to support Free Code Camp, [02:53] join the 10,000 plus people who do so [02:56] each month. Uh, go to [02:58] donate.freecodeamp.org. [02:59] Support our charity. Support our [03:01] mission. You can also pick up a free [03:02] COMAP t-shirt. shipped anywhere in the [03:05] US, $20 total, and you can just print [03:08] one yourself if you'd like. Just grab [03:10] our assets from our repo uh free to use [03:13] our assets for non-commercial personal [03:16] usage. And now that brings us to the [03:19] thing that I'm most hyped about today's [03:21] interview, Dr. Mark Mahoney. He was a [03:24] software developer for years before [03:27] going on to become a computer science [03:28] professor where he has taught for 23 [03:31] years at Carthage College. a [03:34] 180-year-old US university. He's taught [03:37] thousands of developers through his free [03:39] programming courses that he built on top [03:41] of his own custom software platform, [03:43] Playback Press. We'll talk about why [03:45] learning programming the hard way is [03:47] still the right way, how not to deskll [03:50] yourself when you're programming [03:52] alongside LLM tools and why now is a [03:55] great time to study computer science. [03:57] Mark Mahoney, welcome to the Free Code [03:59] Camp podcast. [04:01] >> Thanks for having me. Yeah, I'm hyped to [04:03] have you on here. You've been such a big [04:04] part of the free cooking community for [04:06] years at this point. Uh, as a computer [04:08] science professor, somebody who's been [04:10] practicing uh software development for [04:12] decades at this point and teaching it at [04:15] uh an advanced level. Uh, I want to get [04:17] your thoughts first on LLM code [04:19] generation tools. Something that Nvidia [04:22] founder Jensen Hang said a while back. [04:24] Uh, he said, "Almost everyone says it's [04:27] vital that your children learn computer [04:28] science. Everybody should learn how to [04:30] program." and in fact it's almost [04:32] exactly the opposite seems to be what [04:34] he's arguing. Uh what is your reaction [04:36] to that? [04:38] >> Well that's an interesting point of view [04:39] from him in particular. So I tend to [04:43] think the best way to build like a [04:46] really complex piece of software is to [04:49] already be an expert in uh you know [04:51] software development. Now it's it's true [04:54] for low stakes pieces of software maybe [04:57] it is okay to have you know an LLM uh [05:00] sort of build something for you but once [05:03] you start stacking complexity on top of [05:07] complexity and you start getting [05:10] concerns from multiple people and groups [05:13] of people [05:15] becomes harder and harder I think to [05:17] rely on you know prompts and Jira [05:20] tickets and emails alone. [05:23] Um now having said that I I actually [05:26] encourage my fellow uh faculty members [05:29] to use LLMs to build usually like [05:33] visualizations or uh simulations [05:38] uh in the classroom that normally they [05:40] would maybe you know draw on a [05:42] whiteboard or even handwave uh through. [05:46] Um and I actually do this myself. I just [05:49] recently [05:51] uh did this sort of simulation about [05:56] uh you know how do developers use git [05:59] and you know GitHub to kind of go [06:01] through the process of doing a pull [06:03] request [06:04] and [06:06] um the LLM did great. I didn't I didn't [06:09] even look at the code but there was not [06:12] a lot there was not a lot to it. This is [06:14] something I would have, you know, just [06:15] drawn on the whiteboard, but uh this [06:18] little tool that I vi coded. Um it it [06:23] sort of animated the whole process. It [06:26] was kind of interesting. Maybe I could [06:27] give you a link to put in a show notes [06:29] for [06:30] >> Yeah, I'll include I'll include a link [06:31] to it just to show like this. So, it [06:33] sounds like one of the things you're [06:35] saying is these tools are useful for [06:36] less technical folks who just need to [06:38] get their idea across. They're they're [06:40] good for creating uh data visualizations [06:42] for example or extremely simple uh tools [06:45] where you know it's it's visually [06:48] obvious I guess whether it works or not [06:50] so you don't have to actually splunk [06:52] into the codebase and try to understand [06:53] where things are going ary [06:55] >> exactly so I I'll tell my colleagues to [06:58] start the prompt with you know [07:02] uh create a single page web app that [07:05] does or that shows x y or z like the the [07:09] flow of a pull request on a GitHub or [07:12] something like that. And these are [07:14] definitely low stakes. Uh I I would not [07:17] want my, you know, pacemaker software to [07:20] be uh developed like this. But uh for [07:24] little simulations and visualizations, I [07:27] think those are pretty helpful for [07:28] educators. [07:31] So, when I use LLM's on my own projects [07:34] though, uh, I've noticed, you know, they [07:38] can misunderstand, [07:40] you know, what I'm looking for, and they [07:42] definitely don't know all of my [07:44] preferences for how I, you know, would [07:46] like to do things. [07:49] Uh, and you know, just recently I I was [07:51] using an LLM uh to help me write some [07:55] code with this web- based thing that I'm [07:57] working on, and it wanted to put all the [08:00] data in the like this global document [08:03] object when I had another object that, [08:07] you know, was probably a more [08:09] appropriate place for that data and I [08:12] had to go back and tell the LM, no, [08:14] don't don't put it there, you know, put [08:16] it here And if I hadn't made that [08:20] change, if I hadn't noticed that change, [08:22] um, you know, it would have just added [08:24] some technical debt that somebody [08:27] somewhere down the line would have had [08:29] to kind of realize that that now the [08:31] data is split in these two places. [08:34] So, [08:36] uh, I'm kind of optimistic about LLMs, [08:39] but I I'm I'm not at the point where, [08:42] uh, you know, I think they're going to [08:44] completely take over and do everything. [08:48] uh for us. I just don't think the LLMs [08:50] can kind of read our minds and [08:52] communicate with us well enough yet. [08:54] >> Yeah. Yeah. And just to be 100% clear, [08:57] this is not with like some LLM tool, you [09:00] know, three years ago when these tools [09:01] just came out like early GitHub Copilot [09:03] or or early agent, right? This is just [09:06] like the other day that you encounter. [09:07] >> Yeah. Uh so I have been using Claude [09:10] Code. I pay the the 20 bucks a month for [09:13] it and it's amazing. But um [09:19] you know I rather than have it like [09:22] write code immediately what I what I'm [09:24] doing these days is asking it to come up [09:27] with a plan and tell me the plan before [09:29] it does anything. And then often what [09:32] the first plan is, you know, either not [09:35] the way I would do things or, you know, [09:37] they mis LM misunderstood exactly what I [09:41] was looking for. So I'll iterate a few [09:43] times just on the plan. [09:46] And um then if once it comes up with a [09:51] reasonable plan, I will have it write [09:53] the code. But that back and forth uh I [09:56] don't think would have been possible [09:58] unless I already was, you know, a pretty [10:00] experienced developer and and knew what [10:02] I was doing. [10:03] >> Yeah, [10:04] >> that makes a lot of sense. So, uh, what [10:06] would you say to a learner who is just [10:08] using LLMs to program and they they have [10:12] this they they don't realize they don't [10:14] know what they don't know and uh they're [10:17] just using LLM as their main source of [10:21] learning as they try to get things done. [10:23] Kind of learning just in time by yeah, [10:26] you know, seeing things not work and [10:27] asking the LM what went on here and [10:29] stuff like that. How would you encourage [10:31] them to handle that those kinds of [10:34] interactions to be productive? So, just [10:36] to recap what you just said a second [10:38] ago, you said you'll you'll iterate [10:40] several times on the plan. You'll ask [10:41] them, okay, how are how would you [10:43] implement this before you actually have [10:44] it implemented and that way you can kind [10:46] of reason through and and squash bugs [10:49] before they crop up and things like [10:50] that. How would you how would you advise [10:52] like somebody who doesn't have your [10:53] expertise to interact with LMS to [10:56] achieve a similar effect? [10:58] I mean, if if one [11:02] I'm a huge fan of using an LLM as like [11:05] this infinitely patient tutor. [11:08] Uh, and if the student is really aware [11:12] of how they learn and they're very [11:15] curious, [11:17] maybe that's a good way to go about [11:19] learning, you know, just using an LLM [11:21] alone. But my sense is most learners [11:25] aren't that aware. uh especially about [11:27] something they they don't know much [11:29] about. [11:31] So if an LLM is writing a majority of [11:34] the code and for me the critical point [11:38] is are the students actually stepping [11:40] through the code. So okay if the LM [11:42] generates your code that's fine but they [11:45] should definitely be stepping through [11:46] and watching what that code does you [11:49] know what effect it has. again, maybe uh [11:54] that's uh an okay way to learn, but [11:59] um I would I would really be [12:04] uh worried that you know students who [12:07] learn just with an LLM are are not [12:09] really resilient, [12:11] right? if you're just given the right [12:13] answer or what you think is the right [12:15] answer most of the time and you don't [12:17] have to do that hard work of figuring [12:19] out, you know, why something is not [12:22] working. You know, if I were hiring [12:24] someone, I I kind of would like someone [12:27] who has uh proven that they kind of [12:30] learned the hard way. And I u I [12:34] sometimes joke with people on like on my [12:36] LinkedIn, I I want to put something [12:37] like, you know, I learned to code before [12:39] LLM's. [12:41] >> Yeah. Right. So, I feel like a badge [12:43] like that might uh one day actually uh [12:48] be valuable, right? Because to me it [12:51] says, okay, the the students learned the [12:54] hard way. Uh they went through the hard [12:57] work and they learned about all the [12:58] debugging and and things like that. [13:01] Another [13:03] thing I'm sort of worried about, uh, you [13:06] know, right now I, like I said, I pay [13:08] the 20 bucks a month, uh, for one of [13:10] these LLMs. It's great. I'm not even [13:14] writing code eight hours a day and and I [13:16] run out of, you know, tokens. [13:20] Um, and I wonder, you know, right now [13:23] everything is pretty cheap. And in fact, [13:26] >> venture capital is subsidized. [13:28] >> Yeah. could be even free for uh for some [13:32] people, but I don't know if that's [13:33] always going to be the case. And u I [13:37] wonder if there are going to be [13:38] companies that just can't like literally [13:40] afford to have their developers using [13:43] this uh you know full-time. [13:46] So that's another reason, you know, I [13:48] think it's very important to [13:51] learn, I guess, the oldfashioned way. [13:55] um could be also [13:58] some companies could uh you know worry [14:00] about liability. So I've I've got a a [14:04] former student who told me their their [14:07] uh company doesn't even let them use uh [14:11] these AI tools because they're, you [14:13] know, they're worried about getting [14:14] sued. [14:15] >> Yeah. And who knows in the future, you [14:18] know, some parts of the country might [14:19] not uh or some parts of the world may [14:23] not have act the same access to the [14:25] tools that we have. And [14:28] so I I definitely think it's there's [14:30] value in kind of learning the [14:33] traditional way. [14:35] >> Uh well, let's talk about what that [14:36] traditional way is because uh I in [14:40] increasingly encounter people who are [14:41] like, hey, why do we even need free [14:43] cooking? Why do we even need textbooks? [14:45] Why do we even need programming [14:46] tutorials? We can just do everything [14:48] through talking with the LLM and learn [14:51] just in time. Like this is an [14:52] increasingly pervasive attitude. And uh [14:55] I guess what would you say to those [14:57] people in terms of like first of all [14:59] you've already illustrated some of the [15:00] pitfalls of relying on these tools? Uh [15:03] they will potenti like you're not [15:05] actually doing the work so you're not [15:07] actually learning the process. You're [15:09] just kind of getting uh the shortest [15:10] path through um getting to whatever you [15:13] need done. But you don't necessarily [15:15] like if you took away the tool, you [15:16] couldn't reproduce it. Right? That's the [15:18] definition of tutorial hell is somebody [15:20] who can do lots of things when they're [15:22] being guided step by step by a tutorial, [15:24] but then you take away the tutorials and [15:25] they're staring at the blank page and [15:26] they're not quite sure where to go. Um, [15:29] and they haven't developed the [15:30] confidence and they haven't developed [15:31] the confidence. Well, with LM, they're [15:33] probably going to develop the [15:34] confidence, but they're not necessarily [15:35] going to develop the competence. Right. [15:37] So uh like you have created story uh [15:41] like this really powerful tool uh to [15:44] essentially have like code replays where [15:45] uh you can record um and you can walk [15:48] people through uh code and they can like [15:51] uh it's kind of like this cool [15:52] multimedia experience and it's free and [15:54] free code camp has published many of [15:56] your uh learning resources like how do [15:59] you envision like the ideal interactive [16:02] learning experience even with these LLMs [16:05] like like are LLM even a part of that [16:07] like how how do you how have you [16:08] incorporated LMS into your platform? I [16:12] know that's like 50 questions, but I I [16:14] just want to like load all that stuff [16:16] into the peripheral context window so [16:18] you can like kind of unload some of that [16:20] insight because I know you're thinking [16:21] very hard about this. [16:23] >> I mean, it's a very exciting area this [16:26] this idea of uh again the what I like to [16:29] call the the infinitely patient uh tutor [16:33] who's who's able to answer questions. [16:36] I know from many years of experience [16:39] there are some students who just refuse [16:42] to ask for help and I I don't know why [16:46] uh you know I I try and make it as clear [16:50] as possible I'm here to help you but [16:52] there are a group of people who just [16:54] refuse to ask for any sort of help. What [16:58] I think is if they were if they had [17:01] access to an LLM tutor, they wouldn't be [17:05] uh you know, they would be more willing [17:07] to ask for that help. However, I I [17:10] really do worry about completely leaving [17:13] humans out of that loop, right? I as a [17:18] longtime uh teacher, I know that uh we [17:22] can tailor our uh lessons and what we [17:25] teach each individual student [17:28] uh based on what we know about the [17:30] student. So sometimes you want to be [17:32] very direct. Sometimes you want to not [17:35] give them so much and maybe ask a [17:38] probing question and have them sort of [17:40] figure out stuff on their own. [17:42] I don't know that the LLMs are at that [17:46] point or will ever be able to really get [17:48] to know uh students like uh an [17:51] instructor does. [17:53] Maybe more importantly, uh, instructors, [17:56] I I feel like my my main job, if I'm [18:00] good at anything, it's like motivating [18:03] students to want to learn. [18:05] Uh so [18:08] you know I do that I think through [18:10] enthusiasm and uh passion [18:14] and um [18:17] so [18:19] again I I I don't quite see how uh an L [18:23] lom can kind of inspire that passion. [18:28] >> Yeah. I mean and also your competence. [18:30] You're you're somebody who's been [18:31] practicing for a long time. uh you [18:33] started before the dotcom bubble way [18:36] back in the mid 90s working in industry. [18:38] You yourself have a CS degree. You've [18:40] gone through the proverbial slings and [18:43] arrows of like learning programming [18:45] properly and then applying it and now [18:47] you've been turning around and teaching [18:48] it for decades at this point. And so [18:51] there's that human aspir like that that [18:54] empathy like they're looking at somebody [18:55] who's gone through this, right? the LLM [18:58] hasn't gone through anything other than [19:00] you know a process of uh you know [19:03] reinforcement learning with human [19:05] feedback and and a training process and [19:07] all that stuff like it's not it is not [19:08] sentient there's there's nothing like I [19:10] don't aspire to be an LLM or like an LLM [19:13] right uh where what you said I mean your [19:16] primary job as a professor is to [19:18] motivate people to want to learn that's [19:21] a pretty profound statement uh and I [19:23] don't know that every professor looks at [19:24] their role quite like that but I I think [19:27] it's a really helpful uh kind of lens to [19:30] look at the role of the professor or the [19:33] role of academia like what are you [19:35] getting when you go to a university [19:37] you're getting somebody who has been [19:39] there who knows the stuff who going to [19:42] try to communicate to you how important [19:44] it is that you learn these things [19:46] without necessarily putting you in I [19:48] mean people will learn this the hard way [19:49] when they go out and work in industry [19:51] and they'll realize oh man I really [19:52] screwed up there company lost a lot of [19:54] money now I'm going to get fired or um [19:56] like oh wow I just wasted days of my [19:59] life working on this feature where I [20:00] misunderstood the spec or something like [20:03] that right like like they're going to go [20:04] through these processes and to the [20:06] extent that you can kind of guide them [20:07] around these things so they don't have [20:09] to make all the mistakes for themselves [20:10] that's another function of u going to [20:14] school and that's a function of human [20:17] communication not to get too abstract [20:18] but one of the reasons humans are so [20:20] successful compared to like I don't know [20:24] um cockroaches which are also extremely [20:27] success successful but not individually [20:29] individually they're extremely brittle [20:31] individually humans are extremely robust [20:34] because they can learn from one another [20:35] and I don't have to go you know touch [20:38] the electric fence myself somebody can [20:40] say hey we have human language right we [20:42] have written language we got all these [20:44] things and and I I think the role of [20:46] like learning and the role of teaching [20:48] is to essentially help steer people [20:50] around things and give people context [20:53] like that's my personal perspective [20:55] active as a teacher, but I haven't been [20:57] doing it as quite as long as you have. [20:59] >> And I know I know we talked offline sort [21:01] of about different types of colleges and [21:05] schools, and I'm pretty lucky in that uh [21:09] uh the school that I teach, it's very [21:11] small. I have very small classes. [21:14] Um my students get to know me. I see the [21:17] same students for their full four years. [21:20] And I really believe this is maybe [21:22] corny, but when I tell them, you know, I [21:26] I want you to [21:29] um you know, I'm trying my best to to [21:32] teach you, I I feel like students when [21:35] when they get to know you, uh they want [21:38] to show that yes, they believe in that [21:41] and they uh you know, they uh so for [21:46] example, I'm surprised by how little I [21:50] think my students are using LLM [21:52] just based on what I've heard from uh [21:55] you know professors at maybe bigger [21:57] schools [21:58] and we were also talking about this the [22:01] other day the we never really bought [22:03] into the totally asynchronous online [22:07] uh courses and I think that really helps [22:11] uh you know if the student and the [22:14] teacher have this relationship and the [22:16] student makes it or I'm sorry the [22:18] teacher makes it clear that, you know, [22:20] we don't want you to use the LLM because [22:21] it'll make you learn, you know, more or [22:24] better. So, [22:27] yeah. [22:28] >> Yeah. And and that's something I've [22:29] definitely observed like talking to my [22:31] friends who went to like UT Austin [22:34] versus people who went to like Harvey [22:36] Mud or some school that's like like [22:37] really hardcore engineering uh schools [22:40] like there's a lot more do it by hand. [22:42] There's a lot more back to basics kind [22:43] of focus on fundamentals at a lot of [22:45] schools than there are at other [22:46] programs. Not to pick out any particular [22:48] program. Uh UT Austin is a great program [22:51] too if you want to study CS there, but [22:53] it's not like super duper like I don't [22:57] know like like an engineering school's [22:59] engineering school, you know. Um, and [23:02] like there are programs like that that [23:04] that are like small classes that are [23:06] teachers who are working very closely [23:08] with students and uh who develop this [23:11] long working relationship over like four [23:13] years uh as opposed to just you're [23:16] you're hopping from one lecture hall to [23:18] another. So I can definitely see what [23:19] you're saying there and it might be [23:21] because they're spending so much time in [23:22] proximity to you and they have so much [23:24] access to you that they don't need to [23:25] rely on LLMs as much. Uh, as you pointed [23:28] out, like LLMs can be great for people [23:31] who have like social anxiety or have [23:32] issues with uh, self-confidence or [23:35] issues with uh, just not wanting to look [23:37] dumb in front of other people, which I [23:38] think almost everybody has an issue [23:40] with. I certainly have an issue you [23:41] with. I ask you all those questions I [23:43] probably wouldn't feel comfortable [23:44] asking one of the people on my team just [23:46] because I'm worried, oh man, I'm going [23:48] to look like a total dum dum. Like, am I [23:50] not with it at all? You know, and the LM [23:52] isn't going to judge me, right? Um, so [23:56] >> that's one of the great [23:57] >> one of the great uh aspects of that. I I [23:59] do the same thing uh not only do I use [24:03] it for asking like coding type [24:06] questions, but all types of questions [24:08] throughout the whole rest of my, you [24:10] know, day-to-day life. Uh I I ask it all [24:14] questions all day long. [24:17] >> Yeah. Well, let's talk about what the [24:19] main limitations are and like how long [24:21] you think it's going to be before uh [24:24] LLMs, especially in terms of generating [24:26] actual curriculum, right? Like there [24:28] like in game development, people are [24:30] talking about LLMs are just going to [24:31] generate. This is one of the many things [24:32] that Jensen Hang and a lot of other uh [24:34] hypemen from the uh the AI, you know, [24:37] bubble world frequently say is, oh, in [24:40] the future, you won't actually have game [24:42] developers developing a game. the you'll [24:44] just tell you know some system what you [24:46] want to play and it'll generate that on [24:48] the fly and keep all that in it context [24:50] and and you like turn to the side and [24:52] and you'll see what you're supposed to [24:53] see and so you can have any game you [24:55] want right and um like my personal [24:58] rebuttal to that is you know what the [25:01] world doesn't need that many games and [25:02] it doesn't need a single individual game [25:04] for me there are millions of people that [25:06] are still playing Minecraft to the point [25:07] that the game industry is actually kind [25:09] of in decline right now because there [25:11] just already more than enough good games [25:13] music industry. Uh I listened to Nile [25:15] Rogers say like we've already got so [25:17] much great music that we don't [25:19] necessarily even need more. Why do we [25:21] need to be generating tons and tons of [25:23] music when people haven't even had time [25:25] to go back and listen to like great, you [25:27] know, music from different eras [25:28] throughout history, right? Um and you [25:31] know to the same extent there are only [25:34] so many topics that need to be taught. [25:36] Do you really need to generate [25:38] curricula? [25:40] But uh how long do you think it would be [25:42] based on your understanding of where [25:44] things are and where things are going [25:46] before an LLM could generate something [25:48] like free code camp uh there like the [25:50] full stack curriculum or could generate [25:52] something like Harvard CS50 uh like the [25:54] entire course with like a you know [25:56] generated uh David Men standin [25:58] essentially teaching all the concepts uh [26:00] without and thus avoiding intellectual [26:03] property. That's a big theory of mine is [26:05] that a a lot of the reason behind the [26:06] generation is so they don't have to [26:08] worry about existing intellectual [26:09] property laws and uh licensing and [26:12] things like that. [26:13] >> Yeah. So, I agree 100% on the music. I [26:16] know we were talking about music the [26:18] other day. Uh and I totally agree. [26:23] There's uh [26:26] you know, nobody needs more music. We [26:28] need more time to get through the music [26:29] that already exists. Um the one thing I [26:33] would say about software is maybe maybe [26:35] we do need more software. This is why uh [26:38] I think there's a maybe a small [26:40] distinction here where I'm I am okay. Uh [26:45] again I'm sort of on the fence about the [26:47] LLM but I am using it. Um but you're [26:51] right about the the music there. There's [26:54] a famous quote, you know, I wanted AI to [26:58] do my dishes and do my laundry so that I [27:00] could write the music and work on my [27:02] art. Not not the other way around. But [27:05] that's exactly what we have. We have it [27:06] the other way around. [27:09] Uh when we talk about limitations [27:13] uh especially in computer science [27:15] education [27:17] the one thing I've noticed is uh the [27:20] texton interface is kind of a [27:23] limitation. [27:25] So I don't know about you but when I'm [27:27] teaching I I have to have like a [27:29] whiteboard near me so that I can draw a [27:32] picture and kind of animate and describe [27:34] what's happening. [27:36] >> Yeah. So, not I'm usually not writing [27:38] code on that whiteboard. I'm drawing [27:39] pictures and uh I'm asking students, you [27:43] know, what would happen if we moved this [27:45] thing over here? And sometimes they'll [27:47] get up and go right next to me on the [27:49] whiteboard. [27:52] I've done some experimentation with the [27:55] LMS and they're they're just not there [27:57] yet in terms of like visually [28:01] uh explaining things. [28:04] Uh maybe it's just a matter of time, but [28:06] um you know when I I think about my [28:09] classroom, there's a projected screen [28:11] and there's a whiteboard and there's an [28:14] opportunity for me to sort of work with [28:16] a student. And I don't see how the the [28:20] LMS are going to, you know, completely [28:23] get there. [28:24] >> Yeah. [28:24] >> So how long is it going to be before it [28:26] completely generates a free code camp? [28:30] I'm not optimistic about that. I think [28:33] uh having like I said humans in the loop [28:37] uh humans who get to know the students [28:39] or get to know kind of students in [28:40] general [28:42] uh is just a critical [28:45] uh part that you know I don't say never [28:49] but I don't see that part going away. [28:51] >> Yeah. And uh I'll I'll just add that [28:55] humans learn because we're kind of [28:58] programmed this way from like evolving [29:00] and living on the savannah and uh [29:03] eventually you know spreading out to you [29:05] know Europe and North America and South [29:08] South America and everything, right? [29:09] Like uh just [29:12] we are animals just like the animals [29:14] around us, right? And and like we learn [29:16] experientially like when I learn like a [29:18] new word in a foreign language that I'm [29:20] studying, I can almost always remember [29:21] where I was when I learned that word. [29:23] Why is that information relevant? Like [29:24] it's just part of how humans learn. Like [29:27] I can remember the situation. I can [29:28] probably even remember whether I was [29:29] sitting down or standing up. You know, I [29:31] can remember whom I was talking to and [29:32] that does not help me at all. Like why [29:35] is my information why is my brain like [29:36] wasting space on that context? But [29:39] that's how humans learn. And I would [29:41] imagine there are plenty of students who [29:43] can distinctly remember the moment that [29:45] they got that breakthrough with like [29:46] some particular type of data structure [29:48] or some sort of aspect of uh you know [29:51] some layer of abstraction, right? Uh [29:53] because they were up there standing at [29:55] the front of a whiteboard sketching with [29:57] a a pen thinking really hard while they [29:59] were listening to you and you were like [30:01] you know also writing on the whiteboard [30:04] and say oh actually it's like this. Oh [30:06] what about this? Oh yeah it's like this. [30:08] you know like like there like so much of [30:10] communication is not through just [30:12] language alone and currently these LLMs [30:14] can only really I mean they are large [30:17] language models right that is a [30:18] fundamental uh and you can plug those [30:21] into other systems and build systems on [30:23] top of them and stuff but it's not like [30:25] the world's most powerful calculator is [30:26] going to be powered by an LLM it's [30:28] probably going to be powered by [30:30] something more deterministic or [30:31] something like that but similarly uh [30:33] other things that need to be done will [30:36] probably like different tasks will [30:38] require completely different tools and [30:40] potentially completely different [30:42] approaches technologically. [30:44] And so what it seems like you're arguing [30:46] here is that teaching is just one of [30:48] those tasks that's so multimodal to [30:50] borrow a term that I hear a lot in the [30:52] LLM space um that it's just not going to [30:56] be fully addressed through LLMs that [30:59] that we need additional [31:01] u archetypes essentially or [31:03] >> I definitely think uh uh [31:08] the they're not quite there yet. They're [31:10] they're great with again anything [31:12] textbased, [31:14] but um I've tried to have it, you know, [31:17] draw [31:18] I'm often drawing like this is what the [31:21] memory looks like and this is what we're [31:22] going to move around in memory. [31:25] At least the the tools I have used uh [31:28] I've not been super impressed with um [31:32] them creating visuals to go along with [31:34] their textual explanations for things. [31:37] >> Yeah. So I want to jump to uh like a [31:41] little bit more about LM before we get [31:42] into your history which I'm very excited [31:44] to talk about. But one concern like a [31:46] lot of people have been talking about is [31:47] desklling. essentially learners uh in [31:51] this case maybe even developers who [31:53] finished school and you know spent [31:55] several years of their college career [31:58] using LLM tools essentially deskkilling [32:01] themselves by overly depending on these [32:04] tools especially proprietary tools and [32:06] as a result like their their skills [32:08] atrophy and and to an extent even their [32:10] critical thinking atrophies. Have you [32:12] observed this? [32:16] Well, [32:19] it's all so new. There there's [32:21] definitely a lot happening right now. [32:23] There's a lot of hype right now, but [32:25] there's also there is something I think [32:28] very real about [32:30] uh you know AI as a tool for educators. [32:36] Um, I really wish I knew what was going [32:38] to happen uh in the next few years. Uh, [32:44] in terms of deskkilling, [32:47] again, uh, if if I were hiring someone, [32:51] uh, I I don't I don't know how, uh, [32:54] interviews go these days, but I would [32:56] want someone to be able to prove that [32:58] they can tackle problems and think about [33:01] problems without any sort of AI [33:04] assistant. [33:05] Um, [33:07] and you know, [33:11] again, this is maybe just my personal [33:12] opinion, but I don't think that human [33:14] element is all is ever going to like [33:16] fully go away. [33:19] So, um, you know, I I guess the answer [33:24] is I don't know. Yeah, [33:26] >> I know that the companies that are doing [33:28] the hiring these days, I think they are [33:30] really uh [33:33] fixed on the hype, right? I I feel like [33:36] uh they are kind of waiting it out. [33:40] They're going to try and see uh you [33:43] know, is this thing really gonna take [33:45] over completely? [33:47] Um, [33:49] I wish uh that [33:54] you know the hype would go away so we [33:55] can actually have real discussions about [33:58] this and conversations about it. And I [34:00] think that is happening but it's I feel [34:03] like the [34:05] companies are are kind of sitting and [34:07] waiting to see what's going to happen. I [34:10] mean, that's a rational thing. Uh, like [34:12] to to be charitable to the companies [34:15] that have like kind of frozen their [34:18] hiring or even let go of developers. [34:20] Like, they don't know what's going to [34:21] happen anymore than anybody else does. [34:23] Uh, we can all certainly predict and [34:25] some people with a lot more expertise uh [34:28] can predict what's likely to happen, but [34:30] from the perspective of a manager, it [34:32] might not make sense to hire a whole [34:34] bunch of devs if these tools actually [34:36] pan out. and the risk of hiring a bunch [34:37] of devs and looking like a fool for [34:39] hiring a bunch of devs and getting fired [34:40] yourself. Maybe that is informing a lot [34:43] of the uh managers that are just [34:46] basically just doing like essentially [34:48] hiring freeze. I talk to people all the [34:49] time uh on this podcast and just in [34:51] day-to-day interactions and stuff who [34:53] are just kind of waiting to see what [34:55] happens. And that's totally fine. like [34:58] uh if you are a manager and you've got [35:00] like a budget and everything like that, [35:01] it's a little harder to tell somebody [35:03] who is a developer who's just finished [35:05] their CS degree and is having trouble [35:07] getting interviews uh and and has been [35:10] spending you know months and months [35:11] trying to find a good job like oh uh you [35:14] know like we're you just graduated in a [35:16] very unfortunate time or you transition [35:18] into software development from a [35:20] different field at a very unfortunate [35:21] time where there's just so much [35:23] ambiguity that people aren't able to [35:25] move forward with confidence and this [35:27] will probably be resolved within a year [35:29] or two, but like what is that person [35:31] going to do in the next year or two? My [35:33] advice to them is just uh keep expanding [35:35] their skills and maybe get a job at like [35:37] doing literally anything you can do [35:39] that's within your ethics to to survive [35:43] essentially. Um, but like how would you [35:46] advise like let's say hypothetically you [35:48] have a CS student who um who is [35:51] struggling to find work after graduation [35:54] or maybe they did get a job but then [35:55] they got laid off and now it's them in [35:58] line for interviews behind lots of [36:00] people that have five 10 years of [36:02] experience in the field. [36:05] >> It's really tough. Uh, so I just talked [36:08] to a former student today who graduated [36:10] in December and he took him couple [36:14] months but he he finally accepted a [36:17] entry level job today. He he talked to [36:20] me. Um, [36:23] yeah, I I I really don't have a great [36:26] answer other than like you said, [36:29] continue to learn. Uh, I do think this [36:31] is going to resolve itself. I don't [36:34] quite know how it's going to resolve [36:35] itself, but I'm also thinking in that, [36:39] you know, monthstoyear [36:41] time frame, we're going to have some [36:43] more concrete answers. And I'm at least [36:46] my personal opinion is the, [36:50] you know, we are still always going to [36:51] need developers to kind of drive the [36:54] process. Um, [36:57] so it's tough. I I don't have a great [37:00] answer. [37:03] Awesome. Well, I want to dive into your [37:04] background because you've got a very [37:05] interesting background. Uh, you grew up, [37:08] uh, child number five and a family of [37:10] six. Um, and maybe you can kind of like [37:14] walk us through your early years and how [37:16] what inspired you to get into software [37:19] development. [37:20] >> Well, I wish I could say I had a great [37:23] plan and it all worked according to that [37:25] plan, but I I kind of fell into all the [37:28] good things that happened in my life [37:30] kind of by accident. [37:32] So yes, I'm part of a big family. [37:36] I don't know if you're part of a big [37:37] family, but what tends to happen is all [37:40] the resources tend to go towards the [37:43] oldest uh children. But then, you know, [37:47] I'm the fifth, so when I graduated high [37:50] school, for example, my parents knew [37:52] they could not afford like a traditional [37:55] uh send me away to college experience. [37:58] >> And they themselves didn't go to college [37:59] either, right? [38:01] >> No. No. Um, [38:03] luckily though, I grew up in a place [38:06] that was near a community college and uh [38:11] I I was able to convince them to, you [38:15] know, send me to community college for [38:18] those first couple years. Now, at the [38:21] time, I did not I was not interested in [38:24] software development. I was actually an [38:25] art major back then. But I got so many [38:28] of the uh sort of required courses out [38:31] of the way that when I eventually went [38:33] back, I could focus almost exclusively [38:36] on uh computer science and math. So that [38:39] was awesome. [38:41] But I went for my my two years and we're [38:44] talking this was quite a while ago, but [38:47] I mean the full year might have been [38:49] $1,000. Yeah. I of course lived at home [38:53] >> uh and with books and everything, it [38:55] might have been $1,000 a year. And even [38:57] that was a you know that was a pretty [38:58] big stretch for my parents. But I [39:01] finished and you know I didn't quite [39:05] know what I wanted to do. So I spent a [39:07] couple years just doing weird jobs. [39:11] Uh you know I thought [39:14] you know my dad was kind of a truck [39:17] driver. I thought maybe maybe I'll be a [39:19] a truck driver too. [39:21] >> Yeah. [39:21] >> And uh I did jobs like that. actually [39:24] was driving once and I just fell asleep. [39:28] I almost killed myself and I knew right [39:31] then I I have to do something to keep my [39:34] brain active. I I can't do [39:37] the driving just wasn't for me. [39:41] Uh so by then I was uh you know a little [39:44] bit older and uh I was actually able to [39:49] uh pay my way uh to get the the [39:52] four-year degree. [39:54] And of course uh you know when in high [39:58] school I was at best a C student, [40:01] >> right? [40:01] >> When I went to that community college my [40:03] parents were paying so I I upped my game [40:05] a little. I got B's. But when I started [40:07] paying of course I was a straight A [40:09] student from Uh-huh. Yeah. [40:12] >> Uh, so, uh, like I said, I had, uh, I [40:16] didn't quite know what I wanted to do [40:17] either. Uh, so I went back to that same [40:20] community college and I tried a couple [40:22] different things. So, I I took a couple [40:25] journalism classes. I was fine. Uh, but [40:29] it didn't really, uh, excite me. And [40:32] >> yeah, [40:33] >> totally by accident, I saw that there [40:35] was this computer programming course. I [40:38] honestly I didn't even really know what [40:40] that was, but I got hooked. I mean, [40:44] immediately. [40:45] Uh I was amazed that I could tell this [40:49] computer to do something and it would [40:50] actually do it. And uh that I felt like [40:55] a wizard and uh I I got hooked. Um, like [41:01] I said, I I then went and uh to a [41:04] different school and completed the [41:06] four-year degree. [41:08] Then I got a really good job right out [41:10] of uh college as a software engineer. I [41:13] was doing that for uh a few years. The [41:17] place I worked was very generous with uh [41:20] like sort of educational [41:22] uh benefits. So I was able to get my my [41:25] master's degree in computer science [41:28] while while I was working there. [41:30] >> Yeah. [41:31] >> And the way I started teaching was again [41:34] totally by accident. Uh where I worked [41:38] was only a couple miles away from the [41:41] campus of the the college I went to. So [41:44] one day at lunch, this is the the first [41:47] time I had done this. I thought, "Oh, [41:49] maybe I'll go back to the computer lab [41:51] uh and see who's around." And when I got [41:54] there, the the lab manager [41:57] had told me uh you know, one of our [42:00] professors got ill and like class was [42:04] starting in a day or two and this lab [42:07] manager friend told me they're going to [42:09] make me teach the course for him. You [42:11] know, they're desperate basically. and [42:13] he said, "Mark, would you would you [42:14] please do it because I I definitely [42:16] don't want to do it." [42:17] >> Called the [42:18] >> I said, "I I can't do it. I'm I was a [42:21] student here like a year ago." But then [42:24] I went home and I thought about it and I [42:26] thought it might look good on a resume [42:28] to say, you know, I taught a course one [42:31] time. [42:33] So, I forced myself to do it. And it [42:35] actually went great. I uh I know this [42:40] term wasn't used back then, but fake it [42:42] till you make it. Yeah, I definitely [42:44] decided I'm gonna fake it. [42:46] >> I'm gonna pretend I know what I'm doing. [42:48] >> And the students, they couldn't tell. [42:51] They just assumed I knew what I was [42:52] doing. [42:54] And um so that went well enough where [42:57] the next semester uh I think the [43:00] professor eventually uh recovered, but [43:04] they still needed people to teach. So [43:05] the the the main person asked me to to [43:08] keep doing that and I did that for [43:10] several years just you know one course a [43:14] semester. I was an adjunct uh professor [43:18] and then after a few years I uh I was [43:21] sort of getting getting bored with the [43:23] job I had and I was looking around at [43:26] other jobs and uh my now wife suggested [43:31] oh why don't you look at you know [43:32] full-time teaching jobs. [43:35] because I had, you know, had earned an [43:38] advanced degree and I now had a few [43:40] years experience [43:43] and uh so I found this uh opportunity [43:48] uh at uh my current school and they were [43:51] also kind of desperate so I got lucky [43:53] again. [43:54] >> Yeah. [43:55] >> Uh and uh I wasn't quite sure if I was [43:59] gonna you know if this was going to be [44:01] my career. I just thought maybe I'll try [44:04] it out for a year or two. But now it's [44:08] 23 years later. So, uh, that's sort of [44:12] how I fell into this. If I was five [44:16] minutes later or if I had decided to go [44:19] the next day, my life would have been, [44:21] you know, completely different. Uh, it's [44:25] very I got very lucky, I think, that [44:27] yeah, I stumbled in on that one day. [44:30] >> Is that computer lab still there? [44:32] No, that that's how old I am is uh back [44:35] back in those days, nobody had a [44:37] computer. If you wanted to do your [44:38] homework, you you'd go to the computer [44:40] lab. [44:40] >> Yeah. [44:41] >> So, I'm sure it's not I'm sure it [44:43] doesn't exist anymore. I don't think [44:44] computer labs exist anywhere really. [44:47] >> Yeah. Well, uh that that's pretty good [44:49] serendipity. And uh I mean the [44:52] opportunity presented itself and you and [44:53] you took it. Like a lot of times people [44:55] just eh and they they don't realize [44:57] they're passing up on an opportunity [44:59] that can lead their life in a completely [45:01] different direction. Of course, you were [45:02] promoted repeatedly uh over there uh [45:05] through like the many ranks of academia. [45:08] I think you've also been like, you know, [45:10] in in like a chair type position or [45:12] something like that where you're [45:13] administering your peers uh fellow [45:16] professors and adjuncts. Uh and uh [45:18] that's really cool. One of the things [45:20] that you've done that I think is really [45:22] interesting is you've taught high school [45:24] teachers how to teach computer science [45:28] uh to their students because of course [45:30] like we think in the US like oh we've [45:32] got like all this technology everywhere [45:34] like pretty much every school has [45:36] computers in the US uh but like the [45:38] actual knowhow of using those computers [45:41] is not evenly distributed and you need [45:44] somebody to go out and train those [45:45] teachers and a lot of them didn't study [45:47] they didn't have computer science when [45:48] they were university, right? So, so [45:51] that's like an incomplete aspect of [45:52] their uh own education. So, you kind of [45:55] go in and backfill and help help bring [45:57] them up to speed so they can help [45:59] prepare students to use computers [46:01] effectively and not just consume [46:03] technology, but produce technology, if [46:06] you will. [46:07] >> Yeah. So, I I was a kind of a [46:09] non-traditional aged student, especially [46:11] when I went back. So, I I've always [46:14] liked non-traditional age students and a [46:18] few years ago there was an opportunity [46:20] uh in the Kenosha area where I live [46:24] where the uh local school district they [46:28] wanted to begin teaching more computer [46:31] science but like you said uh most people [46:33] who get a CS degree do not become uh [46:37] teachers. It's just much more lucrative [46:40] to become software developers. [46:42] So, uh, I volunteered to work with, uh, [46:46] like half a dozen. These were mostly [46:48] math, uh, teachers. [46:51] Um, and I basically tried to in like a [46:56] year, year and a half, kind of go [46:58] through the basics of what uh, somebody [47:01] would get at a uh, somebody who's [47:04] studying computer science. But more [47:06] importantly, I wanted them to kind of [47:07] build up their experience [47:10] because I know what I know about [47:11] students is whenever you're teaching [47:14] students about coding, right, they're [47:16] going to get stuck. [47:18] And I guess my number one advice for the [47:22] high school teachers was you have to [47:24] learn you have to get stuck yourself and [47:27] figure out how to get yourself unstuck [47:29] because that's you know mostly what [47:32] you're going to be doing all day is [47:34] these students are going to have good [47:36] questions and the more experience you [47:38] have the better. [47:41] My general thoughts on teaching like I [47:43] I've already said uh my main job is just [47:47] motivate the students to want to learn [47:50] and again enthusiasm goes a long way. [47:54] also coming up with interesting projects [47:57] like projects that they actually want to [47:59] solve or they want to know how well how [48:01] do we uh whatever fill in a a cross word [48:06] puzzle or how do we have a robot move [48:08] through uh like a 2D map uh stuff like [48:12] that um I spend a lot of time actually [48:16] thinking of interesting projects and I [48:19] think that's way more valuable than you [48:22] know being a really polished ed speaker. [48:24] I don't consider myself a really [48:26] polished uh presenter of information, [48:29] >> but I'm enthusiastic and I I try really [48:32] hard to come up with these interesting [48:34] projects and then the students just take [48:36] over from there. [48:38] >> Well, I will say this is your first ever [48:40] podcast interview and I think you're [48:41] doing a great job. Uh so, [48:42] >> thank you. [48:43] >> Yeah, don't don't hurt yourself too much [48:45] in terms of uh being a Polish speaker. [48:48] Uh one thing that I you brought up that [48:51] I think is really interesting is just [48:52] the the focus of projects and uh that's [48:56] one thing that free code camp like many [48:58] years ago we we reoriented the [49:00] curriculum to really be project oriented [49:02] like everything is a project everything [49:04] is like every lesson is in the context [49:07] of you're trying to build this you know [49:10] leaderboard for a video game or you're [49:12] trying to uh like even in the design [49:14] space you're trying to design a USDA [49:16] nutrition label. or something like that, [49:18] right? So that way there there's [49:19] something concrete and there's like a a [49:21] clear goal and same thing with the free [49:22] coamp YouTube channel. Almost all the [49:24] courses are project oriented where you [49:26] code along at home, you clone the GitHub [49:28] repo and then and you try to progress. [49:30] What are some uh ways that you've [49:32] introduced project-oriented teaching [49:34] into your teaching over the years? [49:38] >> So I'd say that's the focus. Uh again, [49:42] um I'm not dynamic enough to I think [49:46] really get the students to want to learn [49:48] just from hearing me. So, uh I pretty [49:52] much center all of my courses around a [49:54] group of projects. [49:56] And of course, there is content that [49:58] that I have to present, but it's I [50:01] almost always uh let the students know [50:04] the goal first, right? We're building X, [50:06] Y, or Z. Okay. So, how do we go about [50:09] doing that? Well, we have to do this and [50:11] we have to do that. So, what what do we [50:13] have to know in order to do those [50:15] things? [50:17] That I think for sure is uh that's [50:20] definitely my preferred uh way of [50:23] teaching. [50:24] >> Yeah. And a question I have then is so [50:27] what proportion of the time uh like [50:29] let's say totally like there are like I [50:31] don't know 20 weeks of class I don't [50:33] know the exact amount and like maybe [50:34] there are you meet like twice a week or [50:36] something like that. I'm not sure how it [50:38] works over at Carthage, uh, the [50:40] university that you teach at, but um, [50:42] like what percentage I guess of the time [50:44] would you spend lecturing versus, um, [50:47] like kind of doing flip classroom where [50:48] they're working on stuff and walking [50:50] around and talking to them? [50:52] >> So, it depends depends on the course. [50:55] Uh, [50:56] there are some courses that uh, it's [50:59] easier to do uh, the project work. Those [51:03] are the courses I prefer. Um, [51:06] and [51:08] you know, there's a couple sayings some [51:11] of my colleagues use. Uh, you could [51:13] either be the the sage on the stage or [51:16] the guide on the side. And I like to be [51:20] the guide on the side. And I would say, [51:24] you know, there's not any course where [51:26] I'm talking 100% of the time. I would [51:28] say I would maybe limit the max would be [51:31] maybe 50% lecture 50% hands-on [51:35] and a lot of courses it's much you know [51:38] much higher hands-on work there's some [51:41] some courses are 90% [51:44] uh you know we have discussions about [51:46] the problems but I wouldn't really call [51:48] that like straight lecture [51:50] >> yeah it's not like PowerPoint and just [51:52] standing up there delivering the same [51:53] lecture that you delivered six months [51:55] ago. Yeah, that's not my particular [51:58] style. I uh I like engaging the students [52:02] and seeing what they can do. [52:05] >> And you know, students change. I was [52:08] just talking to a colleague about this. [52:10] Every six or seven years, the whole [52:14] student philosophy on learning changes. [52:18] And this is not something I knew was [52:20] going to happen, but it's been pretty [52:22] consistent. [52:24] uh you know it was a little bit of a [52:26] shock. my first six or seven years I [52:28] spent a lot of time and I thought I was [52:30] pretty much done but no the students [52:33] changed and then I had to adapt and now [52:38] you know I I pretty much always have my [52:40] eyes open and I'm I'm trying to uh [52:44] adjust and adapt to uh you know students [52:48] the way students want to learn [52:50] >> right [52:50] >> so for example I I used to be able to [52:52] like have a hundred minute class where [52:55] maybe I was talking uh you know a lot of [52:58] that time now forget about it. No no no [53:01] student will sit still for 100 minutes [53:04] and listen to anything. So but that's [53:07] fine. I I know in six or seven years [53:10] something else is going to change and [53:12] that's just part of our jobs I think is [53:14] to adjust and adapt [53:17] uh and you know meet the students where [53:19] they're at. [53:20] >> Yeah. a lot of people like computer [53:22] science has been I think one of the most [53:24] popular uh majors uh for the past yeah [53:28] really two decades or so and it's it's [53:31] certainly I think it's the highest [53:32] compensated like you're you're likely uh [53:36] you know annual income is the highest if [53:38] you study that versus pretty much any [53:39] other subject uh assuming you're not [53:41] going to like medical school or law [53:43] school uh afterward so like like just at [53:46] the bachelor level uh and there with all [53:49] the rhetoric around, oh, you shouldn't [53:51] become a developer anymore. You don't [53:53] need to, you know, just like [53:56] you have some skin in the game. Not only [53:57] are you teaching, of course, but your [54:00] son, one of your your oldest son is [54:01] actually actively studying computer [54:03] science right now. And uh your other son [54:05] is getting ready to finish high school [54:08] and go study computer science. So [54:10] clearly, you still believe that this is [54:13] the place where people should go. This [54:15] is kind of like one of the best fields [54:16] to get into. Would you agree with that [54:18] statement? Definitely uh I mean learning [54:21] about computer science changed my life [54:24] for the better of course uh not only [54:28] kind of career-wise but it really helped [54:31] me think like I'm a much better thinker [54:34] than I was before I started uh learning [54:38] about computer science [54:40] and yeah I'm definitely excited about my [54:45] kids studying computer science. So my [54:47] oldest son is right now uh he's a double [54:50] major computer science and environmental [54:53] studies or environ environmental [54:55] science. This is the one thing I wish I [54:58] did when I was a student is maybe study [55:00] computer science and something else like [55:03] so I I know how to program really well [55:06] but I don't know anything else. I kind [55:08] of wish I I learned something else too. [55:10] >> Kind of like a subject domain area that [55:12] you can apply program because like [55:14] environmental, you know, science like [55:16] you you could potentially take data [55:18] from, you know, like a a water [55:21] filtration plant or I I'm just trying to [55:23] grab examples out of my head or uh you [55:26] know data around like the water runoff [55:29] uh of a reservoir or something like [55:31] that, right? And then you can apply kind [55:33] of like computer science thinking and [55:35] systems thinking and you know all the [55:38] applied mathematics and everything that [55:39] you learn when you get a CS degree uh [55:41] toward being able to do useful things [55:43] with those data or solve problems, [55:45] right? Is that [55:47] >> Yeah, exactly. So, my son is uh he's [55:50] going to be a senior next year and he's [55:52] thinking about his senior thesis project [55:56] and he's been telling me about um his [56:00] interest in in like AI data centers and [56:04] we happen to be right on uh Lake [56:05] Michigan and there's all these uh AI [56:10] data centers that are popping up I think [56:12] because of the large body of fresh water [56:14] that we have here in the Great Lakes. [56:17] So, he doesn't quite know what it's [56:19] going to be yet, but it's going to be [56:20] some [56:22] uh the environmental science part of it [56:23] is going to be some kind of [56:27] uh report or looking into the the [56:31] environmental effect of all these AI [56:33] data centers and he'll have the skills [56:36] to, you know, take data and manipulate [56:39] it and do interesting things with it. [56:42] >> Yeah. [56:42] >> So, I think it's great study computer [56:45] science. my younger son. I I hope he [56:48] studies computer science and something [56:50] else as well. I think that's uh that's [56:53] an awesome uh makes you very powerful in [56:57] that other field if you can do things [57:00] manipulate data and write programs. [57:02] >> Yeah. And I'm sure lots of you know [57:04] scientists and other people that uh your [57:06] son works with are going to be leaning [57:08] on him to help him get their work done [57:09] because he'll have you know the ability [57:11] to leverage these tools. [57:14] I want to talk a little bit about [57:15] playbacks uh because of course you have [57:18] built your own interactive learning [57:20] platform that free co camp frequently we [57:22] we publish uh articles that link people [57:25] to a lot of your coursework and we think [57:27] it's phenomenal and uh I want to talk [57:30] about like the platform and and why you [57:33] chose to build an interactive learning [57:35] platform on your own when there are like [57:37] companies that do this there like [57:39] probably you know entire university [57:41] systems that will build their own like [57:42] learning management system and all this [57:44] stuff and you just built it yourself. [57:47] >> Yeah. So, this all goes back to uh so [57:51] I'm a book lover. Again, I'm I'm old [57:53] enough when I was a student learning [57:55] computer science. Uh pretty much the [57:58] only resource we had were books, you [58:01] know, traditional programming books. I'm [58:03] surrounded right now by 800page [58:05] programming books. Uh and I see some in [58:09] your uh background as well. Yeah, this [58:11] is one we published through Free Code [58:13] Camp Press. It's Yeah, [58:15] >> Omar Rosenbomb's uh getting things done. [58:18] He he just published another book. Uh we [58:20] publish a lot of free programming books, [58:21] but but basically you're talking about [58:23] like the O'Reilly, the uh you know, like [58:26] C++ in 24 days, you know, like all those [58:29] books of your uh like the reference [58:31] manuals. If anybody's seen the movie [58:33] Hackers, they're like geeking out over [58:34] all their different books. Uh like the [58:36] the picture guy, you know. [58:39] >> Yeah. Yeah. Exactly. Um but you know [58:43] younger people are not into books uh [58:47] sadly for me but [58:50] you know there's a reason right so when [58:53] I think about a programming book you [58:55] have of course this narrative that comes [58:58] from the author they're they're trying [58:59] to explain uh some code and then they [59:03] show little code snippets usually these [59:05] programming books I know you've read [59:06] many of them [59:09] but It's a little inconvenient, I think. [59:12] So, you're always sort of going back and [59:14] forth between the author narrative and [59:16] this little snippet of code. And you [59:19] never really get to see the full code, [59:22] right? Because it just would take up too [59:23] much paper, I I guess. And also [59:26] interestingly, I I think publishers [59:30] probably ask the authors to be a little [59:32] more verbose than they probably need to, [59:36] right, to justify that whatever $40 book [59:39] price. [59:40] >> Yeah. [59:40] >> Uh so it's books are fine. Uh I love [59:44] books actually. They're still my [59:46] preferred uh way of learning something. [59:50] But uh again, not not everybody agrees. [59:55] So more recently of course people make [59:58] videos [60:00] and uh videos are great too. Videos have [60:02] those two sources right? They have the [60:05] narrative that comes from the author [60:07] that's the audio and then the code of [60:10] course shown on the screen. So you get [60:13] to see kind of see and hear both of them [60:16] at the same time. You don't have to go [60:17] back and forth. You don't have to scroll [60:19] 10 pages back to uh reread something. [60:24] But videos have some problems I think as [60:27] well. So um [60:31] it's just too darn easy to let the video [60:33] like play on. [60:34] >> Exactly. I mean people like people [60:37] literally will sometimes comment on free [60:38] coance videos. I went to sleep and I [60:40] woke up and I was learning something [60:42] completely different because YouTube [60:44] just kept autoplaying. Uh, and and I [60:46] mean that's great. Like maybe they [60:48] learned something subconsciously when [60:49] they fell asleep on their keyboard or [60:51] whatever. But [60:52] >> yeah, like like I like the fact that the [60:55] book doesn't advance itself. You have to [60:57] be reading at least think [60:59] >> with a book if you get to the end of a [61:00] paragraph and you you kind of wonder [61:02] what did I just read? You could easily [61:04] go back and restart that paragraph. [61:06] >> With the slider, the video slider, it's [61:08] it's harder. Um, also when you're, [61:13] you know, going back for reference, when [61:15] you want to go find that twominute part [61:17] of the video that explained exactly what [61:19] you're looking for, again, that slider [61:21] is not a great search tool. [61:25] And also, of course, if you ever want to [61:27] copy and paste anything, it's it's very [61:29] hard or impossible to do with a video. [61:31] >> Yeah. And to go back to one thing, not [61:33] to derail you with learning on a book, [61:35] like when I'm reading a book, I can [61:36] often remember like kind of about how [61:39] thick the book was on one side of my [61:41] hand and how thick it was on the other [61:42] and maybe even like the position on the [61:44] page that that text or if it had like a [61:47] lot of the O'Reilly type books have like [61:49] different kind of like panels type [61:51] things inside there that are different [61:52] colors or they'll have some sort of [61:54] illustration. Those are just little [61:55] hooks that your brain uses because of [61:57] the weirdness of this associative [62:00] network in our brain that will help you [62:02] quickly find what you're looking for. Uh [62:04] just because of all that additional [62:05] context and you don't get that with a [62:07] video. [62:08] >> Yeah, it's uh amazing how our brains do [62:11] stuff like that. [62:12] >> Yeah. But getting back to the playback, [62:13] sorry, I just wanted to add that and [62:15] like loop back to that previous insight [62:16] that I had uh because I think it is [62:18] important that we acknowledge the way [62:20] humans learn is completely different [62:22] from computers. computers would just [62:23] throw all that extraneous data away. We [62:25] can't. It's just part of who we are, [62:26] right? U so these playbacks combine the [62:30] best of both worlds in a sense that they [62:31] they combine the audio track with the [62:33] textual content uh and the code. [62:37] >> Yeah, that that was my goal is to take [62:39] uh what I liked about books uh is [62:43] uh the narrative part, but also you get [62:46] to see the full code uh available uh at [62:50] any time. So you could uh if somebody [62:53] goes to the website, they can kind of [62:55] click around the tabs. You always have [62:56] access to the full code and the [62:58] narrative is kind of on the side um [63:01] that's describing [63:03] uh that narrative. [63:06] The other thing that I I think is great [63:09] about the code playbacks is not only can [63:12] you add text, but you can draw those [63:14] pictures, those whiteboard style [63:15] pictures. You can attach those right to [63:18] the narrative. So when somebody's [63:20] reading, you can read the code and then [63:22] see the whiteboard style picture uh kind [63:25] of side by side. It's a little harder to [63:27] do that with a book, right? You have to [63:30] usually go back a few pages and you kind [63:32] of [63:33] >> It is hard to do that sideby-side [63:35] comparison because the text is going to [63:36] wrap and you're like you said, it could [63:38] be several pages back and so you're [63:40] flipping back and forth. It's kind of [63:41] like the math books where the answer [63:42] would be in the back and you'd have to [63:43] like flip back and forth. But with you [63:46] have infinite size canvas essentially. [63:49] uh in a browser to put all these things. [63:51] One thing that you've been doing [63:52] recently is incorporating LLM assistance [63:56] and like code analysis into your tool. [64:00] Can you can you walk through like how [64:01] you went about doing that? Um because I [64:03] think there are a lot of lessons there [64:05] as to how to do it and not to be too [64:08] handholdy or to be gimmicky, but to [64:11] actually use it and actually get utility [64:14] for having this LLM looking at your code [64:16] and giving you feedback on it. So [64:19] definitely my goal was to sort of build [64:22] in that uh infinitely patient tutor. So [64:26] as you are kind of going through these [64:28] code playbacks uh you maybe I should [64:32] have said this before you get to see the [64:33] code evolve. So it's uh you don't start [64:36] with a final product. You it sort of [64:39] grows slowly over time. And at any point [64:42] in time, you can uh basically ask this [64:46] AI assistant a question about the code [64:49] that's currently on the screen. [64:52] And um so I I the way I built it was uh [64:57] with you know one of the big companies [65:00] APIs and I send the code and I send the [65:03] author narrative to the LLM and um [65:08] couple ways to interact with this. the [65:11] the student can come up with a question [65:12] like literally, you know, what's the [65:15] head pointer going to uh be set to after [65:18] this function is completed, right? Or uh [65:23] there are a couple ways to have there a [65:26] couple sort of built-in questions like [65:28] uh summarize this code or what could go [65:31] wrong here. Uh there's also the ability [65:34] to [65:36] um to make it a little more interactive. [65:39] you can have the LLM generate these [65:42] multiple choice questions, self-graded [65:44] multiple choice questions that are about [65:47] something that's on the screen, okay? Or [65:49] or something that the the author is [65:52] trying to talk about in their narrative. [65:55] And I was shocked how well the LLM [65:59] actually worked. So, I started [66:01] experimenting with this uh probably a [66:04] year and a half ago. Uh, and I've just [66:08] been shocked that um, you know, it it's [66:12] pretty darn good at analyzing the code [66:14] and the narrative and the student [66:17] question and coming up with a pretty [66:20] good answer. So, it doesn't generate [66:23] code. That's not not the point. It [66:26] really is there to be that tutor, that [66:29] assistant, [66:30] uh, that can help answer questions. [66:35] you know, I I measure how many students [66:38] are actually using that and it's really [66:39] only about 10% of the time do uh does a [66:43] student actually use the that AI tool. [66:47] And what that tells me is 90% of the [66:49] time the human generated content that I [66:51] came up with that's doing the trick just [66:54] fine. [66:55] uh it's only kind of like the edge cases [66:58] and sometimes a a student will be uh [67:02] have a particularly insightful question [67:04] that maybe I didn't think of and so for [67:08] that it's great. Uh but most of the time [67:11] most students just are happy with the [67:13] the human content that I generated. So [67:16] all of my [67:18] uh code playbacks they're code that I I [67:22] write myself. [67:24] So I I'm very specific when I I'm [67:27] writing some code for teaching and it's [67:31] just easier for me to write the code [67:33] than to have the LM write it and then me [67:36] kind of mess with it. So [67:38] >> yeah, [67:38] >> you know, I use the LLM's for to [67:41] actually help me a little bit with [67:43] writing the code for this tool, but when [67:45] I'm doing some when I'm teaching [67:46] something, I you know, I never use the [67:49] LLM to generate code. [67:51] >> Yeah, that makes sense. So uh [67:54] essentially 10% of the usage is using [67:58] the LLMs to address edge cases that the [68:01] other 90% of usage uh which is focused [68:04] on the coursework you've already created [68:05] personally like manually uh that that [68:08] does the trick 90% of the time and then [68:10] the 10% or the LM's there to catch the [68:13] 10% edge cases and then eventually based [68:15] on that you can go and further improve [68:19] the original coursework and it and maybe [68:21] eventually it'll be down to like 1% of [68:23] usage depending on uh how like the [68:26] feedback loop and how um but like I know [68:29] Google for example I talked with like a [68:31] Google engineer working on search and [68:33] they said like something like 15% of all [68:35] Google queries have never been searched [68:36] before. So there's always going to be [68:38] that edge case that unexplored kind of [68:40] frontier. Um I have two more questions [68:43] for you if you have a moment. [68:45] >> Yeah there there's also like remedial [68:47] questions. Sometimes students jump in a [68:50] little too uh they're not quite there to [68:53] handle the the particular program. So [68:56] edge cases, remedial type things. Yeah, [68:59] those are uh [69:01] >> yeah that's really interesting because [69:03] like free co camp like people will get [69:05] their their eyes will be bigger than [69:06] their stomach. They'll like get ahead of [69:08] themselves and then it is nice to have [69:10] that kind of like release valve of like [69:12] they don't have to be like oh damn I'm [69:13] like completely uh you know I have no [69:16] idea what I'm doing right? I'm out of my [69:18] depth. Uh they can kind of chat their [69:19] way back to feeling comfortable or maybe [69:21] the LM could remind them like, hey, you [69:23] know, you might not feel so out of your [69:25] depth if you hadn't skipped this [69:26] section. [69:27] >> Yes. [69:28] >> So I I guess that's something that your [69:30] LLM tool can do, too. Um so my quick [69:34] questions for you because uh you've been [69:36] very patient with your time. I know [69:37] you're a busy prof. Uh what are the most [69:40] durable skills and concepts that [69:42] everyone should still go deep with that [69:43] you don't think LLMs are going to you [69:46] know obsolete like like nobody's going [69:48] to regret spending n hours learning XYZ [69:51] because that's unlikely to be you know [69:54] automated. [69:56] >> I mean I always go back to problem [69:57] solving. uh I don't have uh you know I [70:04] don't think there's any one particular [70:05] language that uh somebody has to learn [70:09] or any one technology but just problem [70:11] solving in general and often that means [70:14] really uh getting comfortable with [70:17] debugging code like when we write code [70:21] it's different than when the code runs [70:23] and the only way to really see what's [70:26] happening is to step through that code [70:28] and see what effect it's having on the [70:30] other parts of the system. So if you [70:33] build something from scratch or you [70:35] build on top of you know something uh [70:41] being able to to debug it and fully [70:43] understand what effect it's having I [70:45] think is the to me that's the core of [70:48] problem solving. [70:52] >> Awesome. So problem solving like nobody [70:56] going to progress [70:56] >> very general and generic but uh [70:59] >> but in the context of programming like [71:01] literally like like just understanding [71:03] what's actually happening in there and [71:04] being able to fix that and like you said [71:06] earlier programming is just getting [71:09] stuck over and over and over like when [71:11] things are going swimmingly and you know [71:12] exactly what to do like those are the [71:14] parts that are actually like pretty [71:15] hashed out and and uh well understood. [71:18] Those are the parts that are going to [71:19] be, you know, potentially automated by [71:21] you just reusing code that you've [71:23] already written, uh, or, you know, like [71:25] grabbing a library for something that's [71:27] very well understood. It's that that [71:29] frontier of our understanding of a [71:31] particular, you know, code base or [71:33] trying to solve a particular problem. [71:35] That's where you're going to spend [71:35] almost all your time as a developer, [71:37] right? [71:38] >> Yeah. And uh again, the best way to [71:40] learn is just build build, [71:43] get stuck, get yourself unstuck, and you [71:46] know, try and take lessons away every [71:48] time you do that. [71:50] >> Absolutely. You've been through the com [71:52] bubble. Um you were like a pretty fresh [71:55] dev in the late 90s when it happened. Um [72:00] I don't know if you want to share any of [72:02] your stories from that period real [72:03] quick, but but like many people would [72:06] say we're kind of in a similar era where [72:08] just like you know kind of unbelievable [72:10] amounts of money are being put into [72:12] building out AI data centers or invested [72:15] in uh training models when you know [72:19] there might be diminishing returns that [72:21] we don't know about. Uh there are lots [72:24] of you know extremely expensive projects [72:26] being built in parallel when likely only [72:29] a few will actually see sustained usage [72:31] over time. Uh just the sheer volumes of [72:35] money uh and the the sheer brick neck [72:38] speed at which people are trying to get [72:39] things done and you have to spend more [72:41] money if you want to get things done [72:42] quicker, right? Like uh it's that whole [72:45] good fast cheap triangle, right? And uh [72:48] you can only have two and and most [72:50] people are opting for good and fast. [72:52] Good and fast, right? Um so the cheap is [72:56] nowhere to be seen and trillions of [72:58] dollars being spent. [73:00] What did you learn from the doco on [73:02] pubble and do you often think about that [73:05] time in the context of like reading the [73:07] headlines about you know investments and [73:10] product announcements and and CEOs [73:12] saying you know kind of grandiose [73:14] things. [73:15] >> Yeah. There have definitely been [73:19] there's been hype, you know, since [73:21] the.com bubble up until, [73:24] you know, this AI era. So, I'm thinking [73:27] of like uh NFTTS, for example. Most [73:30] people knew that was a scam. But at the [73:33] time, there were some people who [73:34] thought, "Oh, yeah, NFTTS, that's that's [73:37] going to change the world." [73:40] with AI there's something different like [73:42] like there's something really there that [73:45] I can't deny. Um [73:49] so I would say of all the you know right [73:52] after the dot uh bubble happened I was [73:55] actually uh just when I started teaching [73:59] and the numbers of computer science [74:01] students went down a lot but it didn't [74:04] go away. [74:06] That was actually one of the the best [74:08] parts of my career because the only [74:10] people who were studying computer [74:12] science were the ones who really wanted [74:13] to uh learn about programming. Uh you [74:18] know all the other people they were [74:20] scared off by the by the bust. [74:24] Um, and then there have been several ups [74:27] and downs over the last, you know, 20 [74:29] years, but there's definitely something [74:31] different about this this AI era that [74:35] we're in. And I wish I had a better [74:38] answer. I I don't know what's going to [74:40] happen, but this one does feel [74:44] different. [74:47] So, one hardening takeaway from that is [74:49] a lot of the fairweather, you know, [74:51] people that would happily end up on Wall [74:53] Street or wherever they could earn a lot [74:56] of money, uh, they're not necessarily [74:58] going to be in the computer science [75:00] departments right now. [75:01] >> So, exactly, at the very least, if you [75:03] genuinely enjoy learning programming and [75:06] building software, uh, then you're [75:08] probably going to be surrounded by [75:09] like-minded individuals if you go pursue [75:12] a CS degree right now. Yeah, I think [75:14] that's that's how it's going to be. [75:17] >> Yeah. Well, this has been a very [75:18] insightful conversation, Mark. It's been [75:21] an absolute pleasure talking with you. [75:23] Uh I hope everybody learned as much [75:25] during this call as I have. Uh I've got [75:28] like 179 lines of notes that I'm going [75:31] to be reviewing as I try to figure out [75:33] how to like process and break all this [75:34] down and distill it into like an intro. [75:37] Um, Mark, uh, I have linked to some of [75:41] your playbacks and some of your free [75:42] Code Camp articles in the description. I [75:44] encourage everybody to check out that, [75:46] uh, learn from this guy. He knows what [75:48] he's talking about. He has been there [75:50] both in industry and in teaching for [75:53] decades. Uh, and there's a tremendous [75:55] amount to learn from him. [75:58] >> Thanks a lot, Quincy. [75:59] >> Yeah, it's been an absolute pleasure. [76:00] Until next week, everybody. Happy [76:02] coding. [76:03] >> Happy coding.