How teaching and learning will change in the age of AI, with C. Edward Watson, PhD, and Beth Schwartz, PhD

Over the past several years, generative AI has moved into classrooms at lightning speed. C. Edward Watson, PhD, author of “Teaching with AI: A Practical Guide to a New Era of Human Learning,” and Beth Schwartz, PhD, senior director of teaching and learning at APA, discuss how students and educators are navigating these changes; how AI can help or hinder learning; how teachers can redesign assignments and assessments; and what schools and universities should be teaching students about using AI responsibly. Learn more about your ad choices. Visit megaphone.fm/adchoices

Speaker names marked ? are inferred from speaking time and the show’s known hosts — they are not voice-verified.

• 0:00Speaker 00

  Are your ad campaigns lighting up the dashboard? But not the pipeline. That's bullspend, and marketers are calling it out in. Dashboard Confessions.

• 0:09Speaker 01

  My boss asked for results, so I opened my dashboard for the only positive sounding metric I had: impressions.

• 0:17Speaker 00

  Cut the bullspend. See revenue, not just reach. LinkedIn delivers the highest return on ad spend of major ad networks. Advertise on LinkedIn. Spend $250 on your first campaign and get a $250 credit. Go to LinkedIn.com/campaign. Terms and conditions apply.

• 0:31Kim Mills?

  Over the past several years, generative AI has moved into the classroom at lightning speed. Students are using AI to brainstorm ideas, write papers, solve math problems, and study for exams. Teachers are experimenting with it to create lesson plans and develop assignments. Meanwhile, schools and universities are trying to adapt to a technology that is evolving faster than many policies and teaching practices can keep up. Today, we're going to talk to two experts who are helping educators navigate these changes using the science of learning to help answer questions about how AI can support and not hinder student learning. So how are students using AI in their work right now? What counts as cheating in the age of AI?

• 1:16Kim Mills?

  What kinds of assignments, projects, and exams can teachers use to make sure that their students are really learning, not just copying AI-generated answers? What are the potential benefits of AI in education? And how should colleges and universities think about educating an AI-literate workforce in the future?

• 1:36Kim Mills?

  Welcome to Speaking of Psychology, the flagship podcast of the American Psychological Association that examines the links between psychological science and everyday life. I'm Kim Mills.

• 1:51Kim Mills?

  I have two guests today. First is Dr. Eddie Watson, Vice President for Digital Innovation at the American Association of Colleges and Universities and founding director of the Association's Institute on AI, Pedagogy, and the Curriculum. His work aims to help colleges move beyond fear or hype toward a thoughtful, evidence-informed integration of AI into teaching and learning. He has focused on how institutions can respond to AI's challenges for academic integrity, student learning, curriculum design, and workforce preparation. He's the author of the book Teaching with AI: A Practical Guide to a New Era of Human Learning, and was recently recognized by EdTech Magazine as one of the top 25 higher education influencers to follow in 2026.

• 2:37Kim Mills?

  My second guest is Dr. Beth Schwartz, Senior Director of Teaching and Learning at APA. She leads the Association's efforts to advance evidence-based teaching and learning across educational settings. Before joining APA, Dr. Schwartz spent decades working in higher education as a psychology professor, university dean, and provost. She's a fellow of APA's Division II, the Society for the Teaching of Psychology, and she's leading APA's efforts to provide educators with practical, evidence-informed strategies on how to use AI in their classrooms, guided by psychological research on how students learn. Dr. Schwartz, Dr. Watson, thank you for joining me today.

• 3:19Speaker 03

  Thanks for having us.

• 3:19Speaker 04

  Yes, great to be here.

• 3:22Kim Mills?

  Let's start by talking about how common AI use in the classroom is right now at the K through 12 and university level. I mean, what do we know from research and surveys? And what are you hearing from the educators you work with? Dr. Watson, maybe you want to grab that?

• 3:38Speaker 03

  Well, it's interesting that the data is a little bit mixed. We can probably estimate that it is a large percentage of students that are using it, especially at the university level. But surveys of students, you know, where they self-report how active they've been using generative AI, the results, depending on the survey, are wildly divergent. I've seen them as low as in the 20% range and in other surveys, up near 90%. So I'm imagining that some students might not want to self-report that they have been using it. But without a doubt, you know, I would imagine it's certainly more than half, maybe three quarters of students are using it. And of course, there's a lot of different ways to use it.

• 4:20Speaker 03

  I mean, some students are using it for, you know, career planning or to help them learn various topics. And of course, there are other ways that they might use it that might diminish their learning or might be outright cheating. But certainly, the vast majority of students are indeed using AI today.

• 4:37Kim Mills?

  And are we talking about mostly high school and college level? Or are kids younger than that using AI right now?

• 4:43Speaker 04

  In the primary schools and secondary schools, a lot of the surveys have focused less so on the students' use of AI and more so on the teachers' use of AI. And when we look at that, yeah, we see that it's obviously increased a great deal in the last year and a half. The use by faculty or educators is about the same in the primary and secondary and higher ed, where it's primary, it's about 43%. In middle school, we see it about 63%, up to 72% in high school and college. So, but the use for students is much less common to be asked on those surveys. So we have much less data on that. There is more in the high school use, but in the primary and secondary schools, educators

• 5:26Speaker 04

  are still trying to figure out for themselves how to use it, let alone how to incorporate it into the learning environment for those younger students.

• 5:33Kim Mills?

  Yeah, and we'll talk about that. But Dr. Schwartz, I want to ask you, what are the biggest concerns you hear from educators about how the students are using AI?

• 5:41Speaker 04

  Mm-hmm. And we see time and time again, I think the top two concerns reflect cognitive offloading, which simply means that the AI is doing the thinking, the reasoning for them. And in other words, they're not learning. And there's the issue of cheating or academic integrity. Those kind of rise to the top when you ask educators about AI. But there's also concerns about the fact that the reliability of AI. So they're speaking less so about the learning part, but just what the content is when they're using it or their students are using it. Educators also speak sometimes about the erosion of trust that happens because whether or not students turn in work, then they have to determine whether or not it's the student's

• 6:26Speaker 04

  work or the work of AI. And that that relationship that exists between the student and the faculty member can be eroded just by those questions. There's issues of equity that's of concern. So does everybody have the access that they need when AI is available, particularly since there are some that are free and some that are where there's costs? And I've heard often from faculty about just the lack of professional development. So they're having, there's an expectation sometimes for administrators that they should be using AI, and yet they are not given professional development on the technical proficiency that they need or the pedagogical strategies that they need to learn to incorporate AI into their everyday classrooms.

• 7:10Kim Mills?

  Well, Dr. Watson, that's something that you're working on, right? Can you talk about that?

• 7:15Speaker 03

  So specifically how faculty are using AI in the classroom?

• 7:19Kim Mills?

  Yeah.

• 7:20Speaker 03

  Well, I guess what we're kind of seeing, that there's maybe 20% of faculty, give or take, that are sort of leaning in to figuring out how to use AI themselves, you know, like leveraging it for the improvement of learning outcomes for their students.

• 7:37Speaker 03

  There's probably this big middle of 70% that are sort of like wait and see, but sort of like learning to use it in a variety of different small ways. And then there's certainly a resistance pocket as well. So it's kind of like the broader landscape of how faculty, at least are self-reporting that they're using AI. But then, when it comes to specifically the notion of leveraging AI for maybe teaching and learning purposes or course improvement, there are literally tools out there that help with every aspect of sort of the teaching and learning lifecycle, from developing a syllabus, to finding course materials, to sorting course materials, to assignment

• 8:18Speaker 03

  design, putting together lectures, developing PowerPoint slide decks, creating active learning

• 8:26Speaker 03

  activities for the classroom, all the way through providing feedback to students and even grading. About a year ago, or a little bit less than a year ago, one of the big learning management system providers came out with tools nested within university learning management systems that purports to be able to grade just about anything. So the question is, it's like, yes, AI can do these things, but do we want AI to do these things? Is this something that we would like to ask AI to perform that kind of task? Or are there certain aspects that really belong within the the identity of what it is to be a professor in higher education? And a lot of that's being hotly debated right now.

• 9:10Kim Mills?

  Yeah, because from what you described, I mean, basically, AI, it sounds like can do almost anything, right? I mean, why do you need a teacher if AI can do all of these things?

• 9:20Speaker 03

  Well, isn't that, that's one of the debated topics. In fact, I had an interview with a press outlet just yesterday about a new program that had been launched in London. It's a master's program that's largely taught by AI.

• 9:36Speaker 03

  So is that a good idea? That was the question for me was like, do you think this is a good idea? And it's like, well, I guess I'm of different minds. I mean, one is very much from a learning aspect. Like, would there be evidence to show that how they've designed and are leveraging AI? Does it result in learning outcomes that are at similar or the same level of proficiency as a face-to-face program? And indeed, these are questions that we ask about online learning 25, 30 years ago. Right? So it's kind of like the same kinds of questions. And then we, of course, through inquiry around face-to-face and online learning, we found that there's nuances, right? I mean, online learning can be done very successfully, but it's like, when done well,

• 10:21Speaker 03

  it can have really good outcomes. And, you know, when done poorly, face-to-face instruction could be worse than an online course. So I think that that's likely going to be the outcomes of some of these, I guess, emerging studies around leveraging AI for instructional purposes. But we don't know yet, right? I mean, those research studies haven't been done, and certainly evidence should guide, you know, the practice that we might employ or where we might encourage our students, you know?

• 10:49Kim Mills?

  Well, Dr. Schwartz, let me ask you this. This is not the first time that people have worried about disruptive technology in education. I mean, AI is just the latest example of schools and universities and students needing to adapt to changing technology. Whether it was calculators or Google, you know, is now we've got AI. Is that fundamentally different from these earlier disruptions, do you think?

• 11:12Speaker 04

  Right. It's a great question and one that I'm often asked, and the calculator example is often brought up, and then the internet. And most educators would would agree that it's fundamentally different. And it's fundamentally different because, well, the calculator, right, it did routine computations, right? It didn't it didn't do any arithmetic thinking. It didn't come to arithmetic conclusions, like concepts or reasoning. The search function, like of the Google search, it just removed the search process from the learning. So and I remember as a faculty member, I would sit there and and think about the times I had to go to the library, open up those onion-skinned abstract books to find the articles for psych info.

• 11:57Speaker 04

  And then I was teaching, and my students could literally just lie there with their laptop and and and just find something instantly. And I thought to myself, but that's not a learning goal of mine, right? That's not one of my learning goals. So it's okay for the search process that's being assisted with with the internet and with Google searches to get in the way of learning. Whereas AI is is a much different type of technology. Right now, it's producing the output. It's producing the evidence of learning that students are turning in. So it's a very different type of disruption. It generates explanations. It writes essays, right? It it does problem solving instantly for students. So between solving problems, summarizing readings, and I with the list can go on.

• 12:42Speaker 04

  It's a very different type of output compared to the former technologies that were considered disruptive in the classroom.

• 12:51Kim Mills?

  Well, so Dr. Watson, keying off of that, do you think AI is going to erode students' critical thinking ability?

• 12:59Speaker 03

  Well, we did a survey at AAC and U. It was titled in November of 2025, published in January of 2026. It's available full text online. It's called the AI Challenge. And we surveyed faculty in higher education across all disciplines, all institution types. And of that corpus of intelligence, 90% of respondents said that they thought that generative AI will diminish students' critical thinking skills, that that was their perception. And more than two-thirds said that it was going to have a significant impact, like a really large impact on students' critical thinking skills. So, you know, that's a that's a lot of opinion that points in the same direction that is

• 13:43Speaker 03

  likely suggestive of an outcome that we will see around student usage of generative AI, if we're not careful, you know? There's there's definitely instructional intervention that we could do to address that, but without that, we could expect that erosion.

• 13:55Kim Mills?

  Well, and what are those interventions? What do you do to ensure that your students are still, you know, using their full brain power?

• 14:04Speaker 03

  Well, it's it's interesting. I mean, it varies from one discipline to the next. But I think, you know, part of it is engendering in students a healthy dose of skepticism regarding the output of generative AI, that AI doesn't do the work for you, but it does work with you, but you're the you you're the executive decision maker. You you see the output, you have to evaluate it, you should read every sentence, you should examine every line of code, and you should anticipate that there will be things that you would need to correct. Or if it's not wrong, that you, as the human partner in this, as the one providing oversight, that you could improve the quality of the output, given the nuance of understanding you have regarding whatever the particular task might be.

• 14:49Speaker 03

  So I think what we need to teach students going forward is a new habit of mind, that is to have this this ongoing, almost moment-to-moment level of critique or criticism of the output of AI, expecting that it will not have accurate outputs, and then to engage with it appropriately. So that's new habit of mind that I think is part of a signature pedagogy for higher education.

• 15:16Speaker 04

  And I'll just add that that requires faculty to to reconsider or redesign their assignments so that that can incorporate what Eddie just described into those assignments. Because a lot of the assignments that faculty for decades have used have not required that reflection because they didn't have that tool that would generate the answers for them. But now, with that tool in mind, we have to sit back and think about the exact assignment, and can it be done with AI? Can it be done without AI? And when is AI removing the effort for learning, right? That we know that and leads to understanding, retaining information, and when is AI actually enhancing the learning?

• 16:01Speaker 04

  So redesigning assignments with those thoughts in mind will help students also in the end with with greater learning outcomes.

• 16:10Kim Mills?

  We're going to take a short break. When we return, we'll talk more about cheating and what constitutes fair use of AI for both students and educators.

• 16:20Speaker 00

  Are your ad campaigns lighting up the dashboard? But not the pipeline? That's bullspend, and marketers are calling it out in. Dashboard Confessions.

• 16:29Speaker 01

  My boss asked for results, so I opened my dashboard for the only positive sounding metric I had: impressions.

• 16:37Speaker 00

  Cut the bullspend. See revenue, not just reach. LinkedIn delivers the highest return on ad spend of major ad networks. Advertise on LinkedIn. Spend $250 on your first campaign and get a $250 credit. Go to LinkedIn.com/campaign. Terms and conditions apply.

• 16:51Speaker 05

  This episode is brought to you by Google Chrome. You think you know a browser, but Gemini and Chrome, that's new. It can help you with practically anything on the web, like restoring a vintage motorcycle from a 50-page restoration block, or finally break down that long article you've had open for weeks. Gemini and Chrome is here for it. Ready to make anything online make sense? There's no place like Chrome. Check responses set up required. Compatibility and availability varies. 18 plus.

• 17:17Kim Mills?

  I want to talk about cheating. A lot of the public conversation around AI in education has focused on this. How should educators and students think about what constitutes cheating with AI versus what is legitimate permitted assistance?

• 17:33Speaker 04

  Mm-hmm. And cheating is a question and a concern that, as I said earlier, is a top concern for faculty. And I always want to mention when when people ask questions about generative AI and cheating is that it's not what created cheating, right? We know that cheating has been around for for and has been a problem. Academic dishonesty has been there long before generative AI came about because we had to deal with cutting and pasting from those searches that students were doing. And then we had the, you know, larger issue of paper mills where students could purchase papers. And I don't know if you've ever gone to those sites, but in the past, when I was researching cheating, and you can put in the topic, the grade level, the grade that you want, the

• 18:18Speaker 04

  date that you wanted it. So they were very sophisticated ways of cheating prior to generative AI. When it comes to defining cheating, what research shows us is that students and faculty define cheating very differently. When you give them examples of academic behavior, you don't get agreement in what they consider to be cheating. And a lot of that has to do with the fact that you have faculty or educators have to define very clearly for their students what their expectations are and have clear policies on when they can use AI and when they should not use AI.

• 18:58Speaker 04

  And what we find is that the best practice that reduces cheating overall is when you talk about academic integrity with students, when you have that conversation and you talk openly and clearly and regularly, and not just one time, because we typically, most faculty have it on their syllabus, and they have a statement about academic integrity and define cheating there. But just staying stating it one time at the beginning of the semester is not enough. We find that if you repeat it over and over again with each assignment and explain to students what is allowable, then you reduce the likelihood of academic dishonesty. And the same is true for generative AI.

• 19:40Speaker 04

  You have to be clear how they can use AI in the assignment that you're giving to them. And if you don't, the assumptions will be made. And this is actually a big concern for students. I hear we hear from students that they don't know when to use it, and it creates great anxiety. And that's the last thing that we want to do. So that communication piece is really important.

• 20:06Kim Mills?

  So Dr. Watson, is there any work toward developing a consensus around the standards? So that if I'm a student, for example, I know, okay, I can't use AI to write my paper, but can I use AI to write the outline, for example?

• 20:19Speaker 03

  Well, that's, that's a really good example. We actually gave, in that survey that I mentioned to faculty, we found that there were large disagreements about what constitutes legitimate and illegitimate uses of generative AI. We gave scenarios and writing, you know, working from an outline was one of them. In fact, we gave, the language was, a student receives a writing assignment and asks AI to provide a detailed outline of an appropriate response. The student then follows the outline to write the paper. Fifty-two percent said that that was an example of how to use AI to cheat, while the other forty-eight percent said, this is faculty again, said that that it was either legitimate

• 21:03Speaker 03

  or they weren't quite sure. So, and to be honest, I think it depends on what the learning goal might be. Like, if you wanted students to have practice writing, and you know, sometimes it's difficult, you know, when you're seventeen, eighteen years old to think of something to write about, and so having an outline and then now you have a, something that you can practice, you know, creating sentences and grammar and syntax and all of those things, for that goal, that learning goal, that's very helpful. But then, on the other side of things, maybe you're not, you know, using writing to learn to write, but you're using writing to learn other things. And then, in that case, you want to see the students' thinking as made transparent by their writing. So I think that's why there's such disagreement, is like, the scenario doesn't speak to what

• 21:48Speaker 03

  the learning goal is. I think the learning goal's gonna vary from one assignment to the next, from one faculty member to the next, and as a result, I don't think we're gonna find a consensus regarding what is the best way to use generative AI, or what is a legitimate versus illegitimate way to use AI. In fact, I think that, you know, most institutions now have some level of policy, and often the policy says, look at the faculty member's syllabus, but I even go further in recommending that we're probably at a moment where we don't just mention it on the syllabus, but probably every assignment we give students, even a discussion board post, should have, should address that question, like, can you use AI for this particular assignment?

• 22:31Speaker 03

  And also address the question of why or why not. I think students are far more likely to follow along if they, if we make our pedagogical choices transparent. Oh, I can see, you know, I don't have enough foundational knowledge to be able to use AI in this way, yet as a partner, I would just be leaning on it too much. And so, it's more of a conversation than we've ever had regarding our, our teaching choices and our teaching practice, to make sure that we're providing students with guidance and understanding, so that they'll be more willing to follow along with the recommendations that we make.

• 23:05Speaker 04

  And I'll just add, because Eddie brought up a great point, is the question of whether or not there should be an institutional policy versus just leaving it up to the faculty and giving the autonomy to the faculty. And when, if you put yourself in the role of the student, if they have to go from one class to the next and figure out what that, each educator's policy is, that gets very confusing. If, if there can be one overarching one, that could be helpful, but as Eddie pointed out, it's the learning goal of each and every assignment that will determine when it's appropriate and how is it appropriate to use AI for that particular piece of work. That's what's gonna learn to, that's what will lead to achieving the learning goals

• 23:49Speaker 04

  and having students, you know, learn to, whatever skill it is or content it is, that you want them to achieve.

• 23:57Speaker 03

  Yeah, I think that, that's a really good point. I think students, though, have been, even whenever all of us were back in college, we navigated the syllabus, right? We'd go to one class, it's like, class participation's worth forty percent of the grade here, and here it's worth five percent, so then I might adjust my behaviors. I know I kind of have to be social today because I'm in that particular psychology class versus my math class, or whatever it might be. It adds additional levels of sophistication to that navigating from one class to the next, but, you know, given that, you know, learning should dictate a lot of the choices regarding technology use and, and indeed, as grade breakdown is distributed within a class,

• 24:42Speaker 03

  it's, it's complexity, and it does make it more difficult for the students, but that just further signals that this is additional change, not so much in student practice, but in faculty practice.

• 24:52Kim Mills?

  What about, for faculty, what would constitute, quote, cheating or an unethical use of AI? For example, you know, could you use AI to read all your, your students' papers and, and assign grades? Can, can you use it to develop the syllabus? And where is the line for, for the faculty member?

• 25:12Speaker 03

  Well, so we included questions in that survey specifically about the profession of the faculty member, like, is it okay to use it to, you know, create your syllabus? Or is it okay to leverage it in some minor ways in, in terms of scholarship? And when we're talking about, you know, creating an, an article to submit to a journal, more than four out of five of the faculty said, no, that is an illegitimate use, an unprofessional use of AI. And as you got sort of further down the food chain of teaching, from syllabus development all the way to grading, there is less and less comfort with using AI for those purposes,

• 25:54Speaker 03

  you know, so specifically, like, we ask a faculty member, uses generative AI tools to grade essays in their course. Seventy-one percent said that that was an illegitimate use for faculty to leverage AI in that way. Ten percent, only ten percent said outright, yes, that is acceptable. Another twenty percent, nineteen percent, they said that they weren't really sure. But it's suggestive that we need to have larger conversations, because there are some that indeed see it as acceptable and are indeed doing it, and then there are others that see it as, you know, really just absurd for a professor to think about allowing AI to do the grading in a course. So there's, again, this, everything is evolving, right?

• 26:38Speaker 03

  And we still haven't gotten to a sense of like, what are the common norms, even for our own practice, within our own disciplines regarding using generative AI?

• 26:46Speaker 04

  Right, and it brings up acceptance and transparency as well. If you think your colleagues believe that use of generative AI is not acceptable, you're, even when you do use it, you're not gonna be transparent that you're using it, and that brings up a whole other issue in the world of education. So that, Eddie's so right, in terms of that conversation is needed to really better understand when is it acceptable, when is it appropriate? Because I've also heard students concerned about their faculty members using generative AI to grade their work. You know, you're, I mean, you're not reading my, my papers, you're using this technology to give me a grade, which for them is, is disconcerting.

• 27:29Kim Mills?

  Sure, sure.

• 27:30Speaker 03

  Yeah, I think that's really, I think, important for faculty to be transparent with their students if they do indeed make that choice. And if they do make that choice for grading specifically, again, the explanation of, here's why I'm doing this, because you're gonna get feedback immediately, here's a process where if you want to debate the grade that you received and would prefer to have a human grade it, here's a path for that. But you know, whatever the rules are, be transparent with students and talk about what the benefits are to them. But there's, you know, also the other side of the question, like, what do students really think? And as we were putting together the second edition of Teaching with AI, we did a number of different student focus groups, and we asked students the question.

• 28:14Speaker 03

  And I, I had made the erroneous assumption that students would not want this, that they would not want faculty to be using AI for grading. And it was a diversity of perspectives that emerged. One student said, you know, I think I would prefer that because I think AI would be more fair. And he said, you know, I kind of have a, I lean more politically to the right, and I'm kind of careful about things that I say in class as a result. But, you know, I, I think if I voice some opinions, I'm afraid that the, that the faculty member might get back at me through grading. You know, I had another student say that they showed up late for the first day of class, and they thought that their professor had it out for them for the entire semester

• 28:59Speaker 03

  because of that first day of, of class. So they like the idea of AI grading as a, as a unbiased arbiter of the graded work.

• 29:09Speaker 04

  Which is interesting, because we know the bias that exists in AI itself, right?

• 29:14Kim Mills?

  Sure.

• 29:15Speaker 04

  For like, people who have English as a second language, or even gender differences, that there's some clear biases. So, yeah, that's an interesting point, Eddie.

• 29:25Kim Mills?

  But AI won't know that you were late.

• 29:31Kim Mills?

  Well, how might AI change the sorts of tests and assessments that teachers use to evaluate their students? I mean, I've read that there's been an increase in orders for those old fashioned blue books that a lot of us used in college. Will teachers need to go back to handwritten or oral tests and assignments?

• 29:53Speaker 04

  There definitely has been an increase in blue books, and that's the one thing I hear from faculty most often is that they're asking for some work that's taking place right in the classroom where it's proctored or supervised, or where no technology, so to speak, is available for students to use for answers. So essentially, many faculty are redesigning those assignments, in some cases, assignments where AI is not allowed to be used, or in other cases where AI is allowed, but there's clear guidelines to ensure that the learning goals are met. But here's really where basic learning science can guide the way, because AI will hurt when you remove the effort that's involved in

• 30:38Speaker 04

  learning. So when there's shortcuts for problem solving, and it leads to what I guess you'll call it passive acceptance of answers, that's when AI acts like a substitute and it's doing the work for you. And faculty are trying to avoid take on tests, for example. I mean, I don't know what many faculty these days, even if you give clear guidance. When I worked at a college with an honor system and an honor code, and all of my exams were just placed on at my door with two envelopes, one that said, here's the test, and here's where you return the test, take it where you will throughout this 24 hours. I wonder how many faculty can still do that, even at an institution with an honor system where

• 31:23Speaker 04

  AI is available. Faculty are more likely to do more assessment of learning in the classroom. So it could be oral exams where the person, the student is speaking, it could be in-classroom work, using blue books or not, as opposed to, I would guess, and Eddie, I don't know if this, I don't recall if the AAC and U survey asked this, but less homework assignments, right? Will there be fewer homework assignments where there's less monitoring of what's taking place when the students are completing the work? Because the big question is, when students turn in that product, can you tell whether or not it's student work or AI-generated

• 32:08Speaker 04

  work? That's the ultimate question there.

• 32:11Speaker 03

  Yeah, I had a colleague this spring, she was telling me that, well, I'm making this interesting observation. My students are doing really good on the homework, but they're doing really poorly on the in-class exams. I don't know what that means. It's like, well, what do you think that means? If, when you're watching them, they do poorly, but when they're home, you think they might be getting assistance in any sort? So there's things that we can do in our class to collect, you know, information to better guide, you know, some of our practice, or at least inform what's actually taking place within the classroom. But, you know, everything that Beth just shared, our observations that we're seeing at AAC and U, and I also have the good fortune of having an in of two in my household. So

• 32:56Speaker 03

  I've got two sons that are college age, and this past summer, my youngest came home from college, and I was doing the dad thing. So, "Son, how was college? You know, how did you do on the finals?" And he said, "Well, you know, it was interesting. I did great on the finals, but for two of them, I had to go to the bookstore and buy a packet of papers stapled together." And I said, "Well, was the cover blue?" And he was like, "Dad, how did you know?" I'm up on the latest trends in higher education, son, that's my job.

• 33:28Kim Mills?

  Well, I didn't have to pay for them. I was lucky, I guess. But speaking again about, basically, being able to suss out what's the student's work and what's AI, I mean, do detectors work? And should faculty rely on those tools to figure out whether their students are doing the work themselves or using AI?

• 33:51Speaker 04

  From the research that I read, the detectors, if I were in a classroom, I would not be using those detectors, because the range of accuracy of the detectors is so great that if I was placing the percentage that was created by that detector on a high-stakes type of assignment, I just don't think it would provide fair grading. So, and the interesting thing is that the people who are generating or developing the generative AI technology is also generating the AI detectors at the same time. And I wonder about that fact.

• 34:31Speaker 04

  Faculty often say, "I can tell when a student has used AI." And I don't believe that there's evidence to date that shows that faculty are very good at differentiating papers written by their students versus AI. Now, I will say that if you're in a small classroom, and you know your students' writing skills, and you have trends in their assignments, and you know their voice, then yes, you are much better at detecting whether or not the paper was written by AI versus the student. But when you go to the larger classrooms, which many of our colleagues are in classrooms with 200 students, and they have writing assignments, then that becomes a much more challenging issue. But I've, just to summarize

• 35:15Speaker 04

  again, I don't believe that AI detectors are at the point where I would comfortably use them to differentiate AI-written work versus student work.

• 35:26Speaker 03

  Yeah, they're definitely not smoking-gun evidence of students cheating. I mean, at best, you might use it as one data point among other points of triangulation, such as, as Beth was describing, knowing the student's voice, like, "This really doesn't sound like this student." But, you know, sometimes even that doesn't give you the evidence that you think that it is. I had one student tell me that they were accused of using AI, and she said that the reason her work was so much better was that where she works, they were doing inventory over the weekend, and she usually works 16 to 20 hours on the weekend. So she had this one weekend where she had all-day both days, and so she was able to do really her best work. And so she turned in her best work,

• 36:11Speaker 03

  and the faculty member was like, "Well, this doesn't sound like your papers from earlier in the semester. You must be using AI." And the AI detector had flagged certain elements of it. So the student had to, when they had the actual, I guess, case heard on campus, her boss came in and said, "Yeah, she was off that weekend, and she usually works every other weekend." And so she was able to win the case, but that's kind of a harrowing experience for students to go through. And I actually had another student in a focus group share late last year that, "Whenever I finish writing a paper, if I think that it's, sounds too smart, I'll go back and dumb it down a bit so that my professor won't think it was produced by AI." And I just found that heartbreaking. Like, that's not

• 36:56Speaker 03

  a skill you need in the real world, is to dumb down your best work. You know, that's not something we want to promote. But some students are indeed trying to walk that line to prevent themselves of being suspected of having used AI.

• 37:12Speaker 04

  There was also an interesting story that I heard about a student who was accused of using AI. There was a percentage that was generated for the paper that they turned in, and they took the faculty member's work, something that the faculty member did, and put it into an AI generator, and then provided for the faculty member a percentage of that work that was detected as AI-generated, as evidence of the popularity of those types of detectors. I just thought that was ingenious.

• 37:43Kim Mills?

  That's great.

• 37:44Speaker 03

  Yeah. You had a book, Teaching with AI, we documented a case exactly like that, where they, you know, went to the honor court meeting, and they took the professor's last, single-authored paper prior to November 2022, and ran it through one of those AI detectors tools. And sure enough, it was flagged as being 33% AI-generated before generative AI existed. So yeah, I think it's probably, you know, if you're going to use a gen AI detector, maybe run some of your own work through it, and that might give you pause before you would actually bring that to your classroom.

• 38:23Kim Mills?

  Is there an age below which you think students shouldn't be using AI at all?

• 38:31Speaker 04

  So I would say that, as is true in any kind of a learning environment, it just has to be age appropriate or developmentally appropriate when you introduce any type of technology or technique. And for younger students, you have to be really careful and aware of protecting learning, making sure that students, particularly younger students, you're not outsourcing the learning of basic reading skills, or writing skills, or mathematical skills, or reasoning, because you want to build those foundational thinking skills, right? So that teaching, but at the same time, you want to start to introduce, if you can,

• 39:10Speaker 04

  an understanding of what AI is, but at a very basic awareness level when it comes to elementary school. And then, when you start to look at middle school, you can look more at the guided use, teach the guided use of AI, where students can begin to learn how AI can give answers, but why the answers can be wrong or biased, and how you can compare AI output to what are considered trusted resources for middle schoolers. The emphasis for middle school should probably be more on the curiosity and safety and responsible

• 39:50Speaker 04

  habit use of AI, not the nitty gritty how AI works. But then, when you start to get to the high school level and even higher ed, there's more of the critical and ethical use of AI. So it just develops over time. And higher ed is more the advanced and workforce-oriented maybe use. How does this transfer to the workforce when I'm in the college classroom and I'm learning how to use AI? How will this work when I've graduated? So it's all about being age appropriate. But I think we're going to learn over time as educators start to learn how to incorporate AI into the classroom. What is age appropriate? And

• 40:31Speaker 04

  what is the appropriate pieces to incorporate from a very early age?

• 40:39Kim Mills?

  Well, just to wrap up, what are the big questions you're trying to answer? What are you each working on in this area right now? Dr. Schwartz, you want to take that first?

• 40:49Speaker 04

  Yeah, well, at APA, a lot of what we're looking at is, well, one of the collaborations at APA and AEC and you is going to start later this year is helping faculty to answer some of these questions and conduct some scholarship on teaching and learning, which means like, what are the questions that they have? And how can they go about critically evaluating what it means to use AI and its impact on learning? So part of our efforts is teaching faculty to ask those right questions and figure out the methodology to answer them, and to really fill the gap in the literature that exists when it comes to understanding

• 41:29Speaker 04

  AI's impact on learning. Because when you look at AI research to date, it's often, it's not long-term learning. So we know more about short-term performance than we know about long-term learning. There are often quiz scores and assignment scores, but it's not retention over weeks or months later. Like, I'm just not finding them. Many of the research these days focuses on perceptions and people's opinions rather than on learning. So that's why filling the gap in the literature was so important. And most studies to date don't really isolate the cause and the effect, you know, the generative AI to the learning

• 42:08Speaker 04

  that's taking place. And so we don't know whether it's the fact that AI is creating better feedback, or more practice opportunities, or is it more time on the task? Some of those questions in psychological science, we ask when it comes to what is leading to the connection between using a certain type of learning tool and the learning that's taking place. And as I said earlier, it's heavily concentrated in higher ed. So we really need to help and support educators for the younger students to figure out how to really start to assess how AI would impact learning for elementary and secondary schools.

• 42:51Kim Mills?

  And Dr. Watson, what's ahead for you? You struggling to update your book? You have to do another edition every six months?

• 43:00Speaker 03

  About every 18.

• 43:02Kim Mills?

  Oh, wow.

• 43:03Speaker 03

  The rhythm that we're on. I guess the other thing that I'm really focused on these days is actually larger curricular structures. Like, you know, if we are incorporating AI literacy as a learning outcome, maybe as essential learning for college students, well, what do we actually teach them? And then, what would be the compass for that? I mean, certainly, what the world of work demands would be a component of that. But then, there's also sort of other moral, ethical, civic responsibilities as well that we would want to incorporate into that definition. But, you know, as you look at some of the AI literacy frameworks that have emerged, almost all of them predate agentic AI. And as we think

• 43:45Speaker 03

  about the world of work, there's a lot of shifts that are taking place. I was literally just talking to a friend last week who was at a, they're in sales, and they were at a sales meeting, and they learned that all of the support staff around the core sales team was going to be laid off before the end of the year. And the expectation was is that they would begin using agentic AI to actually do a lot of those discrete tasks that require some level of decision making that these more entry-level positions were doing. So managing agentic AI, managing agents, you know, that's a different skill set that

• 44:26Speaker 03

  I haven't seen in any of the earlier models, because agentic AI really wasn't present at that point. So there's this evolution of what is AI literacy. And, you know, one thing that we do well in higher ed is curriculum reform. One thing that we don't do well is curriculum reform fast. And the truth is, the target is changing quite quickly these days. So staying on top of that, you know, going through, doing all of the processes that we always do to ensure quality and accuracy for the components of a learning outcome that we would then engender into our curriculum or into our major. We still need to be doing that work. But I think what's really needed of higher ed is a new kind of agility. How

• 45:10Speaker 03

  do we do this work more quickly and recursively? Like, gen ed reform is maybe the biggest curricular reform effort that a campus can do. And on many campuses, they do gen ed reform every 10 to 20 years, because it is such a

• 45:26Speaker 04

  It takes a year to do, right.

• 45:28Speaker 03

  Right, exactly. And so, how do we shorten that timeframe for curriculum reform in service to a learning outcome that is shifting rapidly? You know, you don't build something into the curriculum, and then you don't change it for five years, where we see the rate of change around AI is exceptionally fast. So that whole, the whole challenge of institutional change, agility, curricular agility, and the evolution of what AI literacy means to the population of students that we have, and that will soon be beyond our campuses. Those are among the key things that I think about these days.

• 46:05Speaker 04

  And I'll just add one more thing. And my role from APA's perspective, or from my own perspective, is that we have this foundational learning science, foundational principles that we know leads to strong learning. And we have to keep that in mind when we're reconsidering the assignments and what we're having students do, because we know what leads to a student understanding, a student remembering, a student, you know, moving forward with that information in mind. If we don't keep that in mind, then we're just missing out on this important part of the scientific literature from psychology that can really provide a lot of answers.

• 46:46Kim Mills?

  Well, Dr. Schwartz, Dr. Watson, I want to thank you both for joining me. I think you're doing really important work right now.

• 46:52Speaker 04

  Thanks for having us.

• 46:55Kim Mills?

  You can find previous episodes of Speaking of Psychology on our website at speakingofpsychology.org, or on Apple, Spotify, YouTube, or wherever you get your podcasts. And if you like what you've heard, please subscribe and leave a safe review. If you have comments or ideas for future podcasts, you can email us at [email protected]. Speaking of Psychology is produced by Lee Weinerman. Thank you for listening. For the American Psychological Association, I'm Kim Mills. Tim Mills.

• 48:05Speaker 06

  This episode is brought to you by State Farm. Football has a play for just about everything. You can go deep, keep it on the ground, or draw up something nobody sees coming. And whether you want to run a flea flicker or the tush push, you need a plan to fit your unique situation. That's why State Farm takes the time to get to know you so they can help personalize coverage for your life and budget. Now that's a solid game plan. Like a good neighbor, State Farm is there.