The Writing and AI Walk

Transcript of Helen Sword’s podcast episode The Writing and AI Walk

 Hi, I'm Helen Sword from HelenSword. com, and this is Swordswings, my podcast series for writers in motion. Whether you're out walking, or riding in a car, or on a train, or a bus, or just pottering around in your kitchen, this recording will help you move yourself and your writing to someplace new. I've extracted today's podcast episode from my conversation back in September 2023 with Jane Rosenzweig, the director of the Harvard Writing Center, who's become an expert on what it means to write critically with gen AI, that is, generative artificial intelligence.

Jane teaches an undergraduate course on the subject at Harvard. And shared many insights into the risks and rewards of writing and teaching writing in the age of Gen AI.

…..

Helen: Today we're talking about this issue that is on everybody's minds. AI, Artificial Intelligence, we're learning all the acronyms, aren't we? LLM, Large Language Models. And I've invited Jane to come, and Jane Rosenzweig, who's the director of the Harvard College Writing Center, teaches courses on writing, writes about writing, has a newsletter, which I'll invite her to talk about in a few minutes, and I thank you Jane, you first came to my attention, you published something, maybe it was in the New York Times, an op-ed piece about, um, just kind of demonstrations of, of good writing that Jane points to.

And then, when ChatGPT came out, I started seeing little tweets from Jane about how writing is critical thinking. And if writing is critical thinking, then students who are using AI, anybody who's using AI is ducking the critical thinking part. And AI can do lots of things, but it can't do the critical thinking of the human brain, at least not yet. Or not in the way we do it. Jane, welcome.

Jane: Thank you. Yes. So I, I wanted to say, so in 2019, I published this, this op-ed in the New York Times. It was after the first whistleblower of many had written his whistleblowing statement. And I did a kind of assessment of it as a piece of good writing, and a lot of exciting things happened to me, for that was the first time I'd done it.

It was published in the New York Times, and I got all sorts of emails and, you know, calls and letters. But the most exciting thing that landed in my inbox, seriously, was an email from Helen. I had been teaching your piece about the zombie nouns from the New York Times to my students for years, and it was one of those moments where it was like, this person who wasn't a real person but was like an icon of writing things, suddenly was there in my inbox! And my husband's also a writer, so I went running upstairs: “Guess who sent me an email?1” So that was a memorable moment. So I'm extra delighted to be here to be able to talk to you in person. It's like meeting your heroes, always exciting. It's wonderful.

Helen: And we bonded through our New York Times authorship, right? The zombie nouns has opened up a lot of things for me as well. Oh, well, fantastic! A little mutual admiration. And, so you've become a — I mean, it may not be the, the label you put on yourself — but you've become a bit of a public intellectual, dare I say! Op-ed pieces in the New York times, and a kind of Twitter presence, and commenting on things like AI. So, has that all been pretty positive?

Jane: It has. I don't know how many of you have seen the movie Forrest Gump. It goes back a bit, but bear with me here. I've mentioned to a few of my friends that I feel a little bit like how Forrest Gump kept showing up for major historical events … and suddenly he was there! And the way that I got into AI was a little bit by accident like that. I mean, I think all of us who write and teach writing sort of knew that there was something going on out there. And every now and then — this goes back a few years — I would check on some of these new programs that promised that they were going to help you write with AI. And sometimes, I would get an email from one of these companies, you know, asking me if I wanted to do an event for them. And I would mostly say, ‘I can't do an event that promotes something that is going to take away my livelihood and everything I believe in.’ But I knew it was out there.

And then, in the fall of 2021, I was sitting in the writing centre, and I had just tried, I think it was Jasper AI. I was just in my office, and I was thinking, ‘Oh, no, this can really do things!’ And one of the writing center tutors (These are undergraduate peer tutors) was there and he happened to be a computer science major. And I said to him, ‘Xander, what do you know about this stuff?’ And he came into my office and he made me get an account for what was then called the GPT playground, which is the precursor to Chat GPT. This was a year ago. A year and a half before chat GPT was released. So I was in there and I started becoming, you know, quietly horrified by myself at my computer! But I had that kind of head start on what was going on. And I kept thinking, I'm going to write about this. I need to write about this. Look what this can do. And. So I wrote something about it, which was the first thing I ended up publishing about it. It was ‘What we lose when machines do the writing’ and I sent it to the editor I had worked with at the Times. And they sat on it for a couple of weeks and they were literally like, ‘Oh, I don't know. We have to see.’ And I got the sense that they just weren't sure whether this was that interesting. Right? So eventually we parted ways on that. And I published it in the Boston Globe a week before chat GPT came out. So if I had waited a little longer, I may have done better at the Times, but it also put me in this very strange position of being 1 of the few people at Harvard who at least publicly seemed to know anything about this. And that sort of led to everything that followed.

My editor at the Globe called 2 weeks later and said, ‘now that another chat GBD is out, do you have anything else to say? So I wrote another one pretty quickly, and then people at Harvard were asking me to give presentations. So I had a traveling slide show that I was adding to — this was back when we really didn't know what these things could do. So I was just trying things out. I tried to get CHAT GPT to write my undergraduate senior thesis, which had been about the 365-day combat tour in Vietnam. I was a history major and so I was going around to these different offices at Harvard with these slides that had giant chunks of text that were kind of like my senior thesis, but not really.

And obviously, I had to prompt it. It wasn't like I could say ‘write a great senior thesis’, but we were all trying to figure out what was going on. I ended up on CBS Sunday morning talking about this for 45 minutes, that was boiled down to like 30 seconds, but it was just all that kind of stuff. That was all because… I mean, many of you here, I recognize some of your names, I know you on social media, you know all these things too. Right? Well, I just got there 1st. So it was a strange situation to be in, but it's been very interesting.

Helen: You got there first, but you come in with the expertise of what your position is. You're more focused on undergraduate writing every single day than most of us are, even if we teach writing or teach undergraduates, or whatever. So you're in a good position then for people to trust what you have to say. And, of course, one of the things that's been quite interesting with chat GPT and AI is there's plenty of sort of cynicism and quite a lot of head-in-the-sand stuff. Like, ‘I just don't want to know! I don't want to know!’ still amongst academics. That I know, but there was also this explosion of hype, as there always is with these things. It reminded me of the days of MOOCs, you know, when MOOCs, the massive online free courses, we're going to save the world. And it turns out that they're really great for about 12 percent of the population, which is the highly motivated learners who are going to do, who are going to figure something out anyway.

So if you're a retired engineer in Sheffield, you know, and you want to learn about philosophy from somebody at Harvard. A MOOC is just fantastic for you, but if you're a first generation college student who really needs a human being helping them out, it's a complete disaster.

So AI, I've seen these, these curves, you know, of any new technology, that you get the hype, and then at some point you start to get not just the criticism, but the disappointment as people try to use these tools and they don't actually do everything. And then you get the plunge and then at some point, it comes up and it kind of evens out to a new reality…I don't think we're there yet, are we? And the tools themselves keep changing. So, can you tell us to start out with — because I think this will bring up a lot of these issues — about this course that you've just started teaching at Harvard with the great title.

Jane: Sure. So, since I had been digging into this for so long, I moved away from just being interested in what is this going to do to my livelihood to really feeling like there are just so many fascinating things going on in the various debates about the ethics of AI. About what it means for the environment that these systems require so much water to be powered… About what this means for bias, and how are these systems trained.

So I decided that I was going to try to teach one of these first-year writing seminars on the subject of chat GPT so that I could spend more time thinking about this. I thought it would be really interesting to have a sort of organized way to talk to my students about this and learn from them about it. So, the course that I'm teaching that has met twice so far this semester is called ‘To what problem is chat GPT the solution’.

The goal of these writing courses is that you have a subject because you need something to write about. But it is like all of our writing courses. It's really a course in critical thinking. So it's built around several assignments and ends with a research paper and what we call a capstone project. Seemed like the perfect moment for something like that for a couple of reasons. 1 is that there is so much being published every day about chat and generative AI in pretty much every publication that I wondered if I could ever pull off a semester where my students could really widely possibly get published as 1st-year students. This might be it, but also because simultaneously, as I'm thinking about the subject matter of this course, I'm also thinking — as many of us are — about how I'm going to teach a writing course in a time when people could use these generative AI tools to do their writing for them.

And one of the main things that I've been focused on this, this semester is What makes writing meaningful? Why I think students should be doing their own writing in the first place. Why I think that experience matters for them and how to set up writing situations where it will matter for them, but presumably those who have signed up for this course are ready to come into it with a critical eye. They're Harvard students, you know, they're presumably curious and I suppose what we really worry about are the students who don't have that disposition and are coming in just wanting a shortcut or coming in with just chronic insecurity about their own writing. So that's the problem. ChatGPT is a solution for writing for some of them.

Definitely, I mean, 1 of the things that I thought was so interesting in the 1st, couple of classes…I think I'm in a unique position, right? Because my students are people who want to think and talk about this. They signed up for this class on this topic. Not all of them know that much about it, some of them do. Some of them have used it extensively. Some of them want to work in AI labs and are very interested in that, but not all of them. But one of the things that I found interesting in our early conversations was that I just asked them, ‘what are you using this for? And what do you think it should be used for?’

And I think one of the things we always need to remind ourselves of — those of us who teach students undergraduate or younger — is that some of the very first things they said they were using it for would not be things I would want my students to use it for… like, brainstorming, right? Or more practically, you know, putting in your whole paper and asking for edits, which I have done, I've tested it in a variety of ways. Those aren't the sorts of comments I give on student writing. What they're getting out of the machine is very different.

And this just reminded me of this situation that we find ourselves in: If I use chat GPT, I have a whole set of skills that allow me to assess the output, right? If my first year undergraduate students were to use it for the things that I'm asking them to do, they don't have those tools. So, how are they assessing the output? And when I put this to them in class, we had a very interesting conversation about it. They're open to the idea that it might not be the way to go. But you have to be having that conversation with them at some point because how would they know why I think it's not useful for them to use it in certain situations if I don't tell them?
And again, that's what I meant that, you know, they're, I won't quite say you're preaching to the converted, but you've got a group who's signed up for a class with a provocative title.

Helen: My son's teaching creative writing at UC Irvine and the fiction writers have started coming in with AI writing now. And you think' ‘why are you doing creative writing if not to write your own fiction?’ But they also have to write critical essays about things that they're reading and about bigger topics. And so they're coming in with fake quotes from famous authors. You know, “As Ursula Le Guin says about blah, blah, blah,” and chat GPTs just made it up because that's how generative AI works. It doesn't find you a quote out there on some quote website, it goes and it makes up what seems like the most plausible thing, unless it's given very strict instructions not to do that. But then learning those prompts is a whole art.

So that can be a student who's actually quite curious. Very eager to succeed and they just see this as a useful shortcut to get the information. We've all used Google. We all had the panic about Wikipedia, you know, and how this was going to rot everybody's brains. Now, how many of you use Wikipedia like every day of your life?

Jane: Well, I do, because it's turned out to be quite useful to have a crowdsourced encyclopedia like that, and yeah, I know how to be cautious about what's on there and to ask those questions. Right. So yeah, it's those, it's, it's the students who aren't asking those questions who we're particularly wondering about. And I suppose also potentially the faculty, the teachers, you know. I'm sure many of us have read these accounts of teachers who use programs that are supposed to be able to catch / check GPT-written stuff, and they can't! So they ended up making false accusations of plagiarism and all kinds of things. So yeah, it's a real kind of explosion of things.

Helen: Well, let's take one step back. Can you explain, in lay person's terms, what's different about generative AI?

Jane: The main thing that's different about generative AI from, say, plagiarism and just looking something up and plagiarizing it or using a search engine, is that this is artificial intelligence that can generate images or media or text using predictive models.

So the way it works is it basically predicts the most likely next word in a sequence of words, and it's been trained on ‘the Internet’, but we don't really know. I mean, this is one of the many controversies, right? Open AI, which has created Chat GPT does not say to us ‘This is exactly how we've trained this model. And we took all of your copyrighted beautiful novels and fed them into our model.’ They don't they don't tell us, but the assumption and the understanding is that it's been trained on just a huge amount of data from the Internet. Your writing, my writing, Wikipedia, Reddit, Twitter… and so what happens when these generative AI generative language models are predicting the next word is that it's based on their training data.

So this is why we end up with situations where people are upset because what's been generated seems very biased. So, for example, I did an experiment for my class and for a lecture I was giving. I gave a paragraph of my most recent op-ed, which was in the L.A. Times, I gave it to chat. GPT. and Claude AI, which is another Generative AI model/large language model that you can use. And I said, ‘Rewrite this for an audience of men.’ And then I said, ‘Rewrite this for an audience of women.’ People have been talking a lot about bias and I was curious to know what I would get.

So Claude actually rewrites something for you and then it tells you why. So it rewrote my paragraph — which was just about how we're not going to think anymore if we use large language models for everything — and it brought in all this competitive gaming imagery, it said, ‘generative AI will be your wingman and it will be with you through the highs and lows’ or something or other. It was very competitive language. And then Claude explained to me at the end of this passage that this is how you would write it for men because of competition and imagery about sports and other things like that. So we have a clue there that this large language model has somehow been trained to think that that's what men are interested in.

Then, the women's version of this same text. It was about emotions and something about helping you express your deepest emotions yourself in some way. And then Claude AI proudly told me that was to appeal to women, to appeal to their interest in interpersonal relationships and emotions.

So, I've gone off on a little bit of a tangent about how the technology works. But this is, to me, the most interesting piece when you are generating something. It is predicting the next most likely word in a sequence of words only based on what it's been given, and we don't know what it's been given.

So this has been the source of a lot of controversy and problems. You can think of it as a random text generator. You can think of it as…some people like to talk about it as an ‘average discourse generator’. All these different ways of saying the word that's coming next is not because there's a brain like ours inside these machines that's thinking through things. It's predictive. It's based on these mathematical….There's some debate about this. There's debate about everything. It's fascinating in the field of generative AI. Surely some of you have been reading about this, right? But there are some people who say we are very close to artificial general intelligence, and they talk about the chatbots as if they were human.

You know, I asked chat GPT this and it said, and it thought through and it solved this problem. We're using human language to anthropomorphise these machines. There are other people who say, you know, ‘this is actually damaging and dangerous. We should not be calling these things anything that is in that way anthropomorphizing. We should be calling it an “artificial text predictor”.’ Like ‘I put my prompt into an artificial text predictor and out came X, Y, and Z.’ Those people would argue this is a more realistic way of talking about this technology. So, I don't know.

Some of you may have been following the work of Emily Bender, who's a linguistics professor at the University of Washington. She has written widely on this and has a podcast about a hype theatre, all the hype that's going along with this idea that these machines are poised to replace humans. She's always drawing back that curtain and saying, ‘look, this is what's actually going on.’

Helen: When I write about Gen AI, I've been using a name taken from Inger Mewburn (she writes the long-running Thesis Whisperer blog and is a professor at Australian National University), and she calls it Chatty G. And so I quite like that personification because I imagine then Chatty G as this chatty, you know, reasonably bright research assistant whom I can ask to go and find things for me or do things for me, but I need to check everything, you know, I'm not going to trust Chatty G to go and write my paper. And what I've been doing a bit and in my newsletter is comparing things in the style of this author, including myself. One of my kind of first moments with AI was when somebody sent me something about a topic I knew nothing about written in the style of Helen sword.

And, you know, I'm sure we've all had these moments early on with chat GPT, where you get something where you go, ‘Oh my God, how did it do this?’ Because it was this real sort of leap forward. But then as soon as you start to analyze it, there is not a single sentence that I would have written that way. So, When Chatty G starts using style, it's basically just throwing in lots of metaphors and the use of metaphor is just awful because Chatty G has not been taught what a mixed metaphor is.

But I've wandered off a bit from one of the things you were talking about there, which is bias, cultural bias. And if you're drawing on the cultural, on the reservoir of the information that's already out there, then that's the information you're going to use for your next text. And if you're in a universe where men's magazines and a lot of the source material undoubtedly is more competitive speak for men, and the women's magazines are more emotional speak… Is there a capacity for a predictive language model to reverse that? If you asked it to?

Jane: I don't know. I mean, there are a couple of things that go on. One is that there's a huge human training piece to all this, right?These companies are actually trying to use humans to reverse the bias, right? So, they do some kind of reinforcement training where they try to get the responses they want, and they reinforce those within the machines. There are ways that they can go in and get rid of things. You know, this is why when they first released Chat GPT, people kept trying to break it and get it to say something terrible. And then there's a human level that's going in and trying to prevent that from happening. So there's that piece. Behind the scenes, there's also a lot going on to try to remedy these problems. But one of the problems that we all have as end users of these things is that you're dealing with what they call a black box model, right? So what they do is when they get the output they want from the input they're giving, then they're satisfied that it’s working. That's a very bad, basic explanation of what's going on. But what it means is they don't always even know why they're getting the output they want. So the interest in training these models is to get the output they want. And sometimes they don't know how they're getting the output they want, and we don't really know how they're getting it.

It's hard to know if they're going to reach a point (or even what that would look like) where it was always trustworthy and never biased. Who's making that decision? Unless we're each making that decision for ourselves, there's all sorts of complicating factors that go into that. I think bias is one of the biggest problems. One other bias that we haven't touched on that's huge is, you know, Western and particularly American bias. So much of what's going in the training material is coming from the United States.

Helen: Yeah, absolutely. And then there's this issue of…I read an article about it, and they used the acronym C O W. I've forgotten what it stood for, but it was a conscious reference to mad cow disease. Mad cow disease was when they were feeding the cows food that had bits of cow in it. And so eventually this brain disease developed, a sort of brain sickness. So the cow reference has something to do with Chat GPT/chatbots eating their own brain, in a sense. And that's the idea that as the chat box puts more and more material out into the body of work that Chat GPT is drawing on, its quality or range degrades. Because it's feeding on itself in that way!

Jane: So it's desperate for new content and there isn't any because it's generating all the content based on what it already had.

Helen: Exactly. And apparently, that's already happening. It’s been tested. Researchers are testing this and they're finding it both in the language models and in the image models. People are already complaining that things they could get a year or two ago on some of these open source image generation tools, you know, were quite sort of fanciful images. But now they're starting to get more boring. And so that's quite an interesting issue too, which isn't ethical, but it's more of a technical one in a way, but it comes back to that question of, well, why do we write in the first place?

I wrote a piece called Finding the Why in AI and it was using why as an acronym. Writing Helps You … do what? Well, lots of things! And one of them is writing helps you think. It helps you, as a human, to generate new ideas. So if you're not writing, you're not generating new thought or critical thought.

Shall we just run through some of these other ethical issues that you're looking at in your course? Because I think that's really fascinating and something that maybe hasn't had as much airplay.

Jane: We are going to spend some time talking about the general ethics of doing your own thinking and what that means for an educated person—it’s going to be in a particular context for 18-year-olds who've just come to Harvard.

But some of the things that I think they're less familiar with that I raised at the beginning that I'm hoping we're going to be talking about would include: questions about copyright and intellectual property. I want to talk to them about the environmental costs of this technology. So it's emerging just how much water it takes to operate these systems where, where the physical servers are located — hint! they're in countries that really can't spare the resources that these companies are going to be taking from them.

We also are going to be talking about the human cost of all of this. There are a lot of people making very little money (in countries that are not the United States) to do the human tagging and retraining of the machines. Some of you might have seen there was at least one article in New York Magazine about the workers who had to filter pornographic content out of various systems and the trauma that comes along with these jobs. Why are we doing this? You know, to what end is all of this happening? So that I can get this system to write my undergraduate college paper? I'm hoping to do some connecting of the dots with all of these different issues.

Helen: Can we say something for people who need to leave? Something positive? Do you have positive experiences with Chat GPT or Claude that you can talk about?

Jane: There's certainly going to be really interesting things that people do with generative AI. We've talked to social scientists who are able to use it (or think they're going to be able to use it very effectively) to tag interviews that are not written down but recorded. So means you could actually get so much further with things that really do feel like busy work.

Like, ‘I need to go into this interview script and find out where these things were talked about in these thousands of pages of work.’ I think there's going to be really interesting applications for this. I just worry that we are constantly being drawn back to this idea: Either you embrace it, or you ban it. And I don't think either of those makes sense.

There are many ways in which AI and machine learning can be used for research, particularly for using large amounts of data. But even just, you know, I've been using word searches for two decades, basically, ever since we started digitizing our writing! There's certain things that we already do, and as you say, it eliminates a lot of the busy work. Although people in the social sciences will also often say that actually reading through all that data again themselves with a marker—often that's where the actual idea generation is happening. So even there, there's a fine line sometimes between busy work and, and learning.

…..

That's the end of today's podcast episode, which has been extracted from a much longer conversation. I hope that both your body and your mind moved to someplace new since we started and that you've acquired new insights into the affordances and potential dangers of Gen AI writing tools. You can watch the full version of my conversation with Jane Rosenzweig in the videos section of the WriteSPACE Library. That's my online membership community at helensword.com, where you can also find my full series of Swordswing podcasts and transcripts.

Thanks for listening, and I look forward to walking with you again soon.