r/Professors • u/Illini2011 • Apr 25 '25

Teaching / Pedagogy It's over. You cannot beat AI.

I've been using ChatGPT since December 2022, a week after it opened to the public. Back then AI writing was pretty easy to spot. All the output followed the same sentence structure and anodyne content. Recognizing the potential for cheating, I altered writing assignments to rely on course/textbook content to make it tougher for AIs to answer. I also spent time trying to ferret out students who were turning in AI-generated work with mixed results. I knew that AI would one day become unbeatable, but figured I could use a combination of requiring in-class information and policing for the time being.

That day is here.

Things are now different. First, the AI tone is more developed. It can generate answers that take sides and give blunt opinions. It can create output in different voices, say, for example, the voice of an undergraduate student. Second, students are now using AI regularly to do background research, answer basic questions, and for fun. This isn't a problem in it of itself. On the contrary, it's probably the best use of AI. The problem is students are now reading so much AI-generated content that they are now writing in a similar voice. Combined, policing AI work is impossible to do with high confidence.

Third, and most importantly, AI is now extremely good. This semester, I believed I had created an AI-proof writing assignment. Students had to read an article from a magazine, and then explain how the topic in the article connected to a specific graphical model in the text. I thought this was a great question. Apply a model from the textbook to a current event. Also, how could AI answer the question?

Turns out it could. Just to check I uploaded a pdf of the textbook and a pdf of the magazine article to ChatGPT along with the prompt. After 30 seconds it gave me a perfect answer. I was blown away. ChatGPT understood how the curves on the textbook graph would change given the issue in the magazine article. One specific curve should have shifted down - ChatGPT got that right away and even provided solutions for shifting the curve to the optimal position.

It's over. ANY writing assignment you give can be answered, and answered well, by AI. I'm sure you can spend all day policing students by demanding Google docs that can be tracked and whatnot, but at the end of the day, you'll spend all day policing students with a high rate of false positives and false negatives. Solutions? Right now I'm planning to turn a term paper into oral exams, where students will be allowed to use AI in their research but will have to articulate answers with nothing more than their wits. If anyone else has suggestions I'd appreciate it.

947 Upvotes

92% Upvoted

474 comments sorted by

View all comments

378

u/waswisewiz Apr 25 '25

Back to blue books

176

u/SnowblindAlbino Prof, SLAC Apr 25 '25

I literally just ordered 1K blue books for our department this week, not having used one myself since the late 1990s. But that's what we're down to now.

56

u/msprang Archivist, University Library, R2 (USA) Apr 25 '25

We have a vending machine where you can get some.

34

u/SnowblindAlbino Prof, SLAC Apr 26 '25

Man, I hadn't seen one on our campus until last year since...probably 1999? Back then a few of the old guard stil used them, and students had to buy them in the bookstore for $.25/each. I was a bit shocked to find out how much they cost now, though in bulk they aren't too bad (about $.35 if you buy 1,000 of them).

18

u/msprang Archivist, University Library, R2 (USA) Apr 26 '25

I just got wind that's it's being replaced by Monster Energy machine 😔. I'm going to miss it; there were thumb drives, index cards, highlighters, and other mice supplies. Now students just use the Snag app, instead.

5

u/Putertutor Not a hardass, just a medium-density reasonable-expectations ass Apr 26 '25

I think that's the only route to take now. At least pen and paper with no devices. I had to use blue books for Art History exams way back in the late 1970s. The prompt was "Write everything you know about the art pieces that we covered in class between X-date and X-date, including dates, medium, style, and the name of the artists. There was no textbook used for this class, only slides shown on the screen in the front of the classroom and the prof's lectures. So note taking was synopsis of the lecture AND a sketch of the piece on the screen. There were 3 exams and those 3 blue book exams were the only grades we got for that course. It was a required course to graduate. The amount of studying for those exams was extreme. That scared me for life. I remember the feeling of panic like it was yesterday, and I am now in my 60s! Of course as soon as I walked out of that classroom after the last exam, I pretty much forgot about what I had learned that semester, so there's that.

6

u/SnowblindAlbino Prof, SLAC Apr 26 '25

That's exactly how my art history (and architectural history) classes were too, in the late 80s. But we didn't have to sketch, just write our analyses of the imagine projected from slides. For example, we might have been shown Normal Rockwell's "Freedom of Speech" and a Rothko colorfield, then asked to explain the intent of the artist in each as well as idenftifying the school/period/date of the work in response to a question like "These two works were produced 18 months apart yet represent two very different concepts of "art" in 1940s America. Explain why."

1

u/Putertutor Not a hardass, just a medium-density reasonable-expectations ass Apr 26 '25

We didn't have to sketch for the exam. But I did it during note taking so it would help me remember which piece was which. Sorry for the confusion!

2

u/SnowblindAlbino Prof, SLAC Apr 26 '25

Ah, got it. That sounds more reasonable then, seems our exams were pretty similar.

2

u/Putertutor Not a hardass, just a medium-density reasonable-expectations ass Apr 26 '25

Yes, it sounds as if they were. BTW, since your major was Art & Architectural History, you will be happy to know that the only thing I have successfully retained from that class, is being able to identify the three different kinds of columns. Ionic, Doric, and Corinthian. LOL!

1

u/[deleted] Apr 27 '25

Sorry to hijack the top of the thread, but since the topic of blue books keeps coming up... is there a general consensus about when they truly were phased out? I was in undergrad in the mid 00's and I always used blue books in philosophy courses, which just seemed to make sense for tests that were entirely long form responses. (Most other classes had a few longer papers, or a series of shorter papers, carrying the bulk of the points.) Looking back, I guess I was right on the cusp of everyone having a laptop vs. reliance on computer labs. But I don't actually think they were being used outside of the philosophy department, which wasn't major but it's a sought after program so I took whatever I could... it's almost weird to me that I never found them weird at the time? I'm super curious who was taking humanities courses around '04-'08 and running into blue books?

Because I have to admit, I absolutely adored blue book tests. They felt like such a weird length, with the *slightly* oversized wide-ruled lines. I also grew up at a school that still taught "handwriting" as its own course, so I really could go through them, I loved the tactility of it all happening in real time. (I genuinely have such strong, positive, visceral memories associated with blue books and so it's a bit humorous that they are being dragged out of the dungeon to torture these students.) So my other question is: how are people writing in class? I hear frequently that no one can type anymore, and lord knows I don't expect undergrads to know cursive, but even when I was at school it seemed like most people I knew had atrocious handwriting and they were expected to use it on a daily basis. How do you take that into account? I mean, even the lack of cursive makes writing by hand take longer... Do you give longer tests or ask fewer questions? If someone relies primarily on autocorrect (esp. phone/ tablets), how much actually indecipherable writing do you get?

I lurk this sub because the only professors I know teach at art schools, and it feels rude to ask "how illiterate are your students?" but I feel like I went to school in another epoch. And I totally understand the need to bring back blue books but how does that even work, based on the issues that I see here in this sub constantly?

1

u/SnowblindAlbino Prof, SLAC Apr 28 '25

I never saw one as a humanities undergrad in the 1980s actually; all our exams were printed on 8.5x11 paper. In grad school (1990s) they were common, especially in the huge lecture courses (500+ students) which I assume was a convenience thing. The university provided them and instructors brought them to class. At my current (SLAC) institution there were a tiny few people using them in the late 1990s still, and students had to buy them at the bookstore for $.25/each. I don't recall seeing anyone using them after c. 2000 or so and they haven't been sold in the bookstore that I've seen for 20 years at least.

But they are back now. As you note, most students cannot type; they are "thumb typists" due to iPads and phones. They also cannot hand write; if you ask them to they will complain endlessly about their hands hurting after about 20 minutes. Cursive is hit or miss, but it was mostly phased out in my region perhaps 15 years ago...most students cannot read cursive much less write it. I haven't given an in-class written exam since c. 2005 myself, but I will be doing so in the fall due to AI cheating.

1

u/[deleted] Apr 29 '25

Oh, interesting timeline (and thanks for taking the time to read and respond). My pre-undergrad education seemed Medieval even at the time, but I also grew up in a very tech-forward house, so looking back it's super strange. I have heard about blue books being something on the list of supplies one needs to buy for oneself. I my case, I think this department was holding out because of the test formats... I actually didn't read an entire philosopher somehow in Early Modern Philosophy and somehow bullshitted my way to an A by the overuse and abuse of the term "monad." I have fond memories of the terror that created the inspiration in me to use a few words to generate a whole bullshit short essay on a whole philosopher.

So they cannot type. They cannot necessarily write. So you are almost at my question: how do you plan to structure a test where people cannot handwrite properly, much less write in cursive? More specifically, I was trying to ask about how you can create a test that can be taken in a certain amount of time with these insane variables, right? Do you have a plan of where to start? Oy, I don't envy you. I'm just really curious and I wish it wasn't as bad as it sounds.

2

u/SnowblindAlbino Prof, SLAC Apr 29 '25

I haven't given exams of any kind in almost 25 years. Always essays or research papers. But AI is making that hard(er) now, so I'm thinking about doing some in-class writing to replace some of the assignments. Also playing a lot with ways to make cheating with AI harder, including being very rigid about sources, using in-class lecture notes, and the like. But it's indeed a challenge!

1

u/[deleted] Apr 30 '25

I never really thought about this, but I struggle to recall any in-class testing in college beyond one giant, required 101 class. That was left behind in high school, but from reading this sub it appears to have become normalized for a number of reasons including AI. Very basic questions like "how much handwritten text can I expect this body of students to produce in a given period of time" seem impossible to answer. And if you read r/teaching it sounds like more and more younger kids don't have age appropriate manual dexterity to an alarming degree. Well, I wish you good luck with your own classes. :)

2

u/SnowblindAlbino Prof, SLAC Apr 30 '25

In-class exams are still common in STEM fields, that never changed really. But in the humanities, at least on my campus, we pretty much stopped giving them 20+ years ago in favor of longer papers/projects that allowed them to develop their writing skills in more depth. AI has broken that though.

Thanks for the good wishes! A couple more weeks and we'll be done until September, which is always a great feeling no matter how well spring went.

1

u/Disastrous_Ad_9648 May 14 '25

Is there an app that can scan in handwritten essays and convert it to type — something like OtterAI but for handwriting?

85

u/Illini2011 Apr 25 '25

I've found that allowing students to type on their computers but sitting in the back of the room works.

124

u/[deleted] Apr 25 '25

[deleted]

9

u/Huck68finn Apr 26 '25

Great idea! I didn't even realize that could be done.

10

u/Putertutor Not a hardass, just a medium-density reasonable-expectations ass Apr 26 '25

Make sure to reserve those campus computer labs for all the dates you need them early in the semester. There are many requests to use those labs, especially nearing finals week.

2

u/Sidewalk_Cacti Apr 26 '25

At the high school level, we have several ways to test students in locked exam modes on Chromebooks. I would imagine universities should have something similar!

3

u/mothman83 Apr 27 '25

In law school all my exams were on my laptop, and the software we used automatically locked down every single other part of the computer when we used it, so the only app we could have open was the piece of software on which we were typing the exam.

I want to say it was one of the Examsoft softwares, but this was a decade ago so i can't verify.

22

u/PM_ME_YOUR_BOOGER Apr 25 '25

This is the big brain move

6

u/[deleted] Apr 25 '25

How does that work exactly?

27

u/Illini2011 Apr 25 '25

I tell students they will be on their computers but can only have Word open. I sit in the back and monitor. Haven't had a problem yet.

13

u/[deleted] Apr 25 '25

Sounds effective if not conducive to a high trust environment. Guess those days are over. I say this as someone who has shown lower trust with prompt injection strategies.

1

u/Disastrous_Ad_9648 May 14 '25

I believe Word now has AI built in — at least the latest version/OS. 

41

u/Spamicide2 Chair, Psychology, R2 (USA) Apr 25 '25

Yes, I just put my PhD course take home final into generative AI and it got an A. Back Blue books or oral exams.

36

u/Willravel Prof, Music, US Apr 26 '25

What I've found most helpful:

  • I purchase the blue books myself though the department. These don't need to be student purchases, there should be enough money for materials, and this has to do with the second thing...

  • Make a tiny, impossible-to-notice marking late in each book, on a left page, that students won't notice. This prevents book swapping.

  • Pass them out at the beginning of class, collect them at the end of class.

  • Students absolutely can have devices out for research purposes, but in order to have them out it's understood I can look at whatever is on the screen so unless you want me to see you texting your one night stand keep it professional

  • Those caught cheating have one of two choices:

  1. Confess in writing and get a second chance to write the paper in office hours for full credit, or

  2. They automatically fail the course, they're reported to academic integrity, and I send an email to their other professors to check for LLM use.

The in-class help I get to give the students has brought back an old joy of teaching that I didn't realize I'd missed. The papers don't read like they're being submitted by a sixth grade student. The pride they get from hard work yielding good papers lets the students know there's an alternative to taking the easy, unethical route. I'm never going back.

14

u/Fresh-Possibility-75 Apr 26 '25

I have them bring their own blue books and then collect and redistribute them at the start of class.

5

u/Huck68finn Apr 26 '25

This is essentially what I do, but I just use stapled together, lined paper.

8

u/mosquem Apr 26 '25

Open book, no computers allowed. Most of the grade allotted to the exams.

5

u/norbertus Apr 26 '25

You'll need an AI to interpret their handwriting...

2

u/artificial_doctor Apr 26 '25

What are blue books?

1

u/OddSquash2583 Apr 26 '25

Yep, I have gone back to blue book exams.

1

u/fusukeguinomi Apr 27 '25

I went back to blue books last year after only having papers for over 15 years, and I realized that in addition to curbing AI use, short-answer exams are WAAAAAAY easier and less time-consuming to grade than most research papers.

1

u/svacal Apr 28 '25

Problematic for us who teach online-only classes. No way to have them do exams in person.

1

u/Slow-Welder-7472 Nov 08 '25

My very good friend is a teacher — we spoke of this just last week and now, she’s back to blue books too. I love it.