AI, they say, will revolutionize the practice of law. But can it do anything for my actual practice, as in, the case I am working on right now? Prof. Jayne Woods joins us to explain how she used ChatGPT—the question-and-answer AI interface—to draft a very passable first draft of an oral argument outline. Even better, ChatGPT could event engage (with a little coaxing) in a moot court dialogue, asking questions and follow-ups about legal issues.
Some of Prof. Woods’ takeaways:
Jayne Woods’ biography.
Appellate Specialist Jeff Lewis' biography, LinkedIn profile, and Twitter feed.
Appellate Specialist Tim Kowal's biography, LinkedIn profile, Twitter feed, and YouTube page.
Sign up for Not To Be Published, Tim Kowal’s weekly legal update, or view his blog of recent cases.
Announcer 0:03
Welcome to the California appellate podcast, a discussion of timely trial tips and the latest cases and news coming from the California Court of Appeal, and the California Supreme Court. And now your hosts, Tim Cole and Jeff Lewis.
Jeff Lewis 0:17
Welcome, everyone. I am Jeff Lewis.
Tim Kowal 0:19
And I'm Tim colwall. Both Jeff and I are certified appellate specialists and as uncertified podcast co hosts we try to bring our audience of trial and appellate attorney some news and perspectives they can use in their practice. If you find this podcast helpful, we'd be most appreciative if you would share it with a colleague.
Jeff Lewis 0:34
And if it's not helpful, send it to your opposing counsel. And a quick thank you to our podcast sponsor, he's text. He's text harnessing AI for the very best in legal research documents, summaries, and document analysis. Imagine uploading a deposition transcript to the cloud and getting back a summary in minutes. Check out the latest case tech product co counsel hates text.com. For more information and see our show notes for a link to a discounted sign up at case text.
Tim Kowal 0:58
And just speaking of the CO counsel product of case texts, I found a lot of value already, and how it speeds up the processing and summarizing of documents and helps me get quick and dirty analysis of legal questions that come up in my practice. And you'll recall during our conversation when we interviewed Pablo era Dondo, one of the cofounders of case text, we asked him if AI would be able to assist attorneys preparing for oral arguments such as by, for example, predicting what an appellate panel would ask counsel during oral argument. And so when I saw
Jeff Lewis 1:29
that technology is years away, that can't possibly happen, Tim?
Tim Kowal 1:33
Oh, don't speak too quickly, Jeff, because I recently saw Professor Jane Woods recent blog posts at the appellate advocacy blog. And she ran an experiment with chat GBT doing just that she asked us a bunch of questions about what kind of questions an appellate panel might ask in a free speech issue. And I was delighted to have the opportunity to speak with her today. So today we welcome Jane woods to the show. Jane Woods is an associate teaching professor of law at the University of Missouri School of Law, Professor Woods teaches legal research and writing and advocacy and research. She has also taught appellate advocacy and moot court before becoming a professor Jane was an Assistant Attorney General for the Missouri Attorney General's office in the Criminal Appeals division, where she wrote and submitted approximately 400 appellate briefs and argued multiple cases for both the Missouri Court of Appeals and the Missouri Supreme Court. And before that, she was a law clerk for the honorable Karen King Mitchell on the Missouri Court of Appeals western district. Jane woods, welcome to the podcast. Thanks so much for being with us.
Jayne Woods 2:35
Thank you for having me. Yeah, this
Tim Kowal 2:36
is a very topical issue of AI chat GBT and our sponsor case Tech's product co counsel, I think are on the minds of a lot of attorneys and law students. You're the first law professor we've had on the show, and we're eager to talk with you about your forays into how AI may change legal practice. But before we get into that, can you tell us a little bit more about yourself about your journey from legal practice into academia?
Jayne Woods 3:01
Yeah, sure. And actually, there's just one little correction I have. I'm still a clerk for Judge Karen Mitchell in the Western District. I did that after I did my attorney general stint, and then I've been working for her since 2011. And so I'm still doing that now. Yeah, so I started out doing the practice thing. And then I started clerking for her and shortly after I graduated law school, I started adjunct teaching at the University of Missouri research and writing was always my favorite part. So they let me come back and do that. So I did that for a while. Yeah, sometimes
Tim Kowal 3:29
we ask our guests, what is your favorite part of appellate practice? We have an important exception other than the legal research and writing part. So you made your whole career out of the legal research and writing part. Did you have another aspect of appellate practice that you enjoyed? Well, actually,
Jayne Woods 3:43
my former boss at the Attorney General's office, he always used to say that appellate practice is the purest practice of law, because it's where we don't have to deal with discovery. We don't deal with clients as much, because we're really arguing about the law and the theory of law. And so I agree with him. I think that's the really fun part.
Tim Kowal 3:59
That's right. And if you work for the government, you also don't have to worry about billing, right? But that definitely helps make it or marketing or I still remember coming out of law school with a head full of law, but empty, have any useful skills. And I was telling you before we hit record that it's hard. I still remember coming out of law school like that. But I don't remember where things changed. And so now I feel it would be hard to train a first year law student as anything changed, I guess my own personal experience would say, No, it's not changed. We all grizzled attorneys become set in our ways, but maybe others are better in educating than I am. So how do you prepare law students to go out into practice, you know, ready and raring to go?
Jayne Woods 4:37
Well, it's also very different for me because I remember when I was a well, I started out learning about books. And I was in the library doing books and actually jeopardizing with a real Shepards book. So the whole legal databases thing was new that when when I started and so it wasn't until our second semester that we even got to use Westlaw and Lexus. So it's kind of fun. Now we don't have that restriction on students they automatically Get into the legal databases. And you know, now we have the Google generation. So they're already very familiar with how to do that sort of thing. But just kind of switching their mindset from the Google searching to using these legal databases is is a new and different experience. I think the funnest part, though, is actually showing them a book and how the citations work, because they came from an actual book.
Tim Kowal 5:20
They actually did come from a book. Yeah, we talked about published and unpublished opinions. And it's kind of a misnomer, because everything is online, whether it's published or unpublished, it still comes up in my search results. So what the heck does that mean unpublished? Well used to mean that things were actually published, like in a book with paper and binding and everything. And these opinions are not published. So that's why they're called unpublished. Right? And what about, you know, before we talk about how lawyers can make better use of AI, you have any perspectives on how lawyers can make better use of young legal talent? Because that's kind of what the game is with AI is, you know, how do I replace it? I don't want to have to hire somebody, maybe I can just use a robot AI to produce the same results. But maybe they're just not good at training young talent, like it's difficult to delegate, it takes time, the first couple times you tried to delegate something, you think to yourself, well, I could have done this already by now. So what's the point?
Jayne Woods 6:14
It is a little bit that we would chat up to at first, you have to figure out how to tell it to do things, right. I don't know, I think that young associates still bring a lot to the table. You know, for one, they are much more aware of current events, which obviously some AI really is not current, yet. It only stops. It's like 2021. So definitely have them more attuned to the world around them. And you know, I don't think you can replace the human connection.
Tim Kowal 6:36
Yeah, yeah. There's something that one of our local law schools does here. Chapman law school is actually not renamed to the Fowler School of Law. But they did a clinic, I suppose they still do a clinic. I taught at adjunct for a couple of years. And it's a it's a civil procedure workshop clinic. It's for first year second year students, I think it's maybe after they see their after or concurrent with going through their civil procedure. So if they get practice actually doing a notice of remand, it actually propounding some discovery and responding to discovery. So by the time they come out of law school, and they get hired as a junior associate, they at least have had a little bit of exposure so that they're not looking at a Discovery set that they have to respond to. And they're asking, What the heck is this?
Jayne Woods 7:13
Yeah, it's funny. You mentioned that I remember my first legal job that summer after my one year, they asked me to draft some interrogatory. And I was like, Oh, great. I know what those are. I remember learning that in Civ Pro. And I was like, I don't know how to write one.
Tim Kowal 7:25
Yeah, right. No, it's funny how it's just putting pen to paper. And knowing how something is done. It is really making that connection from the world of the abstract the pure law, as you say, to doing something that is actually going to be filed in a court. Sure. In your opinion, is AI a threat to job security for new lawyers? What are law students, your law school saying about things like chat, GPT and CO counsel?
Jayne Woods 7:51
Well, it's interesting, because a lot of them are unaware of chat GPT or are not interested in learning about it. I've been encouraging my one ELS to go experience it now and play with it while it's still free. Because I think that it's something that's definitely gonna be part of their lives in one way or another. And so now is the time for them to learn about it, when it's you know, they're in a safe environment, they're not going to end up being liable to a client for any sort of malpractice or anything for using it. And so I do think that it's something they should learn. But then again, I also have some law students who are still struggling to learn how Microsoft Word works. So they're in all sorts of different levels.
Tim Kowal 8:26
It is funny, you see older attorneys who are confident in technology and younger attorneys who are not confident in technology. So it's like anything, generalizations are always false at some level, but I wondered if there may be an opening there for young attorneys to leverage artificial intelligence for their favor, because I was at a tourney networking meeting this morning, for example. And we talked about AI and chat GPT. And everyone just kind of looked around you. Have you tried using chat, GBT? No, you know, do you have imagined anyways, you could use it in your practice? No, I don't think I could ever use that. And so law students, if like you said, you're urging them to get up to speed on it, they can probably bring that value to older law attorneys who otherwise would be afraid to touch it with a 10 foot pole.
Jayne Woods 9:07
Yeah, absolutely. And I think one of the skills that law students now can really develop is learning how to put in an effective prompt into AI software to get the results that you actually want, because I think there is a bit of an art to prompting the AI to return a result that's helpful to you. And so if they can start learning that now I think that will put them ahead in the game for getting jobs later.
Tim Kowal 9:27
Yeah. You mentioned an important concept, that prompting that we'll get into a little bit later. But yeah, there's a way to train AI train these chat GPT to give you better and better results. So yeah, it does depend on the skill of the user and getting more useful results out of chat. GPT What are some common mistakes in we're going to talk about oral advocacy, because your blog post about GPT was testing your chat. GPT is ability to give attorneys better results, more information in preparing for oral advocacy. But before let's leave chat GBT aside for a minute Just talking about oral advocacy, because that's something that you train law students, and what are some common mistakes in oral advocacy that you see law students and maybe even practicing attorneys make
Jayne Woods 10:09
being married to their outline? Refusing to yield to those tightly? Yeah, exactly. hanging on to it like a security blanket and refusing to yield to where the court wants to go. Because the questions really are the most important thing. And I think if you fear them, instead of embracing them, that's really going to put you in an awkward position at oral argument.
Tim Kowal 10:29
That's right. Do you have any methods, any tips on how to get attorneys unmarried to their outlines? Because yeah, I mean, those stories abound where thank you for your question, Your Honor, I'll answer that later. It's a little bit further down in my outline.
Jayne Woods 10:41
Exactly. Yeah, I don't know if I have a surefire way. One thing I always tell my students is not to imagine it like being cold called in class, because the judges have a different purpose and asking you questions than your law professors do? Because the judges really want to know the answer to what is the effect of this case? If I decided this way? How is it going to affect the next case that comes down the pipe? And so they really just want to kind of play it out and see, and have that intelligent conversation about the law. Whereas you might have a law professor who's really trying to nail you to the wall and figure out like, Did you do the reading?
Tim Kowal 11:13
Yeah, no, that is probably the toughest thing about oral argument is that when you're preparing for it, I mean, all the while you're doing the briefs, and then preparing for oral argument, the question is always burning on your mind, what could this panel be thinking? How could they possibly disagree with me, right, and when they first opened their mouth, that's that means you're going to get a clue as to where they're going. And if their mind is not right, where you are in the outline, then there's a little bit of a disconnect there. And that's information that you have to be able to use it right on the fly there. But to try to bridge that disconnect. So it can be it's extraordinarily helpful. It's it's the only, it's your last chance, and maybe your best chance to bridge a disconnect between you and the panel. Yeah. All right. So Jane, we brought you on to talk about Chad GPT and preparing for oral argument. Let's back up. And let me ask you what first brought you to an interest in chat GPT. And let me just for my for the audience sake, you wrote a post at appellate advocacy blog, the title of the post, it was in last month in March 2013. It's called can chat GPT prepare me for oral argument. And then we'll describe a little bit later about how you your approach there and how you the questions and prompts you gave chat GPT in the the case that you were asking it to give you prompts about but what what brought you to the idea of doing this experiment using chat GPT to help assist you with an oral argument.
Jayne Woods 12:27
So obviously been hearing a lot about it just like everybody else. And you know, I want to know, how is it gonna be helpful, because if it's this truly life changing world changing thing, you know, I want to use it. And so I started trying to think, Okay, well, everybody talks about writing and how it's going to affect writing and the legal writing community, you know, do we fear it? Do we worry about plagiarism? What is this going to do? And so I started thinking, well, if it's really as powerful, as everybody says, then surely it can do more than just right. So what else could it do? And I was in the process of making lesson plans for my students for oral argument and talking about how we're going to prepare for it. And based on how we tell students to prepare, you know, drafting outlines, thinking of questions and writing summaries of cases. I was like, Well, those are things it probably can do. And so I just decided to test it.
Tim Kowal 13:10
Yeah. So you were testing it to create an outline of questions or an outline that you might be able to follow during the oral argument. Right? Both actually. Yeah, both. And let's discuss what you found to be good uses of chat. GBT, your blog post outlines how you found that chat, GBT could create a good first draft of an outline. Can you tell us a little bit more about the type of outline that have created for you? What was good about it? And what might have been lacking about it?
Jayne Woods 13:35
Yeah, sure. It's seems to be very good at formulaic writing, if you ask it to write like a five paragraph essay, it can nail that. And so I think outlines are very much in its wheelhouse. And so if you tell it, you know, kind of some basic facts of your case and what you would like it to do, and, you know, specify you want it for an oral argument outline, it started out with an introduction, it had a head grabbing sort of liner there, that one liner to catch everybody's attention at the beginning. And then it laid out well, these are the main legal issues in this case that you've presented me. So make sure you talk about this and make sure you talk about this and talk about this. And then it had a conclusion, everything that I probably would generate myself if I were going to create an outline.
Tim Kowal 14:12
And what about what was lacking in the outline? Was there anything about the outline that you thought, well, this could have been done better? I need to tweak this? Yeah,
Jayne Woods 14:19
I mean, it's hard because of, you know, I didn't get to enter a whole lot of facts into the prompt. So the more facts you give it, the more detailed questions and outlines it can prepare. And so what I gave, it was a pretty broad brushstroke of what the problem was about. And so the questions and the outline were pretty general in that sense. But, you know, I knew I would need to fill in the details as the lawyer of course, if I prompted it further and told it add this in. I'm sure it would have.
Tim Kowal 14:44
Yeah, yeah, I'll admit to not not having used chat GPT specifically, a lot. I have used co counsel, which I understand is based on the engine of Chet GPT, but it is made for lawyers, and one thing you can do is you can upload documents, you can upload your briefs and other cases. Do It and and have it give you an analysis based on the information that you've uploaded. So I think that could probably help supercharge that process. And what about the next step that you talked about was talking about the cases that were relevant to your argument that you talked about chat GPT summaries that it provided of the cases and you know, some successes and to chat GPT has credit and some not so successes in the debit column. So can you describe what was good and bad about Chachi PTS ability to create case summaries? Yeah, so
Jayne Woods 15:28
the case summaries that it created that were actually accurate, were really great, because they were short, they got right to the point, they narrowed in on the reasoning of highly relevant facts. And you know, if I could distill a case down to a single paragraph, that's probably what I would have written. And that's what it did. But it did it way faster than I could. The one downside is that one of the cases that I asked it to look at tends to be an outlier in the area of law. And it actually got it completely wrong, like the exact opposite of what the case held. But I think my understanding of how tat GPT works, I think that that actually makes sense, because it is based on kind of a predictive text model. And so it's looking for the most likely outcome of words in the most likely order. And in that particular context, the party in the position of the appellant in that case, typically wins. And they typically win for the reason that they actually lost in that case. So when it predicted the exact opposite. It made sense, because it predicted it in line with the rest of the cases.
Tim Kowal 16:27
That's interesting. That reminds me of again, when we had a discussion with Pablo Redondo that one of the cofounders of case text, he mentioned that there is a problem with AI a well known problem that you it's something to do with the problem of negation is that when you put in a prompt, you know, find me a case that says X, you'll get a whole bunch of cases that say x results to say x and a whole bunch of results to say not x, it's directly on point, just the opposite of the point. And it's very hard for AI to get that quite right. Yeah. So yeah, so that does suggest that you can get good summaries of cases. But how do you make that information valuable, while also knowing that some of those results might just be 180 degrees off?
Jayne Woods 17:05
Well, I mean, as you know, as
Jeff Lewis 17:09
you said, you ensure that you have a decent lawyer, experienced lawyer reviewing the computer generated results. Sam, come on with the softball question.
Jayne Woods 17:19
Yeah, that was actually going to be my response is that, you know, as an appellate attorney, you know that you've already been familiar with all these cases, when you're reading the briefs and preparing for argument yourself anyway. And really, my thought and preparing them as summaries wasn't to rely on them so much as to refresh my memory. If I were to go argue the case, just to keep them all straight in my head, but that's how I recognize the one was wrong, although it's so confident in the way it presented that I really questioned myself. And I had to go back and look at the case again.
Jeff Lewis 17:46
And yeah, yeah, interesting. Well, yeah, really careful with your, the way you formulate the prompts in the chat. GBT. I had an experience, I was trying to write a letter involving a copyright matter. And I typed in the prompt, write me a cease and desist response to a cease and desist letter regarding a copyright claim. And the response I got back was nope, chat. GBT is not gonna replace lawyer won't do that. But if you narrow the prompt, instead of say, write me a letter, say, Does this particular legal doctrine apply in the area of copyright law? It'll give you a nice thing that you can cut and paste and put into a letter. It's just you have to be super careful in terms of how you frame your question to chat, GBT.
Tim Kowal 18:25
I didn't realize they had done that. So is that kind of like an ethics directive that they programmed into it? No, we're not going to replace humans.
Jeff Lewis 18:33
Basically, we're not going to engage in the unauthorized practice of law. We're not gonna you know, we're not that other company. I can't remember the name of it. That was gonna have a one a $1 million reward for any lawyer would walk. Oh, yeah. earpiece.
Tim Kowal 18:47
They're not that. Yeah. Well, that's what I wanted. Yeah. If Chad GPT This is a tangent, but a chat GPT defames somebody or does something unethical? Who's liable?
Jayne Woods 18:58
Great question. We actually were talking about that today, because you had a thing on chat GPT. And some of the concerns that are coming up and one of them was about liability, or things like that. We talked about
Jeff Lewis 19:07
because it dropped discovery. About You know, sometimes, Tim, you and I we do malicious prosecution matters or legal malpractice matters where the legal research of an attorney is put into question. I wonder if chat GBT saves inquiries and outputs whether or not that could be the subject of discovery to legal malpractice or mouth a malicious prosecution case regarding that lawyer's subjective belief in the validity of his claims and adherence to the standard of care.
Tim Kowal 19:33
Oh, great question. So yeah, is the output chat GPT or CO counsel work product protected under the privilege?
Jayne Woods 19:41
I'm not gonna touch that when I'm not a PR professor.
Jeff Lewis 19:44
Yeah, I'm gonna say Yeah, but you know, if you sue, you know, certain cases, when you see a malpractice, prosecution, you waive that protection, or if you have sought advice of counsel, so yeah, stay tuned. I think the chat GBT queries and results the extent there's saved anywhere might be fertile ground for discovering the future.
Tim Kowal 20:03
Yeah, well that yeah, they are saved in CO counsel, all your previous questions and results are saved. So I wonder if it's a good practice to blow them out of there, if they do prove to be discoverable. I had one other question for you before we leave this topic of using AI to create case summaries, Jane. So back to your point about one of these cases, one of the Case Summaries was completely accurate except 180 degrees off, does that have anything to do with legal opinions being written in too subtle manner? Maybe the opinion itself wasn't clear enough. Is this a limitation unique to AI chat? GPT? Or could a human junior associate have made such an error?
Jayne Woods 20:40
Well, with this particular case, I would question whether you should retain a junior associate if they made that same mistake. It was quite clear in the opinion which way it was supposed to go. GPT just really got it wrong.
Tim Kowal 20:53
Got it. Okay. Okay. That's good to know. All right. And I thought this was one of the more interesting things about your piece, you asked chat GPT to generate some possible questions that the appellate panel might ask you during oral argument. Can you tell us a little bit more about that?
Jayne Woods 21:06
Sure. Yeah. I mean, that's one of the things that we tell our students to do all the time. And that's what appellate attorneys are trying to do in preparation was think, what is the court gonna ask me? So I just asked shad GPT. I said, here's the case. This is basically the facts of the case. What do you think an appellate court would ask? And it generated a list of 10 questions. And I was like, Well, those are interesting. Like, can you give me more? And I just kept asking it, can I have more? Can I have more and it ended up with I think, 30 something before the program actually froze, and I couldn't get any more after that.
Tim Kowal 21:35
Oh, really doesn't get fed up after a while. Yeah, I
Jayne Woods 21:38
don't know. I think it was one of those times when it was a lot of users going on or whatever.
Tim Kowal 21:41
Yeah. And you say in your article that you found that while chat. This is a quote, while chat, GBT is very adept at answering questions. It's not terribly inquisitive. But then later on, you indicated that it could be valuable practice in building confidence answering answering questions about the case. Can you talk a little bit more about the process you use? You had a very interesting way of going about it, you actually You ask questions, answered questions, and there was a back and forth you kind of engaged in a mock dialog with a with Chad GPT.
Jayne Woods 22:09
Yeah, I tried so hard to get it to actually go with the back and forth. And I could not get the free version to do it. Now I just actually upgraded right before we started talking. And I got the new version, the GPT, four, eight will do the engaged back and forth with you, which is super cool. But under that free version, the one that I did for my blog post, and that one, I had asked it to roleplay because I heard that's the best way to prompt it to do things like that. And every time I asked it just spit out a transcript for me of other people possibly role playing court and counsel. And so I was like, Well, this is great. I don't want to read an oral argument. I want to practice one. But it did at one point, when I asked it to directly ask me a question. I could get it to respond to my response. But it only did it like once or twice. And then after that, it would ask me a question. I would give it my answer. And it would say that's great. Thank
Tim Kowal 22:56
you. Yeah. Well, were the questions useful, where the incisive, you know, back to your quote, you said that it's good at answering questions, probably better at answering than asking questions. So talk a little bit more about your impressions on the kinds of questions that chat GPT asked.
Jayne Woods 23:11
Yeah, it's very socially awkward in the sense. It's like trying to have a conversation with somebody who answers every question you ask, but doesn't ask you any questions about yourself. And so it's kind of one of those things where just kind of dies if you don't ask anything else. And so I could not get it to engage with me in that way. Now, like I said, the new GPT four would engage with me more. But what's interesting about that is that it got really stuck on a procedural question, and did not get into any of the substance. And I was really wanting it to engage with me on substance, but it was just really hung up on that procedure.
Tim Kowal 23:41
Yeah, no, I think that's fascinating. So what's your takeaway? And in your view, is chat GPT a valuable tool for attorneys right now? Or is it not yet quite ready for primetime? Does it need more work? First,
Jayne Woods 23:52
I think that the two versions that I've tried the GPT for definitely is a lot cooler to interact with, as far as trying to do a roleplay with the court and prepare yourself that way. I'm not sure that either one of them is quite there on the legal reasoning yet, because it's not going to ask you those hard questions about your interpretation or your understanding of a case. It will ask you about cases generally. And so I think it's really good practice, certainly for one else who are probably not going to get those really intrusive questions about you know, what does this case mean? But I do think it might end up there at some point, you know, now that we know GPT, four can pass the bar exam by 90 percentile. Surely, it's getting close to the the reasoning we want. Yeah. I'm wondering,
Jeff Lewis 24:31
and what a great boon for solo attorneys who don't have somebody to bounce ideas off, you know, big firms, they could do Moots. They could do practice arguments, but solo attorneys. Wow, what a great opportunity to just get your wheels thinking in terms of possible questions.
Tim Kowal 24:48
Yeah, and even when you mentioned that, it's not terribly inquisitive, maybe doesn't ask the most probing questions, but sometimes, you know, like we talked about earlier, you're racking your brain trying to figure out what am I missing about this case, but The one subtle angle that I'm missing in your head is so far deep in those weeds, that maybe you're forgetting to train yourself to answer the more basic questions, and so maybe Chet GPT, or CO counsel, whatever it is AI product is training you to think about and to answer succinctly the the more basic questions about the case.
Jayne Woods 25:17
Yeah, for sure. And it definitely asked those kinds of questions. You know, it asked me like, explain how Rosenberger applies to this situation. And I think that's a very legitimate question that a court might ask
Tim Kowal 25:27
Yeah. Is there a learning curve to learning to use chat GPT, anything that practicing attorney needs to know before using chat? GPT? Effectively,
Jayne Woods 25:37
I think personally, the only information I would say is make sure that when you first start playing with it, you ask it things that you already know the answer to, so that you can kind of learn its limitations. Because if you ask the questions that you don't already know, the answer to you don't know if it's telling you true information or not. So that would be my only guidance. Other than that, I think it's really just playing with it and figuring out how to draft prompts in ways that really get you what you're wanting.
Tim Kowal 26:01
Yeah, I'm thinking again, to the meeting I was at this morning, where a lot of attorneys were just, their eyes got big at the idea of using chat JpT for their practice, should attorneys be afraid of using chat? GPT? What's a gateway drug for getting an attorney to try out? GPT?
Jayne Woods 26:16
That's a great question. You know, I personally don't think anybody should be afraid of it, I think that we should really learn about it. Because whether we want it here or not, it is here. And it is going to be a big part of our world. And so you might as well learn about it. Now, while it's still in its infancy, rather than later, when it actually, you know, could take over a job, possibly, I don't know, but I don't think that you should be scared of it. I think instead, just go play with it and ask it things, you might start by asking it to give you a bio of yourself. And that was really interesting to to see what it did for me. I'm a lot more accomplished than I thought I was.
Jeff Lewis 26:50
That's great. You know, Tim, I found that asking questions that you already know the answer to can help develop confidence in the product. And what I mean by that is not necessarily the legal research area. But suppose you're getting ready for oral argument, you know, the record cold, you know, what happened in this case, or you getting ready for trial, you know, these trial exhibits called you throw those exhibits up into the cloud, whether it's a co counsel product or something similar and say, find me all the emails that prove John Doe had malice, and it spits out a list with hyperlinks of all the right answers. And you know, because you're just getting ready for trial, you know, it's correct. So I think the best way to get somebody comfortable using this technology is right, when someone is very familiar with their own case, they have aI prove what they can do.
Tim Kowal 27:34
Yeah, I think that's a good idea, Jeff, and even when you mentioned, you know, what is all you asked a question, what is all the evidence that supports malice or whatever other element? That's key to your case? Maybe it spits out something that you didn't think of before? Maybe, you know, the core issues that the court decided that was really argued at the hearing, you covered all those got those all down cold in your brief, and you're ready for oral argument about it. But maybe there was one other thing that chat GBT picked up and you know, under the substantial evidence standard that could amount to an affirmance. And so you need to be able to pick it that one as well. Now, Jane, you mentioned earlier about using prompting to in chat GPT to help tone its answers. So giving when you give chat GPT a series of instructions, it'll continue refining and improving its response. So for example, if I want to know an answer to something about personal jurisdiction, say, I will want to prompt chat GPT that I want to know the answer based on what Ninth Circuit law will say. And then I want to know about specific jurisdiction rather than general jurisdiction. And then I might prompt it further and say, I want to know, we can do this yet, if it's capable of answering how a particular Ninth Circuit Judge might answer the question or what questions a particular jurists might ask an oral argument involving, let's say, a foreign national, rather than just a citizen, a resident of a different state in the United States. Do you have any thoughts or advice to attorneys or law students about using prompting or training to get chat GBT to produce better information?
Jayne Woods 28:57
Sure. I actually asked chat to PT about this, because I was trying to figure out what can I tell my students? And one thing I was really curious about was whether it operated better with boolean terms, if we did some sort of Boolean searching or Boolean language, and it very clearly told me no, use natural language. And it said to give as much information as possible.
Tim Kowal 29:17
Yeah, that's interesting. I've run some searches. And I forget to say, you know, under California law, and then the results will be great, except that it's, you know, all over the place. So I have to say, run this, run the result again, but give me the answer under California law, or you have to continue refining. So I think that's a huge advantage, because you get to not only do you get to help chat, GPT improve, but based on its response, you can see oh, you know, this wasn't quite what I expected. So let me give it a little bit more information, and it keeps getting better and better. All right. So someone has suggested actually, we're going to interview this person soon on the podcast that AI at some point in the future maybe even suggested, it's already here that AI will be able to draft entire judicial opinions. As such that in the future law clerks may be unnecessary. Are you similarly optimistic about AI as capabilities?
Jayne Woods 30:07
I don't know if I would call it optimistic given that I am still a law clerk. Next replace me. But I would be surprised if it could draft a legal opinion in that way. And maybe it will, maybe it will at some point. But as of now, from what I've seen, it's really good at informative writing. But check GPT at least does not give a whole lot of legal authority even when you prompt it to include legal authority, it might give you one case, and then of course, you have to go check and make sure that case is actually what it says it is. But it's not a really thorough in depth analysis, I would say it's more surfacey. And I think that judicial opinions tend to be a lot more in depth, of course, depending on the issue. And then you have to remember, too, that judicial opinions are not necessarily informative writing, sometimes they're persuasive as well, trying to get the other people on the panel to agree and trying to convince the general public that the answer is what the answer is. So
Tim Kowal 30:57
yeah. And then finally, do you have any opinions about the future of AI and chat GPT and the law and what sorts of things might be inevitable about chat GPT, or AI? Generally, what sorts of tasks will never replace a live human being lawyer?
Jayne Woods 31:15
Well, I think that probably our days of drafting complaints, and petitions are probably going to be gone pretty soon, because I can't imagine that anybody's going to have to do that on their own anymore. Even discovery sort of basic questions, those sorts of things. I imagine what we're really gonna be shifting to is not the drafting so much as the revision as attorneys, I think we're really going to be looking at it kind of as you know, partner would over a lower associate and just checking their work and seeing how it does. But I don't think it will replace us. I think part of the reason I'm just guessing here, but my guess is so why case text calls it co counsel is because they see a world where we're going to collaborate with AI as opposed to being replaced by it.
Tim Kowal 31:53
That is the language that they use. I like the way Pablo put it that we'll be able to redeploy our staff and our junior associates to doing other tasks than some of the grueling tasks that CO counsel can take over and do. But I was surprised a little bit when you mentioned about drafting complaints. And maybe you're right about that, in terms of you know, it can recite a chronology of the case and all the relevant facts, I guess it can recite, you know, what facts and evidence are relevant to a particular ultimate issue. But I thought in terms of that, can you just load up all the facts and tell chat, GBT or CO counsel draft me a complaint?
Jayne Woods 32:28
Well, so I actually tried that with my students with our problem. I hadn't draft three different versions of the complaint, because I asked it to do it. And every time it drafted something slightly different. And so then I had my students analyze the three versions of the complaint to see how it compared with the rules to see how it was persuasively. And I mean, all three of them, I think, you know, you could look at them and combine them and come out with a complaint that is satisfactory. Is it going to be amazing? Probably not. But is it going to do the job? Yep.
Tim Kowal 32:54
Yeah. Well, yeah, complaints are never bedtime reading in any event, they're to serve a purpose. Yeah, so I think I think we'll have to continue, you know, collectively thinking about the types of tasks that AI will be able to replace. Because yeah, I think we are entering into a new era here. Well, one other question. I know that there are I have been following, but I know there's some doom and gloom scenarios or some predictions that AI chat GPT will will turn in some hellscape. Somehow. I don't know if you've been hearing any of that. Is there anything to be concerned about with jet GPT? Other than what we've talked about? You know, we'll replace some of our jobs visit in danger. I guess maybe that's a it's probably a question for a different podcast, I guess. But any other parting thoughts about chat? GPT?
Jayne Woods 33:39
No, I don't think so. It's just I think it's really it's a cool idea. I think it's going to be the future of the law profession. I was talking with some other colleagues about how the railroad changed the legal profession. I think this is probably another one of those big things in the world. That's going to change the direction. Yeah,
Jeff Lewis 33:55
yeah. Big time. You know, I don't have any more questions. But before we conclude, there's one a couple of time we tidbit I wanted to share with you, Tim, that maybe Jay might find interesting as well, before we conclude our interview. I did last week a meeting of the roundtable it was the LA County bars appellate section had an interview with a bunch of Second District research attorneys about their preferences and quirks regarding appellate race, and I thought you might find this one interesting, Tim, you ever submit appellate brief that's fully hyperlinked with citations to the record and citation to Westlaw cases. Right. And you've done that?
Tim Kowal 34:27
Yeah, I've seen those. It seems like it would be immensely helpful. Yeah, my paralegal
Jeff Lewis 34:31
Jason, he whips those together. They look fantastic. My jaw hit the ground when I heard not one, not two, but every research attorney during this MC Lee presentation saying, Yeah, we don't click on those links from a tech slash security point of view. They don't click on those links, meaning you send them the brief. They go to Westlaw look up the case manually. They don't take the risk of clicking on a malicious link, which I suppose is a good thing. But yeah, so I thought you'd be intrigued to hear that tidbit. And then the second
Tim Kowal 34:59
I heard The exact same thing at a conference I was at late last year. And remember Jody and Todd from the Texas appellate law podcast told us about a couple of years ago, early in the pandemic, the Texas judiciary got plagued with some rants, I got hacked. So there are real security risks. They will these guys even go after the judiciary, you know, during the pandemic, you know, these ransomware people, you know, some of them just really give the hacker industry a bad name.
Jeff Lewis 35:26
Yeah, we need to get people from clear brief. I've never used that product, we get people from clear brief on this podcast explain that gap between the product they offer in terms of hyperlinking and justices or law clerks who don't want to click on the link? I'd be interested to hear what to say. So that was one issue. And then the second issue, can you put your professor hat on for a second? Okay, you ready for this? Everybody knows the superior way to cite a complicated ruling is to use the parenthetical cleaned up, right to remove unnecessary periods and commas etc, unnecessary citation. Is this
Tim Kowal 35:56
a leading objection away?
Jeff Lewis 35:59
When asked what these research attorneys were asked, do they have a preference? Should we use this? Should we not use it? What do they think? Does it help them? Do you know what they said? What is cleaned up? They had no idea when I had to explain all of them. So anyway, none of them had heard of it. None of them had heard of it. Wow. So I'm gonna say it's not a bad thing to you that I suppose it's not offending anybody. Whenever I use it. I always do a footnote explaining exactly what cleaned up is, and you know what it's about. But anyway, I thought you would be amused to hear that second one.
Tim Kowal 36:32
What a Jane in the Missouri Court of Appeals do they use cleaned up?
Jayne Woods 36:36
I have seen it a couple times. It's not something I would teach my one else because I think that it's probably dangerous in the hands of people who don't know exactly what it means. But yeah, I have seen it a few briefs. I don't know if I have a general sense for how people feel about it, though. I mean, it makes sense to me, honestly.
Jeff Lewis 36:51
Yeah. Yeah. You know, I'm a purist when it comes to like reading a brief and try to avoid any speed bumps that can affect the eyeballs, you know, reading across the page. And I think it's fantastic. Tim, if I could speak for Tim, I know he loves it. When I do. This seems to feel like it's a great thing for judges to do. But that lawyer should never do
Jayne Woods 37:07
it. I will say as a law clerk, it would probably raise my suspicions immediately. And I would go check the citation.
Jeff Lewis 37:13
But then when you check that citation, you see Oh, yes, perfectly cited. I now have enhanced confidence in this attorney. He's possibly his reputation has increased.
Tim Kowal 37:26
All right. Well, yeah, I'm surprised. Surprised to say, Jeff, you fairly and accurately characterize my view on it.
Jeff Lewis 37:31
At first, yeah, you're wrong, but I'm fair. All right. Well, I think that wraps up this episode. Again, we want to thank our sponsor, Keith tech, the makers of the co-counsel product for sponsoring our podcast each week, we include links to our cases discuss that we discussed using the case tech link, and listen to our podcasts can find a 25% discount available to them to sign up the case. tech.com/help
Tim Kowal 37:54
And if you have suggestions for future episodes, either guests or topics that we should have on the podcast, please email us at info at cow podcast.com. We'll even take suggestions that are provided by chat GPT and in our upcoming episodes, look for more tips on how to lay the groundwork for an appeal when preparing for trial. And thanks again to our guests, Jane woods for being with us today. Thank you. Thank you, Jane.
Announcer 38:18
You have just listened to the California appellate podcast, a discussion of timely trial tips and the latest cases and news coming from the California Court of Appeal and the California Supreme Court. For more information about the cases discussed in today's episode, our hosts and other episodes, visit the California appellate law podcast website at Cao podcast.com. That's c a l podcast.com. Thanks to Jonathan Cara for our intro music. Thank you for listening and please join us again
The California Appellate Law Podcast thanks Casetext for sponsoring the podcast. Listeners receive a discount on Casetext Basic Research at casetext.com/CALP. The co-hosts, Jeff and Tim, were also invited to try Casetext’s newest technology, CoCounsel, the world’s first AI legal assistant. You can discover CoCounsel for yourself with a demo and free trial at casetext.com/CoCounsel.
