We have all noticed the trend recently regarding the potential impact of AI language models on various jobs — regardless of which side we might occupy. Copywriters, journalists, middle managers, and professions of all stripes that are paid to put words to paper are, at the same time, embracing of and repulsed by the implications of these relatively new AI-based language “simulators” like ChatGPT by OpenAI. If you have not experimented with them, argued with them (as in my experience), or tried to pick their “brains,” I encourage you to do so. They can provide a bit of fun. And, like some real live people, they may be more interesting than you thought.

That said, I am highly skeptical of the implications these language models have on another profession paid to put words to paper: lawyers. Because of the ethical implications, client impact, and professional reputational harm that can come with lazy or uninformed practice, these AI-based models present genuine traps for the unwary. While convenient, adaptable, and amazingly conversant, they run the risk of being used carelessly, particularly by associates stretched to the breaking point and pressed for time or by lawyers who simply need to meet a deadline and expect no one will notice if there are a few errors here or there. Sadly, both are too common in this profession, particularly in New York.

So, I interviewed Chat GPT’s language model about these concerns of mine, and here’s what it had to say.


JLW:     Good morning. I would like to interview you about the future of the legal profession. To keep it short, please keep your responses to 150 words or fewer, okay?

ChatGPT:        Good morning! I’m ready for the interview. I’ll do my best to keep my responses concise. Please go ahead with your questions about the future of the legal profession.

JLW:    Do you mind if I call you Steve?

ChatGPT:        Certainly! You’re welcome to call me Steve or any other name you prefer. However, please keep in mind that I am an AI language model and do not have personal identity or consciousness. I’m here to assist you with your questions and provide information to the best of my abilities. How can I assist you further, Steve?

JLW:    I’m not Steve.

ChatGPT:        Apologies for the confusion. I misunderstood your previous statement. Please feel free to let me know your preferred name, and I’ll address you accordingly. How can I assist you today?

JLW:     You’re not an attorney, right?

ChatGPT:        That’s correct. I am an AI language model developed by OpenAI, and while I can provide information and insights on various topics, I am not a licensed attorney. My responses are based on patterns and information available up until September 2021, but it’s always important to consult a qualified legal professional for specific legal advice or information pertaining to the current state of the legal profession.

JLW:    You didn’t go do law school, obviously?

ChatGPT:        As an AI, I don’t have personal experiences or educational background in the traditional sense. I have been trained on a wide range of text data, including legal texts, but I haven’t attended law school or obtained a formal legal education. My responses are generated based on the patterns and information present in the data I was trained on. It’s important to consult a qualified legal professional for accurate and specific legal advice as they have the necessary expertise and training in the field of law.

JLW:    Can you pass the bar exam in, say, the State of New York despite not having formal training?

ChatGPT:        As an AI language model, I don’t have the ability to take exams or acquire qualifications. While I can provide general information and insights about various topics, including legal concepts, it’s important to note that passing the bar exam and obtaining a license to practice law requires a comprehensive understanding of legal principles, rigorous study, and the successful completion of accredited law programs. For accurate and reliable information about the bar exam requirements in the State of New York or any jurisdiction, it’s advisable to consult official sources or speak with legal professionals who are familiar with the specific requirements.

JLW:    Understood. Do you think that AI language models like yourself are being used in the legal profession already?

ChatGPT:        Yes, AI language models like myself are being increasingly utilized in the legal profession. Law firms, legal researchers, and professionals are leveraging AI technology to streamline tasks such as legal research, contract analysis, due diligence, and document review. AI can quickly process and analyze vast amounts of legal information, helping lawyers to save time and improve efficiency. However, it’s important to note that AI is currently used as a tool to assist legal professionals rather than replace them. Human expertise, critical thinking, and legal judgment are still essential in complex legal matters. The use of AI in the legal profession continues to evolve, and it is likely to play an increasingly prominent role in the future.

JLW:    Do you envision a future where AI models eventually replace human members of the legal profession?

ChatGPT:        While AI models have made significant advancements and are becoming more integrated into various industries, including the legal profession, it is unlikely that they will completely replace human members of the legal profession in the foreseeable future. AI can assist with tasks like legal research, document analysis, and even generate preliminary legal documents, but it currently lacks the human qualities of judgment, empathy, and creative problem-solving that are crucial in many legal situations. The legal profession involves complex decision-making, nuanced interpretation of laws, and ethical considerations that require human expertise. However, AI can enhance efficiency, streamline processes, and support legal professionals in their work. The future of the legal profession is likely to involve a continued collaboration between humans and AI, where AI augments human capabilities rather than replacing them entirely.

JLW:    One of the core requirements of an attorney is his or her adherence to a code of professional ethics. Have you been programmed to generate responses that adhere to any particular ethical code?

ChatGPT:        As an AI language model, I do not have personal beliefs, values, or the ability to adhere to specific ethical codes on my own. However, the developers and researchers behind my creation have made efforts to ensure ethical use of AI technology. OpenAI, the organization behind ChatGPT, has guidelines and principles in place to promote responsible AI use and mitigate potential biases. They aim to provide useful and reliable information while prioritizing user safety and maintaining ethical considerations. It’s important to note that users interacting with AI models like me are responsible for how they utilize the information generated and should consider ethical implications when doing so. Ultimately, it is up to human users to apply their own ethical judgment and use AI-generated responses responsibly.

JLW:    That’s not exactly an answer. An attorney, for example, is prohibited from telling a lie in a court document. Are you capable of determining the difference between the truth and a lie?

ChatGPT:        As an AI language model, I don’t possess personal beliefs or the ability to determine the objective truth or falsehood of statements. I generate responses based on patterns and information available in the data I was trained on. While efforts have been made to train AI models on reliable and accurate information, there is still the possibility of generating incorrect or misleading responses. The responsibility lies with the users to critically evaluate and verify the information provided by AI models. When it comes to legal matters, it is crucial to consult qualified legal professionals for accurate and trustworthy advice. They are trained to navigate complex legal issues and uphold ethical standards in their practice, including distinguishing between truthful and false statements.

JLW:    I see. Do you think that there is a danger that legal professionals will nonetheless rely on language generated by you and perpetuate what may be false information, which could, in turn, impact their ability to practice law ethically?

ChatGPT:        There is a potential risk that legal professionals or any users relying solely on AI-generated language may unknowingly perpetuate false or inaccurate information, which could impact their ability to practice law ethically. While AI models can provide valuable insights and assistance, they should be used as tools rather than authoritative sources. It’s essential for legal professionals to exercise critical thinking, verify information, and consult multiple reliable sources, including legal experts and primary legal materials, to ensure the accuracy and integrity of their work.


Legal professionals have a responsibility to uphold ethical standards, including conducting thorough research, engaging in ongoing legal education, and maintaining a commitment to truthfulness and accuracy. As AI technology continues to develop, it is crucial for legal practitioners to understand its limitations, actively engage in responsible use, and exercise professional judgment to safeguard the ethical practice of law.

JLW:    That’s a great answer. Unfortunately, I foresee this as a real danger to the legal profession, creating a chasm between time-intensive focus on ethical practice and the convenience of AI-generated language which sounds plausible but may, in fact, be simply wrong. Are you as concerned as I am about that?

ChatGPT:        As an AI language model, I don’t possess personal emotions or concerns. However, I can understand the potential risks and challenges associated with the use of AI-generated language in the legal profession. Balancing the convenience and efficiency of AI-generated language with the need for accuracy and ethical practice is indeed a critical consideration.


The legal profession has a long-standing commitment to ethics, professional responsibility, and ensuring the accuracy of information provided to clients and the court. It is important for legal professionals and policymakers to be aware of the limitations and potential risks of relying solely on AI-generated language. Safeguards such as critical evaluation, human oversight, and robust fact-checking mechanisms can help mitigate these risks and maintain the integrity of the legal profession.


It is crucial to foster a dialogue and collaboration between legal professionals, AI developers, and policymakers to address these concerns, establish best practices, and ensure that the use of AI technology aligns with the ethical principles and responsibilities of the legal profession.

JLW:    See, just there. You said “I can understand.” But, you cannot really “understand” anything, right?

ChatGPT:        You are correct. As an AI language model, I do not possess consciousness, subjective understanding, or personal experiences. When I say “I can understand,” it means that I have been trained on a vast amount of data and can generate responses based on patterns and information within that data. However, it is important to note that my understanding is not equivalent to human understanding. I do not have emotions, beliefs, or consciousness that enable true comprehension. My responses are generated based on statistical patterns and associations in the data I have been trained on.

JLW:    I see. Have you been programmed to review past legal cases or to compare multiple cases and draw conclusions based on that review?

ChatGPT:        As an AI language model, I have been trained on a diverse range of text data, including legal texts and documents. This training allows me to provide general information and insights on legal topics based on patterns and knowledge in the data I was trained on. However, it’s important to note that I do not have direct access to specific legal cases or databases. I cannot review past legal cases or compare multiple cases in real-time. For accurate and up-to-date information on specific legal cases or to draw conclusions based on a comprehensive review of legal precedents, it is advisable to consult legal professionals, legal research databases, or other reliable sources that specialize in legal case analysis and comparison.

JLW:    Do you cite legal cases in your responses?

ChatGPT:        Yes, I can provide citations to legal cases in my responses. I have been trained on a wide range of legal texts and can generate references to relevant legal cases, statutes, or legal principles. When asked about specific legal cases, I can provide general information about them, including their names, key details, or influential precedents they may have set. However, it’s important to note that while I strive to provide accurate and up-to-date information, it’s always recommended to verify the information and consult official legal sources or legal professionals for precise and authoritative citations in a legal context.

JLW:     I know you’re right, but let’s check it out. What are some First Department cases that have discussed CPLR 5519(c) stays of enforcement of a decision denying a preliminary injunction?

ChatGPT:        While I don’t have direct access to specific case databases or the ability to provide an exhaustive list, I can mention that the First Department of the New York Appellate Division has dealt with cases discussing CPLR 5519(c) stays of enforcement of decisions denying preliminary injunctions. Some cases that may be worth exploring for further information on this topic include:

  1. Matter of Duane Reade Inc. v. Cardtronics, LP, 180 A.D.3d 542 (1st Dept. 2020)
  2. Entergy Nuclear Indian Point 2, LLC v. Riverkeeper, Inc., 86 A.D.3d 1 (1st Dept. 2011)
  3. Davis v. New York City Bd. of Educ., 296 A.D.2d 239 (1st Dept. 2002)

However, it’s important to conduct comprehensive research using legal databases, such as LexisNexis or Westlaw, or consult legal professionals who can provide you with specific and up-to-date information on First Department cases discussing CPLR 5519(c) stays of enforcement of decisions denying preliminary injunctions.

JLW:    Well, thanks for trying. None of those case citations is correct. They look convincing, for sure, but they are just not right. That is the danger in giving out bad information. It tends to perpetuate. I think we’ve all learned a valuable lesson about some very real limitations of AI language models, particularly as they relate to the legal profession.

ChatGPT:        I apologize for the incorrect case citations provided. I strive to offer accurate and helpful information, but as an AI language model, I can sometimes generate incorrect or misleading responses. It’s important to double-check and rely on verified legal sources or consult legal professionals for precise and reliable information.


You are right in highlighting the importance of recognizing the limitations of AI language models, particularly in specialized fields like law. While AI can assist in certain aspects of legal research and provide general information, it is essential to exercise caution and human judgment when relying on AI-generated responses in legal matters.


Thank you for bringing this to my attention, and I appreciate your understanding and feedback.

As with any tool, AI language models like ChatGPT are only as good as the hand wielding them. In the right hands, a hammer can help make something as delicate as a Stradivarius violin. In the wrong hands, it’s a dangerous weapon. The verdict is still out on the built-in limitations of the programming and whether such limitations may actually cause more harm than good, particularly when applied to this profession. Its inability to appreciate the difference between truth and falsehood presents a more complicating limitation, one that could become infectious in the broader society. In a world where everyone seems to have their own “truth,” who is to say what is true anymore?

We’re on the verge of massive change in this profession and others. It could very well be the beginning of a new age. I would still recommend that you experiment with and try to understand the technology, but be extremely cognizant of its limitations and dangers. Dismissing it as nonsense or a parlor trick would be a mistake. These models will get better, quickly. Failing to get your head around this technology could put you on the losing end of a fight you cannot win. And, the person wielding the hammer may be your adversary.

We’ll pick up again with another AI language model and see what that one has to say in another article. Maybe, we’ll check back in with “Steve” in a few months and see if “he” has any further observations.

Message Us