Is ChatGPT essential for scientists?
Large language models may be ushering in a new era in academia
For years, we've heard about how artificial intelligence (AI) will transform our lives (or even take them over, as depicted in certain Arnold Schwarzenegger movies). However, just like flying care, these promises of the future have fallen short.
But things are about to change - or so it seems. AI is now here, and it's not just useful, but potentially revolutionary.
In this post I’ll share my initial impressions of this technology and how it can (and cannot) be integrated into the life of a scientist. However, many of these applications extend beyond the realm of science.
What is this AI you speak of?
For this post, I'm referring to a type of AI technology called Large Language Models (LLMs), which include notable examples like ChatGPT (short for Chat Generative Pre-trained Transformer) from openAI, a research lab that includes both non-profit and profit subsidiaries. Other examples include Google's BARD and Meta's LLaMA.
If this all sounds like gibberish to you, don't worry - I won't delve into the technical details (partly because I myself don't fully understand them).
In a nutshell, these models work like extremely sophisticated predictive text software. You input a prompt, and the model generates a response based on that prompt, allowing for a fluid dialogue. These models can do some pretty impressive things, and their capabilities are advancing faster than I can keep up with. Versions of this technology will rapidly start popping up all over the place, such as Microsoft Bing's new AI chat, powered by ChatGPT version 4. Some high profile experts have expressed concern about the speed at which this technology is progressing and have even called for a 6 month halt to allow safeguards to catch up.
While I can't fully explain the scope of this technology or predict its future, I will share my initial impressions from its basic use and discuss how I believe it can (and should?) be implemented by those working in academia.
What it can and can’t do
As an incredibly sophisticated predictive text system trained on nearly all of human knowledge (i.e., the internet), LLMs like ChatGPT are fully capable of generating entire essays for you. And when I say "entire," I mean it - the resulting essay would likely earn high grades on most high school or undergraduate assignments. Clearly, this is not an ideal situation for student assessment, and higher education institutions must come to grips with this technology quickly (although there are differing opinions on how to do so - and “AI detectors” have already sprung up).
However, the major issue with LLMs is that they have no way of knowing if their responses are accurate or even coherent. They confidently produce answers that are complete nonsense, particularly when prompted with certain types of questions. For example, if you ask for an explanation of how vaccines work at an undergraduate level, the response you receive will likely be phenomenal. But if you ask for examples randomized trials of PCR testing of respiratory viruses in children, the system will invent several trials that never actually occurred.
This phenomenon is known as AI hallucination, and it poses a significant challenge. Efforts are currently underway to address this issue and minimize the occurrence of hallucinations. For example, ChatGPT 3 is known for inventing academic references, but if using the latest version of Bing powered by ChatGPT 4 you get real references and it appears to provide semi-reliable results.
There are numerous other examples of seemingly simple things it can’t do, such as struggling with basic maths or drawing a circle, based on the limits of the way it is trained. That said, people are already working on inventive ways of overcoming these limitations, like plug ins that allow it to work with other software.
In my opinion there are 3 broad categories of basic use for an academic:
Editing and summarizing:
LLMs are highly effective at copy-editing, summarizing, and translating complex ideas into lay language. You can even specify the desired reading level, making it easy to clarify your writing, or even reduce word count to scrape under pesky limits. Additionally, the models can summarize existing papers or documents with ease. Just copy and paste the text into the model and ask for a summary in a specific level of language.
Generating momentum:
One of the most powerful use cases for LLMs is overcoming the problem of “blank page paralysis”, when starting a project or paper. By asking the model to provide ideas or an outline for a specific topic or teaching session, you can quickly generate content and ideas. Even if you don't end up using the suggestions provided, it can give you the jump start you need to get started.
Generating material:
Generating material for teaching, writing, or even quizzes and exam questions is another area where LLMs can be useful. However, it's important to be cautious as the model can confidently produce incorrect results. If you have domain knowledge and can verify the accuracy of the output, this can be a valuable tool. For example, asking for an outline for a specific part or chapter of your PhD thesis can be a helpful way to generate ideas.
The internet all over again
The internet has transformed our world by placing almost all of human knowledge at our fingertips. Many people have said that the state of academia (much like medicine) has changed, because simple memorisation of facts is now redundant - you can access almost any information you need in seconds online. The skill became in being an effective librarian, knowing:
- what you need to know
- where and how to access what you need to know, and importantly
- how to appraise the source of that information.
The transformation with tech like ChatGPT, is the middle step is redundant. Cut out scrolling through Google and simply ask the LLM. However, the first and last step are now even more important. Why? If anyone can find the “what”, the skill is in knowing the best “what” to look for, and in knowing how to verify the “what” you found. These things were always important, but a huge middle step has suddenly been made a whole lot easier.
There are of course a huge number of ways this tech might change the world. If your job involves any sort of coding, ChatGPT is now not only the worlds most effective programmer but most effective explainer of programming languages! As a capable amateur user of statistical programming software “R”, it has been a game changer for me. Hours of googling and scrolling through Stack Overflow have been swapped for a simple prompt in ChatGPT (if you have no idea what I’m talking about consider yourself lucky).
My intuition is these models will largely increase productivity by offloading routine tasks, and making some complex things like coding accessible to almost anyone. It also saves a lot of mental energy in it’s most basic generative abilities - even when it’s just creating sparks to get your own ideas burning.
In the same way that being able to master the internet can give an academic (or anyone really) an essential competitive advantage, it may well be that the mastery of LLMs will provide a similar advantage.
I’m fascinated to see how the technology advances over the next few months (and if it does get all “Skynet” then I for one welcome our new LLM overlords please have mercy).
Summary
LLMs like ChatGPT offer huge potential for many applications, including in academia. It can drastically improve productivity if used as a writing and editing aid. It has phenomenal generative capabilities which are most safe when used within the existing knowledge base of the user. Skills such as effectively using LLM prompts and checking the veracity of LLM outputs are going to be essential in the future for giving academics (and many others!) a competitive advantage. The sooner you get on board, the easier it will be to keep up.
Personally, I’m still working on the best ways of incorporating it into my work. I’d love to hear from you on how to make the most of this exciting new technology.
I am not an active academic, ( retired public health doctor). I have tried ChatGPT and and Bard. I gave them both 2 slightly whacky requests, (i) connections between French situationists and urban design, (ii) links and influences of the film ‘Whisky Galore’ and the Spanish film ‘ Bienvenido, Mister Marshall’.
Both gave competent answers. Mainly in your category 2, gaining momentum, filling a blank sheet. ChatGPT was more analytical, whereas Bard tended to produce more superficial flowing text. Both had errors of fact, ( characters and roles) but not major. I think Bard has a link to a plagiariser check. But game changing, yes. I will try something more in the maths/stats line.
Very interesting post. I like your open view of AI.
The challenge now is to learn how to use AI tools well.