SA Health Library Service FAQs

Answer

If you are thinking about using ChatGPT or other generative AI tools for your professional writing and research, it is important to consider the pros and cons. While this FAQ focuses on the use of ChatGPT, many of the same considerations are likely to apply to other similar tools. 

ChatGPT can assist in a number of areas including:

• Identifying potential resources, such as books and websites.
• Brainstorming or prompting e.g. “I am researching the use of xxxx in yyyy, what types of things should I consider?”
• Explaining concepts, topics or jargon with which you are not familiar.
• Proofreading and editing your work, providing it does not contain sensitive, private or confidential information.

There are an increasing number of articles evaluating or exploring the use of ChatGPT in medicine and healthcare, including in areas such as:

• Compiling medical notes and patient discharge summaries
• Writing content for publication, including scientific articles or sections of scientific articles
• Summarising or writing plain language summaries for complex medical topics (ref 4, ref 5)
   

Limitations and considerations

While a full discussion of the merits and risks of ChatGPT falls outside the scope of this FAQ, there are several key limitations and considerations that should be kept in mind. Chief among them is: How do you know the information is correct or legitimate? ChatGPT bases its response on the sources on which it has been trained. The response to your prompt or question may be incomplete, biased, inaccurate or even nonsensical. It will also only be as good as the prompt that you provide.

The tool is not capable of critical thinking: it will provide you with a human-like response based on patterns learned from the information it has been trained on, and does not review the output it provides for accuracy or truthfulness. Care should be taken when it comes to assessing responses incorporating references, as the references supplied may be incomplete or even fake (ref 1).

ChatGPT is not a search engine, and its ‘training’ is mostly on data from pre-2021, making it less useful for rapidly changing or highly specific areas of research or policy. The source material used to train ChatGPT is unlikely to include the full-text of pay-walled journal articles (ref 1, ref 2).

Security and privacy are key considerations when it comes to the use of ChatGPT. The tool should not be used for, or with, any type of sensitive, private or confidential information. There are also ethical, copyright and ownership considerations when it comes to using content generated by ChatGPT, particularly when it isn’t clearly acknowledged (ref 3).

General recommendations if using ChatGPT

1. Critically review the information it supplies
   ChatGPT is only as good as the source material from on which it was trained. It does not fact-check or verify the accuracy of the information it provides and may at times provide plausible-sounding but incorrect or nonsensical answers. Pre-existing issues with the data used as part of its training, such as bias, inaccuracy or incompleteness, will be reflected in its responses. You need to bring your own critical thinking and analysis skills to any content generated by ChatGPT.
    
2. Acknowledge and be transparent about usage
   If you decide to use ChatGPT for idea or text generation, it is important to acknowledge that you have used the tool. Check the recommended style guide for your discipline or publication for recommendations on how to reference.
    
3. Use it as a tool, not a replacement for your own thought and writing
   It is important that your work is just that: yours.
    
4. Ensure you are abiding by organisational policies
   The use of ChatGPT and other artificial intelligence tools is a rapidly changing space. Always check organisational policies or confirm with your line manager to ensure you are abiding by organisational requirements, particularly around confidentiality, privacy and sensitive data. If you are seeking to publish your research, check with your preferred publication ahead of time to see whether they have requirements around disclosing or using ChatGPT.

References

1. Hillier, M. (2023) Why does ChatGPT generate fake references?
2. OpenAI. (2023). What is ChatGPT?
3. Flinders University. (2023). Using Generative AI (ChatGPT, BARD etc.)
4. Sallam M. (2023). ChatGPT Utility in Healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare, 11(6), 887. https://doi.org/10.3390/healthcare11060887
5. Cascella, M. et.al. (2023). Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. Journal of medical systems, 47(1), 33. https://doi.org/10.1007/s10916-023-01925-4