Despite its benefits, it is essential to understand the potential limitations and ethical concerns surrounding the use of AI in education, and its broader societal impacts.
A common issue in AI models is that they can generate content that is false, nonexistent, or misleading (sometimes referred to as "hallucinations"). Always double-check the accuracy of AI-generated content; see the page on evaluating AI content for tips. If you need up-to-date or factual information, use a reliable library database or news site.
Because AI models are trained on large datasets, they can potentially reproduce and amplify biases that exist within the data. This can lead to the generation of content that reinforces racial, gender, disability, and other biases present in information on the internet. AI models are not neutral, and are skewed toward a Western perspective, and are trained primarily in English.
Gen AI tools do not "think" or understand meaning as humans do; they generate responses based on patterns in data rather than true comprehension or reasoning. AI models will struggle with complex social interactions and nuanced cultural understanding.
Although many generative AI tools are currently free, many companies require a subscription or payment to access them or to use premium features. This creates barriers for those who are unable to afford access.
AI models are trained on vast amounts of data that may include copyrighted materials. There are ongoing debates (and lawsuits) questioning whether AI-generated content should receive copyright protection, and if authors, whose work is being used to train AI models, should be acknowledged and/or compensated.
AI models use prompts and responses to train their machines, so they are neither confidential nor secure. Users may be able to opt-out of data retention, depending on the AI tool. Gemini and NotebookLM, that are a part of the University's Google Workspace, does not use your personal data for training. Consensus, an AI tool for research that is available through St. Kate's, is also committed to not using your data.
The significant energy consumption required for training and deploying AI systems raises concerns about the environmental impact and sustainability of generative AI technologies. This often falls disproportionately on socioeconomically disadvantaged regions throughout the world. While there is an ongoing debate about whether AI can help or harm the environment, that there will be an net increase in energy consumption.
Worker exploitation and the potential for job displacement due to automation by generative AI is an issue, especially in industries where AI can replace human labor.
©2024 St. Catherine University Library, St. Paul, Minnesota, USA
