← Back
Selective Ru(II)/benzoate complexes against triple-negative breast tumor cells and their interactions with DNA and BSA
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
DOI: https://doi.org/10.48009/1_iis_116
Exploring artificial intelligence literacy and engagement in higher
education stakeholders
Christopher P. Daniels, Middle Georgia State University, christopher.paul.daniels@gmail.com
Abstract
Artificial intelligence is disrupting higher education both officially and unofficially. This study aims to
determine higher education stakeholders (students, faculty, and staff), AI Engagement (seeking out
information and utilizing AI tools), and AI Literacy (the foundational skills and knowledge required to
evaluate, use, and create with AI effectively). This study makes a significant contribution to the literature
by providing the perspectives of staff, an underrepresented population in current literature. Consideration
is given to differences in these measures in relation to stakeholders' optimism about AI in higher
education, as well as their gender and generational differences among respondents. The 19-question
survey instrument was administered electronically to students, faculty, and staff affiliated with a higher
education institution in the Southeastern United States. Overall, stakeholders tended to have neutral to
slight agreement on their preparedness with AI literacy skills, but neutral to slightly disagree that they
were frequently engaged in AI usage or discourse. The data shows a moderate positive correlation
between AI Literacy and AI Engagement scores for higher education stakeholders. Those with positive
attitudes toward AI’s effect on higher education are associated with higher AI Literacy and AI
Engagement. Staff and other group difference results are presented alongside recommendations for future
research.
Keywords: artificial intelligence, literacy, engagement, staff, faculty, students, higher education
Introduction
OpenAI provided free access to ChatGPT and DALL-E in 2022 (Introducing ChatGPT, 2022), creating a
revived interest in the capability of AI. By utilizing Natural Language Processing (NLP), OpenAI
democratized access to AI that understands users' natural dialogue inputs and generates unique personalized
text and image responses. Following in their footsteps, dozens of generative AI tools emerged overnight.
While these tools are rapidly adopted in business applications, higher education institutions, whose job it is
to prepare students for work and societal contributions (R. Y. Chan, 2016) have had mixed responses. Some
higher education institutions initially responded by outright banning the use of the tools, citing concerns
about academic integrity (Farrelly & Baker, 2023). In contrast, others encouraged their use because of the
opportunity they provided for student learning (Sullivan et al., 2023). Now, nearly three years after the
initial disruptive event, administrators and researchers must continue to monitor stakeholders’ efforts
directed toward learning and using AI as they grapple with the impact it makes on their institutional
processes (Ma et al., 2024).
201
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
The promise of AI tools is, for better or worse, disrupting learning and teaching practices worldwide as
tools and applications are adopted officially and unofficially (Kelly et al., 2023). The result is transforming
teaching, administrative, and learning practices. AI tools, such as adaptive learning management systems
that promise to personalize learning experiences by adapting to individual learner needs, are marketed to
institutions (Kabudi et al., 2021). Tools marketed as AI personal assistants are available to be adopted by
students, faculty, and staff who feel pressed for capacity (Rillig & Kasirzadeh, 2024).
Without training, these tools present institutional security risks from data leaks of well-intended users as
well as concerns around academic integrity, authenticity, and potential ramifications to cognitive and
creative skills (Francis et al., 2025; Janse Van Rensburg, 2024). A new form of digital literacy, AI Literacy,
is required for the ethical and responsible use of these technologies (Chiu et al., 2024; Knoth et al., 2024).
The ethical and responsible use of AI in higher education demands the attention of those in positions to
effect change. Administrators must tune into the AI appetite and skills possessed by internal stakeholders.
While many other studies have surveyed the AI Engagement and AI Literacy of students and faculty, staff
perceptions are underrepresented in current literature (Ren & Wu, 2025). While some research is beginning
to study the perspectives of instructional designers (Luo et al., 2024) the continued inclusion of staff insights
are crucial since they underpin the operations and governance of AI in higher education (C. K. Y. Chan,
2023).
This study aims to gain a deeper understanding of the perception, utilization, and knowledge of AI in higher
education. The study surveys internal stakeholders at a higher education institution in the Southeastern
United States about their habits of engaging in discourse and practice with AI tools (AI Engagement) and
their foundational skills and knowledge required to evaluate, use, and create with AI effectively (AI
Literacy). In assessing these measures, stakeholders' optimism regarding the potential of artificial
intelligence in higher education, as well as the respondents’ role, gender, and age group, are examined for
differences.
The study results can inform institutional policy and academic guidance while uncovering opportunities for
professional development and classroom instruction. Studying AI Engagement and AI Literacy among
higher education stakeholders ensures that institutions can effectively harness AI’s potential while
mitigating its risks.
Research Questions
Consistent with the purpose of gaining a deeper understanding of the perception, utilization, and knowledge
of AI in higher education, the following research questions are asked:
RQ 1: What is the level of AI Engagement among higher education stakeholders?
RQ 2: What is the level of AI Literacy among higher education stakeholders?
RQ 3: Are there significant correlations between AI Engagement and AI literacy?
RQ 4: Are there significant mean differences between the levels of the independent variable, a)
role, b) age group, c) gender, and d) AI optimism, and the dependent variable, a) AI
Engagement and b) AI literacy?
RQ 5: Are higher education stakeholders optimistic about the impact of AI on higher education,
and are there significant associations between the stakeholders' roles?
202
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
Literature Review
An examination of the literature reveals that academics are researching, discussing, and reporting at an
astonishing rate to keep pace with the rapid advancement of AI innovation and its widespread adoption.
The recent disruption of AI is demonstrated by the exponential rise of peer-reviewed articles on the subject
that are available in GALILEO, Georgia’s online library, which hosts over 100 databases indexing
thousands of journals (Board of Regents of the University System of Georgia, 2024). While AI has been
researched since the 1950s, the number of peer-reviewed articles on AI has more than doubled (N = 598,312,
50.8%) since the release of OpenAI software in 2022 to the researching for this paper in 2024. The
exponential growth in the availability of peer-reviewed publications is joined by numerous professional
development opportunities, including conferences, communities of practice, and workshops, which provide
participants with opportunities to learn more about this disruptive technology.
AI Engagement
While a plethora of information on AI is available, this study aims to discover the extent to which higher
education stakeholders are engaged with AI. While previous studies have explored the usage of AI among
stakeholders, this study conceptualizes a broader picture of engagement as an active discovery and
application of knowledge (Duderstadt, 2003). By applying this definition in context, AI Engagement refers
to active participation in the discourse around AI and interaction with AI technologies. This includes using
AI tools and platforms in various contexts, such as education, work, and daily life. AI Engagement pairs
hands-on experience with AI applications with the consumption of information about AI from news, social
media, professional development, and peer conversations. Research suggests that information acquisition,
dissemination, and use have positive associations with attitude and task performance (Kunst et al., 2018).
Assessing AI Engagement quantifies participants' utilization and involvement in the discourse surrounding
AI, providing a more comprehensive measure of stakeholder interest and use.
Researchers have studied students' and faculty members' knowledge acquisition and use of AI tools, but a
significant gap remains concerning staff engagement. A chronological examination of studies on students'
use of AI tools hints that engagement is on the rise. An Australian study from March 2023 reported that
41% of the student respondents knew little to nothing about AI tools such as ChatGPT, and less than 50%
of the students who reported having any knowledge had never utilized AI tools (Kelly et al., 2023). A month
later, in April 2023, a study of Hong Kong undergraduate and graduate students showed that 33.3% of the
respondents had never used GenAI technology (C. K. Y. Chan & Hu, 2023). The number of students who
are inexperienced with AI continues to decline. According to a recent study, ChatGPT was used for
academic purposes by all 499 students surveyed (Acosta-Enriquez et al., 2024).
Similarly, research indicates that faculty members are becoming increasingly familiar with AI and using AI
tools in the classroom. In a sizeable Bulgarian study of 2252 teachers, 72% reported some familiarity with
AI technology, while 51% were actively using it in their teaching. This study found four factors that showed
a significant association with AI training attendance. Females, rurally located, with connections to STEM
or high usage, were more receptive to AI training than their male, urban, humanities, and low usage
counterparts (Kurshumova, 2024).
While faculty remain skeptical, students reported optimism about integrating AI tools into their academic
and professional lives, but are apprehensive about accuracy, transparency, privacy, and ethics (C. K. Y. Chan
& Hu, 2023). Kelly et al. (2024) reported that usage rates correlated with increased confidence in using AI
ethically. Researchers such as Lee et al. (2024) emphasize the importance of involving students in the
communities of practice surrounding AI and engaging in discussions about AI. However, despite the ability
for AI to transform higher education business practices, staff perspectives are underrepresented in research.
203
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
AI Literacy
The ability to use and evaluate applications that use AI is critical due to its disruptive force. Chiu et al.
(2024) found unanimous agreement among teachers that AI Literacy education must include engaged
experiences. However, current research shows that only a minority of students have been exposed to an AIspecific lecture, indicating a need to establish AI education for all students, including informal learning
spaces (Hornberger et al., 2023). Research has shown that individuals with AI literacy can critically evaluate
AI systems, understand their impact on society, and use AI tools effectively (Ren & Wu, 2025). AI literacy
encompasses not only technical skills but also ethical considerations and an understanding of AI’s potential
benefits and risks. This study will examine higher educational stakeholder AI Literacy, or the foundational
skills and knowledge required to evaluate, utilize, and create with AI effectively (Berry, 2023).
As these tools are officially and unofficially integrated into the educational process, higher education
stakeholders must be equipped with AI literacy. Long and Magerko (2020) provide a definition for AI
literacy as “a set of competencies that enables individuals to evaluate AI technologies critically;
communicate and collaborate effectively with AI; and use AI as a tool online, at home, and in the
workplace” (p. 2). Assessing AI Literacy enables researchers to quantify progress toward foundational skills
for understanding, utilizing, and critically evaluating AI systems and their outputs (Lintner, 2024). This data
is valuable for HEI institutional administrators who guide communities of practice and professional
development opportunities.
Research has shown that students with higher levels of AI literacy will have a better understanding of
technological capabilities and limitations. Ongoing initiatives to improve AI literacy can demystify the tool
and provide a higher level of satisfaction and creativity (Al-Abdullatif & Alsubaie, 2024). Recent research
has shown moderately high AI literacy rates among HEI stakeholders (Asio, 2024). AI literacy research
informs the training needs of users to develop the competencies necessary for detecting the use of AI,
critically evaluating AI outputs, engaging in practical applications of AI, and understanding its limitations.
Researchers have crafted multiple scales to assess AI literacy. Laupichler et al. (2023) explored the
development of a Scale for Assessing the AI Literacy of non-experts (SNAIL). Through a review of research
and SME discussions, the researcher developed a pool of 47 items. The final 31-item questionnaire is
categorized into three subfactors influencing AI literacy: technical understanding, critical appraisal, and
practical application. Another measure, the AI literacy Test, consists of 30 multiple-choice questions that
provide objective literacy scores (Hornberger et al., 2023). However, because of the rapid development of
AI tools, these objective questions will be constantly under scrutiny (Lintner, 2024). For this study, Wang
et al. (2022) AI Literacy Scale (AILS) was selected for its “robust quality evidence” for validity, reliability,
and consistency (Lintner, 2024, p. 7). AILS self-report items over four dimensions: awareness, usage,
evaluation, and ethics.
Presentation of the Contributions of the Research Questions
This study will examine the two dependent variables, AI Literacy and AI Engagement, to identify
significant relationships. Exploring these relationships can provide insights into how exposure to AI can
enhance literacy. For administrators, evaluating these factors offers an understanding of stakeholders' needs
and desires, which can inform decisions on policy and training while driving innovation.
Additionally, this study analyzes the data based on independent variables: gender, age, and educational role
(student, faculty, staff). It will also investigate group differences based on respondents' self-classification
of AI optimism, which is the belief that, considering all risks and benefits, AI will yield more positive than
negative outcomes when implemented in higher education. Understanding the contributions of independent
variables may highlight differences that can shape strategies for improving AI engagement and literacy
training.
204
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
Finally, this research will examine the relationships between each internal stakeholder role in higher
education and their perceptions of whether AI will be more beneficial than detrimental. The research will
significantly contribute to our understanding of differences between stakeholders’ perceptions of AI
benefits vs detriments. Specifically, it enriches the literature by including the perspective of staff alongside
of students and faculty for a more comprehensive stakeholder view.
Examining AI Engagement and AI Literacy among higher education institution stakeholders is vital for
making informed decisions about resource allocation and policy recommendations. Grasping these
measures is crucial for ensuring that all higher education stakeholders can fully leverage AI's potential while
minimizing risks such as information bias, hallucinations, and concerns related to academic integrity. These
insights will guide the adoption of AI in higher education, help target training efforts, and permit the
evaluation of the effectiveness of information dissemination about AI in higher education.
Methodology
Sample
Following approval by the Institutional Review Board, the survey was distributed via institutional email
listservs to students (n = 3,435), faculty (n = 205), and staff (n = 297). The sample participants are from a
state college in the southeastern United States, where the author is employed. This convenience sample
population was selected because of the practical application of the research and its availability. All
participants were presented with a disclosure statement of informed consent, assured of the confidentiality
and anonymity of their responses, and were 18 years of age or older. Participation was not required by
institutional administration. One follow-up invitation to participate was sent one week after the initial email.
Responses were collected for two weeks.
Instrumentation
The instrument distributed was composed of 19 questions (available on request). The first section assesses
both dependent variables, AI Engagement and AI literacy. It includes five statements adapted from Kunst
et al. (2018) to obtain respondents' AI Engagement and ten statements adapted from Wang et al. (2023)
Artificial Intelligence Literacy Scale (AILS). All statements ask participants to rate their level of agreement
with the statements using a five-point Likert scale, where 1 = Disagree, 2 = Somewhat Disagree, 3 =
Neutral, 4 = Somewhat Agree, and 5 = Agree. Four additional questions assess the participant's independent
variables categories, including Age (1=18-24, 2=25-34, 3=35-44, 4=45-54, 5=55-64, 6=65+), Gender
(1=Male, 2=Female, 3=Other), HEI Stakeholder Role (1=Student, 2=Instructor, 3=Staff), and AI Optimism.
Participants' AI Optimism is categorized based on a single five-point Likert scale response to the belief that
GenAI will produce more good than bad in higher education, considering both the risks and benefits (1 =
Disagree, 2 = Somewhat Disagree, 3 = Neutral, 4 = Somewhat Agree, 5 = Agree). To increase response
validity, the Likert response questions were randomized.
Procedures
Data was collected for two weeks with a follow-up invitation to participate at the start of the second week.
Descriptive statistics and statistical analytical methods were employed to answer research questions one
and two. Additionally, research question three is assessed by calculating a correlation to measure the linear
relationship between AI Engagement and AI Literacy (Knapp, 2018).
The data will be subjected to group variance analysis to test the significance of group differences between
one independent variable and one continuous dependent variable, allowing exploration of research question
four. The independent grouping variables are Age, Gender, HEI Stakeholder Role, and AI Optimism. The
205
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
dependent variables include respondents' AI Information Acquisition Orientation and AI Literacy. The most
common variance analysis technique is the one-way analysis of variance, often abbreviated as ANOVA
(Knapp, 2018). For statistical tests such as ANOVA to provide robust results, the data must meet the criteria
of normality, size, and homogeneity of variance. A nonparametric test will be utilized where significant
deviations from these assumptions occur. For analysis where there are more than two groups, a post hoc
test will be conducted (Koohang et al., 2024).
To explore research question five, the association between two categorical variables, HEI Stakeholder Role
and AI Optimism, the researcher utilized a chi-square test for independence. This robust measure allows
the comparison of two categorical variables, even with unbalanced sample sizes, as long as each cell in the
analysis has five or more values (Knapp, 2018). For all statistical tests, an alpha value of p = .05 will be
used.
Results
Survey data were collected from 251 participants. Forty-four (44) results were excluded due to incomplete
responses or declining to participate after being presented with an informed consent statement. The
participants' demographic frequencies are presented in Table 1 for the studied independent variables: role,
gender, age range, and optimism toward AI in higher education. The response rate for the survey varied,
with faculty being the most willing to participate (30.24%), followed by staff (17.17%), and only 2.73% of
the student population returned a completed survey.
Literacy and Engagement
Regarding research question one, to assess the level of AI literacy among higher education stakeholders,
participants were asked to respond to ten statements about their ability to detect, evaluate, utilize, and
comprehend the limitations of AI. Responses to these questions were averaged to compute participants' AI
Literacy score (M = 3.33, SD = 0.82, Skewness = -0.46, Kurtosis = 0.02). Overall, participants generally
fell between ‘neither agree nor disagree’ and ‘somewhat agree’ that they were prepared with AI literacy
skills.
Regarding research question two, AI Engagement scores were assessed based on participants' responses to
five statements about their engagement with AI, including use, discussion with peers, and information
consumption. Responses to these questions were averaged to compute participants' AI Engagement score
(M = 2.64, SD = 0.82, Skewness = 0.10, Kurtosis = -0.68). Overall, participants fell between ‘somewhat
disagree’ and ‘neither agree nor disagree’ when asked if they frequently engaged in AI usage or discourse.
To answer research question three, a Pearson correlation coefficient test was used to examine the strength
and direction of the relationship between AI Engagement and AI Literacy. A significant correlation was
found between the two dependent components, AI literacy and AI Engagement, r (207) = .52, p < .001. The
data shows a moderate positive correlation between AI Literacy and AI Engagement scores for higher
education stakeholders.
Table 1. Frequency, Means, and Standard Deviation
Variable
All
Role
Group
Faculty
Staff
Student
N
%
207
62
51
94
100
30.0
24.6
45.4
206
AI Literacy
Std.
Mean
Deviation
3.33
.82
3.28
0.83
3.04
0.87
3.52
0.75
AI Engagement
Std.
Mean
Deviation
2.64
.82
2.88
0.85
2.56
0.90
2.54
0.79
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
Variable
Group
N
%
Age
18-24
25-34
35-44
45-54
55-64
65 or older
Male
Female
Non-binary/ Third Gender
Strongly Disagree
Somewhat Disagree
Neither agree nor disagree
Somewhat agree
Strongly agree
83
21
30
32
32
9
77
127
3
46
42
49
37
33
40.1
10.1
14.5
15.5
15.5
4.3
37.2
61.4
1.4
22.2
20.3
23.7
17.9
15.9
Gender
Optimism
AI Literacy
Std.
Mean
Deviation
AI Engagement
Mean
Std.
Deviation
3.52
0.73
2.50
0.81
3.46
3.34
3.33
2.83
3.09
3.32
3.36
2.50
2.97
3.19
3.23
3.50
3.98
0.79
0.76
0.97
0.83
0.72
0.79
0.85
0.17
0.75
0.77
0.75
0.68
0.88
2.84
2.71
3.13
2.42
2.42
2.76
2.61
1.33
2.24
2.37
2.66
2.85
3.30
0.80
0.88
0.91
0.75
0.67
0.90
0.79
0.23
0.68
0.67
0.90
0.68
0.91
Group Variances
Regarding research question four, to assess significant mean differences between the levels of the
independent variables, responses were grouped by the reported independent variables to compare calculated
scores for AI Literacy and AI Engagement. To obtain robust results with ANOVA, the data must meet the
criteria of normality, sample size, and homogeneity of variance. A Shapiro-Wilk test was conducted to
measure the normality of the distribution of the dependent variable value for each category, and the results
are reported in Table 2. In some instances, these tests confirmed deviations from normal distributions.
However, Knapp (2018) suggests examining group frequency histogram graphs to determine if the use of
nonparametric tests is more appropriate due to extreme deviations in normal distributions. Knapp’s
recommendation is in line with the findings of Blanca et al., (2017) who demonstrated the validity of the
F-test in various non-normality conditions and suggested examining graphical representations of data to
aid in interpreting the results. Therefore, unless noted, the deviations from normality were determined not
to affect AVOVA results.
Role
Responses were grouped by reported role to compare calculated scores for AI Literacy and AI Engagement.
The results of the ANOVA indicated significant group differences for the independent variable of
stakeholder Role and the dependent variables of AI Literacy, F(2, 204) = 6.03, p < .003, and AI Engagement,
F(2, 204) = 3.55, p = .03. Levene’s test for homogeneity of variance failed to reject the null hypothesis for
AI Literacy and AI Engagement, which was p = .25 and p = .36, respectively, indicating the variances are
likely similar. Tukey post hoc tests reveal significant differences in AI Literacy and AI Engagement, as
shown in Table 3. Staff report significantly less AI literacy than students, while Faculty are significantly
more engaged with AI than Students.
Table 2. Shapiro-Wilk Normality Test for each Independent Variable Group
Role
Faculty
Staff
Student
df
AI Literacy
Shapiro-Wilks
Sig.
AI Engagement
Shapiro-Wilks
Sig.
62
51
94
0.98
0.96
0.97
0.97
0.95
0.98
.57
.06
.04
207
.16
.048
.08
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
AI Engagement
AI Literacy
df
Age
18-24
25-34
35-44
45-54
55-64
65+
Gender
Male
Female
AI Optimism
Strongly disagree
Somewhat disagree
Neither agree/ disagree
Somewhat agree
Strongly agree
df
83
21
30
32
32
9
Shapiro-Wilks
Shapiro-Wilks
0.97
0.93
0.98
0.96
0.94
0.91
Sig.
Sig.
.10
.16
.89
.32
.09
.34
Shapiro-Wilks
Shapiro-Wilks
0.97
0.95
0.96
0.96
0.96
0.95
Sig.
Sig.
.08
.33
.32
.30
.27
.71
77
127
0.99
0.96
.65
< .01
0.96
0.98
.02
.11
46
42
49
37
33
0.95
0.97
0.97
0.90
0.87
.04
.37
.29
< .01
< .01
0.96
0.98
0.96
0.91
0.94
.09
.54
.06
< .01
.07
Table 3. Multiple Comparisons – Role
Mean
Difference
0.24
Literacy
Std.
Error
0.15
.25
Mean
Difference
0.32
Engagement
Std.
Error
0.16
Sig.
Sig.
Faculty
Staff
0.1
Staff
Student
-0.48
0.14
.002
0.02
0.15
.99
Student
Faculty
0.24
0.13
.17
-0.34
0.14
.03
Age
Responses were grouped by reported age to compare calculated scores for AI Literacy and AI Engagement.
The results of the ANOVA indicated significant group differences for the independent variable of Age and
the dependent variables of AI Literacy, F(5, 201) = 3.74, p = .003, and AI Engagement, F(5, 201) = 3.60,
p = .004. Levene’s test for homogeneity of variance failed to reject the null hypothesis for AI Literacy and
AI Engagement, with p-values of .36 and .35, respectively, indicating that the variances are likely similar.
Tukey post hoc tests show significant differences in AI Literacy between 18-24 year-olds and 55-64 yearolds (p < .001) and AI Engagement between 18-24 year-olds and 45-54 year-olds (p = .004), and 45-54
year-olds and 55-64 year-olds (p = .009). Younger stakeholders reported higher AI literacy, while those
aged 45-54 reported higher AI Engagement.
Table 4. Multiple Comparisons – Age
Literacy
Engagement
Mean
Difference
Std.
Error
Sig.
Mean
Difference
Std.
Error
Sig.
18-24
25-34
0.06
0.20
1.00
-0.34
0.20
.54
18-24
35-44
0.18
0.17
.89
-0.21
0.17
.84
18-24
45-54
0.19
0.17
.86
-0.63
0.17
.004
18-24
55-64
0.69
0.17
.001
0.08
0.17
1.00
208
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
18-24
65+
0.43
0.28
.64
0.08
0.29
1.00
25-34
35-44
0.12
0.23
.99
0.13
0.23
.99
25-34
45-54
0.13
0.22
.99
-0.29
0.23
.81
25-34
55-64
0.63
0.22
.06
0.42
0.23
.46
25-34
65+
0.37
0.32
.86
0.42
0.33
.80
35-44
45-54
0.01
0.20
1.00
-0.42
0.21
.34
35-44
55-64
0.51
0.20
.13
0.29
0.21
.74
35-44
65+
0.25
0.30
.96
0.28
0.31
.94
45-54
55-64
0.50
0.20
.13
0.71
0.21
.009
45-54
65+
0.24
0.30
.97
0.70
0.31
.21
55-64
65+
-0.26
0.30
.95
0.00
0.31
1.00
Gender
Responses were grouped by reported gender to compare calculated scores for AI Literacy and AI
Engagement. Responses indicating “Non-binary/Third Gender” were excluded from calculations due to an
inadequate sample size (Knapp, 2018). As shown in Appendix B, the Shapiro-Wilk scores for Male AI
Literacy and Female AI Engagement differ significantly from the normal distribution. Additionally,
Levene’s test for homogeneity of variance failed to reject the null hypothesis for AI Literacy (p = .62),
indicating the variances are likely similar. However, for AI Engagement, significant results (p = .06)
indicate the variances between optimism groups for AI Engagement are significantly dissimilar. A nonparametric test was chosen as a suitable alternative, and since there are only two categories, a MannWhitney U test was selected and performed. The test results failed to reject the null hypothesis of gender
differences for the dependent variables AI Literacy (z = 5253, p = .37) and AI Engagement (z = 4354, p =
.19). Male and Female AI Literacy and AI Engagement scores did not differ significantly.
Optimism
Responses were grouped by reported optimism to compare calculated scores for AI Literacy and AI
Engagement. Levene’s test for homogeneity of variance failed to reject the null hypothesis for AI Literacy
(p = .61), indicating the variances are likely similar. However, for AI Engagement, significant results (p =
.013) indicate the variances between optimism groups for AI Engagement are significantly dissimilar. As a
result of these deviations from ANOVA assumptions, the Kruskal-Wallis non-parametric test was selected
for comparing group scores. A Kruskal-Wallis H test revealed a significant difference in the dependent
variable AI Literacy, H(4) = 35.41, p < .001, and AI Engagement, H(4) = 35.35, p < .001, between the five
optimism groups.
Pairwise comparison, adjusted by the Bonferroni correction for multiple tests, show higher AI Literacy
scores for those who “strongly agree” than those who “neither agree nor disagree (p < .001),” “somewhat
disagree (p < .001),” and “strongly disagree (p < .001).” “Somewhat agree” scores for AI literacy
distributions are significantly higher than those who “strongly disagree (p = .025).” For AI Engagement,
participants who responded, “strongly agree” to optimistic outlooks of AI in higher education had
significantly higher score distributions than those who responded, “strongly disagree (p < .001),”
“somewhat disagree (p = .004),” and “neither agree nor disagree (p = .027).” Participants who responded
with “somewhat agree” scores distributions were higher in AI Engagement than those who responded with
“strongly disagree (p = .004)” or “somewhat disagree (p = .04).”
209
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
Stakeholder Role and Optimism
To answer research question five, a chi-square test of independence was performed to evaluate the
relationship between the participant's role and reported AI Optimism. The relationship between these
variables is significant: χ²(8, 207) = 18.25, p = .02. Post hoc comparisons revealed that faculty reported
more ‘neither agree nor disagree’ responses and fewer ‘somewhat agree’ responses than students did for AI
optimism scores. Staff reported fewer ‘strongly disagree’ responses and more ‘somewhat disagree’
responses than did students for AI optimism. This suggests that students a largely optimistic about AI in
higher education, whereas faculty tend to hold neutral opinions, and staff a more often skeptical.
Discussion
This study sets out to examine the AI Engagement and AI Literacy of higher education stakeholders. The
study aimed to contribute to the understanding of students', faculty's, and staff's perceptions of their abilities
and current experiences. Overall, the calculated means and distributions indicate that stakeholders, although
showing a slight agreement, were neither in agreement nor disagreed that they were prepared with AI
literacy skills or the ability to evaluate AI systems critically, understand their impact on society, and use AI
tools effectively. These results align with previous research, which has reported similar mean scores (Asio,
2024; Wood et al., 2021). Additionally, higher education stakeholders were neutral but slightly tended to
disagree that they were frequently engaged with AI. This finding could reflect the uncertainty of AI’s future
role in education, as discussed in K-12 AI research on faculty perceptions by Velander et al. (2024). Further
study should be conducted to determine differences in stakeholders' lack of confidence in the tools or the
uncertainty of policy requirements.
The data shows a moderate positive correlation between AI Literacy and AI Engagement scores for higher
education stakeholders. These findings align with a growing body of research that suggests that individuals
with higher AI Literacy are more engaged with using or learning about AI. Long & Magerko (2020)
indicated that individuals with formal AI training were more confident in interactions with AI systems.
Higher education institutions aiming to cultivate a more AI-savvy academic community should utilize
communities of practice for faculty and staff training (Hur, 2025). Course designs that emphasize
engagement and literacy instruction would prepare students for the future workforce (Cruz, 2024; Cullen
& Kirkpatrick, 2024; Ng et al., 2024). Based on the findings, increased engagement with AI tools might
lead to enhanced literacy skills. Higher education institutions that wish to cultivate a more AI-savvy
academic community could benefit from workshops to engage their stakeholders with the tools.
Differences in stakeholder roles for higher education were found concerning AI Literacy. Staff scored
significantly lower than students, indicating that they feel unprepared to detect, use, and evaluate AI tools.
While these results could indicate that students overestimate their AI literacy, they could also represent a
lack of staff training opportunities and transparent policies regarding the use of AI in higher education
administration. (Moorhouse et al., 2023). Institutions could benefit from engaging with students while
crafting policy and training to enhance institutional understanding of these new tools. Additionally, this
study found that Faculty report significantly higher AI Engagement scores than students; however, student
scores did not differ from Staff AI Engagement scores. Utilizing faculty to mentor staff and students by
providing introductory workshops and training opportunities could build a collaborative relationship
between stakeholder groups. Additional value could be provided by framing these workshops based on a
defined institutional policy.
Younger respondents in the 18-24 age range reported higher levels of literacy than those in the 55-64 age
range, suggesting that younger individuals are more confident in using and evaluating AI tools. Research
210
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
shows that ‘digital natives’ have more exposure to AI technology through education and daily life (Brown
& Lewis, 2022). Participants aged 45-54 had the highest AI engagement scores, indicating that these
stakeholders are more cognitively involved in the AI discourse. This finding aligns with historical findings
of Generation X as early adopters who came of age as technology matured (Roddy, 2024). Some researchers
found that these ‘digital immigrants’ confidently adopt emerging technology (C. K. Y. Chan & Lee, 2023).
While other studies have found significant differences in AI literacy and engagement, this study did not
find any differences between male and female participants. These findings suggest an inclusive environment
with a balanced perspective and equal access to opportunities for engagement and learning about AI.
Participants who expressed stronger optimism about AI’s potential effects on higher education reported
significantly higher AI literacy and AI Engagement scores, suggesting that their positive outlook is related
to their preparedness to use and understand AI tools. These findings are in line with research on the
technology acceptance model, which indicates that perceived usefulness and ease of use influence adoption
(Wang et al., 2024). Future AI literacy training should identify and address the concerns of participants who
are less optimistic.
The participant's role was found to have significant associations with AI Optimism. Students were more
optimistic, while faculty were more likely to report neutral opinions, and staff were more likely to be
skeptical about AI in higher education. This significant finding underscores the importance of future
research on the perspectives of higher education staff.
Limitations and Future Research
This study gathered a significant number of participants. However, the low response rate from students
raises concerns about the generalizability of the sample. Low response rates might have introduced nonresponse bias, meaning the opinions and experiences of non-respondents could differ.
Additionally, the study relies on self-reported data. Respondents' perceptions of their ability to analyze,
evaluate, use, and engage with AI may not reflect their actual skill level. In addition, stakeholders may
engage with embedded AI features more frequently than reported due to technology normalization, as
discussed by Bax, as the point when technology becomes invisible and taken for granted in everyday life
(Zimotti et al., 2024). Future research should evaluate stakeholders using objective tools to measure AI
Literacy and AI Engagement. Practical assessments or observational studies would be a valuable addition
to self-report surveys, providing a more accurate picture of stakeholders’ skills and engagement.
Oher research strategies such as incorporating qualitative methods such as interviews and focus groups
could aid in the exploration of factors contributing to these results. Studies of this structure would benefit
from analysis of individual question differences. This analysis could uncover specific aspects of AI literacy
and Engagement with the greatest ity between stakeholder groups.
Finally, future research could adopt a longitudinal research design to track changes in AI Literacy and
engagement over time. Although unattainable within the current study's limitations, longitudinal research
would provide valuable insights into how exposure to AI tools and educational interventions affects
engagement, optimism, and literacy. This would be particularly useful for understanding the long-term
impact of AI integration in education.
By addressing these limitations and exploring these future research opportunities, scholars can gain a deeper
understanding of how to foster effective engagement with AI in higher education and promote AI literacy
211
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
across diverse academic communities. Continued granular analysis is needed to pinpoint areas where
targeted interventions and tailored training opportunities would be most effective.
Conclusion
This study examines the varying levels of AI Literacy and AI Engagement among higher education
stakeholders. Its notable contribution to the literature is the examination of staff roles in addition to the
highly researched roles of faculty and students. The study found that participants in different roles report
significantly different AI literacy and AI Engagement. Specifically, staff AI Literacy distributions are lower
than those of students and faculty. This addition provides a more rounded discussion of institutional
opinions and identifies an underserved population in need of AI Literacy training.
Other notable findings include that faculty are significantly more engaged with AI than students or staff.
For age, younger individuals tend to report higher AI Literacy, while middle-aged individuals report more
frequent AI Engagement. Additionally, those who are more optimistic about AI's potential in higher
education demonstrate both higher AI Literacy and AI Engagement. Notably, gender did not significantly
impact either variable, indicating that AI-related skills and engagement are not influenced by gender in this
sample.
This research demonstrates that increased engagement with AI tools can significantly improve literacy
skills. Therefore, to foster an AI-savvy academic community, higher education institutions should conduct
targeted initiatives to promote active stakeholder participation. Specifically, they should use communities
of practice for students, faculty, and staff training and develop courses that focus on AI engagement and
literacy. Since staff scored notably lower in AI literacy, institutions should prioritize training opportunities
and transparent policies for this group, possibly involving students in policy and training development.
Additionally, faculty could mentor staff and students through introductory workshops and training, ideally
guided by clear institutional policies. Lastly, future AI literacy efforts should address the concerns of less
optimistic participants to enhance overall adoption and understanding.
Acknowledgement
Grammarly Pro was used to assist with the final edits of this paper. The author reviewed the suggested edits
and carefully selected suggestions that improve the readability of this paper. The core content of the paper
is the author's original thought unless otherwise cited.
References
Acosta-Enriquez, B. G., Arbulú Ballesteros, M. A., Huamaní Jordan, O., López Roca, C., & Saavedra
Tirado, K. (2024). Analysis of college students’ attitudes toward the use of ChatGPT in their
academic activities: Effect of intent to use, verification of information and responsible use. BMC
Psychology, 12(1), 255. https://doi.org/10.1186/s40359-024-01764-z
Al-Abdullatif, A. M., & Alsubaie, M. A. (2024). ChatGPT in Learning: Assessing Students’ Use
Intentions through the Lens of Perceived Value and the Influence of AI Literacy. Behavioral
Sciences, 14(9), 845. https://doi.org/10.3390/bs14090845
212
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
Asio, J. M. R. (2024). Artificial Intelligence (AI) Literacy and Academic Performance of Tertiary Level
Students: A Preliminary Analysis. Social Sciences, Humanities and Education Journal (SHE
Journal), 5(2), 309–321.
Berry, D. (2023). The limits of computation: Joseph Weizenbaum and the ELIZA chatbot. Weizenbaum
Journal of the Digital Society, 3(3). https://doi.org/10.34669/WI.WJDS/3.3.2
Blanca, M., Alarcón, R., Arnau, J., Bono, R., & Bendayan, R. (2017). Non-normal data: Is ANOVA still a
valid option? Psicothema, 4(29), 552–557. https://doi.org/10.7334/psicothema2016.383
Board of Regents of the University System of Georgia. (2024). The Initiative. About GALILEO.
https://about.galileo.usg.edu/about
Brown, L. S., & Lewis, K. (2022). The elementary forms of digital communication. PLOS ONE, 17(9),
e0273726. https://doi.org/10.1371/journal.pone.0273726
Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and
learning. International Journal of Educational Technology in Higher Education, 20(1), 38.
https://doi.org/10.1186/s41239-023-00408-3
Chan, C. K. Y., & Hu, W. (2023). Students’ voices on generative AI: Perceptions, benefits, and challenges
in higher education. International Journal of Educational Technology in Higher Education, 20(1),
43. https://doi.org/10.1186/s41239-023-00411-8
Chan, C. K. Y., & Lee, K. K. W. (2023). The AI generation gap: Are Gen Z students more interested in
adopting generative AI such as ChatGPT in teaching and learning than their Gen X and millennial
generation teachers? Smart Learning Environments, 10(1), 60. https://doi.org/10.1186/s40561023-00269-3
Chan, R. Y. (2016). Understanding the purpose of higher education: An analysis of the economic and
social benefits for completing a college degree. Journal of Education Policy, Planning and
Administration, 6(5). http://www.jeppa.org
Chiu, T. K. F., Ahmad, Z., Ismailov, M., & Sanusi, I. T. (2024). What are artificial intelligence literacy
and competency? A comprehensive framework to support them. Computers and Education Open,
6, 100171. https://doi.org/10.1016/j.caeo.2024.100171
Cruz, I. F. (2024, January 11). AI Literacy: A Prerequisite for the Future of AI and Automation in
Government. IBM Center for The Business of Government.
https://www.businessofgovernment.org/blog/ai-literacy-prerequisite-future-ai-and-automationgovernment
Cullen, M., & Kirkpatrick, M. (2024). Embracing Artificial Intelligence: Incorporating Artificial
Intelligence Into Classroom Instruction. Journal of Nursing Education, 1–2.
https://doi.org/10.3928/01484834-20240626-01
Duderstadt, J. J. (2003, October 7). The Importance of Liberal Learning for an Increasingly
Technological World. University Consortium for Atmospheric Research.
213
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
Farrelly, T., & Baker, N. (2023). Generative artificial intelligence: Implications and considerations for
higher education Practice. Education Sciences, 13(11), 1109.
https://doi.org/10.3390/educsci13111109
Francis, N. J., Jones, S., & Smith, D. P. (2025). Generative AI in Higher Education: Balancing Innovation
and Integrity. British Journal of Biomedical Science, 81, 14048.
https://doi.org/10.3389/bjbs.2024.14048
Hornberger, M., Bewersdorff, A., & Nerdel, C. (2023). What do university students know about Artificial
Intelligence? Development and validation of an AI literacy test. Computers and Education:
Artificial Intelligence, 5, 100165. https://doi.org/10.1016/j.caeai.2023.100165
Hur, J. W. (2025). Fostering AI literacy: Overcoming concerns and nurturing confidence among
preservice teachers. Information and Learning Sciences, 126(1/2), 56–74.
https://doi.org/10.1108/ILS-11-2023-0170
Introducing ChatGPT. (2022, November 30). OpenAI. https://openai.com/index/chatgpt/
Janse Van Rensburg, J. (2024). Artificial human thinking: ChatGPT’s capacity to be a model for critical
thinking when prompted with problem-based writing activities. Discover Education, 3(1), 42.
https://doi.org/10.1007/s44217-024-00113-x
Kabudi, T., Pappas, I., & Olsen, D. H. (2021). AI-enabled adaptive learning systems: A systematic
mapping of the literature. Computers and Education: Artificial Intelligence, 2, 100017.
https://doi.org/10.1016/j.caeai.2021.100017
Kelly, A., Sullivan, M., & Strampel, K. (2023). Generative artificial intelligence: University student
awareness, experience, and confidence in use across disciplines. Journal of University Teaching
and Learning Practice, 20(6). https://doi.org/10.53761/1.20.6.12
Knapp, H. (2018). Intermediate Statistics Using SPSS. SAGE Publishing Inc.
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024). AI literacy and its implications for prompt
engineering strategies. Computers and Education: Artificial Intelligence, 6, 100225.
https://doi.org/10.1016/j.caeai.2024.100225
Koohang, A., Sargent, C. S., & Svanadze, S. (2024). Students’ perceptions of benefits and opportunities of
artificial intelligence (AI). Issues In Information Systems, 25(2), 438–450.
https://doi.org/10.48009/2_iis_2024_134
Kunst, E. M., Van Woerkom, M., & Poell, R. F. (2018). Teachers’ Goal Orientation Profiles and
Participation in Professional Development Activities. Vocations and Learning, 11(1), 91–111.
https://doi.org/10.1007/s12186-017-9182-y
Kurshumova, D. A. (2024). A snapshot of Bulgarian school teachers’ familiarity with, use of, and
opinions on artificial intelligence at the threshold of its incorporation into the educational process.
Discover Education, 3(1), 138. https://doi.org/10.1007/s44217-024-00225-4
214
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023). Development of the “Scale for the
assessment of non-experts’ AI literacy” – An exploratory factor analysis. Computers in Human
Behavior Reports, 12, 100338. https://doi.org/10.1016/j.chbr.2023.100338
Lintner, T. (2024). A systematic review of AI literacy scales. Npj Science of Learning, 9(1), 50.
https://doi.org/10.1038/s41539-024-00264-4
Long, D., & Magerko, B. (2020). What is AI Literacy? Competencies and design considerations.
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–16.
https://doi.org/10.1145/3313831.3376727
Luo, T., Muljana, P. S., Ren, X., & Young, D. (2024). Exploring instructional designers’ utilization and
perspectives on generative AI tools: A mixed methods study. Educational Technology Research
and Development. https://doi.org/10.1007/s11423-024-10437-y
Ma, D., Akram, H., & Chen, H. (2024). Artificial Intelligence in Higher Education: A Cross-Cultural
Examination of Students’ Behavioral Intentions and Attitudes. The International Review of
Research in Open and Distributed Learning, 25(3), 134–157.
https://doi.org/10.19173/irrodl.v25i3.7703
Moorhouse, B. L., Yeo, M. A., & Wan, Y. (2023). Generative AI tools and assessment: Guidelines of the
world’s top-ranking universities. Computers and Education Open, 5, 100151.
https://doi.org/10.1016/j.caeo.2023.100151
Ng, D. T. K., Xinyu, C., Leung, J. K. L., & Chu, S. K. W. (2024). Fostering students’ AI literacy
development through educational games: AI knowledge, affective and cognitive engagement.
Journal of Computer Assisted Learning, 40(5), 2049–2064. https://doi.org/10.1111/jcal.13009
Ren, X., & Wu, M. L. (2025). Examining Teaching Competencies and Challenges While Integrating
Artificial Intelligence in Higher Education. TechTrends. https://doi.org/10.1007/s11528-02501055-3
Rillig, M. C., & Kasirzadeh, A. (2024). AI Personal Assistants and Sustainability: Risks and
Opportunities. Environmental Science & Technology, 58(17), 7237–7239.
https://doi.org/10.1021/acs.est.4c03300
Roddy, K. (2024, May 29). The ABCs Of XYZs: The Evolution Of Technology Across Generations.
Forbes. https://www.forbes.com/councils/forbescoachescouncil/2024/05/29/the-abcs-of-xyzs-theevolution-of-technology-across-generations/
Sullivan, M., Kelly, A., & Mclaughlan, P. (2023). ChatGPT in higher education: Considerations for
academic integrity and student learning. Journal of Applied Learning & Teaching, 6(1).
https://doi.org/10.37074/jalt.2023.6.1.17
Velander, J., Taiye, M. A., Otero, N., & Milrad, M. (2024). Artificial Intelligence in K-12 Education:
Eliciting and reflecting on Swedish teachers’ understanding of AI and its implications for
teaching & learning. Education and Information Technologies, 29(4), 4085–4105.
https://doi.org/10.1007/s10639-023-11990-4
215
Issues in Information Systems
Volume 26, Issue 1, pp. 201-216, 2025
Wang, B., Rau, P.-L. P., & Yuan, T. (2023). Measuring user competence in using artificial intelligence:
Validity and reliability of artificial intelligence literacy scale. Behaviour & Information
Technology, 42(9), 1324–1337. https://doi.org/10.1080/0144929X.2022.2072768
Wang, M., Chen, Z., Liu, Q., Peng, X., Long, T., & Shi, Y. (2024). Understanding teachers’ willingness to
use artificial intelligence-based teaching analysis system: Extending TAM model with teaching
efficacy, goal orientation, anxiety, and trust. Interactive Learning Environments, 1–18.
https://doi.org/10.1080/10494820.2024.2365345
Wang, Y.-Y., & Wang, Y.-S. (2022). Development and validation of an artificial intelligence anxiety scale:
An initial application in predicting motivated learning behavior. Interactive Learning
Environments, 30(4), 619–634. https://doi.org/10.1080/10494820.2019.1674887
Wood, E. A., Ange, B. L., & Miller, D. D. (2021). Are We Ready to Integrate Artificial Intelligence
Literacy into Medical School Curriculum: Students and Faculty Survey. Journal of Medical
Education and Curricular Development, 8, 23821205211024078.
https://doi.org/10.1177/23821205211024078
Zimotti, G., Frances, C., & Whitaker, L. (2024). The Future of Language Education: Teachers’
Perceptions About the Surge of Large Language Models like ChatGPT. Technology in Language
Teaching and Learning, 6(2), 1–24. https://doi.org/10.29140/tltl.v6n2.1136
216