Surveying employees about AI at work

[EricKnudsen]

The rise of AI at work

With the rise of artificial intelligence (AI) technologies in the workplace, a transformational shift is occurring in the way work gets done. Today, organizations are increasingly integrating AI tools into their operations to enhance productivity, decision making, and overall efficiency of their workforce. In moments of change, Viva Glint recommends that organizations stay tuned into the experiences of their most valuable asset: their people. Organizations are always transforming and evolving. This is particularly important because when organizational change fails, as 40-70% of

organizational changes do (Burns, 2000), it costs organizations time and money, and can increase the rate of employee turnover (Yue et al., 2019). In fact, Viva Glint’s own research has found that employees who do not feel their organizations communicated effectively about an organizational change are 80% more likely to leave the company within three months, compared to employees who felt the organization communicated effectively (Glint, 2023). Further, effective change communication can reduce the impact of change fatigue and reduce overall change resistance (Frahm & Brown, 2007; Yue et al., 2019). While the deployment of new tools and technologies at work are not often considered groundbreaking change, the weaving of new AI tools into everyday work does actually represent a paradigm shift that warrants close attention to employee experience.

AI offers some substantial benefits to workforces globally, ranging from automating routine tasks, to analyzing vast data sets for insights, and generating novel content. These tools offer employees the opportunity to increase the quality and speed of their contributions at work, as well as improve their relationship to their work by supporting them in spending more time on the work of their choice. This shift has the potential to enhance their day-to-day and provide greater fulfillment at work. Yet, along with these benefits come potential challenges. These include effective deployment of new AI tools, enablement of the workforce to use them, and employees' comfort with both their use and the potential impact of these new tools on individual work. Failure to acknowledge and address these concerns can jeopardize the success of AI initiatives and hinder the organization's ability to fully harness the potential of AI. In order to effectively address emerging challenges, organizations must prioritize employee engagement and feedback as essential components of a people-centric AI adoption strategy.

Why surveys are a critical part of AI rollout

Surveying employees during the rollout of AI tools has the potential to yield a number of advantages. By actively seeking input from employees, organizations can proactively highlight the value of a new tool, and address challenges that may otherwise go unnoticed in near real-time. Here are a few important reasons organizations should consider surveying employees during the rollout of a new AI tool:

1. Early implementation issues: Employee surveys serve as an early warning system for a problematic roll-out. Effective survey items can uncover potential issues, such as misconceptions about the tool, enablement gaps, and other emerging concerns employees are experiencing. Identifying these issues early allows organizations to course-correct their implementation quickly.

2. User insights: AI tools may impact the work of different employees in different ways. Employees who actually use the tool are uniquely positioned to share insights into its usability, strengths, and weaknesses for their particular function (e.g., Marketing or Sales) or job level (e.g., individual contributor, manager, executive). This feedback may be invaluable for refining implementations and ensuring enablement efforts are aligned with real-world needs of the workforce. For example, it helps an organization identify real examples of success with a new tool, which it can share more broadly and try to replicate in other parts of the organization, helping the tool to “stick” better.

3. Employee empowerment: Surveys empower employees by giving them a voice in the AI adoption process. This empowerment fosters a sense of ownership and responsibility, which can lead to more active engagement with AI technologies. Involving employees in decision-making processes and demonstrating that their opinions matter can significantly accelerate AI adoption. When employees feel heard and valued, they are more likely to embrace change.

4. Continuous improvement: The use of AI at work is not static; its use is continually evolving and will continue to evolve over time. Employee survey items related to the implementation of new technologies may be repeated throughout a longer implementation period to track changes in perceptions, identify emerging issues, and guide ongoing improvements in the implementation of the tool.

Glint's Recommended Survey Items

Glint's recommended AI survey items fall into two categories:

1. Implementation perceptions: these items ask employees to evaluate the tool's utility, their organization's process, and enablement practices. These items are designed to support organizations in understanding "how well it's going" with an AI tool roll-out.

2. Outcomes: these items ask employees to reflect on the task- or work-oriented benefits they experience as a result of using the tool. These items can help establish the productivity and employee experience return-on-investment (ROI) that an AI tool has.

Note that not all perception or outcome items will be relevant for all tools - for example if an AI tool being implemented is unrelated to enhancing the work meeting experience, an organization should not include the AI - Meetings item ([NAME OF TOOL] helps me maximize the value of meetings.). Additionally, these items may not capture nuance in specific outcomes the organization cares about - if a tool is believed to improve time savings for employees, that organization should consider if they want to write their own item asking a respondent to estimate the number of weekly hours saved as a result of using the tool.

Given the frequent employee surveying occurring in many organizations, they may not always be able to administer a separate survey specific to AI experiences. In such cases, Glint recommends selecting a subset of the recommended items based on organizational goals of the AI rollout, and incorporating them into an existing survey touchpoint with employees (e.g., a quarterly or annual engagement survey, or a more local team pulse. When surveying about AI tool deployments, Glint also recommends focusing on internal improvement rather than external comparison, and as such Glint will not offer benchmarks for these survey items.

Implementation Perceptions - Survey Items

Item Name	Item Type	Item Text
AI – Confidence in Leadership	Rating	I believe in [COMPANY NAME]'s ability to deploy [NAME OF TOOL] successfully.
AI – Fair Decisions	Rating	I believe decisions about who gets access to [NAME OF TOOL] have been made fairly.
AI - Impact	Rating	I understand how [NAME OF TOOL] is relevant and useful to me.
AI – Input	Rating	I feel comfortable sharing my thoughts about [NAME OF TOOL].
AI - Questions	Rating	I know where to find answers to my questions about [NAME OF TOOL].
AI - Well Informed	Rating	I am well-informed about [NAME OF TOOL].
AI - Training	Rating	I have received the information or training I need to feel skilled with [NAME OF TOOL].
AI - Policies	Rating	I understand the rules and expectations for use of [NAME OF TOOL] at [COMPANY NAME].

AI Outcomes - Survey Items

Item Name	Item Type	Item Text
AI - Accuracy	Rating	[NAME OF TOOL] helps me process information more thoroughly.
AI - Speed	Rating	[NAME OF TOOL] helps me find or process information faster.
AI - Meetings	Rating	[NAME OF TOOL] helps me maximize the value of meetings.
AI - Fulfillment	Rating	[NAME OF TOOL] allows me to focus on more fulfilling work.
AI - Productivity	Rating	[NAME OF TOOL] allows me to feel more productive.
AI - Creativity	Rating	[NAME OF TOOL] allows me to feel more creative.
AI - Self-Efficacy	Rating	[NAME OF TOOL] allows me to feel more confident in the work I deliver.
AI - Org Innovation	Rating	[NAME OF TOOL] helps me feel empowered with leading edge tools and technology.
AI – Benefit	Open Text	What are some of the biggest benefits of [NAME OF TOOL] to you?
AI – Concern	Open Text	What are some of the biggest concerns you have about [NAME OF TOOL]?

We’d love to hear from you

Are you surveying about AI tools in your organization? What are you learning? Have these items been helpful to your organization? Post below in the comments.

References

Burns, B. (2000). Managing Change: A Strategic Approach to Organizational Dynamics.
Frahm, J., & Brown, K. (2007). First steps: linking change communication to change receptivity. Journal of organizational change management.
Glint (2023). Global attrition driver analysis. Unpublished document.
Yue, C. A., Men, L. R., & Ferguson, M. A. (2019). Bridging transformational leadership, transparent communication, and employee openness to change: The mediating role of trust. Public relations review, 45(3), 101779.