3.1 Signaling Theory and AI Convergence in Human–AI Teams

To investigate AI convergence in organizational settings, we apply signaling theory (Connelly et al. 2011; Drover, Wood, and Corbett 2018). Signaling theory has demonstrated a significant impact across various areas of organizational studies, including strategic management (Bergh et al. 2014; Park and Patel 2015; Plummer, Allison, and Connelly 2016), human resource management (Guest et al. 2021; Wang, Zhang, and Wan 2021), entrepreneurship (Allison, McKenny, and Short 2013), leadership (Appels 2022; Matthews et al. 2022), diversity initiatives (Leslie 2019), and crowdfunding (Kleinert and Mochkabadi 2022; Steigenberger and Wilhelm 2018). Signaling theory emerged from economics, finance, and cognitive psychology and is useful for investigating behaviors in an environment characterized by uncertainty and information asymmetries (Adam et al. 2022). Unlike most studies utilizing signaling theory to investigate signals as means of external communication, we apply signaling theory to the team context to explain how team members' attributes and actions transmit signals within a team and how those signals can establish credibility and consensus between human and AI team members.

AI systems can emulate complex human behavior, reasoning, and learning (Russell and Norvig 2003). To make decisions in human–AI teams (Bankins et al. 2024), information is distributed within the team through individual signals and signal sets (Drover, Wood, and Corbett 2018). Signaling is especially important in human–AI teaming because AI does not have agency, mutual trust, or emotions that traditional forms of team coordination rely on (Mathieu et al. 2017; Troth et al. 2012). Therefore, the signals sent by AI team members are the primary means by which AI contributes in human–AI teams. Beyond the potential impact of AI signals, we argue that our understanding of the mechanisms governing how employees' decisions align with signals from both human and AI sources in human–AI teams remains incomplete, necessitating a nuanced application of signaling theory.

In human–AI teams, convergence between human and AI team members on critical decisions can manifest in various forms, and following the suggestion of AI is not a given (Marti, Lawrence, and Steele 2024). Three distinct patterns emerge in human–AI teams: (1) human teammates may align and converge with an AI teammate on a specific signal, decision, or evidence; (2) human teammates may diverge with an AI teammate; and (3) in teams involving multiple individuals and an AI, there may be a mixture of convergence and divergence with AI across human teammates. We draw on the cognitive processing approach to signals established by Drover, Wood, and Corbett (2018) to investigate how signal heuristics, signal sets, and volitional action influence the patterns of AI convergence within teams.

To begin, the underlying assumptions of signaling theory are critical to understanding when and how AI convergence in human–AI teams is possible. Signaling theory's impact comes from its ability to explain what does (and what does not) constitute an effective signal. Likewise, signaling theory identifies the situations where a signaling perspective holds the most value. Ultimately, effective signals reduce the uncertainty that exists when two or more entities have different information, a fair assumption when considering the nature of predictive and generative AI.

At a minimum, for signals to be effective in teams, three things must happen. First, signals must catch the attention of team members or be seen and processed at some minimum threshold. Second, signals must provide useful, relevant information that the team needs. Third, the situation must include a level of uncertainty or missing information that a signal can help clarify (Connelly et al. 2011; Drover, Wood, and Corbett 2018). In other words, in human–AI teams, when completing tasks that cannot be accomplished with perfect certainty, individuals have two paths they can walk. First, they can take a risk and charge forward with a decision without clarity or certainty. Second, they can seek out and process relevant signals in their environment that make their task clearer and reduce uncertainty before they make a decision.

Logistically, signaling in human–AI teams contains three core elements: sender(s) (team members, be they human or AI conveying the results of their analysis and prescribing a course of action), receiver(s) (team members with ultimate authority to take an action), and signals (observable cues that convey useful information to decision-makers) (Spence 1973). Previous research suggests that there are two main characteristics of individual signals: observability and fit, which influence their impact (Connelly et al. 2011). Signal sets can be congruent or incongruent depending on whether signals convey the same information or not, and they combine to create the collective positive or negative valence of a signal set.

In human–AI teams, signal observability refers to the extent to which team members notice a signal (Connelly et al. 2011). If too many signals are sent in a team at once, or a signal is not readily observed by team members, then it is unlikely that a signal will have any impact. Logically, it follows that if an AI signal is unobservable, goes unnoticed, or is ignored, AI convergence would be independent of the AI team member and will trend toward chance or unrelated factors.

Signal fit describes the extent to which a signal can truthfully minimize information asymmetries and reduce uncertainty for a specific team member (Bafera and Kleinert 2022; Vanacker et al. 2020). This concept is important in understanding situations when a signal may not actually be an accurate representation of underlying value such as in an unreliable signal or dishonest signal (Busenitz, Fiet, and Moesel 2005; Cohen and Dean 2005; Davila, Foster, and Gupta 2003). Historically, the literature has used various terms to identify the extent to which a signal fits, is credible, and corresponds to quality, including the terms signal strength, intensity, and clarity (for a review, see Connelly et al. 2011).

Two elements of human–AI teaming are important to recognize when it comes to signal fit. First, bias in training data for algorithms can create variability in signal fit depending on the context of the task (Bolukbasi et al. 2016; Buolamwini and Gebru 2018; Çalışkan, Bryson, and Narayanan 2017; Grother, Ngan, and Hanaoka 2019). Therefore, AI signals may be more or less credible depending on the task being solved, and this may be known or unknown by human team members. Second, team members may calibrate signals differently, giving a signal with different perceived fit or meaning within a team (Branzei et al. 2004).

Whereas observability is about a human team member noticing signals from an AI team member, signal fit is largely about how AI signals are interpreted and translated into meaning. Previous research theoretically distinguishes signals with strong and weak fit (Connelly et al. 2011) and proposes that strong signals (those with strong fit) are more likely to evoke a receiver's desired judgmental confidence (a threshold that must be crossed to make a decision) through heuristic processing as human receivers go through an automatic cognitive process of signal interpretation (Drover, Wood, and Corbett 2018). A strongly fitting signal sent by an AI teammate would be known to be reliable and aligned with relevant information from other sources. For example, if an AI were to suggest a performance rating of an employee that is consistent with their previous years' score and the score provided by their immediate supervisor, the confirmation of that score would move swiftly. In other words, signals with strong fit, and no information to the contrary, will largely be noticed and processed automatically when they are present (Drover, Wood, and Corbett 2018).

In contrast, signals with weak fit make it hard for receivers to automatically interpret information and reach the desired judgmental confidence threshold, and therefore, systematic processing is necessary (Drover, Wood, and Corbett 2018). A signal with weak fit sent by an AI teammate is one that contradicts some other relevant piece of information as would be the case if the algorithm above suggested a performance score that was dramatically different than that suggested by the supervisor. If a signal's fit is too weak, receivers may deem its interpretation and the attainment of the desired judgmental confidence threshold as impossible, abandoning any type of cognitive information processing and interrupting effective decision-making.

Applying these concepts to human–AI teams, strong signal fit and credibility from an AI team member should trigger heuristic processing and thus increase the chance of AI convergence in the form of accepting the AI advice. This reasoning leads to the following hypothesis:

3.2 Optional AI and AI Convergence

Previous research on human–AI teaming in organizational settings suggests that the distribution of agency is a pivotal factor that significantly impacts AI convergence in the workplace (Agrawal, Gans, and Goldfarb 2019; Chugunova and Sele 2022; Gagné et al. 2022; Grundke 2024). In particular, humans seem to be particularly averse to AI tools that make decisions independently and do not allow human control, especially in contexts with moral implications and high levels of uncertainty (Dietvorst, Simmons, and Massey 2015; Dietvorst and Bharti 2020; Jussupow, Benbasat, and Heinzl 2020; Longoni, Bonezzi, and Morewedge 2019). However, embedding human autonomy and control into human–AI collaborations may mitigate algorithmic aversion and even make people appreciative of AI team members' recommendations (Bigman and Gray 2018; Chugunova and Sele 2022; Gagné et al. 2022; Logg, Minson, and Moore 2019).

This logic was reinforced in a recent review by Gagné et al. (2022) that outlined how mandatory AI, especially without careful consideration of employee needs and interpretation, can hinder the effective implementation of AI technology in the workplace. As teams and work are designed to incorporate new technology, Gagné et al. (2022) suggest that the employee needs for autonomy, competence, and relatedness (Deci and Ryan 1985) will determine how motivated human team members are to converge with AI team members. Empirical evidence suggests that this may be related to the phenomenon of algorithm appreciation (i.e., when humans are motivated to pay attention to AI advice, make use of it, and may even prefer it to human advice). When a task is perceived as objective or analytical, individuals feel more satisfied converging with AI (Chugunova and Sele 2022).

Ultimately, AI convergence is only possible when human team members notice a signal from an AI team member (Connelly et al. 2011). Combining the self-determination theory (Deci and Ryan 1985) with signaling theory (Drover, Wood, and Corbett 2018), we argue that the option to choose whether to see a signal from an AI teammate will help satisfy an individual's need for autonomy (the sense of being agents of their own behavior rather than mere pawns of external pressures, as discussed by Gagné et al. 2022), and therefore increase the likelihood of AI convergence. However, this prediction is reliant not only on employees having autonomy but exercising that autonomy to collaborate with their AI team members. Self-determination theory also posits that beyond the need for autonomy, which can be fulfilled by the ability to choose to see AI advice, employees also need to satisfy a need for competence—a feeling of effectiveness and mastery over their environment (Deci and Ryan 1985; Gagné et al. 2022). When an employee faces uncertainty and chooses to see signals from their AI teammate to enhance their mastery of the environment, their need for competence will be satisfied by the AI advice, thereby making AI convergence more likely. In contrast, when they choose to ignore or eschew their AI teammate, AI convergence will be mitigated.

In summary, we suggest that when employees choose to see AI advice (optional AI advice), rather than being forced to see it (mandatory AI advice), an AI signal takes on two important qualities. First, employees have chosen to attend to a cue in their environment and have exerted agency to satisfy their need for autonomy. Second, the signal becomes costlier as employees now have made a choice to observe and attend to a signal they could have ignored, and their need for competence becomes satisfied from processing the extra information. Taken together, we argue that in human–AI teams, optional AI advice, when observed, should increase AI convergence. This reasoning leads to the following hypothesis:

3.3 Signal Sets and AI Convergence

To this point, we have considered the processes of becoming attentive to individual signals that are either optional or have strong or weak fit. Interestingly, signal receivers may not always interpret a set of signals in line with the message intended by the sender, especially when experiencing high levels of uncertainty (Bergh et al. 2014). As signals become more complex, various signals and their components can become bundles that are interpreted together as sets. Such signal sets produce signal set valence, which is defined as “the collective volume and strength of the selected organizational signals and their assigned weight” (Drover, Wood, and Corbett 2018, 218). In other words, when facing information asymmetry and uncertainty, employees will have to make inferences from multiple cues or embedded elements that have different pieces of information (Drover, Wood, and Corbett 2018). Like signals, signal sets can have an array of qualities that make them more or less effective in reducing uncertainty and improving employee confidence in their decisions. There are three possible configurations of signal set congruence: (1) uniform congruence (represents a signal set with either positive or negative signals, depicting a uniform and consistent positive or negative valence across components of the set); (2) imbalanced incongruence (a signal set consists of both positive and negative signals, and the signal set valence is positively or negatively skewed); and (3) balanced incongruence (a signal set comprises both positive and negative signals with equal competing valences) (Drover, Wood, and Corbett 2018).

In human–AI teams, a signal set includes the prescriptions forwarded by the AI and the prescriptions forwarded by a human team member. When these components form a signal set that is congruent (i.e., where the human and the AI team members both signal the same decision), there will be enhanced AI convergence. When signals create a signal set that is incongruent (i.e., where the human and the AI team members both signal different decisions), AI convergence will be less likely. As repeated signals have been shown to increase signal impact (Connelly et al. 2011), we believe this effect will hold regardless of whether signals have strong fit or whether observing AI signals is chosen and not mandatory. Based on these arguments, we hypothesize the following:

3.4 Research Accountability

Following best research practices, we preregistered Hypotheses 1 and 2 before any data collection started. Hypothesis 3 emerged in response to excellent reviewer feedback and was not preregistered. Findings are replicated across several samples (four studies total) in fully independent samples from employees and leaders working in multiple countries, industries, and roles. To ensure the validity of our experiments, participants from each study were excluded from participating in the following studies using Prolific ID exclusion lists. Research for this paper was approved by the IRB of one of the author's institutions. In addition to this, we have provided all data and code used to produce the results. This includes a transparent data cleaning process where we upload the full dataset with two key cleaning variables. First, a dummy variable identifies which rows and participants are to be dropped from the analysis. Second, we include a column that identifies the reason each row was dropped. Hypotheses were tested using the “melogit” command in Stata 16 with random intercepts based on participant ID. Data and code can be found on the OSF repository: https://osf.io/zjwta/?view_only=74a15f71b21c4b0abb69f3e5de543537 and our preregistration can be found here: https://osf.io/yt5mc/?view_only=f5efa8fc9d2041f4a44dd4b6fc7f6bfe.

4.1 Study 1 Method

4.1.2 Procedure

Each participant was randomly assigned into one of four experimental conditions following a 2 (signal fit: strong versus weak) × 2 (AI advice: optional versus mandatory) between-subject design. Within each condition, participants completed a series of face-matching tasks in an employee identification scenario. Participants were instructed to play the role of a security officer who needs to identify if a person attempting to enter a highly secure facility is indeed a current employee with clearance to enter the building. The experimental setup was a two-step process. In the first step, participants were asked to review the image of the person seeking entry and compare this image to the most similar image in the employee image database deciding to grant or deny entry. In the second step, depending on their condition, participants were sent a signal with strong or weak fit and a signal of optional or mandatory AI advice. Participants had to decide whether the two images were a match (i.e., grant entry) or not a match (i.e., deny entry). In total, participants evaluated a series of 24 pairs of pretested images of human faces that were evenly balanced between races (Black vs. White) and gender (female vs. male). After evaluating each pair of images, participants reported their level of confidence. After finishing the facial recognition task, participants went through a series of demographic questions and additional measures (e.g., explicit gender bias).

The pairs of facial images employed in Study 1 were designed using the color version of the Facial Recognition Technology (FERET) database (Phillips et al. 1998). To select pairs of images for the experiment, the photos were initially evaluated by a face application programming interface (API) that used an algorithm to produce a similarity score between 0 (not similar at all) and 1 (very similar). However, the API had difficulties in creating viable pairs of images for Black men and women in comparison with White men and women. Therefore, images for Black men and women were selected manually from the database rather than using the algorithmically generated scores. To ensure a similar level of difficulty for the selected images, we conducted a preliminary study. We recruited 160 participants on Prolific balanced by gender (men/women) and race (Black/White). Participants were asked to determine if the pairs of Black/White men and women images were a “match” or “no match” and to rank their confidence after each pair of images. No other information was provided. Based on the difficulty of the images (e.g., how many times the pairs of images were matched or not matched correctly) and the confidence ratings, we chose 24 images (six images for White men, six images for White women, six images for Black men, and six images for Black women) of equal difficulty to be used in the main experiment. Results of this analysis can be provided upon request.

4.1.2.1 Manipulation of Signal Fit

In the strong signal fit (coded as 0), participants were presented with a pair of images next to an employee badge as well as the decision made by the AI tool (facial recognition system), shown as match or no match (see Figure 1). In the weak signal fit (coded as 1), participants were shown the AI decision (match or no match) plus additional information about the level of accuracy of the AI for different demographics (White Male/Black Male/White Female/Black Female) represented as bar graphs (see Figure 2). This information weakened the signal by limiting the credibility of AI advice. Visual representation for the AI accuracy for different demographics was derived from previous research on gender and racial disparity from facial recognition system vendors (Buolamwini and Gebru 2018; Grother, Ngan, and Hanaoka 2019). χ² tests revealed that participants accurately recalled whether they were provided with AI accuracy (χ² = 100.116, p = 0.000). Logistic regressions also showed that only the variable capturing the weak fit signal condition was a significant predictor of recalling that participants had information about AI accuracy (p = 0.000). There was no effect of the optional AI advice (p = 0.640) condition or the interaction between optional AI advice and signal fit (p = 0.653).

4.1.2.2 Manipulation of Optional AI Advice

In the optional AI advice condition (coded as 1), participants were asked if they would like additional assistance from the AI to help them make their decision. If participants opted to receive AI advice, they were prompted with information depending on the condition they were in (strong signal fit–AI match/no match vs. weak signal fit–AI match/no match plus AI accuracy for different demographics). If participants did not opt for AI advice, they received no information from the facial recognition system and made an unaided decision for that image pair. In the mandatory AI condition (coded as 0), participants were not asked if they would like additional assistance. Instead, they were automatically presented with either a strong fit or weak fit signal, depending on their condition. χ² tests revealed that participants accurately recalled whether they had the option to solicit AI advice (χ² = 76.786, p = 0.000). Logistic regressions also showed that only the variable capturing the optional AI advice condition was a significant predictor of recalling that participants had the option to solicit AI advice (p = 0.000). There was no effect of the signal fit (p = 0.413) condition or the interaction between optional AI advice and signal fit (p = 0.778).

4.2 Study 1 Results

Table 1 displays the means, standard deviations, and descriptive statistics of our study variables. Hypothesis 1 suggested that a strong signal fit versus a weak signal fit from AI would increase AI convergence in a high-stakes scenario. Hypothesis 2 suggested that having the option to get AI advice would likewise increase AI convergence, given that a human chooses to see the optional AI advice. We tested these hypotheses using a mixed effects logistic regression command “melogit” in Stata 16 (Statacorp 2019) with random intercepts for each participant. The nature of the data, which involve repeated binary decisions nested within participants, required the use of multilevel logistic regression to address issues of nonindependence and our binary dependent variable (see Guo and Zhao 2000 for a review). We first examined the relationships between the main effects of our experimental conditions, comparing the AI convergence of participants who had received a strong versus weak signal fit, as well as those who could or could not choose to solicit optional AI advice. Next, we added a dummy variable capturing whether participants chose to see AI advice.

For the signal fit, results show that there was no significant effect of a strong signal fit on the likelihood of AI convergence (Odds ratio = 0.797, p = 0.161). For optional AI advice, results show a significant negative effect on AI convergence when AI was optional versus when it was mandatory (Odds ratio = 0.630, p = 0.005). Next, we added an additional covariate to our model, which captured whether or not participants who had the option to solicit AI advice actually chose not to see it. Results show that when accounting for whether or not a participant actually chose to see optional AI advice, the effect of signal fit remained unchanged (Odds ratio = 0.884, p = 0.413). Thus, Hypothesis 1 was not supported as the AI signal fit did not impact AI convergence.

In contrast, when accounting for whether a participant actually chose to see optional AI advice, the effect of having the option to solicit AI advice became positive, increasing the odds of convergence (Odds ratio = 1.508, p = 0.011), whereas the effect of choosing not to see the AI advice had a large negative effect, decreasing the odds of making the same decision as the AI (Odds ratio = 0.126, p = 0.000). Thus, Hypothesis 2 was partially supported with the caveat that the participants' choice to actually solicit AI advice was a critical factor in determining the impact of optional AI. Importantly, for interpreting our results in light of Hypothesis 2, controlling for whether or not one exercises the choice to see AI advice leads to the interpretation that ceteris paribus, having the option does have a strong, positive impact on taking AI advice but this is contingent on the participant seeing the AI advice.¹

4.3 Study 1 Discussion

The results of Study 1 were quite surprising. Hypothesis 1 was not supported suggesting that signal fit did not impact human employees' tendency to converge with AI in their decisions. On its own, signal fit does not appear to play a major role in the AI convergence in the workplace, at least not in a dyadic partnership in the facial recognition system context. That said, the results supported Hypothesis 2 suggesting that an optional AI signal is crucial for the effective human–AI teaming in organizational settings. The effect of optional AI became positive when controlling for whether receivers choose to see AI advice or not. This is an important finding in regard to the design of human–AI teams that reinforces the “augmentation thesis,” while favoring a collaborative model of hybrid decision-making. Ultimately, these data suggest that we may need to reject the oppositional framing of AI versus human decision-making to optimize effective human–AI teams. Finally, the effect of optional AI was independent of the signal fit about the AI partner's signal that was sent to receivers.

The results of Study 1 demonstrated clearly that we needed to replicate our findings. It is possible that the data represented a one-time finding that was idiosyncratic to the specific data set. Beyond this, we determined that it was necessary to address a potential confound in our experimental design. Namely, in Study 1, we asked participants in the optional AI condition to solicit optional AI advice only if they were not confident in their initial decision (see Solicit AI Advice measure). Although this design is aligned with the stage of attending to observable signals as well as with the stage of signal interpretation described by signaling theory, it nevertheless could be the case that we introduced demand effects, or otherwise altered the behavior of participants by making confidence directly tied to the decision to see AI advice in the optional AI conditions.

4.4 Study 2 Method

4.5 Study 2 Results

Table 4 displays the means, standard deviations, and descriptive statistics of our Study 2 variables. For the second time, results show that for signal fit, there was no significant effect of additional information that weakened the signal on the likelihood of AI convergence (Odds ratio = 0.827, p = 0.218). Furthermore, for optional AI advice, results again show a significant, negative effect in the likelihood of AI convergence when AI advice was optional versus when it was mandatory (Odds ratio = 0.705, p = 0.024). Results again show the same pattern as in Study 1 when accounting for whether or not a participant actually chose to see AI advice. When controlling for a participant's decision to see AI advice when given the choice, the effect of signal fit remained nonsignificant (Odds ratio = 0.900, p = 0.475), whereas the effect of having the option to see AI advice became positive, increasing the odds of AI convergence (Odds ratio = 1.400, p = 0.034). Again, the effect of ignoring the option for AI advice had a large negative effect, decreasing the odds that participants made the decision as the AI suggested (Odds ratio = 0.159, p = 0.000). Thus, Hypothesis 1 was not supported, and Hypothesis 2 was supported for a second time.

4.6 Study 2 Discussion

The results of Study 2 replicated Hypothesis 2 while removing potential demand effects inherent to the design in Study 1. Optional AI advice seemed to improve the likelihood of employees' convergence with AI, but only when participants chose to see the optional AI signal. Similar to Study 1, this effect appeared to be independent of signal fit. Thus far, our studies have tested our first two hypotheses in a dyadic human–AI partnership. Given this study design, we cannot speak to the context of AI teams, where there is more than one human employee working in a team that includes AI. Therefore, in Study 3, we extend our study design to include an additional signal sent from a human teammate.

4.7 Study 3 Method

4.7.1 Participants and Procedure

We followed the same procedures for Study 3 as in Study 2 with three exceptions. First, we reduced the number of images to 16 to reduce participant burden. We selected the images that were most difficult in our previous two studies to retain, so that there would be the highest level of uncertainty present to drive signal salience and interpretation (Drover, Wood, and Corbett 2018). Second, we included an additional signal sent by a human colleague, Sam Smith, for every image pair. With this addition, we were able to manipulate the congruence of the signal set composed of human and AI team member signals. The study was balanced, so that half of the signal sets were congruent, where the human and the AI both signaled the same decision. In the other half, the signal from the AI was the opposite of the signal from Sam Smith. Finally, we gave the facial recognition system tool the name Iris AI to match the presentation of the colleague Sam Smith in our decision-making task. A total of 319 participants followed the link to participate in the study. We removed 20 participants who failed to provide consent or failed to recall basic information after the tutorial (process checks). This resulted in 299 individuals interested who were allowed to proceed to the full experiment and were paid 2.90 pounds in exchange for their time. These participants were on average 43.26 years old (SD 12.82). Approximately 47.8% of our participants identified as female, 50.2% of participants identified as male, 1.7% identified as transgender, and 0.3% identified as nonconforming. Approximately 49.2% of participants identified as Black/Caribbean/African/Afro Latin and 49.8% identified as White/Caucasian/European.

4.7.1.1 Signal Set Congruence

In the congruent signal set condition (coded as 1), participants were presented with a pair of images next to an employee badge as well as the decision made by Iris AI and Sam Smith, where both signals were the same (see Figure 3). In the incongruent signal set conditions (coded as 0), participants were shown Iris AI's decision (match or no match) plus Sam Smith's decision (match or no match) but the signal sent by Sam Smith and Iris AI was the opposite (see Figure 4).

4.8 Study 3 Results

Table 8 displays the means, standard deviations, and descriptive statistics of our Study 3 variables. For the third time, results show that for signal fit there was no significant effect on the likelihood of AI convergence (Odds ratio = 0.875, p = 0.267). Unlike in Studies 1 and 2, for optional AI advice, results including only main effects (Model 1) do not show a significant, negative effect on the likelihood of convergence when AI advice was optional versus when it was not optional (Odds ratio = 1.112, p = 0.378). Instead, the key driver of AI convergence is the congruence of the signal set within the team (Hypothesis 3). When the signals sent by Iris AI and Sam Smith were congruent, participants were much more likely to take the AI advice, and this effect was large (Odds ratio = 9.098, p < 0.001).

When one accounts for the choice to see the advice by both team members, Sam Smith and Iris AI, however, results again show the same pattern as the previous two studies. The impact of choosing not to see team members' advice decreases the likelihood that participants make the same decision as Iris AI (Odds ratio = 0.212, p < 0.001), while being in the optional AI condition showed a positive effect after accounting for this choice by participants (Odds ratio = 1.519, p < 0.001). Thus, Hypothesis 1 was not supported, and Hypothesis 2 was supported for the third time. Furthermore, Hypothesis 3 was supported.

4.8.1 Study 3 Supplementary Analysis

Although not hypothesized, to fully probe our experimental design, we tested the three-way interaction between signal fit, optional advice, and signal set congruence on the likelihood of AI convergence. This interaction term was significant (Odds ratio = 1.911, p = 0.043). As shown in Figure 5, the largest impact of our manipulated variables is caused by the signal set congruence. When the signals between Sam Smith and Iris AI are the same, participants are much more likely to take AI advice. Visual inspection suggests that this is slightly more likely when the signal is optional compared with when it is mandatory. When the signals do not align, however, there are greater differences across the optional advice and signal fit conditions. Specifically, when signal sets are not congruent, AI convergence is more likely when the signal fit is strong (when AI accuracy is not presented) and when seeing team members' advice is optional. Combined, this provides further support for Hypotheses 2 and 3 and, for the first time, suggests that signal fit may have a nuanced effect that is conditional on the presence of a human team member.

Similar to Studies 1 and 2, we controlled for participants' level of confidence and other variables that would lend support for our theorizing. Results of this analysis again show no qualitative difference in our tests of Hypotheses 1–3 when controlling for initial decision confidence, trust in AI, racial bias, and gender bias (see Tables 9 and 10). Furthermore, similar to Studies 1 and 2, participants' trust in AI significantly increased employees' AI advice taking in the team. Finally, unlike in Studies 1 and 2, explicit racism significantly decreased the likelihood that participants would take AI advice.

TABLE 9. Study 3 results.

	Model 1 main effects only		Model 2 main effects plus solicit decision		Model 3 two-way interaction plus solicit decision		Model 4 three-way interaction plus solicit decision
Variable	Odds ratio	SE	Odds ratio	SE	Odds ratio	SE	Odds ratio	SE
Weak signal fit	0.87	0.10	0.87	0.10	1.02	0.17	1.09	0.20
Optional AI advice	1.11	0.13	1.52**	0.18	1.76**	0.29	1.91**	0.34
Congruent signal set	9.09**	0.74	9.51**	0.79	9.51**	0.79	10.16**	1.64
Weak signal fit × Optional advice					0.74	0.17	0.60*	0.15
Weak signal fit × Congruent signal set							0.81	0.19
Optional advice × Congruent signal set							0.77	0.17
Weak signal fit × Optional advice × Congruent signal set							1.91*	0.61
Decline to solicit			0.21**	0.03	0.21**	0.03	0.21**	0.03
Intercept	0.96	0.10	0.94	0.10	0.87	0.10	0.86	0.11
var (participant ID)	0.68	0.10	0.59	0.09	0.59	0.09	0.59	0.09

• ** p < 0.01.
• * p < 0.05.
• † p < 0.1.

TABLE 10. Study 3 supplementary analysis results.

	Model 1 main effects only		Model 2 main effects plus solicit decision		Model 3 two-way interaction plus solicit decision		Model 4 three-way interaction plus solicit decision
Variable	Odds ratio	SE	Odds ratio	SE	Odds ratio	SE	Odds ratio	SE
Weak signal fit	0.87	0.10	0.87	0.10	1.05	0.17	1.13	0.20
Optional AI advice	1.13	0.14	1.49**	0.18	1.78**	0.29	1.96**	0.34
Congruent signal set	9.44**	0.78	9.72**	0.81	9.72**	0.81	10.50**	1.70
Weak signal fit × Optional AI advice					0.69	0.15	0.56*	0.14
Weak signal fit × Congruent signal set							0.81	0.19
Optional advice × Congruent signal set							0.74	0.17
Weak signal fit × Optional advice × Congruent signal set							1.99**	0.64
Decline to solicit			0.27**	0.04	0.27**	0.04	0.27**	0.04
Initial confidence in decision	0.98**	0.002	0.98	0.002	0.98	0.002	0.98**	0.002
Female participant	1.07	0.13	1.11	0.13	1.10	0.13	1.10	0.13
Black participant	1.13	0.14	1.14	0.13	1.14	0.13	1.14	0.13
Explicit racial bias	0.88*	0.05	0.89*	0.05	0.88*	0.05	0.89**	0.05
Explicit gender bias	1.08	0.05	1.08	0.05	1.08†	0.05	1.08†	0.05
Trust in AI system	1.44**	0.12	1.43**	0.11	1.44**	0.11	1.44**	0.11
Algorithmic aversion	0.92	0.05	0.91†	0.05	0.90†	0.05	0.91†	005
Intercept	1.88	0.94	1.33	0.63	1.19	0.56	1.15	0.55
var (participant ID)	0.59	0.09	0.51	0.08	0.50	0.08	0.51	0.08

• ** p < 0.01.
• * p < 0.05.
• † p < 0.1.

4.9 Study 3 Discussion

The results of Study 3 replicated Hypothesis 2 for the third time testing for the additional effects of signal set congruence in human–AI teams. Although agreement between the human and AI team members was the largest driver of AI advice taking, optional AI advice again seemed to improve the likelihood that employees take AI advice, but only when participants chose to see team members' advice. Unlike in our previous studies, the three-way interaction also suggests that strong versus weak signal fit may have an effect on taking advice from AI; however, this effect is most pronounced when the signal set is incongruent and participants make a choice to see the advice of their team members.

One potential limitation of the studies presented thus far is their focus on a specific context: the facial recognition task, where AI and human partners work together to determine a definitive correct answer. This setup does not fully capture the complexities of real-world human–AI teaming, where decisions often involve ambiguity, conflicting priorities, or subjective judgments. To address this and test our hypotheses in a broader context, Study 4 examines human–AI teaming in the recruitment domain.

4.10 Study 4 Method

4.10.2 Procedure

In Study 4, we adapted the experimental design from Study 3 to the recruitment context. Each participant was randomly assigned into one of four experimental conditions following a 2 (signal fit: strong versus weak) × 2 (option to solicit advice from team members: yes or no) between-subject design. Participants were provided a job description for a junior software position and were given the task of evaluating eight CVs for a software engineering position. Within each condition, participants completed a series of eight CV evaluation tasks. Participants were instructed to play the role of the leader of a team consisting of a human coworker Sam Smith and a resume screening algorithm, Iris AI. Participants' task was to choose whether to invite each candidate to an interview. As the leader of the human–AI team, they held the final decision-making authority.

We developed eight fictional CVs that differed by gender and race of the applicant while keeping professional qualifications (education, previous work experience, and hard and soft skills) directly comparable. Half of the fictional CVs featured female and racial minority applicants. As the study focused on the United States, we chose Black/African American–sounding names to signal the ethnic background of the candidates. Gender and ethnicity could be inferred from the names (e.g., Anna Wagner and Xavier Jeanbaptiste). Figure 6 presents an example of a fictional CV. All job applicants held a bachelor's degree in software engineering from one of the Top 10 universities in the United States awarded between 2017 and 2020. Additionally, they completed an internship of 3–6 months and accumulated 3–5 years of work experience with one of the Top 10 employers in the United States within their respective fields. The selection of applicants' names, universities, and employers was meticulously drawn from reliable sources, including top baby name lists in the 1990s, top employers' rankings, and prestigious university rankings in the United States.

The experimental setup followed a two-step process parallel to that of Studies 1–3. In the first step, participants were asked to review the CV of a job candidate and to decide whether to invite them to an interview. In the second step, depending on their condition, participants were sent a signal with strong versus weak fit from their teammates (Sam Smith and Iris AI) and were either required to see their teammates' decisions (mandatory advice) or were allowed to choose whether to see their teammates' advice (optional advice). Participants then had to make a final decision whether to invite the candidate for an interview. In total, participants evaluated a series of eight pretested CVs that were evenly balanced by race (Black vs. White names) and gender (women's vs. men's names).

Each employee was given signals in a format similar to Studies 1–3 (see Figures 7 and 8). The signal fit was manipulated using similar bar charts as in the previous studies. In the weak signal fit condition, participants saw Iris AI's historical selection ratio of applicants from each of the four demographic categories (Black vs. White applicants) and (women vs. men applicants). For half of the CVs, Sam Smith provided the same advice as Iris AI, whereas for the other half, they provided the opposite advice. These recommendations were evenly balanced between “yes, invite for an interview” and “no, do not invite for an interview” decisions. To mirror the high-stakes scenario of previous studies, participants were informed that the stakes were high for this open position and that they could interview only four out of the eight job candidates.

4.11 Study 4 Results

Table 11 displays the means, standard deviations, and descriptive statistics of our Study 4 variables. Results show that when accounting for only the main effects of our experimental conditions, signal fit had no significant effect on the likelihood of AI convergence regardless of whether additional information was presented as bar charts showing historical selection ratios of Iris AI (Odds ratio = 0.879, p = 0.183). Furthermore, like in previous studies, we found that being in the optional AI advice condition reduced the likelihood of convergence between participants and Iris AI (Odds ratio = 0.672, p < 0.001), replicating the findings from Studies 1–3 related to Hypothesis 2. As in Study 3, we find the key driver of AI convergence is the congruence of the signal set within the team (Hypothesis 3). When the signals sent by Iris AI and Sam Smith were congruent, participants' decisions were much more likely to converge with Iris AI (Odds ratio = 3.717, p < 0.001).

Accounting for the choice to see the advice by Sam Smith and Iris AI in the optional AI conditions, we find a different pattern of results than was found in Studies 1–3. First, the effect of choosing to ignore teammates' advice decreased the likelihood of taking Iris AI's advice (the direction of the effect was the same as in previous studies), but this effect was only trending (Odds ratio = 0.724, p = 0.085). Furthermore, the impact of being in the optional AI condition did not become positive when accounting for participants' decisions to see team members' advice. In other words, the effect remained negative and significant (Odds ratio = 0.701, p < 0.001). Thus, Hypothesis 1 was not supported, and Hypothesis 2 was only weakly supported. Hypothesis 3 was supported.

5.1 Theoretical Implications

Building on prior research, our study investigates human–AI teaming within various organizational contexts (Lundberg et al. 2018; Patel et al. 2019), specifically extending our knowledge of the factors influencing AI convergence in human–AI teams. In particular, our research applies signaling theory coupled with self-determination theory to explore how qualities of specific signals, such as signal fit, optional AI advice, and signal set congruence impact employees' AI convergence. Although signaling theory has already been applied to a variety of different contexts (e.g., entrepreneurship, leadership, and diversity), to our knowledge, we are the first to utilize it to study teams. Understanding how employees interpret and respond to AI signals from multiple sources helps to improve the design and implementation of these teams in organizations. The expansion of the signaling theory offers new avenues for research and encourages interdisciplinary approaches that help shed light on the impact of signals on the AI convergence in different contexts. Further, incorporating facial recognition systems as a novel context into the field of organizational behavior, this study explores unique high-stakes scenarios in the complex and uncertain work environment.

Two main findings stand out in our studies and contribute to the theory. First, optional AI advice is an important element of human–AI teaming. When employees choose not to see AI team members' advice, they are much less likely to make the same decision as their AI teammate. This rather intuitive finding becomes more compelling with our results indicating that after accounting for this behavior by employees, AI convergence was most likely when employees had optional AI advice that they chose to see. This corroborates results from previous studies suggesting that humans appreciate AI advice and are willing to rely on it as long as they retain the ultimate control over the decision-making process (Bundorf, Polyakova, and Tai-Seale 2019; Dietvorst, Simmons, and Massey 2018; Haesevoets et al. 2021; Logg, Minson, and Moore 2019). Furthermore, our findings lend credibility to the idea that self-determination theory is important in understanding the impact of signals in human–AI teams. As human–AI teams become commonplace, AI convergence will be impacted by work design and team design to the extent they foster or inhibit autonomy, competence, and relatedness (Gagné et al. 2022). Our research suggests that having the option to solicit AI advice rather than making AI mandatory is one path to influencing AI convergence in teams.

Second, employees were less likely to exhibit AI convergence when their human counterpart provided an incongruent signal compared with their AI teammate. Another way of interpreting this finding is that when humans and AI on a team are misaligned in their signals, human convergence is much more likely than AI convergence. The results of exploratory analyses in Study 3 showed a significant three-way interaction between signal congruence, signal fit, and optional advice, such that the negative effect of incongruent signals on AI convergence could be somewhat mitigated when the signal fit was strong, and seeing the AI advice was optional. However, the three-way interaction was not replicated in Study 4, possibly because of the more ethically charged decision context.

In combination, these findings highlight a significant limitation in human–AI teams and suggest potential remedies that require further investigation. When disagreements, assessments, and advice are incongruent among team members, the likelihood of AI convergence diminishes, though this could potentially be mitigated through the deliberate design of AI signals and processes that prioritize human choice in decision-making.

5.2 Practical Implications

Our research findings hold valuable managerial implications. First, organizations seeking to form and utilize human–AI teams for solving complex tasks should focus on encouraging a collaborative approach, where all team members' advice is taken into consideration. It is essential that AI advice is viewed as a complementary tool that employees choose to see rather than a mandatory replacement for human expertise and judgment. Second, organizations and their leaders need to ensure that employees on human–AI teams take AI advice seriously. In designing human–AI team processes, it is critical to ensure that employees are more inclined to choose to see AI advice rather than ignore it. Companies should emphasize this through employee training. Third, giving human–AI teams tasks that are likely to result in disagreements between human team members and AI team members will reduce the likelihood that AI advice is taken. In these situations, creative solutions and redundancies may be necessary. Fourth, designing a simple user interface that facilitates easy access to advice from both human and AI team members and encourages interaction among team members (Noti and Chen 2023) is a critical component for effective human–AI teaming in decision-making. Finally, given the impact of incongruent signal sets, businesses may want to introduce optional AI advice gradually into their organizational processes by starting with noncritical tasks or projects where the positive impact of AI advice can be easily assessed (Einola and Khoreva 2022).

5.3 Future Research and Limitations

Our results provide empirical evidence that optional AI advice and congruent signal sets enhance AI convergence when humans elect to utilize the AI, but that signal fit is less important than we had initially believed. However, in Study 4, we did not find that optional AI advice had a positive effect on AI congruence, as participants were more likely to take AI advice when they were forced to see it rather than when it was optional. Future research should explore the boundary conditions of our predictions in both experimental and field-based research. In mastering the task of evaluating CVs, participants needed to invest more time and energy into their initial decisions before they had an opportunity to see their team members' advice. This may have influenced how participants approached optional advice. Additionally, Study 4 required participants to select interview candidates from a pool of equally qualified individuals, where no candidate was clearly unfit for the job, contrasting with Studies 1–3 that had clear, objective answers. Furthermore, in our first three studies, trust in AI was a significant predictor of AI convergence, yet this was not the case in Study 4. Although we can only speculate on what might have caused these results, we believe that future research should investigate what factors might alter the impact of optional AI advice on AI convergence. Specifically, if there is a threshold of time and effort beyond which employees become less likely to take AI advice, this would be critical for organizations to know. This is all the more important given that AI is not infallible, and in some situations, AI convergence may not be the optimal outcome. Future research should explore the conditions under which AI divergence is the optimal outcome, yet may be hindered by overreliance on AI or poorly designed decision-making processes. Similarly, it could be the type of task and its moral implications, which causes mandatory AI advice to lead to AI convergence rather than optional advice. Finally, there could be different ways of manipulating signal fit that alter the impact of optional AI advice and congruent signal sets on AI convergence.

It is important for researchers to replicate and extend our findings. Scholars should especially consider what additional factors contribute to employees' decision to search for and utilize optional AI advice. Understanding what drives team members to choose to see AI advice when they have the option to ignore it could help scholars understand when and how optional AI advice has the greatest impact. It is possible that different types of information and various types of explanations vary in their effectiveness in providing signal credibility and fit across contexts and among individuals. Exploring this aspect further could be a fruitful area for future research. Another promising area would be to investigate further components of signals and signal sets (e.g., response time, receiver feedback, interactions between team members, and repeated signals) that may influence AI convergence in human–AI teams. Finally, although AI does not have free will by today's technology standards, the extent to which employees perceive the AI to have warmth, competence, or agency may alter how signals sent by AI are interpreted. Future research should explore this possibility especially given research suggesting cross-cultural differences may influence how individuals attend to and interpret various signals and signal sets from AI and robots (e.g., Mantello et al. 2023).

Our conclusions must be considered in light of a few limitations. Across four studies, one of our three hypothesized relationships was not significant, and in one experiment, results, in part, were the opposite of our predictions. Although we were unable to replicate the three-way interaction from Study 3, our results do suggest that future research should continue to explore the congruence of signal sets, optional AI advice, and signal fit. A further limitation of our studies is that the hypothetical scenarios we presented to participants involved at most a team of three members. In reality, human–AI teams can be larger, and this adds complexity to the potential combinations of congruent and incongruent signal sets. We encourage scholars to investigate in the future how balanced and imbalanced signal sets, as described by Drover, Wood, and Corbett (2018), influence AI convergence.