Hosted by Jeff Walter, Founder and CEO of LatitudeLearning
Artificial intelligence has rapidly become one of the most discussed topics in learning and development. Organizations everywhere are experimenting with new tools, exploring automation, and asking how emerging technologies might improve training outcomes. Yet amid the excitement, one question remains remarkably consistent: How can AI actually help people learn better?
That question sits at the center of a fascinating Training Impact Podcast conversation between Jeff Walter and Ashley Etemadi, Managing Partner of Research at Motiva Education. Rather than approaching artificial intelligence as a replacement for instructors or human expertise, Ashley presents a far more balanced vision. Throughout the discussion, she explains how machine learning, learning analytics, and generative AI can strengthen education by helping organizations understand learners more deeply, personalize support, and create realistic practice environments while preserving the uniquely human judgment that technology cannot replicate.
For learning and development leaders, the discussion offers an important perspective. Technology should not simply make learning more efficient. It should make learning more effective. The objective is not automation for its own sake but creating environments where learners receive the right support at the right time and develop skills they can confidently apply in the real world.
That philosophy has shaped Motiva Education since its founding and provides a compelling glimpse into what the next generation of workforce learning may look like.
Ashley Etemadi’s journey into learning technology reflects an unusual combination of enterprise technology, academic research, and educational innovation. After beginning her career implementing large-scale technology projects, she discovered that the educational initiatives she worked on were the most rewarding. That realization eventually led her toward graduate research focused on artificial intelligence, immersive learning, and workforce education before helping launch Motiva Education.
Today, Motiva Education works with universities and increasingly with corporate organizations to improve learner success through learning analytics, machine learning, and generative AI. Their work spans predictive analytics, simulation-based learning, and intelligent learning environments designed to help organizations better understand learner behavior and improve educational outcomes.
Unlike many conversations surrounding AI that focus almost exclusively on automation, Ashley repeatedly emphasizes augmentation. The goal is not replacing human expertise but strengthening it. Artificial intelligence excels at recognizing patterns across enormous amounts of learning data, while instructors, coaches, and leaders provide the judgment, empathy, and contextual understanding that machines still cannot replicate.
That distinction becomes one of the defining themes throughout the conversation.
One of the most interesting areas explored during the discussion involves Motiva Education’s work with predictive learning analytics.
Ashley explains how machine learning models can analyze activity within a learning management system to estimate whether a learner is likely to complete a program or disengage before finishing. Rather than relying solely on quiz scores or assessment performance, these models evaluate a much broader collection of learning behaviors. Activity levels, participation in discussions, completion patterns, interaction with course materials, and dozens of additional behavioral indicators combine to create a more complete picture of learner engagement.
This approach challenges a common assumption in education. Poor assessment scores do not automatically indicate that a learner will fail. Some learners struggle academically but remain highly motivated. Others earn excellent grades while quietly disengaging because the material no longer challenges them.
By recognizing these different behavioral patterns, organizations can intervene much earlier and much more intelligently.
Instead of contacting every learner with the same generic message, instructors and student success teams gain insight into who may actually need assistance and, perhaps more importantly, why. A learner who has stopped logging in may require personal support. Another learner who consistently performs poorly may need additional instructional guidance. Someone else may simply require greater intellectual challenge because the material has become too easy.
The result is a far more personalized learning experience.
For learning professionals, this represents an important evolution. Learning analytics become less about measuring performance after the fact and more about identifying opportunities to improve learner success while there is still time to act.
Although Ashley clearly believes in the value of learning analytics, she is equally thoughtful about its limitations.
The conversation turns toward emerging technologies capable of collecting facial expressions, body posture, eye movement, and other forms of multimodal data. While these technologies generate significant excitement within educational research, Ashley expresses caution about relying too heavily on these signals as indicators of learner engagement.
A learner leaning back in a chair may appear disengaged while actually thinking deeply. A serious facial expression may reflect concentration rather than frustration. Cultural differences further complicate interpretation, as classroom participation and communication styles vary dramatically across regions and backgrounds.
Her perspective highlights an important principle for organizations adopting artificial intelligence.
Data should inform human decision making rather than replace it.
Learning analytics can identify patterns that deserve attention, but interpreting those patterns still requires empathy, context, and professional judgment. Effective learning organizations recognize that every learner represents far more than the data they generate.
That balanced perspective appears repeatedly throughout the discussion and serves as an important reminder that responsible AI implementation requires both technological sophistication and human understanding.
As the conversation continues, Ashley shifts from predictive analytics to one of the most exciting applications of artificial intelligence in learning today: simulation-based skill development.
Many organizations have become comfortable using AI to answer questions, summarize information, or automate repetitive work. Motiva Education is taking a different approach. Instead of replacing people, Ashley and her team are using generative AI to create realistic practice environments where learners can safely develop human skills before applying them in high-stakes situations.
Negotiation became the perfect testing ground.
Unlike technical procedures that often have clearly defined right and wrong answers, negotiation requires judgment, adaptability, emotional intelligence, and communication. Success depends not only on understanding facts but on understanding people. That makes it an ideal example of a skill that learners struggle to develop through traditional classroom instruction alone.
Ashley explains that many negotiation courses rely on lectures, videos, or occasional role-playing exercises with classmates. While these methods introduce important concepts, they rarely provide enough realistic practice for learners to develop genuine confidence. Every human interaction is different, making it difficult to standardize learning experiences or provide meaningful feedback at scale.
Generative AI changes that equation.
By creating AI-powered negotiation partners, Motiva Education can place learners into consistent, realistic scenarios where they practice conversations, receive feedback, and experience situations that closely resemble the workplace. Every learner encounters the same scenario, allowing instructors to evaluate performance more consistently while still giving learners opportunities to explore different approaches.
Rather than simply telling learners how to negotiate, the simulations require them to negotiate.
That distinction represents a significant shift from knowledge acquisition toward skill development.
One of the most thought-provoking moments in the conversation comes when Jeff asks Ashley to explain the difference between reasoning and judgment.
Artificial intelligence, she explains, has become remarkably capable at reasoning. It can recognize patterns, process enormous amounts of information, make predictions, and generate logical responses based on available data.
Judgment is different.
Judgment depends upon context, culture, relationships, prior experience, subtle emotional cues, and countless forms of information that humans process almost instinctively. It reflects not only what someone knows but how they interpret situations based on years of lived experience.
Ashley illustrates this using negotiation.
Negotiating with a colleague differs from negotiating with a parent. Negotiating in one country differs from negotiating in another. Even within the same culture, expectations change based on geography, organizational norms, and personal relationships. A successful negotiator constantly adjusts based on signals that are often difficult to define, let alone encode into software.
This is where Ashley believes humans continue to hold a distinct advantage.
Rather than expecting AI to replace human judgment, she sees technology as a partner that strengthens preparation while allowing people to exercise their uniquely human abilities during actual interactions.
It is an important distinction for organizations investing in artificial intelligence. The objective should not be eliminating people from the learning process but giving them better opportunities to develop the skills that only people can provide.
One of the strongest messages throughout the conversation is that meaningful learning requires repeated practice.
Ashley shares findings from early pilot studies where learners participated in AI-powered negotiation simulations. Participants reported feeling genuine emotional tension during conversations with AI-generated managers, even though they fully understood they were interacting with software. When simulated supervisors became dismissive or challenging, learners experienced many of the same emotions they would encounter during real workplace discussions.
That emotional realism matters because confidence cannot be developed through observation alone.
Watching someone negotiate does not automatically prepare another person to negotiate effectively. Reading about difficult conversations does not eliminate anxiety when those conversations become real.
Instead, confidence develops through repeated exposure.
Ashley emphasizes that one simulation rarely transforms behavior permanently. Like any other complex skill, negotiation improves through multiple experiences presented under different circumstances. Learners gradually recognize patterns, build confidence, and develop judgment by encountering similar challenges in varied contexts.
This observation resonates far beyond negotiation training.
Sales conversations, coaching discussions, customer interactions, leadership conversations, conflict resolution, and performance reviews all depend upon repeated practice rather than one-time instruction.
For learning leaders, the implication is significant. Training should increasingly focus on creating opportunities to practice rather than simply opportunities to consume information.
Perhaps the most encouraging aspect of the discussion is that it never frames artificial intelligence as the hero.
Instead, Ashley consistently returns to the same principle: technology should strengthen people.
Machine learning helps identify learners who may benefit from additional support. Learning analytics provide instructors with better visibility into learner progress. Generative AI creates realistic environments where people can practice difficult conversations before those conversations affect careers, relationships, or organizational performance.
The common thread running through every example is empowerment.
Technology becomes valuable because it allows educators, coaches, and organizations to provide more personalized support while giving learners greater confidence to apply their skills in authentic situations.
That philosophy reflects a mature perspective on AI adoption. Rather than pursuing technology because it is new, Motiva Education demonstrates how emerging technologies can solve genuine learning challenges while preserving the uniquely human qualities that make education transformative.
The conversation with Ashley Etemadi offers a refreshing perspective on artificial intelligence in learning and development. Instead of asking how AI can replace instructors or automate education, she asks a more meaningful question: How can AI help people become better learners?
By combining learning analytics, machine learning, simulation-based learning, and human judgment, Motiva Education is helping educational institutions and organizations create more personalized, engaging, and effective learning experiences. Their work demonstrates that the future of learning is not about choosing between humans and technology. It is about designing systems where each contributes what it does best.
For learning leaders exploring the next generation of workforce development, customer education, or higher education, this episode provides valuable insights into how artificial intelligence can move beyond efficiency and become a catalyst for deeper learning, stronger decision making, and more meaningful learner success.
To learn more about Motiva Education and their work, visit https://motivaeducation.com/.
For more from the Training Impact Podcast, follow us on Social Media: https://t-sml.mtrbio.com/public/smartlink/trainingimpactpodcast
Jeff Walter (00:04)
Hi, I’m Jeff Walter and welcome back to the Training Impact Podcast, where we explore scaling performance through training infrastructure. My guest today is Ashley Adamati. She is the managing partner and research at Motiva Education, where she helps educational institutions apply learning analytics, machine learning, and generative AI to improve student engagement, learning outcomes, and success. Ashley, welcome to the program.
Ashley Etemadi (00:28)
Thank you, Jeff. It’s great to be here.
Jeff Walter (00:31)
And so Ashley a as as all my listeners know, I’m always interested in the autobiographical. How did you become a managing partner at Motiva? And and what is what is Motiva and how did you end up as
Ashley Etemadi (00:42)
Sure. So I’ll start with what Motiva is first and then I’ll backtrack. So Motiva is a higher education consultancy and we work with generative AI, machine learning and data analytics to support universities in providing the best learning outcomes for their students. And we also work, we are starting to work outside of the higher ed space with corporate as well. And so.
Our focus typically is bespoke at-risk prediction models that use machine learning. And then we also do a lot of work in simulation-based learning. So creating simulations using generative AI and some of the emerging technologies and for improving student success outcomes. And I think we’ll talk more about that later.
Jeff Walter (01:30)
Yeah, so yes, the and you you s you said bespoke what was that bespoke what?
Ashley Etemadi (01:37)
at-risk prediction models, so.
Jeff Walter (01:39)
Okay, so I understand simulation. what is a bespoke at risk mo no bespoke at risk model.
Ashley Etemadi (01:48)
Yeah, so we can use the LMS data to understand whether students are going to complete the degree or the certificate that they’re enrolled in and or whether or not they’re going to drop out. And so as they’re going through the curriculum, we can gauge, OK, the student is on track to complete or actually the student has a 75 percent likelihood of dropping out of this curriculum.
And so we can create prediction models that give estimates as to whether or not a student will drop out or not.
Jeff Walter (02:25)
Interesting. What and what characteristics is that just what what what characteristics of the student help predict that?
Ashley Etemadi (02:33)
Yeah, so we don’t use actually any demographic data. We only use learning data. So for example, some features in this model might be your activity in the LMS in the last 24 hours or 48 hours, two weeks, whatever timeline it may be,
Jeff Walter (02:51)
Okay.
Ashley Etemadi (02:52)
or whether or not you’ve completed certain exercises, what marks or scores you’ve received, whether you’ve posted in a discussion group or not.
what the content of your posts, so for example, how long your posts are, whether or not your posts actually address the questions that were posed by the instructor, those can all be features, and features are very much tailored to the student group or the learner population. And so the features for one organization or one institution might be completely different than those for another.
And though we do see a lot of overlap, and there might be 75 or 100 features that get factored into these models.
Jeff Walter (03:39)
interesting. So I mean, as as you were talking about it like the you know, a as a layman I would sit there and go, Okay, one would think that poor scores on quizzes and tests and all that would be a predictor of you know a dropout or or you know, failure to complete.
But you’re you actually start to get into behavioral things. Your features have a lot of behavioral elements in there, like posts,
Ashley Etemadi (04:03)
Correct.
Jeff Walter (04:03)
quality of posts, length of posts, stuff like that. And so you’re looking at, you know, digitized behaviors and then using
Ashley Etemadi (04:11)
Correct.
Jeff Walter (04:11)
digitized behaviors in addition to assessments. it’s interesting. It’s very interesting. And then what do you guys
Ashley Etemadi (04:16)
Yeah, because assessment
on its own might not be the best predictor of whether
Jeff Walter (04:21)
Right.
Ashley Etemadi (04:21)
or not a student is going to drop out. They might have received a 5 out of 10 on the last quiz or the last three quizzes, but they’re very adamant about improving their performance or completing the certificate. Or it might not be. They might have received 5 of 10 on the last three assessments, and that might have
discourage them from continuing in the certificate program. So it’s very much student-based and that’s why there’s multiple features that get factored in, not just score, assessment performance.
Jeff Walter (04:54)
and and then I guess the the next question I’d have on that is okay, once you build that model that says
Hey, you know, based on these factors, this person has a seventy five percent probability of of discontinuing. What type of intervention strategies do you see that actually have so what do you do about what once you know that, what do you do about it?
Ashley Etemadi (05:16)
That’s actually an interesting question because you could do a lot of different things and the interventions that you have could depend on the reasoning behind what the student’s high likelihood of dropping out is. So for example, if we, the model says that there’s a 75 % chance of this learner dropping out and then you use some AI explainability that says what’s driving that 75 % is the fact that, you know, the student
performed really poorly on the last three quizzes and that’s driving the high dropout likelihood. Then what you might do is you might contact the student and see what supports you could provide or what they’re finding difficult in these assessments. Or for example, if it’s login activity, so for example, they haven’t logged in in the last two weeks, you could contact them and see, hey, are you on vacation? Is it something happened in your family?
and you have some sort of a background as to what’s driving the student’s high likelihood of dropout. And so that might color the intervention that you end up carrying out for that student. And so coupled with, it’s not just the high likelihood of dropout, you also might want some explainability as to what’s driving that high likelihood.
Jeff Walter (06:36)
Interesting. And and then and then nudging them basically it’s an early warning s system to then engage in some way to see if everything’s on track or do you need help doing this or are you struggling with that? Like again
Ashley Etemadi (06:50)
Correct. Yeah.
Jeff Walter (06:52)
getting back to the explainability, it’s like just checking in, making sure everything’s okay. You know, and and it could be like, you know, I was sick for two weeks and you know, or
It could be like or something else. You know, they you know, behaviors change before a bunch of different reasons, right?
Ashley Etemadi (07:07)
Yeah, and
what we hear from our clients is that, you know, before they had the prediction models, they were just contacting students and, you know, trying to figure out is the student happy? Is the student going to continue in the program? But they had no real benchmark for who to contact or why they were contacting that student. But now with the prediction, they know, OK,
this student has a high likelihood. And when they contact, it can be sometimes creepy, because the student is like, actually, yes, I was about to drop out, and this is what I’m struggling with. And if it’s content, then they can refer that to the faculty member to maybe support that student more. If it’s something outside, like, this is a personal issue, then they can have, you know.
interventions of how to figure out to support that student outside the classroom environment. And so they have much more information to provide a tailored experience to the students and also to prioritize which students to contact. Because before, you’re just, you know, student success coaches are contacting everyone. They have a list of students that they manage. Whereas now they can better prioritize their workload. Okay.
Joe Smith has a 75 % chance of dropping out. And so I’m going to contact Joe Smith before I’m going to contact Susan, because Susan has a 30%. So maybe Susan is at the bottom of the pile. And then maybe Susan increases in likelihood over the next two weeks. And then Susan becomes priority.
Jeff Walter (08:50)
Yeah, yeah, it that’s really it’s really interesting and where my brain is going and just going down into this little rabbit hole is it’s and it’s kind of something we talked about earlier, which was the mass digitization of so many things. You know, in this case like you wouldn’t have had that digitized data in a classroom setting where it’s like, Hey, we’re we’re capturing how many times you raise your hand, how many tim like
I mean maybe y yeah like we’re we’re capturing so much more data about the behavior.
Ashley Etemadi (09:18)
Mm-hmm.
Jeff Walter (09:20)
that it’s you know right.
Ashley Etemadi (09:21)
Specifically with online activity is what we
work on. I don’t know, I’m sensitive about capturing data in classrooms because, but there’s actually interesting work being done by Bertrand Schneider at the Harvard Graduate School of Education on this using multimodal analytics to
Jeff Walter (09:41)
Yeah.
Ashley Etemadi (09:41)
understand, okay, based on students’ positions, how often they raise their hand, their facial expressions, how can you gauge
learning and learning outcomes using that multimodal data, which I find very fascinating.
Jeff Walter (09:57)
Yeah,
you know, it’s this really interesting I think it’s one of these things that we as a society are going to grapple with, not to go deep off the rails here, but that we’re gonna have to grapple with as more and more of our behaviors get digitized. Yeah.
Ashley Etemadi (10:13)
Yeah, and that’s something
that I wanted to point out is the reason why I’m not as interested in multimodal learning analytics is because, you know, me slouching or me lying back in my seat for someone else might be considered disengagement. But for me, I might still be very engaged. I might just, you know, have back problems or, you know,
Jeff Walter (10:37)
Right.
Ashley Etemadi (10:37)
might just be relaxing and
pondering as to what the instructor is talking about. But in terms of when you put everyone on an average, then that type of bodily behavior might be considered disengagement. And that’s why I struggle with that type of multimodal.
Jeff Walter (10:57)
Yeah.
Ashley Etemadi (10:58)
And not only is it personal, but I also wanted to mention it’s cultural too. So I’ve read studies where East Asian students are expected
to raise their hand less in certain cultural settings. And the instructor is much more of the sage and the holder of the wisdom that you’re supposed to be listening to. And so there’s less question asking while the instructor is talking and more processing of the information. And then if you do have any questions, then you formulate those after the instructor has stopped talking. So.
There’s a cultural element to asking questions and posture. And so I struggle
Jeff Walter (11:38)
Right.
Ashley Etemadi (11:38)
with that type of data being collected and used as a precursor for understanding student engagement and student outcomes because there’s so many other variables. And I’ve used facial recognition data for actually my class with Bertrand when I was a grad student. And it was…
The facial recognition would say that I was upset or I was mad or disengaged just because I had a flat face and I wasn’t smiling. And that wasn’t true at all. That was just my resting face. so everyone’s
Jeff Walter (12:12)
Right.
Ashley Etemadi (12:13)
resting face is also different depending on the day too. You have different moods and, but I don’t know how much that necessarily impacts your engagement or your learning outcomes. If.
One day you’re learning and you’re very pensive and one day you’re learning and you’re very happy. You just had the best meal of your life for lunch. And so I struggle a little bit with that, but.
Jeff Walter (12:38)
Yeah,
well it yeah, it well it’s interesting because it’s on the one hand you could see how that information can be used to support the individual, right? Like like you know, to get better learn well, to get better learning outcomes or just outcomes in general. And then on the other hand you can see how that information could be used to stifle the individual and enforce conformity.
Right.
Ashley Etemadi (12:59)
Correct.
Jeff Walter (13:00)
Yeah, like like I just got a new infinity and it’s self-driving and it’s really cool. Except it watches my eyes.
Ashley Etemadi (13:08)
Yeah.
Jeff Walter (13:09)
Yeah, and and like and if I, you know, turn my head for more than a few seconds to look at like the passenger, I get a I get a warning light saying, Look forward. I’m like I’m like I’m not sure I like that. Right.
Ashley Etemadi (13:22)
Yeah.
Jeff Walter (13:22)
But on the other hand, it’s
Keeping me safe, right? Like it’s just it it and and but but it’s interest it it’s it’s really interesting because on and on the other hand it’s you can have more effective intervention in terms of, hey, you’re here at this university, you’re or you’re in this program, you’re trying to get to the end, getting to the end gets you a certificate that has is you know, a diploma or certificate or whatever that demonstrates some level of knowledge and skill and and therefore is you know, the
Has benefit for you and if we can do an intervention, if we can pick up on these behaviors and do an intervention that helps you over to overcome something. Because it’s the flip side of you know, you’re whenever anybody is struggling, you always feel alone. It’s like nobody else is struggling. I’m having
Ashley Etemadi (14:11)
Mm-hmm.
Jeff Walter (14:12)
I’m having this type of struggle. And I’m the only person in the universe that’s ever had this type of struggle.
Ashley Etemadi (14:17)
Yeah.
Jeff Walter (14:18)
It’s
like, no, we’ve got some pattern recognition. Thousands of people before you have had that type of struggle. And we know that if we make this little adjustment over here, it helps get you back on track. And so
Ashley Etemadi (14:29)
Sure.
Jeff Walter (14:29)
you know, so it’s it’s that’s that’s really interesting to use the analytics to build those types of models to ha ’cause then you get more successful outcomes. Right? Which which is
Ashley Etemadi (14:40)
Yeah,
that’s the idea that also for universities or for institutions, organizations that need to manage their support systems for students, that they can better triage students that need support versus those who might be doing fantastic. And there’s another thing as well that I find really interesting is that students have a zone of
Approximal development so it could even
Jeff Walter (15:09)
Right.
Ashley Etemadi (15:09)
be that the student is finding the material too easy and that’s why they want to drop out So it’s not just necessarily a
Jeff Walter (15:15)
Yes.
Ashley Etemadi (15:16)
student that’s struggling with the material or finding it challenging, but it’s also those who are finding it you know too easy and so it’s not challenging enough and That’s what I find also really interesting. So we think a lot about at-risk students as those who are falling behind in the curriculum, but it could also be those who are
you moving so quickly that you know they find this they’re disengaged because it’s too easy or it’s not challenging
Jeff Walter (15:43)
Yeah, that
Ashley Etemadi (15:44)
enough
Jeff Walter (15:44)
that well that that’s that’s yes, that’s interesting because usually when as soon as you say at risk, my I think my mind and most people’s mind go to, they’re failing. They’re failing the you know, whatever the assessments are, they’re failing. Right? They’re
Ashley Etemadi (15:58)
Yeah.
Jeff Walter (15:58)
and and and so nobody wants to be failing, so then they drop out. But you’re going the other side, which also it which is I am so bored because this is so simple.
And I and and then I get restless and then I become disruptive or I just wanna leave because and it’s the other end of the spectrum. But meanwhile they’ve got straight A’s, right? Yeah. yeah, interesting.
Ashley Etemadi (16:22)
Yep, exactly. So at risk could be people on
any end of the spectrum, essentially those who are at risk of dropping out, but it could
Jeff Walter (16:31)
Yeah.
Ashley Etemadi (16:32)
be for a multitude of reasons.
Jeff Walter (16:33)
Yeah. Well and then the intervention in that latter one would be, well, we really need to get you to a more advanced level of you know, don’t take physics one two. Let’s jump you into a physics two hundred course. Let’s you know, let your next one let’s get you let’s let’s get you more challenged, right?
Ashley Etemadi (16:50)
Yeah,
or maybe the instructor providing more challenging assignments or assessments for the learner,
Jeff Walter (16:55)
Yeah.
Ashley Etemadi (16:56)
providing a range that people could go through. Or for example, there were some courses in my master’s program where they would provide the material for everyone and then say, you know, if you want more challenging material or if you want to read more studies that go deeper into this topic, here are some additional papers that you could look at.
And so you had the option to look at higher level, maybe doctorate level work
Jeff Walter (17:22)
Right.
Ashley Etemadi (17:23)
that maybe isn’t necessarily for a typical entry level master’s student, but that option is there if you want it.
Jeff Walter (17:31)
Yeah, very interesting. And so that you don’t even go into that course because you’re you’re like, you know, especially at the post grad level where it’s like everybody’s c you you know, especially if you’re coming at w a f with a few years of experience. It’s like look if you
Ashley Etemadi (17:53)
And that’s where I find
simulations really interesting, because you can actually put someone in a simulated scenario and see where their skills are. So there’s one thing where someone tells you, I would consider myself an expert level negotiator. Or you could put them in a simulation and see what are their negotiation styles? Where do they excel in? And what are areas that they could improve on?
Jeff Walter (18:19)
Yeah,
well well it’s it’s it’s that’s interesting ’cause it’s a more real world test out
Ashley Etemadi (18:26)
Mm-hmm.
Jeff Walter (18:27)
you know, instead of in yeah, I that’s interesting. So on simulation, what what have you seen on simulation? Well actually let me back up ’cause there’s we skipped over so how did you end up as managing director? What was your what’s your what’s your path? We jumped right into you know, at risk prediction models.
Ashley Etemadi (18:45)
Yeah, so I started in technology at Accenture. I was at Consul and Accenture. I was working on large deployments for corporations. And one of my projects was on health education. And that was my favorite project. And I decided then and there that I wanted to work in education in some capacity, and specifically in workforce training.
Jeff Walter (19:09)
Mm-hmm.
Ashley Etemadi (19:09)
And so then I went on and did a work.
with a organization in Columbia that was working on creating certifications for the largest informal labor market in Latin America, which is Mechanics. And
Jeff Walter (19:24)
Okay. Yeah.
Ashley Etemadi (19:24)
then I went to Hugsie and started doing research with Professor Chris Deedy, and he was exploring AI and immersive media with education. And we started looking at, back then, this is
pre-2022, so OpenAI hadn’t launched ChatGPT yet, but we were looking at, you GPT-3s and what were some of the capabilities that AI had that humans didn’t have, and then what did humans have that AI didn’t have? And so what was the competitive advantage of each? It’s a concept we called intelligence augmentation. where both…
human and machine working together produce better outcomes than machines having their own reasoning and humans their own judgment in silos. And then we started working on this project in negotiation because we, in our research, we came to the conclusion that human judgment is something that machines, even AI, can’t replicate in the near term. And
Jeff Walter (20:31)
Mm-hmm.
Ashley Etemadi (20:31)
so machines have a lot of,
reasoning abilities so they have they can calculate they can predict we can create you know machine learning models that can you know predict or give some sort of percentage of student dropout we can We can predict engagement and courses, but there’s a lot that that comes with being human and understanding context and culture
And so we identified negotiation as a skill that was particularly human. while bots can bargain on
price
Jeff Walter (21:09)
Right.
Ashley Etemadi (21:11)
and very objective measures, it’s much harder for them to navigate the complexities of a negotiation to come at an agreement where it’s
the outcome is greater than what each individual was coming in to the negotiation with. So basically, one plus one is greater than two in the definition of negotiation that we were trying to train for. And so we started actually using generative AI to create exercises for learners around negotiation. And so actually putting them in the scenarios that they were reading about, or
watching on videos because a lot
Jeff Walter (21:52)
Uh-huh.
Ashley Etemadi (21:52)
of training is you know watch this video and When you watch the video the assumption is that you have acquired the skill you are now ready to go out and negotiate or a lot of the negotiation curriculum in Masters programs was okay. Here’s a role play exercise. You’re going to be assigned a partner in the class now the partner
Jeff Walter (22:13)
Right.
Ashley Etemadi (22:14)
is also trying to figure out how to negotiate the scenario and so
There’s not much control that a instructor has in terms of how the negotiation is unfolding. And they have no insight into how learners are doing. And so what we were able to do is we were able to use generative AI to create these negotiation exercises, have insight into how learners are doing, create standardized interactions that an AI was doing as a role-play partner.
And then we did research on that and that brought me to Motiva. So Motiva does a lot of this work. And after graduation, I didn’t see many organizations that were working at the intersection of emerging technologies and learning. And so that spun off Motiva and we collaborate a lot with professors. also, our partner is Dr.
Ryan Baker at the University of Adelaide, formerly at the University of Pennsylvania. And he still runs the Penn Center for Learning Analytics. And we collaborate a lot still with Professor Chris Deedy.
Jeff Walter (23:22)
Inter very interesting journey. I have a question on something you said with AI. You said very good on reasoning and not so much on judgment and and I I take that at face value, but what do you mean by what how are you defining judgment? Yeah.
Ashley Etemadi (23:38)
Yeah,
that’s a great question. So judgment is a collective of understanding culture, context, and bodily cues to
Jeff Walter (23:54)
Uh-huh.
Ashley Etemadi (23:55)
form a, I guess, not an opinion, but a deliberate,
How would I describe it? Like a deliberate action for how you should move forward based on those cues. so oftentimes judgment is intertwined with wisdom. And so for example, if you see something over and over again, you will kind of know that, okay, this is how I’m supposed to act in this social situation or something. I
Jeff Walter (24:30)
Right.
Ashley Etemadi (24:30)
can give an example. So in the…
in negotiation, for example, there’s ways of negotiating in a US context. then
Jeff Walter (24:38)
Yeah, thank you
Ashley Etemadi (24:39)
oftentimes, this is the extreme example that’s given. There’s ways of negotiating in a Japanese context. And
Jeff Walter (24:45)
Okay.
Ashley Etemadi (24:45)
so the AI, if it’s not actually prompted to understand a new culture, a new context, it might just go about
negotiating in a way, or not negotiating, but it might go about responding in ways that would be appropriate in one context, but not in another. And so, for example, I come from a cultural background where, you know, your elders aren’t supposed to be respected and you’re supposed to speak a certain way to your elders.
Jeff Walter (25:18)
Mm-hmm.
Ashley Etemadi (25:19)
And, and
But I come from a context within a context. So the city that my family is originally from has very different cultural mannerisms and expectations than the neighboring city. And so if you tell a bot that, you know, act as if you’re from this country, country X.
It’s not going to know the nuances between what it’s like to be in a small town versus a bigger city, and then this city versus another city. I always give the example of, say you’re negotiating with someone from New York, it’s going to be very different than having conversation with someone from California. And even
Jeff Walter (26:01)
Right.
Ashley Etemadi (26:01)
within California, everyone is so different. And so there’s a lot of judgment that goes into figuring out what’s the right approach in this scenario.
Another example that I give is negotiating with your kids or with your parents. With your dad, it’s very different than with your mom, at least in my family it is. There are things that work with my dad that don’t work with my mom. And there are things that, vice versa, that work with my mom that don’t work with my dad. And a but, unless it really would understand that person specifically, it might not understand the nuances of what it’s like.
to interact with one person over another, what it’s like to convince one person over another. It will give a generic response
Jeff Walter (26:44)
Mm-hmm.
Ashley Etemadi (26:46)
that might be void of the context, whether that be, and it also doesn’t have a body, so it can’t feel, it can’t taste. If you say, hey, is this mozzarella better or the other one? That’s also very much based on human preferences and.
culture, about which one you like more, but a bot won’t be able to analyze that. It doesn’t have the judgment to tell you, it doesn’t have the cues and human, like human senses to be able to tell you this mozzarella is better than this one. And so that’s what I mean by human judgment.
Jeff Walter (27:24)
Well that’s yeah, very you know, it’s a as you’re going through that, I w I was like, it it just reminds me that
As humans, I I got the where my brain went was the difference between conscious thought and all your subconscious. Right? As as humans, we are absorbing billions of bits of information a second through our senses, but we can only think in terms of bits per second. Like single right.
Ashley Etemadi (27:50)
Yeah, and you filter out a lot, right, when you’re
thinking. And so try feeding all of that data to a bot and saying what is most important.
Jeff Walter (28:01)
And that well and that judgment becomes that that feeling that’s reaction reacting to all that stimuli. Right? It’s like like take your example of Japanese negotiation versus American negotiation. Being an American and not being steeped in Japanese culture or any cult you know, any other culture. If I was to start a negotiation with somebody who is steeped in a Japanese culture
I would approach it similar to I would anything else, but very quickly pick up on very subtle cues that I may not even be conscious of that this is not going the way I thought it would. Because
Ashley Etemadi (28:40)
Mm-hmm.
Jeff Walter (28:41)
maybe I was too aggressive, too direct, too. Whatever it is, right? Whatever that context difference is, you would kind of like sense it. But you couldn’t put your finger on it. You couldn’t consciously necessarily unless you had been trained
be like, yeah, that’s right. I’m in an I’m in a Japanese negotiation, I have to behave this way. But you would sense something
Ashley Etemadi (28:58)
Exactly.
Jeff Walter (28:59)
and you would change your beh you’d you know, and it’s in humans you have mirroring, right? And you would just automatically subconsciously start mirroring the other person to pick up on their l on their particular context, you know. And it’s it it’s interesting. And whereas when I think when when I was listening to you talk about the bot, it’s almost like it’s only that conscious mind.
Right, it’s only that rational mind that it’s not picking up necessarily all that other context necessarily.
Ashley Etemadi (29:24)
Correct.
Jeff Walter (29:26)
Unl unless it has some way of being fed that context. You know. It’s i interesting.
Ashley Etemadi (29:31)
Yeah, and someone
asked me once, you know, if I just feed a chatbot all the context around me, I think it would be just as good as a human in terms of judgment. But there’s so much that we filter out that we subconsciously make that decision that we don’t even recognize that we’re making that decision. It reminds me of system one and system two from Daniel Kahneman. There’s these automatic…
that we use. For example, eye contact is a big one. Or whether or not someone is fidgeting when they’re talking to you. That’s not something you can fully feed to an AI because some people just naturally fidget more than others, but you can see if someone, you know, first started off as not fidgeting and then started fidgeting more, then you can, your judgment
kicks in and you’re like, this person is uncomfortable. But whereas a bot might feed it a bunch of pieces of information and might not be able to filter out that that is, you know, that the person is uncomfortable.
Jeff Walter (30:36)
Yeah, it’s it’s it’s it’s in it’s interesting to think about ’cause then you get the thought of experiment, the thought experiment of well what if he could capture all that contact? What if he could arm it with those senses, right? To pick up all that contact. But then you’ve got the anyway, now you’re way down the, you know, s nine seconds picture.
Ashley Etemadi (30:54)
Yeah, but then you have to figure
out if you’re arming it with all those senses, it also has to realize what’s important and what’s not important.
Jeff Walter (31:02)
Right.
Ashley Etemadi (31:03)
And so I think that’s the in that specific context. So I think that’s what’s more difficult with with AI displaying judgment is that figuring out what is important versus what is not important. For example,
Jeff Walter (31:19)
So if we
Ashley Etemadi (31:20)
when when we’re in
building simulations and coming up with the characters, if I was coming up with characters and saying, you know, this character is Afghan. And if you ask the AI to create an Afghan character, it creates a stereotypical Afghan
Jeff Walter (31:35)
Right.
Ashley Etemadi (31:36)
character and, you know, just what you see on TV and what’s displayed. But it doesn’t create one that’s complex that might have, you know, actually have new, be nuanced.
It’s much harder for it to create that nuance. So that’s where I think the human comes in is that there’s a lot that You know AI in general is is putting everything on an average, right? So it’s
Jeff Walter (32:01)
Right.
Ashley Etemadi (32:01)
predicting generative AI is predicting what’s the next most likely word and so it it’s much harder to create nuance and exception on Exceptions and I think that’s where human judgment comes in
Jeff Walter (32:14)
But you know, that’s an interesting way of putting it because and it goes back to what you said earlier about the the context when we’re talking about the prediction models, right? It’s like well, based on this, the average would be a seventy five percent probability of dropping out. But none of us as average we’re all on the distribution. Right? And and
Ashley Etemadi (32:31)
Mm-hmm, exactly.
Jeff Walter (32:33)
and and and I think and and
And it’s it it’s almost like the distribution is the judgment part, right? It’s it’s the it like you said, like it’s predicting what the next word would be, but I have characteristics that are on a we humans are not the average, we’re on a distribution. It it kind of reminds me of economics and like homo economicus, right? Like the not as opposed to Homo sapien. It’s like
The rational actor in the economy would behave this way. And yes, there are a subset of humans that will behave that way in that situation. But most will behave differently. Like one thing, as you were talking about negotiations, and I wonder how you p bake this in, is the the the the kind of cultural fairness. Like I forgot what the name of the experiments were, where they would give s you know somebody a hundred dollars to s to divide between two people. Like
You get to divide the money between me and you, however you want, and I get to decide whether or not I’m going to accept it.
Ashley Etemadi (33:35)
Mm-hmm.
Jeff Walter (33:36)
And the theory being, well, with a hundred bucks, you should be able to say, I’ll keep 99 and give Jeff one. And of course, Jeff will accept it because if I don’t accept it, I get zero. We both get zero, right? So like one dollar is better than zero dollars. So I should accept every conceivable.
splitting of the hundred dollars, except you keep a hundred and I keep zero, right? And even that I’m not better I’m not worse off. But yet
Ashley Etemadi (34:02)
Mm-hmm.
Jeff Walter (34:03)
culturally there’s like a fairness doctrine that if you don’t get close to fifty-fifty, and there’s different cultural norms, but in different c I remember the studies were, you know, different cultures had different deviations, but it’s like you gotta get close to fifty-fifty or sixty forty. You gotta give the other person their fair share.
Otherwise the probability that they would reject it was very, very high. Right? And which makes no logical sense because they’d be better off with ten dollars, right? But they’ll say no just to punish you for being unfair. And and and which makes no rational sense, but but it’s it actually makes total sense because in that culture you’re you’re playing the long game. And why I might forego ten dollars today.
I’m teaching you a lesson of how to treat me tomorrow. Right?
Ashley Etemadi (34:55)
Yeah, and so it should be even on, there should be some sort of middle average 50-50, but then
Jeff Walter (35:01)
Right.
Ashley Etemadi (35:02)
there might be deviations too, because some people might understand that, you know, $1 is better than none. And so maybe I should accept it.
Jeff Walter (35:09)
Well and and some people will take the one dollar.
Ashley Etemadi (35:12)
Yeah.
Jeff Walter (35:12)
And it was interesting when I re read the stud I mean I this was decades ago, so I apologize for not re reminding, but it but but but it was also cultural, like you talked about Japan and US and and there were different cultures that would have a where the a larger spread was acceptable,
like
Ashley Etemadi (35:28)
Yeah.
Jeff Walter (35:28)
a seventy thirty spread versus other cultures where it pretty much had to be fifty fifty or it would get rejected. Yeah.
Ashley Etemadi (35:35)
Yeah, and I’m curious
if socioeconomic background also has a factor to do with whether or not people accept or reject, especially in societies where there’s more inequality. So maybe that’s also expected that I would get less.
Jeff Walter (35:52)
Yeah, there w there was some of the yeah, and there was other there was another piece of it that there was another piece of it also when I remember going through it was like, well, how much are we talking?
Ashley Etemadi (36:02)
Mm-hmm.
Jeff Walter (36:02)
Because for me to give up a dollar to teach you a lesson is not a large cost for me.
Ashley Etemadi (36:08)
Yeah.
Jeff Walter (36:09)
But if you were dividing up a hundred million dollars
Ashley Etemadi (36:11)
Mm-hmm.
Jeff Walter (36:12)
and and I would get a million and you would get ninety nine million, well for me to teach you a lesson about fairness would cost me a million dollars.
Ashley Etemadi (36:20)
Yeah.
Jeff Walter (36:20)
And so there was also that kind of like, were the stakes high enough? Right? And if if if
Ashley Etemadi (36:25)
Yep, exactly.
Jeff Walter (36:26)
we increase the stakes, does the gap spread, right?
Ashley Etemadi (36:30)
Yeah,
and then how many people are offered that too. So is everyone being offered that or are you just a select few where one person might get 99 and you a million, in which case, you know, maybe a lot of people would take the million if it’s just a few people. If it’s like, okay, you’re a select group of folks. So I think there’s also an exclusivity component as well.
Jeff Walter (36:51)
Right. Well that and that gets
into the socioeconomics. If I’m Elon Musk, I’ll give up the million to punish you for being unfair.
Ashley Etemadi (36:59)
Yeah.
Jeff Walter (36:59)
If I’m, you know, living off of welfare, I’ll be pissed at you, but I’ll keep the million. Right? Like I’ll I’ll still think you were unfair, but I’m not gonna forego the million for myself because it would have such a huge impact on my life, right?
So it’s but it’s but it’s interesting. That I I think those are some of the nuances. So so let’s talk about the the negotiation stuff because it’s very interesting. Cause I I yeah until you just mentioned it, I never thought about it, but it’s a very human-centric thing and we’re negotiating all the time. For all you know,
like
Ashley Etemadi (37:31)
all the time.
Jeff Walter (37:33)
like con like it’s basically human interaction. Wha from who’s gonna cook dinner tonight, where are we gonna go, what are we gonna have.
How much am I gonna w you know, how much are you gonna pay me for this job? what route are we taking to grandma’s house? just it it’s it’s interesting. So
Ashley Etemadi (37:51)
And what is negotiable too is also on the table. I think what we consider negotiable versus not is up for debate. And I don’t think we talk enough about it. I remember I was speaking to someone and they said that their father would negotiate at Walmart. So most people would go in and you you expect you got to pay the price that’s listed, but he would find items that were coming up for expiration and say, hey,
can you give me a 50 % discount because this milk is expiring in a day or two days. And
Jeff Walter (38:25)
that’s a fun.
Ashley Etemadi (38:26)
was very successful doing that. And so very few people
Jeff Walter (38:29)
Interesting.
Ashley Etemadi (38:30)
would consider going to Walmart as a negotiation or something that could be negotiated, but it was.
Jeff Walter (38:38)
That’s that’s really interesting. I I had a friend that didn’t do that, but whenever we went whenever we would go to a hotel or something, he’d always ask for the nice guy discount. And he goes and
Ashley Etemadi (38:49)
I’m a nice guy.
Jeff Walter (38:51)
he goes, Well, you know, do you guys give a nice guy a discount? And they’re like, What? He goes, Well, I’m a nice guy, so you know, is there anything you can do? And more often than not, they would give him like the triple A discount, you know, like we’re talking like hotels, right?
Ashley Etemadi (39:05)
Yeah.
Jeff Walter (39:05)
They’d g they’d they’d they’d give him some preferred rate that he didn’t qualify for. Because he’s just like, I’m a nice guy and you know, do you have a nice guy discount? it’s you know, to your to your point, it’s just kinda funny. Like I would have never thought that about Walmart. Like I would have thought like the last place you can negotiate is a Walmart.
Ashley Etemadi (39:24)
Yeah, me too. That’s why the story stuck with me because I said if
Jeff Walter (39:27)
Yeah, that’s like crazy. But
Ashley Etemadi (39:29)
someone can negotiate Walmart, then we can negotiate anything.
Jeff Walter (39:33)
Yeah. So so so let let’s talk about the negotiation stuff that you did because I I think it’s really interesting because it’s it it’s a really uniquely human thing, but now you’re using simulation environments with AI avatars to to do that and everything that we just talked about about context and all that. How have what kind of impact have you seen with with what you’ve been doing there?
Ashley Etemadi (39:55)
Yeah, and actually I wanted to point out the fact that we’re using generative AI to simulate characters to build human skills. I think that’s a non-intuitive concept and structure. And that’s what I find really interesting about it is a lot of people are talking about, how to use generative AI to automate my work or to remove humans. And what we’re looking at is actually how to use it to build skills and empower humans.
with the agency to be able to negotiate or feel comfortable and confident negotiating. think that’s a big element that we see when talking to folks is that there’s a lack of confidence in negotiating. So people don’t negotiate because either they don’t feel that their skills are good enough or they feel hesitant about negotiating in that scenario.
whether or not they have the power or whether they’ll be successful or if it’ll hurt them in the end. And another one I think that we think about is negotiation is an interaction where the combined outcome should be greater than the two individuals’ outcomes. So what you come together to solve is greater than the individual parts. And so I think that’s also a uniquely
human trait. So it’s not just an A plus B equals C, it’s a 1 plus 2 equals 3 or 4 because the outcome is much greater than the individual parts.
Jeff Walter (41:33)
Well, it and I think it’s one of those things that a lot of it gets lip service that it’s win win. Like that’s what you’re talking about, like win win. W one plus one equals three. But I o but and I think a lot of people think of negotiations a zero sum game. If
Ashley Etemadi (41:46)
Yeah.
Jeff Walter (41:46)
if I you know, if I get an extra dollar, you lose the dollar. Like i y like kind of more that you know, how much are you going to give me for this milk? Right? And
Ashley Etemadi (41:56)
And that’s we define
as bargaining in our work
Jeff Walter (41:59)
Right.
Ashley Etemadi (42:00)
specifically, is we say that, and this is not how negotiation is defined in all textbooks, and it’s not how all faculty members think about the concept, but we think of if it’s just a money game, it’s a bargaining exercise. So,
Jeff Walter (42:16)
Right.
Ashley Etemadi (42:17)
okay, how much lower can I get the price? And I don’t know if you saw, but a few years ago, in light of…
the launch of generative AI, Walmart started using chat bots to negotiate its contracts with vendors.
Jeff Walter (42:31)
No, I really
Ashley Etemadi (42:33)
Yeah. And they actually found it really successful because Walmart was negotiating down because chat bots were so, they were so, I don’t know if the word is vicious or ruthless, that’s the word.
Jeff Walter (42:44)
Ruthless.
Ashley Etemadi (42:47)
They were so ruthless, they just kept negotiating down the prices. Well, that’s bargaining. And
Jeff Walter (42:54)
Right.
Ashley Etemadi (42:54)
so there’s definitely a winner and a loser in that case. So
Jeff Walter (42:59)
Yeah.
Ashley Etemadi (43:00)
Walmart is the winner and the vendor is losing on the margins. But what we talk about is where if Walmart were to come up with a unique structure where actually both parties get greater rewards, you know,
augment like for example Walmart partners with one of their vendors to expand their business in an area. Now that is the type of negotiation that we’re exploring.
Jeff Walter (43:26)
And and what kind of results have you seen from the this y use of this technology for that you know for for that type of build building those skills. You know, one of the biggest I think unknowns right now is you we’re we’re seeing a lot of, you know
AI avatar based skill development, especially on these power skills, these human skills, right? As and then and then in simulation also t more technical skills, right? but I think there’s still a question of how how much how big it what kind of impact is it moving the needle? And what what have you seen there?
Ashley Etemadi (44:02)
That’s a good question. And I can mention some of our pilot work and what we saw there. But I think also more broadly, I have some, I guess, some thoughts on how the needle can be moved and what are some factors that it relies on. So I’ll start off first with some of our pilot results. it was, we conducted a few small pilots.
with folks in Ohio as well as in Texas and at the Harvard Graduate School of Education. And what we saw is that folks started to, and I’ll outline specifically the negotiations, so it was what they call a power play negotiation, so where there is a power difference between the two parties and where one party has to basically advocate for themselves.
And it was a, in this case, a promotion, but the promotion wasn’t strictly a salary increase. It was you have, you’re taking on two roles and you have to negotiate with your boss about moving full-time into one of the roles because…
you know, someone left the company and you had to assume their role as well. So now you’re doing two jobs at once, which this is a very common occurrence in the workplace
Jeff Walter (45:23)
No.
Ashley Etemadi (45:23)
and you’re feeling burnt out. And so either you leave or you negotiate to get moved in a trial basis into this new position full-time. And so you have to navigate your salary, but also more importantly, your position.
And it has to be both beneficial for you and the company, for the company to be able to accept. And in this case, you’re negotiating with the CEO, who is also your boss. And so folks who just negotiated salary, they actually receive lower scores on one of our metric, which was creativity. And so we were assessing people on creativity, how they manage the negotiation process.
their ability to manage emotions because the boss was actually prompted to be condescending to the person. And so how do you navigate scenarios where, you know, the boss might get angry. And another thing is in future studies, we want to actually put people in the position of the boss to be able to see how do you navigate a conversation with someone who’s asking for a promotion because
In no way do we advocate being condescending to employees, but there have been in the past situations where employees have had to manage those conversations and
Jeff Walter (46:43)
Uh-huh.
Ashley Etemadi (46:44)
manage the emotions of someone who might be upset. And so how do you manage emotions? And in general, how persuasive are you in terms of getting what you’re looking for? And so what we noticed from putting people in this scenario was that one,
people said that they found themselves, had found themselves in the same situation in previous roles, and what they had opted for is leaving the company rather than negotiating with the boss. And in that case, that’s actually a lose-lose situation, because you give up your position and the company has to find a replacement for you. And so, which, you know, it could have been better for you just to get promoted, but if you didn’t feel that you could have the conversation,
then you were more likely, you felt more comfortable leaving than having the conversation. And so people said that, wow, I’ve been in this situation in the past and actually I’ve just resorted to leaving the company rather than actually negotiating with my boss. People also said that it made them realize what’s negotiable in terms of in their environment. So we were working a lot with people who are not from the United States.
and who were learning about what’s appropriate, and they didn’t realize that these are things that you can negotiate because in their culture, for example, the boss comes to you to promote you, offer you a promotion. It’s never you going to the boss to ask for a promotion. And
Jeff Walter (48:18)
Right.
Ashley Etemadi (48:18)
so there are certain stigmas around you asking for more in their cultural context.
So I thought that was very interesting. And so they said, you know, I’d never even knew that I could negotiate this. So this has opened up my world for what I can negotiate. And then lastly, people mentioned that they actually felt, for example, when the boss was condescending, they felt the emotions of what it’d be like to be in a real negotiation with someone, even though they were just speaking to an avatar. And so a lot of times,
I speak to institutions and they say, well, you know, it’s just an avatar. you know, not going to, it’s not going to evoke those same emotions that it would if someone were talking to a human. But what we see from our studies is it actually does. People felt tense when the
Jeff Walter (49:08)
Mm-hmm.
Ashley Etemadi (49:09)
boss was condescending. They were worried about, you know, how to approach the boss and how to word what their requests, because they didn’t know if the boss was going to accept it or not.
And so there was a lot of uncertainty that they felt that they would feel in a normal negotiation with a human. And so I thought those three results were really interesting. And I’d be happy to chat further. There were other things that we noticed in this study, I know I wanted to also talk about, there’s a second point that I wanted to talk about, the second part of your question.
Jeff Walter (49:45)
Well b but before we jump onto that, th what’s what’s interesting about what you just said on the emotional side is it it’s interesting ’cause from a physiological standpoint, the body the the mind doesn’t know the difference. And the the evidence of that that everybody can go to in their in their own lives is did you ever read a book or ever watch a movie and get emotional?
Did you ever watch a horror movie and jump back when the guy with the axe jumps from behind the tree or whatever?
Ashley Etemadi (50:16)
Yeah.
Jeff Walter (50:17)
It’s like you know it’s just a movie, it’s right there, yet your body reacts because it doesn’t it so your subconscious doesn’t know that it’s fake. Right? And and
Ashley Etemadi (50:29)
Yeah, and that’s
why people feel, you know, they feel, what’s it called? When they’re in immersive experiences and there’s a large drop or something, they feel fear of heights and they
Jeff Walter (50:41)
Thanks. Right.
Ashley Etemadi (50:42)
feel all the same emotions that they would and feelings that they would in a real setting.
Jeff Walter (50:47)
And so it it it it I understand why people would go, why it’s just an avatar. But just put yourself in a movie theater watching something that’s highly charged emotionally, whether it’s a a love story, a horror movie, or anything, and you know, if you’ve ever had any type of emotional reaction, that that’s what we do. That’s what we’re doing.
in
in movie theaters or watching TV or you know watching a video is we’re putting ourselves into the characters and feeling what they’re feeling. And this is a much more immersive interactive and that’s passive, right? Yeah, and this is in and and so it that that that makes complete sense to me. And
Ashley Etemadi (51:27)
And this is active.
then, so your follow-up question was around efficacy and
Jeff Walter (51:32)
Yeah.
Ashley Etemadi (51:32)
does this actually lead to behavior change? So what the research suggests in the learning sciences is that one experience will likely not radically change someone’s ability to negotiate. It might change certain
Jeff Walter (51:45)
Mm-hmm.
Ashley Etemadi (51:46)
elements. Like people said, they feel more confident asking for things or they feel more confident opening a negotiation after they went through the training.
But in order to have long-term negotiation skills and to build the disposition for negotiating, that requires multiple experiences. So being
Jeff Walter (52:09)
Right.
Ashley Etemadi (52:09)
put into multiple simulations and seeing the similar scenarios but under varied circumstances. So it might be, you know, you have a scenario where it’s a rent.
You know, there’s a rent dispute. Your landlord wants to kick you out. You have recently lost your job. You’re trying to ask for a rent decrease. Now, how do you do that in a way where your landlord actually gets some benefit, even though like monetarily, it’ll be lower. But there’s other things that you could give your landlord that would allow
Jeff Walter (52:45)
Mm-hmm.
Ashley Etemadi (52:46)
for the rent decrease. And so that could actually.
benefit your landlord even greater than maybe the $300 that they would be losing per month. And so that’s a similar structure, but it’s a different scenario, right? Now there’s still a power imbalance because your landlord could kick you out and you’re in a position of where you need to ask for something, but you have other things that you can offer the landlord. And maybe you’ve been very much on time with your rent to date.
And so
Jeff Walter (53:19)
huh.
Ashley Etemadi (53:19)
putting someone in that scenario, that helps build the skills longer term. And so I think in the L &D world and even sometimes in education, we think, okay, if we give someone one example and they do it well, then when they go out into the real world, they can identify that they need to negotiate in this scenario and they can do it well. But that often doesn’t happen. Sometimes folks don’t identify the scenario
because they haven’t seen it under various circumstances, or they feel unprepared to do it well because they’ve only seen it once. You see this a lot in the medical education space as well. People will be like, you know, I was in nursing education and I only saw this once. And so once I got on the job and I was in the hospital, I didn’t know if I was doing a good job. And so how do we prepare people, I think,
providing them multiple examples and allowing them to build their skills over time, which is not fun because that means it’s not a one and done, you you do this training, you get completion and you are, you’re ready and you’re gonna do a fantastic job. It might be,
Jeff Walter (54:30)
Yeah, well
Ashley Etemadi (54:31)
but it might not be.
Jeff Walter (54:32)
well on skills development I I I often use like a a a a sports metaphor. It’s like it’s like you’re you’re you know, you’re constantly practicing. You’re you’re constantly trying to hit you know, learn how to hit the curveball better. You know.
Ashley Etemadi (54:45)
Correct.
Jeff Walter (54:46)
You’re c you you’re constantly throwing yourself into scenarios to figure out how you can perform better within that scenario. you know, one one of the interesting things, I remember
It’s a helic helicopter pilots. It was training other helicopter pilots, combat pilots. And it this goes to the learndo teach. They would learn so much more about how to be a better helicopter pilot when they were teaching others because they couldn’t even imagine the mistakes, the compound set of mistakes that a novice would make, and they would make them in all different ways, and all of a sudden there was
you know, problem resolution in real time happening to your you know, so even that they’re teaching, but they’re still learning because they could have never imagined they would string together this sixth set of failures, right? Or this sixth set of mistakes, right? And now all of a sudden you find yourself in a pickle that you’ve never would have imagined you’d be in. And and how you know, but it’s but so that makes complete sense to me because you’re if you use a any other type of skill-based
take AI out of it, you’re you’re constantly putting you know you’re constantly practicing. You know, a championship team doesn’t go, well, we won the championship last year. No more practice. We’re done. Right? Like,
Ashley Etemadi (55:57)
Yeah, exactly.
Jeff Walter (55:59)
you know, you’re always thinking of the next thing and and what can it come and how can we take it to the next level. And and and so that makes complete that makes complete sense. And then you know, on the on do you have any data on on the negotiations where it’s happened
Does it lead to lower turnover? Does it lead to happier like, you know, h more satisfaction? Like I’m just trying to sit sit there and go step back. Like have have we have you been able to get that level of data yet?
Ashley Etemadi (56:26)
That’s a great question. I would love to run a study on that if anyone
Jeff Walter (56:31)
Yeah.
Ashley Etemadi (56:32)
who listens to this podcast is interested. But it could be, I could build negotiation simulations that are relevant to a specific context and we could test, does it lead to higher satisfaction? Does it lead to better outcomes in the workplace? One thing that I’m particularly interested in is a lot of…
your interactions with your colleagues are also negotiations, negotiating
Jeff Walter (56:57)
Yeah.
Ashley Etemadi (56:57)
responsibilities and division of labor. And so can that improve? Can your interactions, your relationships improve? The hypothesis is yes, but I would love to study that if anyone is interested.
Jeff Walter (57:11)
Yeah, it well it it’s interesting because it’s it’s i at the end of the day, to and we’re we’re focusing on negotiations, but I find it a very interesting topic because
To your point on relationships, it’s it’s it’s you’re you’re you’re always negotiating because you’re always working with other human beings to coordinate activity. And when and coordinating activity is basically, I’ll do this, you do this, we’ll we’ll do this together, we’ll hire somebody. Like and and so it would seem to me, my hypothesis would be that you would not only lower turnover, but you actually increase satisfaction because there’s more communication.
As I’m listening to you talk about this, it’s like what you’re really doing is increasing authentic dialogue within the organization. And y
Ashley Etemadi (57:56)
Yeah, and making
sure that, you know, I guess everyone’s needs are met. So in the negotiations that I build, it’s not just one person, you know, wins and the other person,
Jeff Walter (58:09)
Right.
Ashley Etemadi (58:10)
you know, kind of gets what they want, but not really. It’s that how do you find solutions where both people are winning? And yeah, this is something we do every day.
Jeff Walter (58:19)
Yeah,
it’s like like even going back to that splitting the hundred bucks, you know, thing. If if even if you only give me a dollar, if then you come and say, Well, I need ninety-five dollars to get this for there’s this thing I wanna get for my daughter and you know, I don’t have money and I need ninety five dollars for it plus four dollar bus fare to get there. Well all of a sudden it’s like
Okay, it’s you’re not being unfair. You’re trying like I’m giving you the opportunity to do something pr like I still only I still don’t get my fair share, but I understand it. And I y you know what I’m saying? Like
Ashley Etemadi (58:56)
Yeah, it could be that in exchange that person gives you something else. Maybe they cook you dinner, maybe they give you a ride or something. Something that you need that might be invaluable
Jeff Walter (59:09)
Yeah.
Ashley Etemadi (59:10)
that doesn’t necessarily have a price tag. Or maybe it does. Maybe you have to pay.
Jeff Walter (59:14)
Or or you just
you you just understand their context
Ashley Etemadi (59:17)
Yeah.
Jeff Walter (59:18)
and and what they’re what why and and you can empathize with it. And it’s like doesn’t it doesn’t mean you’re happy but you’re maybe doesn’t mean you don’t wish you got what you didn’t you know, you didn’t get what you didn’t get, but at least you have a be you know, it’s not you know, I just find authentic conversation leads to better outcomes. And if you can have those
conversations. Anyway, we we gone down the negotiation
Ashley Etemadi (59:43)
Yeah, and Mike Wheeler at
HBS has a great graph. So on one line, on one axis, he has the outcome. And then on the other axis, he has the relationship. And so I think that’s a great way to depict how negotiations are a constant juggle between managing outcomes and managing the relationship. And so he…
Jeff Walter (1:00:10)
Uh-huh.
Ashley Etemadi (1:00:11)
In his book, he provides, In the Art of Negotiation, he provides some great examples of people that had great relationships in the negotiation but didn’t think about the relationships when they were negotiating. And not only were they unsuccessful in getting what they wanted, but they also hurt the relationships. And so I think it teaches us that there’s…
multiple things that we need to keep in mind when we’re negotiating. It’s not just, you know, the $95 or the $6 that I’m going to get, you know, the monetary value, but it’s also the relationship. So, for example, you said, you know, I need the $94 to do this. Well, anyone whose friend says, you know, where they’re splitting $100
Jeff Walter (1:00:58)
Yeah.
Ashley Etemadi (1:00:58)
and so, someone gives them $100 and they say, you know, I need the $94 to give to my daughter to do this.
I don’t think any friend would say no.
Jeff Walter (1:01:08)
Right.
Ashley Etemadi (1:01:09)
I want my 50 % share. And so they would likely give it to their friend. And they know that in the future, their friend might do something for them that would be similar if they were in a similar situation. so I think knowing that that relationship is there and that it needs to be considered in the negotiation, I think is also something.
interesting that we should always keep in mind.
Jeff Walter (1:01:35)
Yeah, well and I I think it’s also not just if you have an ongoing relationship with that individual, but if you’re part of us even part of us a society, right? You’ve got this
Ashley Etemadi (1:01:43)
Correct.
Jeff Walter (1:01:44)
you’ve got this relationship with the with the world outside and you wanna w live in a certain world that behaves in a certain way. Yeah, it’s the whole well the whole concept of paying it forward, right?
Ashley Etemadi (1:01:55)
Correct.
Jeff Walter (1:01:55)
And you know, it’s not that I’m going to do this for you so that I get something from you tomorrow.
But then you do something for somebody that does something for somebody. And next thing you know, something good happens for me because I’m in an environment where people are anyway, it’s just f I I just find it very fascinating. very, very fascinating.
Ashley Etemadi (1:02:14)
Yeah, that too, right? You want
to live. So we work a lot with examples that are in the in a community. think, you know, one of the beautiful examples from one of the pilots that we ran was a recent immigrant to the U.S. and he had to pick up his daughter at school and he mentioned that both him and his wife worked and they had to take public transportation by the time they would get to the school. The
you know, daughter had already been left outside because the school had closed and the principal, you know, they’d missed the chance to pick up their kid. And so the daughter would be waiting alone outside. And so they’re very concerned and they went to the principal and they didn’t know that they could negotiate. But after they did our curriculum, they realized that, maybe this is something I could negotiate. So then they went to the principal and
presented their case like, this is something that, I mean, it was very important to me and my wife, because we both work and we need to work to make ends meet. Could you keep my daughter in your office for an extra 30 minutes until we get to the school? And the principal was more than happy to do that. And they said that, you know, if I hadn’t done the curriculum, I wouldn’t have known. And so that’s not something that necessarily the principal is going to get a reward from or.
that is gonna be repaid, but the principal is doing a good service. And so I think they were more than happy to help out the family. And I think a lot of people think of negotiations as this tug of war, where,
Jeff Walter (1:03:48)
Right.
Ashley Etemadi (1:03:50)
you know, if it’s like, I get something, you get something, but they forget that, you know, there is acts of kindness. And if you just ask, you can get.
And so, but if you don’t ask, people don’t know. And I think that’s one of the beauties of this curriculum is that it allows people to even know that they can ask, which a lot of people don’t even open up the doors for negotiation because they don’t know that things can be negotiated.
Jeff Walter (1:04:19)
Well and that goes to what you said before. They’d they don’t f feel that that they can voice their concerns, so they leave the environment and go somewhere
Ashley Etemadi (1:04:28)
Right.
Jeff Walter (1:04:28)
else. And that’s a lose lose. You know, could be. what’s the future hold for Motiva? Where where are you guys going in the future? You you you’re done you’re doing a lot of really cool cutting edge stuff.
Seeing what works, what doesn’t work. where where’s where where’s the future taking
you guys?
Ashley Etemadi (1:04:44)
That’s actually a great question.
So I think I would summarize our future in terms of that we’re constantly exploring the boundaries of the affordances of this technology. So I think the technology is changing quickly, less so in the recent months, but still I think that most of the benefits of the technology haven’t been unlocked. I oftentimes compare it to the early or late 90s in terms of the internet.
Jeff Walter (1:05:11)
No.
Ashley Etemadi (1:05:12)
companies that were popping out and then, you know, then we hit the 2000s and we had that recession and then everything that came after the 2000s kind of capitalized on the affordances of this new technology. I look at some of the things that were created in the late 90s and I just wonder, wow, that’s what people were thinking this technology would, that would be the useful applications. And I say it’s similar to the
social media boom in mid 2010s or 2010,
Jeff Walter (1:05:42)
Right.
Ashley Etemadi (1:05:43)
2012 era where everyone was creating an app for everything. think a lot of people are creating AI for everything. eventually we saw a narrowing down of the use cases and what people are actually going to use versus not. And so people were expecting that you’re going to have 50 pages of apps and an app for everything.
But a lot of companies just mostly went back to browsers and then have an app just in case people want to download it on their phone, but they now have a dual strategy. And
Jeff Walter (1:06:14)
Right.
Ashley Etemadi (1:06:15)
so I think we’re going to see that with AI, where the affordances of the technology and the use of it is tapped in greater depth. And for use cases that are really going to move the needle.
rather than AI for everything and, you should have a bot for this and you should have a bot for that. And I think we’re going to see a simplification and just better use cases of the technology. I think also for just energy usage as well. Of course, we’ll see more efficient AI. But as you mentioned, there’s a data center opening near you. There’s a data center that just recently opened near me.
and all of a sudden, you know, water shortages. so, and people have reported that the temperatures near the data centers are much higher than in other areas. And so I think we’re going to have to be more cognizant about our use of AI. And so I think that’s where it’s going to lead the area. And so I think we’re just more and more interested in
tapping into the true affordances of the technology and how it can help learners improving those outcomes and improving effectiveness. And so not just, I would say, not using the technology for the sake of the technology and saying that you use AI, but actually using it to improve student outcomes is kind of what we’re going to continue to explore. We have a few things in the pipeline, but I think we’ll stay posted with you, Jeff, about when those manifest.
Jeff Walter (1:07:45)
Yeah. Well, I I think that’s an interesting a very healthy way of looking at it. Yeah, when you when you’re comparing it, it’s like you get this new technology and you don’t know you really don’t know where you’re gonna get the biggest bang for the buck for it. And so people try and apply it everywhere. And eighty percent of that is gonna f fail. We just don’t know which eighty percent.
Ashley Etemadi (1:08:09)
Mm-hmm.
Jeff Walter (1:08:09)
But the other twenty percent is going to change how we live.
And we don’t know what the which the twenty percent are. So I think if we w I I like the way you position that in that it’s gonna be tried everywhere. We’re gonna see what sticks, what doesn’t stick. It sounds to me like you guys are doing that. Like let’s try it here. Does it work? Does it not work? Let’s let’s track its impact. Okay. It has an impact here, let’s go forward. It doesn’t have an you know, we tried over here, you know, modest impact, not worth the effort.
And and and we we we grope our way into the future by seeing what works and what doesn’t work. And so I think that’s a very healthy way of looking at it and I think Botiva’s gonna have some very fun things coming down the pipe that are gonna be really interesting.
Ashley Etemadi (1:08:56)
Yeah, I think one thing that I stray away from is there are some people that try it in certain contexts and then blanket statements say, you know, AI doesn’t work or it’s
Jeff Walter (1:09:06)
Yeah.
Ashley Etemadi (1:09:06)
not a good technology. Well, in that context, I could tell you it was not going to be a good application of that technology. But there are other contexts where it’s actually making great strides and helping in big ways. And so
How do we identify those areas? I think it’s just going to be constantly experimenting with new applications and new use cases and testing. Always, think research is really important to understand how is the technology improving or maybe it’s not improving. I think one of the most interesting examples is the application of generative AI for doctors sending emails. And they found that actually having machine learning
So the doctor getting a template and then having, based on what the patient has emailed, the doctor gets a template and then with suggested inputs is better
Jeff Walter (1:10:01)
Right.
Ashley Etemadi (1:10:01)
than having the, is more efficient than having the doctor have a large language model produce the email because they spent
Jeff Walter (1:10:08)
interesting.
Ashley Etemadi (1:10:09)
so much more time editing the email and editing the context than they did just selecting
from the machine learning, having a template and then selecting from what the machine learning algorithm suggested. And so, you know, that’s an application where generative AI was not great, machine learning was, but they found that between generative AI and the doctor typing the email from scratch, it was actually more efficient, the doctor typing the email from scratch, which is completely counterintuitive.
you would think that it would be faster just having the LLM do it. But these are the nuances that I think we’ll be discovering over the coming years that are really important for what applications thrive in the next decade and which we kind of sunset or opt for more human touch or human interventions than having AI generate it.
Jeff Walter (1:11:07)
Excellent. And and Ashley, if if somebody wanted to get a hold of you or a Motiva education, what’s the best way to get a hold of you guys?
Ashley Etemadi (1:11:15)
Yeah, the best way, well, you can get a hold of me, Ashley, at MotivaEducation.com. You can also send a note on our website, www.motivadeducation.com. And that’s Motiva like motivation, M-O-T-I-V-A.
Jeff Walter (1:11:31)
So before we go, anything else you wanna share with anybody or any other thoughts before we go?
Ashley Etemadi (1:11:39)
I mean, it’s always great chatting with you, Jeff. I think we only scratch the surface of what’s possible.
Jeff Walter (1:11:44)
All right. Hi.
Ashley Etemadi (1:11:47)
And I’m really excited to see where our collaboration continues and how we grow this area and continue to improve education for the better.
Jeff Walter (1:11:59)
Thank you. And I it’s i always a joy talking to you. I it’s well it’s it’s so interesting ’cause you have such an interesting perspective on all of this and and what what you guys are doing. And it’s just I I it’s just very fascinating and I I love chatting with you. So thank you for your time. I greatly appreciate it.
Ashley Etemadi (1:12:18)
Of course, thank you for having me.
Jeff Walter (1:12:20)
Yeah. And to everybody out there, thanks for joining us. w you know, this this is why we do it and and
Have a great day. We’ll see you next time.