Joe Petro, Executive Vice President and Chief Technology Officer at Nuance Communications, has seen the company’s continual evolution over his twelve years there. As the head of a global team of more than 1,600 researchers, developers, and engineers, Joe has a unique perspective on the technology, business, and people who make Nuance a leader in conversational AI, security, and biometrics.
We had the opportunity to chat with Joe about Nuance’s technology, the impact of COVID-19 on the business, and the challenges of managing a large, remote team.
What’s the elevator pitch for Nuance?
At Nuance, we build state-of-the-art conversational AI technology designed specifically for healthcare, financial services, telecommunications, retail, and government agencies that amplifies our customers’ ability to help others. Essentially, our technology works in the gap between human beings and machines, and we’re focused on augmenting what humans bring to an experience in terms of their set of experiences and best practices.
This whole notion of using AI to amplify the contribution of people and augmenting what they can do is an exciting space and it’s something we’ve been working on for a very, very long time. That said, while we’ve been developing and using AI and machine-learning approaches for the last 20 years, we never really talked about it widely. Over the last three years—since our new CEO, Mark Benjamin, came in—we’ve really transformed the company 
top to bottom, honed our strategy, and refined the ways we approach and develop solutions for our markets and customers. That’s around the time that I became CTO and we pulled together all of the various R&D teams from across the organization so all the products, all of the research, and our centralized tech stack would be under one person with a focused agenda. Part of that transformation has been a renewed sense of energy in terms of explaining to the outside world what we do and all the various techniques that we bring to bear on solving the problems facing our customers. As a result, over the last couple of years, we’ve started legitimately being recognized as an AI powerhouse.
Let’s talk about enabling professionals by bringing some AI to the system. There is a lot of conversation surrounding the idea of AI displacing high-knowledge workers and what it could mean for them. Does AI enhance things or take away from them?
There certainly is no shortage of discussion around AI and automation, but our focus is not on using technology to replace humans, rather we design and build solutions that amplify and augment human capabilities. Think about it as serving up the right information, at the right time so that you can do your job even better and drive better outcomes.
That said, as is the case with nearly every other industry, there is certainly discussion around the implications of automation. For example, in the radiology community, some might be asking: “Why would you go to school for radiology now because it’s going to be replaced by AI?” The reality is that radiologists are not going to be replaced by AI because it is a very complex field and there are things that a human can do that a machine cannot; however, a radiologist using AI is going to be better than a radiologist without AI. It’s about augmenting their knowledge and ability to detect, as well as increasing their efficiency.
Similarly, we have more than 550,000 physicians using our products. They’re holding the mic while dictating. It’s their knowledge and data inside of our applications. Because we’re privileged to what’s happening in that clinical conversation as well as some of the things that are happening within the electronic medical record (EMR), we are able to bring real-time intelligence to that experience and help the physician create better clinical documentation and support better patient care. That’s really the way we think about and approach AI applications.
The year 2020 has really shaken up the healthcare industry. Have you seen an impact on your business or how people are using your products in new and different ways that you didn’t anticipate?
Yes, a few things have happened. The disruption to the hospitals in general lasted a few months, so we did see the transactional volume going through our systems decrease due to the cancellation of non-emergency and elective procedures. At the same time, some things really lit up. For example, increased interest in what we call the “digital front door of the hospital,” which refers to all the various ways you can interact with your hospital without physically walking through the front door. Because of the pandemic and the focus on keeping people safe and healthy, the physical front doors are starting to turn digital for hospitals. So, whether you’re coming in from text, you’re coming in from a website, you’re talking to a call agent, you’re calling the call desk— we have technology to help support those different channels and create a seamless experience.
Another area where we’ve seen some change is an increase in telehealth adoption. We have technology that we can put in that telehealth channel that can securely transcribe the doctor-patient conversation, populate the medical record, and help track follow-up actions. Since the pandemic, more patients and providers have understandably been using this type of technology. In fact, we spoke to a large healthcare system in Boston just a few weeks ago, and they said that prior to COVID, they had about 2,500 telehealth visits a month. At the peak of COVID it increased to 300,000 visits a month. They’re estimating that when the pandemic is finally behind us a certain extent, that volume is going to level out around 150,000 visits a month.
On the enterprise side where we serve large businesses across several industries like financial services and telcos, one of the things we saw happen is that call centers needed to send their employees home to keep them safe, however they weren’t equipped to work remotely. We started getting phone calls from our customers almost overnight. In many cases, they needed to accelerate their digital transformation plans and wanted help building and standing up virtual agents so they could continue to support their customers while getting their employees set up to work from home. This has driven increased demand for intelligent engagement solutions, as well.
In a strange way, 2020 has accelerated trends that would’ve taken a bit longer to manifest. Technology adoption that we predicted would occur over the next three to five years has accelerated over the last few months and become almost a staple of doing digital business today.
What about security with everybody working from home and businesses pushing the business to the home? What do you see happening there?
Security, biometrics, and fraud detection are really big. For example, one of the things that banks and financial institutions realized early in the pandemic was that fraud attacks on senior citizens was increasing dramatically. So, we have seen an increased interest and adoption from our customers who want to implement our security and biometrics technology so they can help protect their customers. For instance, we worked with a forward-thinking Australian bank that asked us to build a system that could prioritize calls from senior citizens. To help them, we created a system that uses voice biometrics to detect the caller’s age range, so the bank can escalate those calls to be sure these individuals are not being preyed upon by fraudsters.
Another example is an increase in the adoption of voice biometrics, which is basically the ability to use someone’s voice to detect who they are. Financial institutions are now implementing this type of technology for an easy and streamlined identification and authentication process. It takes between half a second and four seconds to identify that it’s you, without having to remember pins or passwords, which results in a better experience for you as a customer. There’s actually a bank in the UK that uses our technology to enable customers to complete an entire transaction – including moving $5,000 at a time – without having to step foot in a bank or talk to a human.
We’re also seeing increased use of behavioral biometrics. Not only can we listen to your voice and identify that it is you in a highly accurate way, but now if someone is attempting to pretend to be you by texting, we can query the phone itself and check: Are you typing with your thumbs? Are you typing with one hand? Are you swiping left or do you swipe right? What angle do you hold the phone? What is the phraseology that you use when you communicate? Using that type of behavioral data, we can better detect and identify someone committing a fraudulent act. It’s really, really cool when we bring behavioral biometrics to the table.
As you look toward 2021, what are the things you think are going to take off in the next couple of years within the business and the industry?
I think conversational AI in general will take off. And when we say “conversational AI,” we really are talking about speech, natural language processing, and something called dialog. It’s basically a human having a human-like interaction with a computer, being able to ask for things from a system, and having the system be able to ask you questions in a very natural way. And the reality is that our ability to interact naturally with machines is here. It’s not “Star Trek” anymore, it’s not 20 years out, it’s actually right here at our fingertips. It’s the convergence of the last 20 years of tech, and it’s lighting things on fire right now in a really positive way.
For example, in the healthcare space, we’re doing something called ambient clinical intelligence with a solution called Dragon Ambient eXperience (DAX). It’s ambient sensing technology inside of the treatment room. That technology securely listens to the conversation, and it figures out what the physician and what the patient are talking about and, as a byproduct of the natural interaction, it creates a highly accurate clinical document, and it puts that document in front of the physician to review. This is really targeted at reducing physician burden, improving the physician-patient experience by removing the need for laptops and keyboards, increasing financial integrity, and increasing clinical integrity of that whole process. That’s a big focus right now for our healthcare business.
For our enterprise business, the digital front doors of companies are extremely important now, and will continue to be in the future. We are seeing increased interest from organizations looking to implement conversational AI technology, security, and voice biometrics solutions to provide a better, more secure digital experience for their customers.
So the biggest thing we’re excited about right now is definitely continuing to create state-of-the-art experiences using conversational AI to revolutionize healthcare, financial services, telco, retail, and government industries.
You’ve been with Nuance for twelve years. What big changes and evolutions in the company or the development organization stand out?
The only thing that doesn’t change is that we are always changing. We’ve managed to stay relevant by staying really, really modern which means we’re always evaluating and continually evolving. If you look at the mathematics that was behind speech when I originally joined twelve years ago, there were things called Hidden Markov Models, basically a pipelining approach. Now we’re going with end-to-end neural math. Our research teams keep pace with the state-of-the-art and are very active with the global research and academic communities. We test against what’s coming out of the best practices of all that science that’s being published, and then we try to go past it by applying our vertical-specific data. We’re forever challenging the status quo and pushing the state-of-the-art.
This is something we’ve done so many times over the years, and that’s the reputation that we have. It plays a major role in connecting our employees with our company Purpose and it’s what keeps our teams engaged, happy, and committed to the important and meaningful work they do.
The events of 2020 have made it clear that, for a lot of people, working remotely works well. How has that changed how you go about hiring people?
We have always been flexible in terms of respecting how our employees prefer to work, and especially with researchers who are often very unique in their skillsets. That said, collaboration and community building is critical to our success, so we do try to pull people towards our various locations across the globe where we have our offices, so they have the opportunity to interact with and “run-in” to colleagues on other teams whom they might not otherwise interact with.
Interestingly, in some ways, this year has been a great equalizer. In a strange way, it’s raised the productivity of the organization. In the old days when we were in the office, I would have a team of ten people around the table and then there were another twenty on the phone, and then we’d end the video conference and inevitably the ten people who were in that room would keep talking for a few minutes. Unintentionally, those ten people who were in the room with me were always advantaged because we’d have those extra minutes to finish the conversation. Now, when the call ends, the call ends. In a weird way, it has raised all the boats in the harbor, and I feel like we are actually more aligned and productive as a global team.
The challenge that it brings is that I’m not exactly sure of what we’ve lost. There’s a certain something special about interacting in a live, ad hoc way, and being able to walk by somebody’s office and have a short conversation. We definitely are losing something there. I’m not sure we can measure that part yet.
There’s also the issue of bringing new leaders on in a fully virtual environment. Connecting with people on a global scale while fully remote is definitely a challenge. We just brought somebody in to run a fairly big team and a very key function, and when I interviewed him and I talked to the people that were interviewing him about whether or not we should hire him, I said, “Is he going to be able to handle reaching through the computer and really captivating and motivating people?” Because it’s different when you’re remote— you can’t walk around the office, meet people serendipitously, and have those quick conversations in the hallway anymore. It’s a totally virtual thing, so there’s a layer of abstraction between you and the people that you’re leading. So I think leadership is going to change as we evolve through this and it will be permanently different when everything “goes back to normal.” It’s going to be a new and different normal.
That’s an interesting focus on leadership. You’re not running into people and having conversations that lead to things, but there does seem to be increased productivity on the other side. I don’t think many people think about the challenges of leadership.
I’ve got the benefit of being here 12 years, so people know who I am, and how I react, and how I interact. Because of that, I’m established and have a visible role as the CTO. I worry about the new folks who are coming into lead in a purely virtual environment. There is a total lack of serendipity, right? That is literally completely gone. You won’t be making a cup of coffee or grabbing something to eat and then just happen to run into someone. It’s a very different environment. I think it’s going to change the way leadership is actually done, how people are measured, and how you measure productivity.
What is it like to manage a global team of 1,600+ remote workers?
I’ll never forget the first team I managed. I had no idea how to manage a team. I didn’t know that I could ask them to do stuff. I didn’t know if I could tell them to do stuff. No one taught me how to do that— I’m an engineer by training, right? Over the years, I figured out that my job is to amplify the effect of everybody and help them do their best work. If I have ten people on my team, and I can support them with tools, resources, and access in a way that makes them feel supported and happy and that results in productivity of thirteen people actually being accomplished by those ten, then I feel like I’m doing my job.
One of the things I tell my most senior leaders is what got you here is not going to keep you here. If you keep doing it the way you did to get here, you’re probably not going to be effective. You need to empower people and make people accountable. You need to measure people in a different way. You need to coach people in a different way. My role now is about trusting but verifying. It’s about making sure we have the right people in the right jobs. It’s all about talent. It’s about removing roadblocks. It’s about trying to keep an eye on the horizon, and then turning back to the organization and making sure that there is a strong connection so they can see the powerful impact an individual has to moving the needle on our biggest most important projects. And that they feel proud of their work—because they should!
It’s very, very different than what I thought I was going to be doing 25 years ago. I think that’s part of the heaviness of leadership: you have to keep evolving yourself in order to be a good leader for those you serve—your team.
What else do you want to share about Nuance?
These are very exciting times for us. I have been here for a long time, and at no other point have I been this jazzed about the future. It seems like there’s so much stuff converging all at the same time. We’ve got an incredible team of employees across the globe all marching toward the same goals and it feels like we’re harnessing the moment with projects like DAX, AI Marketplace, patient engagement, our Intelligent Engagement platform, and Security and Biometrics solutions. That’s what makes it really exciting, and it’s kept me engaged and at the company for as long as I’ve been here.
