Ep 14, S2 - Ethical AI in Public Service: Keeping Humans in the Driver’s Seat - Ernesto Aguilar

We're thinking about the use of AI and using these tools, but it's always with an eye to figure out how can we bring the audience a little bit closer to the content that we're creating? How can we use AI to help the audience feel a deeper sense of belonging with what we do? This podcast is for city communications, teams and video professionals in government. We talk about expanding service delivery with video and streaming, media accessibility, gear, broadcast and streaming workflows and more. It's all right here on the Government Video Podcast. Hello and welcome back to the Government Video Podcast where we talk about all things government video and communication, like workflow, tools, emerging issues like accessibility, and like what we're gonna talk about today, which is AI. Specifically, ethics and practicalities around AI, which we need to revisit on a regular basis because as we know, this is a technology that is rapidly developing. And I'm gonna be your host this week. Michelle Alimoradi, and I am joined today by Ernesto Aguilar, who is the Executive Director of Radio Programming and Content Innovation and Initiative at KQED in the San Francisco Bay area. Ernesto hails from Houston, Texas, and he's also a graduate of the journalism program at the University of Houston. And Ernesto has worked in print, broadcast media, and all over with daily newspapers, alternative weeklies, as well as public radio news. And at KQED, he now oversees radio broadcast content and a range of initiatives in the organization's content division, including leading its AI working group and overseeing AI implementation. Prior to KQED, Ernesto served stations nationwide as executive director of the National Federation of Community Broadcasters, and he's a decorated leader in public media and AI use in journalism as a Sulzberger Executive Leadership Fellow, a Newmark School of AI Journalism Leadership Fellow, and Robert C Maynard Institute for Journalism Education Fellow. And in his spare time, believe it or not, he still has spare time. He writes AI and Public Media Futures, which is a newsletter on AI and public radio and television spaces, and Oigo, or OIGO, a newsletter on public media and Latino audiences. Ernesto, so happy to have you here on the podcast today. Welcome. Very happy to be here, so excited for this conversation. Did I, did I get it right? Is it OIGO or do you say Oigo? close enough. and what does that stand for? that means I hear I hear, oh, okay. So it's not an acronym. I saw it in all capital letters and I thought and Spanish, yeah. Okay. If I would've not seen it in all caps, I think I would've understood. I, I know a little bit of Spanish, very rusty. Well, so thank you again for joining us today. As I mentioned before, it's very important that we continue to revisit, ethics and practicalities in the use of AI in, in your case, public media. In the case of most of our listeners, public information, for use of video content or and beyond. Social media, content that you put on your website. People are looking for ways to use AI and there are also people trying to pick apart AI and say like, this is bad, this is scary. We shouldn't be using it. And so we're here to kind of comb through that and, and see what good we can take and what we need to be concerned about. We've heard about your journey across the journalism field. Tell us about how you gravitated now from journalism into specifically AI in journalism. How did that happen? Well, I can tell you being here in the San Francisco Bay area where there is a lot of AI engagement around basically anything. Technology companies, the municipal space and so many other industries are looking at AI, in particular where KQED is situated in the city of San Francisco. We have a lot of AI companies around us. So open AI is across the street from us, literally just next door across the street. And there are a number of AI startups which have mushroomed around this area of the Mission District of San Francisco. And then for me, I've been very interested in the potential for this technology, to accentuate what nonprofit media does for its audiences, writ large, and then also how it might also enhance the work that we do and trying to scale up the work that we do to serve those audiences a little bit better. And so that's where I've engaged mostly in relationship to KQED. I like that you used the word enhance. I feel like that's something that's coming up a lot in the AI conversations is, replace versus enhance. Right? A lot of people are worried about replace, they don't like to hear the word replace. But if you say enhance, they are, they're intrigued. Right? So what's, what's been your experience there? So to me, I think it's important for whoever you are in the media space, whoever you're in, the nonprofit media space in particular, to really think deeply about what your business model is. I oftentimes remind people that, for KQED and just to situate this, KQED is the top broadcaster of News Talk in the country. We have the largest audience in the nation when it comes to news and talk content, and I have reporters that are certainly concerned about the emergence of AI. I have audience members that are concerned about the emergence of AI and how that is going to impact the media space, but, again, I go back to what is it that we do and why do we do it? For me, the conversation is that we are not simply in the information business. We're not in the news business especially. We are in the relationship business. We're here to build on the bonds of trust that we have with an audience. There's tons of information already out there. AI is creating a lot of information. A lot of news organizations are already producing a lot of information, but people come to nonprofit media 'cause they want someone who they trust can explain to them the world around them. And so, for me, as someone in charge of programming, I'm not interested in replacing a reporter. That reporter is the basis by which donors and listeners and audience members as a whole come to this broadcaster because if I know you and I understand your perspective, I am more likely to trust it. And then what? You can help me as an audience member to understand my community a little bit better. However, are there ways that AI might be able to help me as a journalist do that work a little bit more and a little bit better? And I love that you, that you put it that way, relationship first, because this is what I hear on the public information end too, right? We've got folks that need to establish trust, and that's built through, you know, a journey of, a relationship that they're building over time so that residents then know where to go in a time of crisis, right? Or emergency, or maybe it's just that they, they need a city service. And now they know where to go because, this is a place that has put out a lot of useful content and now like they're aware that that resource exists, they know where to go. Right? And it's, so it's parallel missions in that you're building a relationship that's based on accuracy, timeliness, urgency sometimes. And, and just being there, being a presence that's, that's always, being useful and, and has a particular, a particular voice. That part I think is interesting in terms of AI, right? Because i've heard so many social media managers in particular, wanting to push people or social media consultants even, wanting to push people to actually use AI to create content. And you've got a different approach to content creation at KQAD. Can you talk about that? Certainly so, and I think I want to go back to something you said about the information you provide, and the trust that's built that way. For me, it is incredibly important to, for organizations to really begin to think about how you double down on that. I've done a number of trainings for listening posts collective for the Maynard Institute and many others talking with nonprofit in these publishers and all of them have a very similar point that you've made. How can we use this to help us create more content? This is only intended, this tool is only intended to help or should only be intended to help you to enhance and scale up what you've got and help you bring attention to the content that you're already developing there in the spaces that you're in. Simply because, I mean, you could certainly have AI just swirl up a lot of social media posts and articles and other kinds of things, but ultimately, those aren't really about the trust that you have in your communities and the finer personal touches that you can add to each piece of that content that calls back to that trust and reminds people that you have been here in this community. A good, a short example, i've got a very dear friend of mine who is a very talented journalist, but is also kind of terrible at a lot of things on social media. And when this person uses AI to enhance air quotes, that social media presence, by creating the content, it's very obvious. And the, if I, as somebody who knows this person can see this, your audience members can see this too. They know you, they appreciate what you do, and they want to see you in your most realistic voice. If and if you're going to consider using AI in those kinds of instances, think about it in a more strategic way to help you build up what you already do and build up on that relationship. Right, and I think. I think a thing that we're touching on here is that before you can even think about the ways that you're going to engage with AI in your everyday workflows or even in your organizational policies, you have to think about your why first, and that why probably hasn't changed, right? Your mission, your goal, those things probably haven't changed. It's just that now AI is here to intersect with that in some way. And so it shouldn't change your why, right? I think that's, that's where a lot of people need to remember to start. And I say remember to start, because I think it can be so easy today to even forget all the ways that, that we're actively or even passively engaging with ai. Every single day. Like I, I forget, I forget sometimes, for instance, we're, you know, we're recording this podcast as a video, right? And there's a lot of AI happening in video tools now, and I got on here today. So for our audio only listeners, you know, Ernesto and I are just talking heads right now. I am sitting in front of a microphone with my headphones on, but I am sitting in front of a virtual background that's actively keying me out. And I also noticed when I looked at my face, I was like, oh my gosh. That's not what I really look like. There's some sort of filter on here, you know, like something, something is lying on my behalf without my permission. And, and so that, that can happen, without us even thinking about it because it's become so ubiquitous. So starting with our why, and then even just like taking account of all the places where AI is active, even in a passive sense in what we're doing is a great place to start. Has that, has that been anything that's come up in your work? So I will say for a. Audio only listeners, Michelle looks marvelous. AI did not enhance that. I look marvelous too. AI did not We both look great. I'm just saying. look fantastic. I can tell There's some enhancement. There's some enhancement there's no enhancement happening. AI's not doing a thing. We, we naturally look outstanding. So I, I think that you're talking about something that I think is very important, related to why, and I think it, it oftentimes when you're thinking about AI or when we're thinking about AI internally, I can tell you. We often have, have to take a step back and ask what our goals are, what we're trying to get accomplished, who we're trying to serve, and how we're trying to serve them. And what exactly is going to be, say, a one month goal, a three month goal, a six month goal, a one year goal, to any of these tools. And what's an immediate benefit to the audience that a use of a tool is going to, provide? An example of this is we developed internally a tool that will take our two hour morning program, and we'll listen to all of that audio, and then out of that audio, we'll surface out a description of what the content was, what topics were brought up, and then what areas might have been discussed. We're in the San Francisco Bay Bay area, and you oftentimes hear about San Francisco, Oakland, Berkeley, but there are many, many smaller communities outside of those large cities that are, have somewhat been supported because there is a lot of income inequality and people mo move further out. But the reality is people also want a different lifestyle. And so we wanna try to figure out how can we bring out the interests of each of those communities a little bit more. Uh, in addition, there are immersion communities like San Jose, uh, Silicon Valley. Those are areas that are not emergent necessarily, but they're not necessarily always what people associate with the San Francisco Bay Area. But we cover them a lot and we ask if we have this tool. What could it be used for? Well, it could be used to help our social teams understand what we're doing on a regular basis on our broadcast, whether it's the two hour show, or maybe it's within newscasts. How can that then support their goals to reach the audience? Well, if we have these descriptions, now we can decide how can we begin to target, say, Silicon Valley with social media content. We might be able to target Oakland a little bit more, or Berkeley or Richmond or any of the other areas in the Bay Area. Again, we're thinking about the use of AI and using these tools, but it's always with an eye to, figure out how can we bring the audience a little bit closer to the content that we're creating? How can we use AI to help the audience feel a deeper sense of belonging with what we do? I go back to what I mentioned at the very beginning in that we're in the relationship business. Not only do I wanna provide you information. But I want you to feel like you can come to me with your questions. Me as the broadcaster with your questions. You can come to me as the broadcaster if you're curious about what is happening in your area because you don't know. There are a lot. We're not an induced desert in the Bay Area at all. There are so many broadcasters out here, so many nonprofit newsrooms, so many other entities that are happy to provide you information. How can we make you feel like you have more of a seat at the kitchen table here than you would anywhere else? And how can these tools help us get to from the why to that sense of belonging that we want to continue to build on, so that we continue to build not only the trust relationship, but also the donor relationship and a long-term conversational piece for you as an audience member where we're not only that, person, that individual, that entity, that reporter, that organization that you trust. But we're also serving, for lack of a better word, the smart friend that helps you be the smart one within your friend circle because we giving you information that you found valuable in your life and that you're sharing out with others that is the larger. Chain, the ripple bond that we want to build with the audience, and we want to use AI to help us build that. And I love that you brought up using AI to, to process long form information. Right? Because, as a, as a public media organization or as a city, you probably have on hand a lot of transcripts. Those are again, things that you could use AI to summarize, just as you mentioned and say like, Hey, what are, what are the topics that are coming up? And that's a great starter for your social media, right? Like, what are some things that we need to highlight it, or reiterate in short form. From these most recent meetings that happened. Another great thing is a lot of cities are using, AI chatbots or forums on their website, and there's a lot of data coming from that, and you could summarize that in a useful way to help you move forward on, on next action steps to respond to feedback. Very useful ways to enhance something that was already part of your mission, right? Like you already had the mission of, of collecting feedback. You already had the, the, mission of recording the meeting that took place, and now you are giving people small bites of that longer form media that, that you had to post somewhere else, right? And someone might not have three hours, but they have three minutes to watch, a summary or a highlight from a video, right? So, practical ways that you can hopefully save time, highlight priorities in a faster way. And, and, and like you said, just give people a high volume of information that's based on like, what am I trying to say? Demand, right? Like, you understand, you understand what they want, because you're using these tools to help you process even more data. And I think that that is a great example of, of how we can enhance and not replace, because you still need people to feed the prompts into these tools, right? You still need people to say, like, tell me, you know, what the, the top five things were in this, in this meeting. We're not having it simply generate content from nothing. So always to be, yeah, it's always gonna be based on your goals first. And I think that something you bring up here that I, I think is, I wanna underscore because what you're saying is so accurate. I think AI has wonder, wonderful things like transcription. To be able to not have staff dedicating hours upon hours, having been somebody who's had to do that, transcribing, is a good example. But to what you've mentioned at the very end of this. It's really important to think about people as the connective tissue in your community, of your relationship to your community, because you, I oftentimes hear this, and I'm sure you've heard this as well, and probably many of the listeners have heard this too. We wanna have AI to do, the boring tasks. We want AI to do the tasks that are not as exciting, but. Again, if we're in the relationship business, I don't think I'm sharing breaking news here at all, but sometimes relationships are boring. Sometimes there are not the most exciting things that we all have to do with our, for our friends and our loved ones and our family members, but they're the things that bind us. They're the things that we remember the most. And so rather than have AI cough out of your emails or have AI do all the things that are those touch points, I wanna encourage you to please remember that those memories and those ways that you connecting with your community and your voice to, as you pointed out with transcripts to make sure that those are shined and brought out and contextualized in a way that are most relevant to the audience, to the people that you serve, are gonna be the ways that we as individuals maintain the work that we care so much about, so that we are not, as you pointed out, replaced because we are that connected tissue that brings these communities to the work that we do. Right. And, and in the city context, and pub well, public information context, city level, county level, state level, whatever it is. The communication that happens in a crisis like the bad news is often the most important work that you have. And if you think about leaving that to AI as opposed to having a human in the driver's seat, I think most of us would shake our heads, right? That we, none of us feel like that's a good idea. So, I think that's a good way to look at it, right? When we talk about AI for relationships, humans should always be in the driver's seat. Because the most important stuff, the most critical stuff, and sometimes the most boring stuff, like you said, really needs to come from a human, for people to still, understand the value and understand the point of why you're sharing that. The work that your constituents do is frankly, not terribly dissimilar than what, than what we do in public radio and television in the sense that when there is a crisis, when people are most unsure, when people just don't know where to go. They turn to your constituents for answers. They, they're not sure what's happening. They're not sure what to trust. There's, as we've, we've talked about so much information out there and you just don't know where to go. When that that worry for families is at its highest, they turn to your constituents for it, and this is the moment where you have to be completely dialed in. This is the moment where you have to be looking over every little bit of copy to make sure it is exactly right, because this is the time the audience will remember you. And not forget if you make a mistake, but they will absolutely also remember you if you help give them the information they need the most, when they need it the most, and you allay all their fears. You just are able to help them feel a sense of calm when there is so much going on in the world when it is incredibly stressful. We see this in public radio and television every single day. There is a lot of signal to noise and there's a lot of noise to signal, and people turn to us when they need answers that they feel like these have been vetted, and I'm not having to wonder if there's a lot of spin here, and I suspect with public information officers and their teams, people are coming to you because they really want to make sure that you're on it and that they can under understand this is information that they can trust. Exactly. So, so far we've been talking about the why. And the reason that I started us out there in this headier place was 'cause I eventually wanted to turn to, we're in a place now where people need to start thinking about putting AI in their SOPs or in some cases just having an AI policy. I mean, no shame to anyone who doesn't have one yet. It's been a, a thing that feels difficult to wrangle because things are so, are changing so fast, right? So if you don't have one yet, perfectly fine, but we're gonna talk about some of the things that you might wanna consider as you are forming, operating processes that include AI in some way. Organizational policies, departmental policies, and even just your everyday interactions with AI. So, so let's talk about that a little bit. When we, when we talked about this before, Ernesto, we had a little, little pre podcast chat. You had mentioned the presence of stigma like that some people don't even wanna admit that they use AI at work. Like what's happening there? I, I think that there is absolutely a worry for staff who are using AI and perceptions of what that is. In some, and the perception can be legitimate, per what I mentioned among my friend, who probably is the best writer and is using it and maybe not in the best ways. However, there's also just a perception from people that if you're using these tools, you might or might not be doing all of your own work. If you're not doing your own work, you should be doing your own work. I don't wanna encourage you or would discourage you actively from using AI to do all of your work. You're not seven, and it's good for you to think about how this can enhance your knowledge already. But there is that perception. There's a stigma to talking about it because people feel like they're going to be judged for it. And so the thing that we're oftentimes, running into here at KQED with an AI working group that we have, is how can we de-stigmatize that? How can we incentivize people talking about the ways that they are using AI and the questions they have about AI so that we might be able to advance the organization forward, not only to understand the questions, the real time questions people are dealing with, but also ensure that we're seeing more of the innovations that are happening and thinking, how could we grow this? How could we find a funder that might be interested in supporting growing this to a much greater degree that might support our audience goals that we might not really have ever perceived when this initial innovation took place. It is a learning process. I will tell you that because there is still a lot of fear. People just assume that, if I'm using AI, my coworkers will judge me harshly or there will be a feeling about any particular thing related to AI? A lot of feelings, as you alluded to across the board with AI that have nothing to do with people who use them, but that impacts staff members. And so it's been a journey for us. We haven't quite gotten there, but we're starting to really try to figure out how can we incentivize ways the staff can, can talk more about it. We've had a lot of in-person conversations with staff showing them tools, how they work, but taking post-it notes and asking people what kind of questions, what kind of things you're dealing with, and you can post them up. You don't have to say anything in the room. You don't have to be the person that calls yourself out, but if you wanna make some notes and leave them up, the working group can look at them later on and we can talk about them and figure out how can we solve this problem. It is just one way we've been trying to incentivize this and also trying to get it out more in the open AI use and conversations about such for the rest of staff. Right. I think getting rid of that stigma somehow, and it, it sounds like it's, it's a multi-step process and a long journey at that, is probably the only way that we're ever going to get people to feel comfortable being vulnerable, like truly vulnerable and transparent about the ways that they do or want to use AI. Right? And until we get there that we can't really perfect the process, right? Not that we're ever going to perfect it, but refine the process, right? And, and have good ethical and safe practices. I think safety is another thing we have to talk about. And on other end of that, oh, go ahead. Mm-hmm. I was gonna say, I think, and part of the other piece of that is just getting people to talk more openly about it. Getting people to understand what the possibilities are. Because oftentimes we found a lot of people have heard a lot of things. I think this is everywhere, this is every organization. A lot of people heard things about AI, but have, and it may have tried something, oh, I made a cat on a skateboard, or something like that. But have never really tried, a work use of it or workplace application of it. And so in the absence of a little information, confirmation bias sets in, you see headlines, you hear about things. You see the cat on the skateboard, but it doesn't do anything type of thing. And so you just don't really understand what it could practically be used for. So being able to share more real world examples. Being able to talk out questions you have related to your workspace has been very freeing for a lot of staff that we've, that I've talked to. Being able to get a chance to understand, oh, I didn't know I could use it for this is just a light bulb moment for a lot of people. And so if we do more of that, I think we're starting to see a little bit more openness and a little bit more conversation starting to happen, which I think is, very exciting. Before we get back to this episode, here's something you'll wanna hear. Picture this: seamless video access for residents seamless compliance for you. Whether it's city hall or at home, MediaScribe brings every resident into the conversation. With MediaScribe, captions are everywhere you need them: on live streams, the web and beyond, so no one misses a word. MediaScribe delivers accessibility straight to personal devices with live captions and translations wherever residents follow along. No extra apps to build, no barriers. Just clear compliant video wherever your community tunes in. And for your team, it's just as simple. MediaScribe automates the heavy lifting captions, transcripts, translations, and compliance delivered in real time. The result, seamless video access for residents, seamless compliance for you and behind it all MediaScribe is doing the work so you don't have to. MediaScribe: Compliance made simple, video access made easy. Learn more@mediascribe.ai. And on the other end of that, well, you also mentioned some AI gatekeeping, right? That there's people out there coming up with really great things and they, and they know what they're doing and they're not sharing them. Maybe on purpose or, or not on purpose? What? What have you seen around that? I think that it's one of the big challenges that I have noticed from time to time is that, there are, there's a lot of innovations happening and it's very exciting, but also for many media spaces, a lot of information can be proprietary. Those of you who are in the government or in nonprofit space, this happens a little bit less. But certainly in the commercial space, there is absolutely a lot of creation that's happening that's driven by competition. And as a result, you're having a real challenge, with trying to bridge the gap for non-commercial and governmental broadcasters and those who are providing information to the public that might not have the same level of access and that has a lot of downstream. Effects for the public that we might not really perceive as much, but are going to have big effects in the next five to 10 years. As that information gap and as the access gap to information and access to technology gap begins to widen, we are going to see a lot of challenges for, non-commercial nonprofit governmental information providers are going to just not be able to jump on these technologies as quickly as possible. And it's certainly a big worry of mine and a big worry of my colleagues in the public media space who are seeing, any number of commercial broadcasters that are just leaped ahead, leaps and bounds in some cases, far ahead of us. And the race to catch up is going to be something we're gonna have to contend with. And I even see gatekeeping as something that can be a problem, you know, even within one department or one organization, right? If those expectations are not communicated, that when you discover something. You would be encouraged to share it with your team or with your department, right? And I think sometimes that can just be something you don't think about, right? You're, you're busy, you've got a lot of output that you need to get done today, and you found something that works, great. You never thought to take the time to share it. And I think that's where, when we get really intentional, as as we're allowing our departments and our organizations to fully embrace AI, that we take the time to think about those situations as well. Like you mentioned that at KQED, you created your own internal tool that does these specific things, and I think that's really useful in certain cases like. A department that might have to respond to like a Freedom of Information Act request or in, or in your case, somebody might act, ask for, you know, fact checking around something you reported and, and to know specifically how information is being treated, how it's being processed, where it's being stored. Those are gonna be really important things to know moving forward. Right? Has that come up for you? Yeah. No, absolutely. And, and just speaking to this point about, siloing within organizations, this is relatively new technology for a lot of workplaces, relatively, of course, for a lot of workplaces and in the moment it's easy for us to get focused on one or two people who might know or understand some of these tools and are able to apply them in an effective way for an organization. But this is also a really critical moment for organizations to begin to think about how do we start to ensure that there is coalition building that's happening? How do we understand that there is cross-departmental communication that is happening, actively. So that the information isn't as siloed, you're not gonna be able to prevent it entirely. Every organization has its small or large silos, and that that happens, just in the course of work, especially if you work for a complex organization with many, many departments. But how can we foster and spark more active cross-departmental collaboration so that we together as an organization are learning actively around how these tools can be applied and then what are some of the guardrails, what are ways we don't want it used as an organization. Because when you do have those silos, you will oftentimes have use that might not necessarily fit as much with the mission, then we as an organization as a whole might have intended. So that intentional internal conversation is so crucial to get out on the table. What do we see that we as an organization want to be doing? But then where do we not want this to go? I was on a panel recently with Jon Zilkha from the BBC, and one of the things that Jon pointed out is that the BBC, we are very, very careful to say what tools are approved, what tools are not approved, and what tools are still under evaluation, and what is the process for someone being approved to use those tools? And the BBC is a very large, very complex international organization, but that's kind of what I mean. An organization has taken the step back and said, we've got people that want to use a tool or may not want to use ano another tool, but what's a good way to vet these tools in a way that ensures that we are meeting the mission and what's the process by which we can approve use of these tools so that everybody is on the same page, that these are being used. It's not in a silo where Ernesto is using it or Michelle is using it and we don't even know we're using it. We are communicating actively within our organization and we're trying to make sure that everybody is learning together around best practices so that we do the best work possible. Now you work at a very large organization today. But I know you've worked at several very small organizations and small departments, and I'm sure in some cases have been a one person department. Tell me what you, what's your best advice for smaller teams trying to deal with this? So going back to the very beginning of this conversation, thinking about the why, thinking about what we would like to address with these tools or how these tools could help us serve our mission. What are some goals that we can set? If, if it's a two person team, like for example, you and I, what are the goals that we would like to accomplish for our organization? And how could this tool, how could these tools help us scale that and then think, more broadly about what kind of goals we would have to demonstrate what we've accomplished or how we could accomplish it? For example, you and I may say, well, we'd like to increase our social media presence. We would like to grow, well, if we wanna increase our social media presence. What's the goal by which we could evaluate whether we've increased our social media presence? Well then we could say, well, we wanna increase by 500 followers. I'm just making up a number. We could say 10,000 if we wanted to, but let's go modest for this moment. Well we wanna increase by 500 followers. Well, then what would we need to do? Well, something you could potentially do is you could go to any of the large language models that are out there and you could input your information. We have on our Instagram account, 800 followers. We'd like to increase this to 500. Based on best practices and making up a prompt here, but this is a pretty accurate prompt based on social media best practices, how could we increase it? What are means by which we could do that? Would it be video? Would it be daily posts? What kinds of posts would we be doing? Who Then naturally, the question would be, who's our audience? What are their needs? What, how could we serve those and then build it from there? We could use that prompt then to help us potentially inform some focus groups. And when I mean focus groups, I, for those of you who are in the public information space, or either are probably glazing over because you're thinking, oh my gosh, to bring all these people together. Your focus group just needs to be one person. Or two people, or three people you mentioned in my bio about some of the things that I've done. But one of the things that I did was, the Sulzberger Executive Leadership Program, which is at Columbia University, and we advocated the idea of one person focus groups. I, Ernesto, am going to talk to you, Michelle, and I'm going to ask you about problems that you have within the work that you do. How you solve those problems, where you will look to solve those problems. And then I use my wonderful notepad. I still have an old school notepad that I use. I will make all the notes about the problems that you're dealing with in your work and how you're trying to solve them in ways that where you go to try to solve them. And then I build my tools from there because. The fundamental issue here is not AI. It's the problems that people are dealing with. So how can I ask within my prompt? Using my focus group, in this case, you, to start to solve problems and to start to address problems there that serves as content that we can then begin to build out, a strategy to grow those followers to 500. Solving some more to those problems. Now, your focus groups, your air quote, focus groups can be two more people, three more people. You can talk to a few people in your circle and ask 'em about the things that they're trying to work out. That can be part of the basis of your strategy. Again, it doesn't need to be very complicated. It doesn't need to be a team of 20 social media experts. It can be just one. You might also have things like, I need legal help in my space. Our teams might have an attorney or probably not. Could I ask, a large language model for guidance around illegal issue? That I'm looking for. You don't want her to obviously replace your hr, you don't want her to Or your lawyer. Yeah. wanna get sued, however you want it to be adversarial with you, to ask you to, for you to ask it a provocative question around a legal question and ask it to challenge you, and to be adversarial in a way that will push you to see beyond what your blind spots are and to you. And, and jumpstart your research potentially Yeah, absolutely. No, absolutely. One thing that I did recently was, and I've talked about this at at the AI panel I had with Jon, was I built a GPT. And a GPT is not very complicated. It's basically a very long prompt, for those of you who who go, oh my gosh, what is that? It's basically a very long prompt that you can do with Chat GPT, around underwriting content. So I have to review a lot of underwriting content, basically sponsorships and other kinds of things, but there are particular rules around that. And all of you probably have a very similar kind of experience in your own organizations and ask you to challenge me, I think that this might be right, or I think this might be wrong. Tell me what you think. And sometimes it's identify things that I completely missed because I just had an assu, made an assumption and, that's where I think AI can be incredibly beneficial, challenging our assumptions, helping push us beyond our comfort zones a bit, and seeing things that we don't always see because we see what we see because we've been in our businesses for a long time. As you pointed out at the very beginning of this, have been public radio a while. But there are biases that I have because I've been here for a while, a fresh set of data and some, some other technology that's seeing things in a different way that doesn't come at it from those years of experience, but comes at it from the hard data, is gonna see things in a different way. And I, I think in that way it can be really beneficial for a small team or a one person shop to have that push. If, if and when you wanna avail yourself of that. As you were saying that, it made me think of some ways that you could structure a workflow to deal with, feedback, right? So, so say you were trying to grow your social media audience and, and you had that one person focus group, or you had a large amount of feedback from a forum on your website or some other source where you're collecting resident feedback, right? And then you use AI to, to summarize that for you. And then you could maybe take your transcripts and, and compare the two and say, Hey, these are the things that people feel, the areas where people feel underserved or they have questions. Right? Can I answer any of these questions from this? Data that we already have from content that we already have, and start to connect the dots, right? So you can identify a things that you're not sharing enough that already exist, right? And, and B, things that you need to clarify because there nothing exists to clarify those things yet, right? So there's, yeah. So you're enhancing, you're enhancing your research, right? You're getting things done faster and you're able to consider a larger amount of data. There's a fascinating development, the, in the, the audience engagement space around AI, called synthetic focus groups that speak to exactly what you're talking about. You take your audience data. Or you take your surveys, you take all the things that you're talking about, and you can feed it into a large language model. I would encourage you to try several of them because one, just because it's ubiquitous doesn't necessarily mean it's the best one. Sometimes you'll get better answers, from another one, but you'll feed in a bunch of audience data and then you'll ask it, give me some profiles based on this, this information that you've got about these sur from these surveys or whatever data you might have, and profile a few of these groups based on this information, give me three. And so you may get three focus groups or three synthetic focus group audience groups out of this that are completely composed of the information profiled out of each of those groups. And then it's another test out of that synthetic focus group you can say, here's a piece of copy that I am going to put up about, some issue in the city. There's going to be a park renovation. I'm making up things now, of course, park renovations, health access, other kinds of things. Tell me how these synthetic focus groups, based on the information that you have, would react to the way that this is worded. And I have some friends at a, a wonderful organization called Verso, and one of the things they pointed out is you have to continuously keep this information refreshed. It's not going to live on forever to be accurate because people's attitudes change. But based on their tests of the synthetic focus groups against the real focus groups, you probably had about 75% accuracy. Which isn't too bad when you think about anything you wanted to feed into this, these synthetic focus groups to ask them about how they would react if three outta four times they reacted the same as the real focus group, the real individuals for which this focus group was created. It might be, it might give you some ways to begin to think about how your information is presented, and that's created solely out of AI And I can see that being a resource for small teams that might not have the time or the resources to do anything besides, you know, collect a lot of existing data. Right? So that's where you start to see the combination of, of many automated tools, potentially AI driven automated tools, that can really start to pull information together to again, enhance what you do. And, And there are tools like Notebook LM, that you could use for a very spiff specific small data set. Notebook LM is free with your Google account and it will do the drill down for you around a specific data set. Now if you give it, say, park reno history of park renovations, and you give it a set of data and you can prompt it and ask questions about that. Now, if you suddenly have questions about, sidewalks. It's probably not gonna have all that information because it's almost an LLM designed specifically around that set of data that you've got. And so those kinds of, you can experiment a little bit with these kinds of things to just develop a level of comfort ,and also begin to begin to understand how the tools are used related to your work. Because I think we alluded to this at the very beginning of the conversation, all these tools can be very overwhelming. They do so many different things that when you can find these moments where I have an application that I can really try out and get a level of comfort with, then I can begin to open up the box and have more questions and begin to test out more tools that might be useful to me as a small team rather than be overwhelmed with the six than by the 600 AI tools that are out there. 599 of which I might not ever have a use for. Right, so like for smaller, for smaller organizations that are just getting started, finding the most purpose built tools that you can find will often be the best place to start. If you're a larger organization who has the resources to develop your own purpose-built tools, like you did at KQAD, that can also be really helpful because you could have all of the data coming into that tool in a central place as well. It, and it just, it really streamlines everything, right? We know that this is what we use it for. This is what it's gonna output. And I don't have to spend a bunch of time prompting because it's already trained to do this specific thing. Right? And I, and we could have a whole other episode about, information bias in AI. Right? But that's, that's another episode. But that could also really, help mitigate that problem as well. Well, as you alluded to, it's, you always need to have someone reviewing everything. I oftentimes have told people that for me, AI is the anxious intern. It wants to please you. It's gotta, it can only deal with the question you've given it. But it might not understand the context 'cause it's, it's completely new to this information. So it's gonna try to give you the answer that you're looking for, but you as the expert know much more. You have the context, you have the nuance, you know, a great deal more so if you're going to ask it for an analysis, if you're gonna ask it to look over a data set and point out things, discrepancies or point out anything that you might be curious about. You have to be very, very thorough with your prompt, and you also have to be very thorough about your review, to fact check it frankly and verify that what it is saying is as accurate as possible, because it just may not understand every bit of detail that you do as the expert. And that can be, that can range from, you know, detrimental in terms of inaccuracy to just like to an annoyance. I mean, I, in a recent, collaboration with a leader in the digital accessibility space, we were hearing that one of the biggest complaints across the board, in terms of like digital accessibility is people using, AI captioning without correcting it. Like just letting the AI do its thing and never updating it and like. You should be using a tool that lets you have a human review and go back and fix it. You know, next level, on top of that, having a tool that you can train ahead of time, for your specific purposes is great too. But it's just, it's things like that, right? Even the most simple thing that we, we think like, oh, AI transcription is so basic and it's so ubiquitous now. Even that still needs human review to really serve the purpose that it's supposed to serve. So trying to, that's another thing you have to work into your procedurals and just your everyday decisions with AI, right? Is like, what, where am I really gaining efficiency? Because everything that I use AI for, I also have to have some human at some point review, right? Yeah. Now, and you're, you're so on point about just ensuring that you have that person you make the time available for. If you're on a small team, it's gonna be just you. So make sure to be very intentional about that. Something else I always have to remind people about. I think the, the assumption and the bias is that AI is going to help us make work easier or make it quicker. However, that's not always the case. There's a really great study by an organization called metr, METR, that pointed out, especially in coding in some other areas where people were actually spending a lot of extra time fixing what AI gave them than they would've spent just doing it. So there are some times where it's just, and you have to be very discerning about this as you do your testing and realize, is this just gonna be more time than it's worth? Is this, is it time better spent just to do it on my own? Or in cases, as you pointed out, where you can scale it to do much more. Where doing that front end work is actually going to save you much time, much more time later on. It's just an equation you really want to be very eyes open about. As you begin to think about the AI journey, because I do think that there is a, a lot of hand waviness about some of this, that it's going to fix everything and do everything and make our lives and jobs much easier. And sometimes it turns out to be a lot more work and sometimes it can be a lot of work on the front end and much be much easier on the backend. But you just have to be very, very thoughtful about, what it is you want it to do. Right. I think it's still important to be discerning and, and pay attention to that. Because you can get excited, it can be fun to interact with AI too, right? So you can get lost in that and forget like, oh yeah, this actually took like twice as long as it would've if I just wrote it myself. But I was having fun going back, seeing what it would give me. It's almost like, it's, it's that, that social media effect, right? You, you don't know what it's gonna churn out. So, so you're excited to just keep iterating and, and see what it gives you. It's like a, it's like a AI slot machine. They, they're miraculous tools. They truly are miraculous tools in the sense that you can go in there, you can, as a history dork. I, I love history and having somebody who can someone something. To tell me about little pieces of history that I had, would have to go look up very down to the library, would be much, e is much easier to get it that way or to get just facts about science or biology about all these kinds of things. Mathematics so fascinating to be able to get this as just this game changer for a lot of people. But this is, I think why it has been such a terror in the education space because there are a lot of teachers who are very concerned and a lot of fa parents who don't know how to, to direct it, for their young children and, and teachers who don't know how to deal with this in classrooms. Our education department here at KQED has put together a number of curricula for teachers and parents. To help them talk to kids about how these tools are used for this exact same reason, because they are magical tools. But, yeah, they can definitely cause a little bit of havoc, in different, depending on how it gets, applied. Yeah, and we're getting into that meta space. So that's something where, you know, news and public information also will probably find themselves saddled with the responsibility of educating people about AI safety, right? Because so many people have been introduced to this as an adult, like it might be easier to help kids understand, and then they'll have grown up with it, and I think that they'll hopefully be able to be a bit more discerning about it. But adults and particularly older adults, they need a lot more, regular interaction, you know, regular training, rather regular chances to discern whether or not something is AI or quote unquote real, right? Because AI is just, as we mentioned at the beginning, it's, it's shaping our reality in so many ways that we aren't, conscious of most of the time. But, No, it's absolutely true and I think that there, yeah, there are so many communities right now that are also trying to figure out how do we get information to people as publishers, and I'm sure as public information officers and staff are really trying to think about how. We connect with the, the public. It used to be we could go to a search engine and our information will pop up immediately, but now as search engines just are becoming answer engines, because ai,AIe are just having a harder time breaking through. So I think that our role is very important, but it's also no more than it's ever been, than today. These are all, these are all complex questions, complex conversations, and we are going to have to continue to revisit this, so I'm sure, I'm sure we'll have to have you back in a year or probably less than that, right? To see what you've discovered in the meantime. Thank you so much Ernesto Aguilar, Executive Director of Radio Programming and Content Innovation at KQED in San Francisco. This has been the Government Video Podcast. I'm your host, Michelle Alimoradi. If you found this episode useful, we hope that you'll please share it with a colleague. Please like us and give us a review on your favorite podcast platform. And please tune in next time for our next episode on Best Practices in Government video. Thanks so much.