Bridging the gap between content design, content ops, and AI

Podcast: Play in new window | Download

Subscribe: Apple Podcasts | Spotify | Amazon Music | Email | TuneIn | RSS

Is AI the catalyst that will finally bring content design and content ops together? In this episode, Sarah O’Keefe speaks with Kate Agena, Vice President of Content Design at SAP, about why fluency is cheap while meaning remains expensive in an AI-driven world.

Kate Agena: But users don’t care if content is coming from marketing, product, or whatever: they want whatever it is that they need at the time they need it. If everything is just kind of in one pile of information, that’s not going to come out correctly. So I think ops is just so incredibly important now, and content ops in particular are going to be far more important than they were.

Related links:

• KateAgena.com
• Paz Parez: Context architecture
• Digital sovereignty in the age of AI
• From ad hoc to autonomous: The AI content ops maturity model

LinkedIn:

• Sarah O’Keefe
• Kate Agena

Transcript:

Disclaimer: This is a machine-generated transcript with edits.

Introduction with ambient background music

Christine Cuellar: From Scriptorium, this is Content Operations, a show that delivers industry-leading insights for global organizations.

Bill Swallow: In the end, you have a unified experience so that people aren’t relearning how to engage with your content in every context you produce it.

Sarah O’Keefe: Change is perceived as being risky; you have to convince me that making the change is less risky than not making the change.

Alan Pringle: And at some point, you are going to have tools, technology, and processes that no longer support your needs, so if you think about that ahead of time, you’re going to be much better off.

End of introduction

SO: Hi everyone, I’m Sarah O’Keefe. I’m here today with Kate Agena, who is Vice President of Content Design at SAP and also founder of the Content Design Leaders Membership Community. She has a PhD from Purdue, and in doing her PhD, she studied how tech writers could use iterative development processes to impact software design, otherwise known as, at least to me, basically waterfall versus agile. So she’s got some really interesting background as a tech writer and kind of started out there via I think information development at IBM and then landed in content design. So welcome, Kate.

Kate Agena: Thank you so much, Sarah. Happy to be here.

SO: Yeah, we’re glad to have you and to connect some of the maybe more technical content enabling content side of things to content design. our focus today is actually on that, on the fact that content design and content ops are related, or maybe should be, but somehow aren’t connected. And Kate has a theory that AI is actually going to for once be a force for good and maybe bring these disciplines together. So we should talk about that. But first, give us your definition, your short definition of content design. What is that?

KA: Yeah, so for me, UX content design is a UX design practice, first of all. And it is the practice of designing the information and the language within a product experience so people can understand what’s happening, they can make decisions better, and they can accomplish whatever it is that they’re trying to do.

SO: Yeah, and I think I would argue that content ops is the tooling and the processes that sit underneath that, that this content design process or discipline flows over or inside. so I was reading up on some of your stuff, and I’ve included Kate’s personal website in our show notes. One of the things that you said, and I’ve seen this in longer phrasing from other people, but you had a very succinct definition of this. You said fluency is now cheap and meaning is not. 

So, we’ve said for many, many years, that it is not true that anyone can write. However, it turns out that with AI, if you know, if writing counts as let the AI generate the text, then I guess it is actually true that anyone can write. The problem is that we evaluate content on fluency. If it’s poorly written, if it’s hard to understand, if it’s ungrammatical, we assume it’s low quality. But that isn’t true anymore because AI will always generate something that looks good, looks plausible. So it sounds good, it’s well written for values of the mechanics are correct, but it’s not necessarily accurate. So when you say that fluency is cheap because of AI, but meaning is not. What what is meaning and why is it still expensive?

KA: Yeah. So I mean, for me, I think coming from the techcomm world where, you know, and from rhetoric where obviously I’m always worried about things like audience purpose and context. What is it that the person is needing? What are they trying to do? And then design really makes you think about psychology and you know, where is this in a user’s journey and maybe in the user’s day and additional, you know, a lot of different context around whatever the content is. And that emphasizes so much user research. You understand if you’re, you know, if you have experience doing content design, you have to like, you know, changing one phrase, changing one word can make a huge difference on, you know, even product success, but certainly on how a user reacts to that content. 

And so it’s, you know, it’s very different for an AI to just kind of generate a phrase that has the general meaning, supposedly, that you need in that piece of UI, than to really do all of the thinking about the human context that’s around that. And I think there’s a lot of discernment that happens that a content designer brings in, or even a technical writer brings in to documentation, maybe. 

As they’re kind of interpreting a lot of times what devs have put in, that any tech writer will tell you that they do a lot of work to get maybe specs or you know, JIRA information or a description that a developer gave them into a format that users can actually understand. And with content design, it’s the same that you really have a lot of human context that you need to take into account. And that’s a lot of discernment that I think. You know, you can’t give all of that context to the AI so easily. and if you try to, I think you it’s it’s gonna get lost in the context window at some point.

SO: So then as we’re working through this and we’re saying, well, you know, writing and discernment and humans are more important than ever, you know, I had yet another note from a colleague who’s facing a layoff. And it’s pretty clear that at the individual level, people are losing their jobs because of AI. the AI can do it, we’re getting rid of our entire team. The AI can do it, you’re out. So there are lots of content people looking for work on sort of both sides of this fence that we’re talking about, but there are also new opportunities. So acknowledging that there’s huge disruption and a lot of pain, where do you see some potential and some new jobs and some new specializations evolving?

KA: Yeah, I think there’s a lot. And I I feel like there should be kind of niches that people can find something that feels a little bit closer to their own skill set to move into. In content design, a lot of folks are moving into content systems, which is a little closer to content ops. Also, content ops, I think, should be something that people could be moving into more content strategies, you know, kind of ideas that we would have thought about as content strategy a while back.

So there’s definitely the system side, which is just expanding because now we’re talking about things like SAP calls it a compositional design system that would allow a machine to use that design system to generate UIs on the fly based on user intent. And to do that, you need an awful lot of information to help that machine kind of go through a tree of decision-making.

And find patterns. What are the good patterns for doing this specific thing? How do you understand what the user is actually looking to do? And then things that you know used to be a human would be involved, and so they would bring in all of their human knowledge. You need to encode that into components and things as well. So there’s, you know, kind of system-level voice and tone, personality, behavioral things, but also, you know, on the component level, there needs to be a pattern for what the copy should be on that specific component. So I think content systems is a huge new area. Where, you know, before I think we always wanted to put resource into that kind of work and there wasn’t any appetite for resourcing it. And now it’s kind of table stakes that we need to get there. If they want AI to do these things, we need to have a lot more effort put into content systems and governance. You know, it’s one thing to build it, and it’s another thing to keep it going. 

SO: A bad word.

KA: So content systems, yes, content systems is a big one. A second big one for content designers is model design, or there’s a lot of different names going around for it now. Paz Paris called it model design when she first kind of identified it when she was at Google working on, actually, I think it was working on Bard at the time, before it was Gemini. And that’s work that content designers have been kind of really a natural fit for. So that’s things like system prompts and skills and evals and determining all of that stuff around kind of how the model will behave with folks. so model design is a second one, content engineering people call it, that kind of thing. So that’s a second big one, I think, for content designers. 

I don’t think static UI content design is going to fully go away, but I don’t think anyone will be doing exclusively that. I think there’s a new need that you have to have patterns and guidelines that fit your static UI, but you have to have all of those also articulated for the machines. And then everything has to go together. So I think on the content design side, that’s where most people are trying to learn skills. And then I see this additional area that I feel from my background would be ideal for folks with a tech writing background, which is, you know, you just have such a greater need for structured content and for kind of interpreting what maybe a developer or a data scientist or a PM or someone else, even a designer, articulates as this is the need, and we need this put into clear structured content.

That the machine can read. And I think that there’s a big space for that. Because I also feel like there’s, you know, I mean, I see in SAP, we have a ton of data scientists, we have a huge amount of folks that are working on ontologies and knowledge graphs and things like that. But from the design perspective, I feel like coming from the user’s side of things, there’s probably a gap in between somewhere where there needs to be some kind of like interpretation layer that connects with how the user sees things to how the data sees things. And I have a feeling that, you know, that needs to be designed. It’s not, it’s not objective. And so I feel like there’s a space for designing that level, that’s a new level that I am not sure until we’re really trying these things and finding that we’re not quite connecting. I don’t think companies will realize they need that. But I have a feeling that’s going to come. And I think content folks are best equipped to do that.

SO: So one of the things that sends me into sort of incandescent rage is, you know, we talk about all these content groups and all this investment that’s being done and structured content and all the rest of it. And then you go talk to the AI team and what you’ll discover is that they go over to SharePoint and they grab a bunch of PDFs and they dump them in their LLM. And then they’re like, yeah, we have to figure out how to do the metadata and it’s real challenge. Really, you don’t say. okay, allow me to show you the back end of where these PDFs come from. And they’re like, that looks hard, just I need a PDF-based solution. And then I just start foaming at the mouth, and that doesn’t go well. So I’m still working on that one. 

But as we, you know, as content design, it sounds like evolves a little bit towards systems, which means content ops to a certain extent. And then content ops, you know, needs to connect better with content design because just automating it, see my example of PDFs being dumped, is not actually thoughtful necessarily, and it needs to be done better. So where do you see opportunities for these things to connect via the I guess like catalyst of AI or something?

KA: Yeah, I mean, I think so content designers can bring language, they can bring interaction patterns, they can bring meaning into structured content, terminology, taxonomy, and the context layer. And I think that, you know, all of that needs to go into systems, and those systems need to be scalable, they need that they’re that bad word again. They need to be governed, they need to be maintained. there’s, you know, I mean. We know how difficult it is in any org, I think, of any size to get any kind of handle on all of the content. And there’s this idea that, you know, with AI, you can just like magically throw it all in and it’s gonna come out in some way that makes sense. And that’s, you know, just not going to be the case, I think. so I I think that there’s there’s room already in content design for there to be more content ops.

I think that content designers kind of define what good content experience is. And at this point, to make that scalable, it has to show up in a lot of different places. And there has to be a really complex architecture for that context to be given to the machine in a way that’s going to be effective and efficient. So, you know, there’s this architecture of there’s, you know, system level prompts that are going to apply to everything. There’s skills that need to be pulled at the moment that that specific skill is needed. And, you know, and we’ve got a lot of UX writing skills and things like that. But I think that this like large UX writing skill is not the answer. It’s going to be pretty granular skills that are needed and an architecture to help the machine know when to use which one.

And then evals and also the whole process of doing evals and getting results and doing things that will improve the results next time around. To me, those are all really ops-centered things. And we’re going to need better systems to run all of that. And I think we also are going to need better systems to help deal with those silos and silos of content and disagreement on content, because you know, if a user is interacting with an AI from a product, you know, it’s not so different. I in so many ways nothing has changed. But they don’t care if it’s you know coming from marketing or from the product or from whatever, they want whatever it is that they need at the time. And if you know everything is just kind of in one pile of information, that’s not going to come out. Correctly. So I think ops is just so incredibly important now, and content ops in particular are going to be far more important than they were. 

SO: Mm.

KA: I think, you know, at least in orgs that I’ve been in, that’s been kind of more, you know, there’s marketing content ops, and they have marketing CMS and all of a whole system there, and maybe, maybe also with like experience, like journeys and things like that. So they know all about the customer and what they’ve done and what they haven’t done and what question they asked and et cetera. But that’s completely siloed from product content. There’s customer support content, that’s a whole other thing. You know, all of these things have been separate. And we’re going to have to find more ways to unify that entire picture in a way that actually can make sense.

SO: Right. And the interesting thing to me is that fundamentally, if you think about it, the reason that they’re siloed is because in the olden days they were siloed, right? 

KA: Mm-hmm.

SO: You go to a trade show, and you get marketing paper, collateral, handouts, and then you buy the product and the product comes with documentation like a book on paper. And maybe you read it and maybe you don’t, but that’s where the technical documentation lives. And then you have a problem. So you call up on the phone or your field services tech comes on site and helps you out. So then along comes digital, just broadly digital web online stuff. Well, now everything is on the web page, but it’s like training.yada.com and docs.blah.com and well www.yada.com, which is of course the marketing content. So even there though, once you sort of find yourself in the learning management system, the experience is different. And so you’re signaled that that’s where you are. Fine.

KA: Right.

SO: And then there’s a docs portal, we hope. And then there’s the marketing, the dot-com. But now the way this is going to work is that I, as a reader, as a content consumer, am going to ask the AI for information. And it’s going to give me this flat text. And by flat, I mean not designed by the originator.

KA: Mm-hmm.

SO: Right? It just shoves it through the LLM and spews out a bunch of text at me and says, here’s the answer you’re looking for, here’s the information you’re looking for, maybe. And so therefore, that whole signaling mechanism that says, I am training content, or I am marketing content, or I am tech support content is basically gone. It’s just here’s the answer to your question, and it’s conversational, it’s interesting, and maybe it’s accurate; we don’t know. It’s probably accurate at a high level in the sense that the first question you ask is pretty accurate. As you start drilling down, it gets less and less and less accurate because it has less and less and less information to average in order to give you that answer. So with all of that, I was gonna ask you how the next couple of years are gonna go, but then I decided that was dumb. So how do you think the next six months, or maybe six weeks, or you know, even next week? Like pick a time period and tell us what you think.

KA: Yeah, I mean, from my point of view, and I talk to a lot of folks from across content design, and that’s kind of been my focus the last months is to really think about future of UX content and where we’re headed. I do see what you were saying earlier of you know, there are have been a lot of layoffs. I expect there will probably continue to be a lot of layoffs in content roles and design roles and everything else probably as well.

So I think that in the short run, things are gonna be pretty bumpy. I do think that companies are getting to the point where the development that they’ve been doing on their products is getting further along. They’re closer to releasing something or at least using something with customers. And I also have a feeling that there’s going to be we’re getting closer to this point where companies are going to start seeing the way that we thought we could do this is not working. And they’re gonna have to start trying to figure out where are the points that it’s breaking. Now, whether or not they will have the folks at the table who will know where to point as, you know, these are the kinds of skills that we need back here. I don’t know. I guess I’m in some ways I’m not terribly optimistic about that in the short term. but I do think eventually they’re going to have to figure out what’s actually required.

To build these products that they want to build. And I think that a lot of content people are going to be required to do that work. And so I’m hopeful, I don’t know if in the next six months, but in the next year anyway, that we will see a major turnaround of companies starting to really hire back content folks. I’m not sure if they will have the same titles they had before, or there’ll be new titles. But I do think within a year, we’re going to start seeing the turnaround on content folks having a lot of work. but I I have a feeling that the success of the products themselves are going to be the thing that’s going to cause that turnaround to happen.

SO: What’s your best advice to the people that are being displaced right now?

KA: Yeah, I mean, I think there’s so much hype about AI, and a lot of us would probably wish that we could tell folks, don’t worry about it. You don’t have to learn it, you can choose not to. And I mean, there’s a lot of talk now in design at least of people maybe wanting to leave tech. And I think that can be a valid choice. But if you do want to stay in tech and you do want to stay in user experience or even in techcomm, I think you really need to be upskilling, a lot of upskilling. And there’s a ton of information out there. I think there’s courses are just starting to come out. A lot of them are not necessarily that good. now I think the better courses are starting to come out. 

SO: Because ironically, if you write them with AI, they’re not going to be that good.

KA: That’s the problem. That is the problem. So find the folks that have actually done it who are teaching things because they’re gonna be the ones that have, you know, better advice for you and more realistic advice of kind of, you know, this is what this is what I’m seeing while I’m doing it, or this is how it came about, as you know, I mean, model design is a whole new field. It’s not that they’ve necessarily figured out the best way to do these things, but the same kinds of skills that we have as content folks are the things that are going to stay the same, even if the format changes. And you know, now we’re thinking about all these markdown files. And, you know, I think in design systems there’s a lot of talk about adding context to JSON files and things like that. All of that stuff can change, but the ability to know what good looks like doesn’t change. 

So I think it’s a combination of trying to get really clear on those fundamental skills that you have. So, you know, are you, are you a good writer? Are you a good designer? And what makes you a good writer or a good designer? Can you articulate those things? And can you get better at being able to articulate those things? Because I think that’s what’s going to be needed. We’re going to need to define what good looks like and be able to create an eval for what good looks like in a way that we didn’t have to before. So one thing is double down on your fundamentals and make sure that you’re really you know, not just kind of a surface-level writer, but you’re getting down into the depths of why what you’re doing is right. 

And then second is, you know, that upskilling on could be on model design, could be on content systems, could be on, you know, I mean, Mike Iantosca is constantly putting out new stuff on how DITA can be expanded to use AI. So I think depending on where your kind of comfort is, there’s plenty of ways that you can find things, even for free. And that’s one of the things I would tell people is there’s lots of courses, but you don’t have to spend a fortune, especially if you don’t have work right now. There’s so much good information out there for free. So just, you know, go out there, start digging into it. And also use AI in your, you know, daily activities because a lot of times you it’s something you’ve said many times, like if you talk to an AI for five minutes, it might sound really smart, but the more you use it, you’ll figure out that it’s it doesn’t get it right so easily.

SO: Yeah, I think the best advice I have right now is that while I think we have a recognition that there’s some really problematic stuff going on, and we haven’t talked about, you know, the problems with bias and ethics, but you cannot in in this universe that we live in, which is largely, you know, tech and software, you cannot be the anti-AI person. That is not going to work.

KA: Mm-hmm.

SO: So my advice to people is to do two things at the same time, which are try nudge your AI strategy, your organization, assuming you still have a job, into a place of using it intelligently and away from a place of using it unthinkingly and unethically, without putting yourself in the position of becoming known as the person who is sort of sounding the alarm. So it’s like look like an advocate, but nudging it in a direction it needs to go. 

KA: Absolutely. I love that.

SO: Because people are making some really, really dumb decisions. I don’t mean us people, but at a at a very high level, there’s a lot of people that perceive AI as the solution to all of their problems, which are basically, goody, we’ll never have to hire anybody again. And that’s real dumb. And I worry a whole lot about the products that we work on and what would happen if and when those products are AI-ified in non-strategic ways and what could happen.

KA: Mm.

SO: I think about medical devices and bad AI, and it kind of makes me twitch.

KA: Mm-hmm. Mm-hmm.

SO: So you know I see risks there. And I think I wanted to close with that. Either what do you see as the biggest risk, or maybe that, and also what’s the biggest opportunity here? Like what are the two sides of the coin of this, you know, innovation, new thing that we’re in the middle of?

KA: Yeah, I think there’s a ton of risks. As you say, it’s not popular to be the person who’s talking about those all the time. But all those things you mentioned are huge risks. I’m not even sure I would call them risks. They’re inevitable at this moment. A lot of, you know, the issues with ethics, issues with inclusion, issues. I mean, in content design, I feel like content designers care so much about the users. 

They’re just completely full of empathy for users. And they’re often the ones who come in and argue with a PM or argue with someone about the way that they’re, you know, suggesting something is done in a product. And, you know, I don’t know that we can even, if we try to encode that into some kind of discernment for a machine, I’m not sure that it can do that. So I think there’s a huge risk of taking humans out of these processes because I just think humans bring so much to them on that kind of risk side. I also think there’s a huge risk that these products will never quite get to the level that the companies need them to get to. 

You know, if you’ve worked in content, you know all the things you need to deal with legal and all the compliance issues that you have and how difficult it is to get people to trust your product enough to want to adopt it. If if you’re in consumer work, even more so that you’re, you know, constantly focused on how do we get people to trust this product and want to use this product. and I’m just not sure. Well, I just feel like it’s we’re very, very far from the orchestration, the mechanisms, the context, the data: All the different pieces that would need to be in place and working pretty well, working pretty smoothly in a way that orgs have never been able to make all of those things work really smoothly. And I don’t think that AI can magically clean that all up. And so I think there’s definitely a risk that companies are putting so much effort into AI and expecting it to get to a level that it’s just not going to get to.

So I think that’s a huge risk. In the meantime, my take on it and my take for my team and others in content is use this time to be learning and we’ll see where it goes. You know, but in the meantime, if you’re learning and, like you say, I think if you have an opportunity to nudge something in a better direction, do it. You know, speak up with some ideas. Yes, you don’t want to be the person who’s anti-AI at this moment in your company, probably. But that doesn’t mean you don’t have small opportunities where you can nudge things in a positive place. so I think that’s a lot of risk, a lot of danger. 

And that’s not even touching on all of the real dangers that I think we all can kind of see coming with AI. And then to move on to the biggest opportunity, I think I mean it’s a challenging moment for people in content. And I feel like we have our work cut out for us. And if this is ever going to get to a functioning place, it’s going to require a lot of really smart people who work in content and who have the skills that we have. And so in that way, I feel there is a ton of opportunity for for folks who have a background in content fields. And so I think you know, I think that’s wonderful and to maybe tie them together. One thing I love about that is that content folks tend to care about ethics and tend to care, you know, they have a good set of values that they want to bring to things. 

SO: Yup

KA: And so if a future necessity at some point will be a lot of content folks, I think that’s going to be a positive for all of these AI experiences eventually.

SO: I think that I heard this I think over the weekend, and I can’t remember from whom, but it was 100% not me. So let’s start there. There was an interesting discussion around the idea that code is effectively an artificial language. It’s a language in a box. It has a set number of things that you can do with it, which means that effectively, it’s a solvable problem for a machine. Content language, not code, but language, is not that. 

Because people can use language in interesting ways, you can use it in ways that’s never been used before. You can analogize the orca sitting off of Gibraltar to something interesting that nobody’s done before. That’s not a thing you do with software code, right? So language itself has simile and metaphor and all these other fun things, and it has different forms and shapes and things that you can do with it. And so fundamentally, solving for code automation is a different and much easier problem than solving for language automation. But we’re accustomed to thinking that code is hard and language is easy. And I think what we’re gonna find in an AI world is that the opposite is actually true. So that’s the opportunity that I see is once we get to that point, and I hear you saying a year, and I’m, you know, it could be next week, who knows?

KA: Who knows?

SO: Who knows? Okay, so yes, so our concluding thought is who knows? Yeah. It depends. That’s always

KA: It depends. That’s always the content answer, right?

SO: So Kate, well, thank you. This has been really, really interesting. And I hope you’ll come back at some point in six months or six weeks or whenever the next, you know, disaster strikes us, and we’ll take it from there.

KA: Perfect. Absolutely. There’s so many more things we could chat about.

SO: So many. Thank you.

Conclusion with ambient background music

CC: Thank you for listening to Content Operations by Scriptorium. For more information, visit Scriptorium.com or check the show notes for relevant links.