ConVEx 2024: Advice for AI in content operations

At the 2024 ConVEx conference, Scriptorium CEO Sarah O’Keefe was part of a panel of content experts including Dawn Stevens, Val Swisher, and Rob Hanna. The panelists discussed the pros, cons, and cautions of using AI in content creation. 

“What are the advantages and disadvantages of AI as a writing tool?”

AI is good at patterns, and it’s good at extracting meaning from existing content. If you have this huge volume of stuff, you can ask it to summarize what’s in there, so this is a process of taking existing information and summarizing or consolidating it down. It’s really bad at creating new content from scratch. That is really the key; if you have content and you ask it to summarize, analyze, assess, and so on, it’ll do that pretty well. If you don’t have the content you need and you ask it to impute or infer, it’s going to make stuff up. You have to be really, really careful as you get into those kinds of things.

The other thing I’ll say is that you have to really watch out for the bias that is introduced by the content that you’re feeding the AI, which is also biased. It is going to extrapolate from what you give it. Whatever bias is already in that content is going to get emphasized in many cases. It’s not that the AI is bad and terrible and biased. It’s that what you are feeding it is biased and therefore it will produce biased content—so pay attention.

— Sarah O’Keefe

“How can AI be used by teams today?” 

I think there’s a lot of pressure from upper management executives who say, “Oh, this AI thing is awesome. We’ll just do that and fire everybody.” It’s important to appear to be cooperative with the AI strategy while simultaneously making sure that your company doesn’t do something extraordinarily stupid.

Let’s back up for a second and talk about car manufacturing. When the car assembly line came in, the cars rolling off the assembly line were objectively worse than the custom-built cars, but they were cheaper and faster. Over time, the manufacturing assembly line process introduced guard rails in the sense that you’re going to produce a higher-quality car off an assembly line today than you are when you custom build it. Tolerances got a lot finer, they introduced standardization of parts, and they had components that you could use in multiple car models.

Think of this AI piece as something that’s going to force you into that manufacturing model. It will force you to have higher quality and lower tolerance for variance in your content so that you can deliver high-quality assembled content deliverables.

Ultimately, I think what AI is going to do to the content process is force us into a model that is much more rigorous in terms of the actual content creation and production. Focus on that, and not on, “AI is bad and I don’t want to use it.” You’re not going to win that argument.

— Sarah O’Keefe

Assessing risk with AI

I would start with your risk profile. What kind of products do you have? What are the implications of getting it wrong? This looks different for a medical device than it does for a video game. What’s your risk profile? What happens if something goes wrong? Who do you get in trouble with? What happens? Is it that you won’t be able to sell? Or is it that you’ll get shut down by the FDA?

— Sarah O’Keefe

Beware the foraged mushrooms

During the panel, Dawn Stevens explained the unique design of the slides. “You may wonder why our slides have mushrooms on them. The reason is that when we met to talk about this panel, an article was posted by The Washington Post that using AI to spot edible mushrooms could kill you.” 

The next day, this sign was displayed at the hotel buffet. 

Beware the foraged mushrooms. 

Other questions about AI and the future content? Ask us!