When or If AI Meets Indigenous Knowledge?

An Ideation Series: Part 3, What Would It Mean to Say AI-Prohibited?

In the last part, I talked about classification, about sorting what AI can touch from what it can't. I ended by saying there's one category I keep circling back to, the hardest one of all. This is the one where I have to ask myself whether there is knowledge that should never be shared with AI. Not "not yet." Not "with permission." Never.

I've been calling this AI-prohibited, and I want to be honest that the phrase itself makes me uncomfortable. Prohibited sounds like a law. It sounds like a sign on a fence. And maybe that's exactly what it needs to be, but I'm still sitting with what it means to say that about knowledge our communities hold.

Here's what I mean. There are things I can picture right now that I would not want in any system, no matter how careful, no matter how well governed. A story that is only told when the season is right, and only by certain people. A teaching that belongs to a family, that was given to them, that is theirs to carry and theirs to decide about. A ceremony that is not for description, not for documentation, not for a model to learn from. These things are not secrets in the way a password is. They're sacred in a way that has nothing to do with hiding. They know that knowledge only lives properly inside a relationship, and a machine has no relationship. A machine can't be accountable to a family or to a place or to the people who will come after us. So there are things it should never hold.

But then I start second-guessing myself, because I know how these conversations go. The moment we say some knowledge is AI-prohibited, someone will ask: how do you enforce that? And that question, reasonable as it sounds, actually makes me a little angry, because it assumes the only real protection is a technical one. It assumes if we can't build a perfect fence, we shouldn't build any fence at all. I don't accept that. We don't stop teaching our children right and wrong because we can't guarantee they'll never do wrong. We teach them anyway, hold them accountable, and build the community around them that makes the teaching mean something. I think AI governance needs some of that same humility. The rule matters even if it can be broken. The rule says what we value. The rule tells the people building the systems where the line is, and it tells our own communities that we take their knowledge seriously enough to draw a line in the first place.

And yet. And yet. I have to be honest that enforcement is genuinely hard, and I don't want to pretend otherwise. Once something is in a model's training data, it's not really removable in any meaningful way. Once a system has seen a pattern, the pattern is in there, in ways even the builders can't fully trace. That's the part that scares me because it means the decision to prohibit must be made before anything enters the system, not after. There is no delete button for knowledge. There's no undo once a model learns a song it was never given. So the prohibition has to be built into the doorway, not the exit. That's why I keep saying classification has to come before innovation. The doorway is the only place we actually have power.

I've also been wrestling with who gets to decide what's prohibited. And this is where I feel the weight of it most. I don't think it's my place to decide what the boundaries of any community are. I have my own communities, and even within them, I would want to sit with elders, with knowledge holders, with families, with the people who carry these things, before I ever presumed to name what should be off limits. The model I keep coming back to is one where the standard gives communities the room to make that call themselves and backs it up with rules that public institutions must respect. Not me deciding for everyone. A framework that lets each community say: this is ours, and this is where the machine stops.

I know some people will hear this and think it's about keeping Indigenous communities out of the benefits of AI. It's the opposite. It's about making sure that when we do step into AI, we do so with our dignity intact, and that we're not trading our most precious things for access to tools. I refuse that trade. I've said it before in this series, and I'll keep saying it: we should not have to choose between being excluded and surrendering control.

So here's where I land, for now, and I want to be clear this is a landing, not a conclusion. I believe AI-prohibited has to exist as a real category with real consequences, and it has to be defined by the people who hold the knowledge, not by the systems that want to use it. I don't know yet how to make enforcement as strong as I want it to be. I don't know how to keep a determined actor from crossing the line. But I know the line has to be drawn, and I know drawing it is an act of respect, not an act of exclusion. In the next part, I want to talk about the questions we should ask before AI touches community knowledge at all, because I think the act of asking might be where the real protection lies.

— Olani Lilly, CoMission LLC · mission.consulting