Kickoff recap · Sovereign AI · 1:20 p.m.
Sovereign AI, she said, is who controls the environment where the model operates, and who answers for the outcome.
Every day brings a new story of AI writing code and taking on human tasks. The other side is risk. Models hallucinate. Responses come back biased. Customer data is exposed. That is where governance starts.
Governance enforces the security controls that keep a model operating responsibly. Sovereign AI asks who controls the environment: where the data resides, where the infrastructure is, where customer data sits, who can see it, and who is responsible for the outcome.
Her example was a bank's mortgage agent that keeps denying applications from people with a strong credit history. If the training data is biased, including a preference for some locations, the responses will be biased too. That is the gap governance is meant to close.
The controls make the output transparent. When something looks wrong, the system looks at the evidence and escalates to a person. If the person finds a problem, they ask for a retrain, and the system stays safe.
People think governance will slow innovation. She said it is how you scale with confidence and trust. She remembered the early fear of putting a credit card online. Anti-fraud controls made the payments ordinary. The same shift follows once the controls are in place.
Full Transcript
Cleaned up transcripts.
Keynote
Hello. Good afternoon everyone. I'm Sunita. Every day we are hearing new stories about AI. AI is writing code. It is doing human tasks. But there is another side of the story, too. It is adding risk. Models are hallucinating. It is giving biased responses and customer data is exposed. And that is where AI governance becomes very important, and that is the journey we are going to explore today.
AI governance enforces security controls that will make sure that AI models are operating responsibly. Now, who controls the environment where the AI operates? That is where sovereign AI becomes important. It will enforce meaningful controls around data, infrastructure, and environment, making sure where the data resides, where the infrastructure is, where the customer data is, who has access to the data, and who is responsible for the outcome.
Let me make it real with an example. Let's say a bank has set up a mortgage AI agent and it is continuously denying a few mortgage applications even though the customer has a very good credit history. Now, models are trained on biased data, and there could be a possibility that the data is biased on location preferences. If the data is biased, model responses will also be biased, and that is the gap AI governance is trying to solve. It will enforce security controls, making sure that the output and the responses are transparent. Whenever there is any escalation or something is wrong, it will look at the evidence, and if something looks suspicious, it will escalate for human review. If humans find something wrong, they will request to retrain the model and make sure that this entire system operates in a safe and responsible manner.
Now, people think that AI governance will slow down AI innovation, but it will help to scale with confidence and trust. Remember when credit card payments were launched, people were scared to put credit card details on online platforms? But slowly we built anti-fraud mechanisms, and everyone is comfortable using online payment systems. The same kind of shift is going to happen when we have the right governance and controls in place. We can scale with confidence. Thank you so much.
Panel
Francis Wang asked the questions.
What is the first distinctive thing we can do by borrowing from patterns abroad? What are the patterns we can accommodate in Canada?
Sunita Kumari.
Sovereign AI is basically enforcing meaningful control, ensuring that data resides in the same country, and that we have the right control. We know who accesses the data, who governs it, where the server resides, and who is responsible for the outcome, or who is accountable if something is wrong. If we have all these questions answered, I think we are in the right place to implement sovereign AI.
What evidence should decision-makers demand before calling an AI deployment here safe, secure, trusted, or publicly beneficial?
Sunita Kumari.
Governance is a continuous process. If we want to build trust, we need to make sure that governance is running along with every process. Every response getting generated has to be governed, and that is the only way we can build trust and confidence.
At the kickoff
-
On enterprise risk and who answers for a model's outcome.