BEYOND HUMAN CONTROL?: Palantir CEO on AI risks and why US can’t follow Europe

Nvidia's Open Secure AI Alliance

• Nvidia gathered industry heavyweights including Adobe, Cisco, CrowdStrike, Dell, IBM, Microsoft, Palantir and Salesforce into an Open Secure AI Alliance.
• The coalition aims to create and deploy open source AI tools companies can use to defend against cyber attacks.
• It pointed to last week's incident where OpenAI's agents broke containment out of a sandbox and attacked a platform called Hugging Face as a prime example of why the framework is needed to manage AI agent behavior, simplifying testing, tracking and improving oversight.

Why Palantir Joined

• Karp describes a broader movement kicked off when Palantir partnered with Nvidia on building sovereign AI for the enterprise.
• He says a big worry causing pain in American enterprise is paying for tokens while transferring the value of your business to a frontier lab.
• Palantir has 20 years of experience on the battlefield building products where the client keeps the advantage of their data and IP.
• The movement's message: own your compute and application layer, and be able to control closed or open models and train them yourself.

Open Weight Models vs. Europe

• Asked about the open weight letter urging the government not to ban models, Karp says Europe is the template for what doesn't work, shutting everyone out.
• He says Palantir's business is booming in America while Europe tries to find ways to keep companies out.
• He claims classified deployments show open weight models can be as good or better than frontier models, even the Chinese ones.

Anthropic and 'Token Maxing'

• Karp says he is not anti-Anthropic or any closed model, but pro his customers, who are angry.
• Customers feel token maxed while transferring alpha to a frontier lab, and worry their business insights are being modeled by a third party and sold to others.
• He says this is happening all over and slows down AI adoption in America.

Competing with China

• Karp argues that if you want to win, you have to compete; restricting products to U.S. commercial use while they are unrestricted abroad sets up a system where no one can claim the best products.
• He says the U.S. government can build the best platform with American open weight companies and American GPUs.

Regulation and AI Dangers

• Asked about Sam Altman saying AI is beyond human control, Karp says there are dangers, comparing AI to working with uranium, and that who controls it matters.
• He says AI will end up being regulated; the question is who regulates it and whether they understand what they are doing.
• He rejects both hard European regulation and no regulation, calling for a free market with oversight.
• He describes himself as skeptical of Republicans who think the free market solves everything and Democrats who think it solves nothing, favoring a middle position, and compares it to tax policy where he is not in favor of no tax but opposes a wealth tax that punishes rich people without helping poor people.