The U.S. Government classification system, woefully outdated since the 1950s, is about to have its AI revolution. In this blog post, we lay out an achievable vision for modernizing the classification system to finally turn the tide in what the government itself has described a “tsunami of digitally created classified records” that is getting exponentially worse each day.
“As chairman of the Senate Intelligence Committee, I think it is clear that our security classification system is badly in need of change.“ — U.S. Senator Mark Warner
Conceived in the 1700s and formalized in 1951, classification of government documents was intended to protect information relevant to national security while contributing to an open accountable government. However, the notoriously manually intensive classification, declassification, and review functions coupled with under resourcing of departments and agencies — each with their own distinct processes — have led to an epidemic of overclassification. The status quo impedes information sharing, diminishes transparency and confidence in government, and costs taxpayers billions of dollars.
Congress, the Director of National Intelligence, the Public Interest Declassification Board (PIDB), the National Archives, and academics have all studied the problem and come to the same conclusion: the classification system requires a digital age upgrade. Despite years-long agreement the government had made no substantial progress against this challenge.
“The Government needs a paradigm shift, one centered on the adoption of technologies and policies to support an enterprise-level, system-of-systems approach.” — The PIDB
The time to update the classification system has arrived, and Palantir’s Artificial Intelligence Platform (AIP) is the technology that can help fix this decades-old problem.
Palantir AIP will help the U.S. Government revolutionize the classification system by delivering capabilities to:
- Operationalize traditional information management processes by translating its security controls (e.g., security classification guides, foreign disclosure rules, Freedom of Information Act rules) into an ontology — turning the rules into tools;
- Implement discrete controls on AI large language models (LLMs) to achieve its desired outcomes while mitigating the risks of AI hallucinations;
- Direct LLMs to quickly decipher and connect the context of documents, implementing superior search and discovery which links documents back to relevant rules — enabling more informed decisions;
- Simulate, inspect, and audit workflows in real-time to ensure the software is producing desired and consistent results and make improvements for accuracy and efficiency.
Human+Machine principals are foundational for AIP. As a result, AIP enables humans to make decisions with scale, speed, and accuracy that is unachievable with today’s processes. Most importantly, AIP will empower humans to improve the safeguarding of sensitive information while simultaneously increasing government transparency.
Reimagining Classification
Now that long-sought technology is available, the Intelligence Community (IC), the Department of Defense (DoD), and the Federal CIO can set and achieve an ambitious vision. For instance, by 2025, the government could have:
- Declassification software engines: Automatically process large backlogs of classified records. Identify, at speed and scale, the likely classification of sentences and paragraphs and map them to policy guides and previous declassification decisions — exponentially increasing the confidence and pace of declassification decisions.
- A FOIA Front Door: A single interface for Americans to request access to government records. The interconnected back-end identifies potentially responsive records and enables government agencies to coordinate the response — expediting notoriously slow, redundant, and manual process that exist today.
- Intelligence Diplomacy: Embassies, CIA stations, and Combatant Commands who need to share critical intelligence with foreign partners, convert NOFORN intelligence reports into draft tearline reports for human review with a single click of a button.
- Improving our Defenses: IC and DoD components test how adversaries like Russia and China are likely using AI to discover insights from previously declassified information. New declassification guidance can then address the weaknesses discovered.
- Improved Security Classification and Disclosure Guides: IC and DoD components will use the same AIP technology to identify, consolidate, and streamline overlapping and conflicting policy guides — a necessary enabler for the envisioned mission outcomes.
By 2026. the government should have:
- Software-powered bots, leveraging improved security classification guides to automatically recommend portion markings for emails and documents as officers draft them. These services will have massive downstream effects — making future discovery, review, declassification, and release processes more accurate and efficient.
Once achieved, the classification system will strengthen America’s national security. It will enable stronger public trust in government institutions; stronger accountability over government decisions; and hopefully decrease the more than $18.4B in taxpayer dollars spent annually on classification.
Palantir’s Artificial Intelligence Platform
LLMs are not, by themselves, a panacea for these issues and to effectively operationalize LLMs and achieve the Human+Machine potential, proper tooling and controls are necessary. Palantir AIP is the integrated platform to meet this challenge.
- Data & model integration: AIP includes Palantir Foundry as a data management platform, enabling organizations to ingest, clean, fuse, and operationalize data both across the enterprise and within enclaves. AIP’s model-agnostic and scalable infrastructure allows users to seamlessly bring multiple LLMs to bear on the same problem and combine the best results from each.
- Data operationalization: AIP enables organizations to turn classification guides and disclosure rules into digital enforceable tools. Combining LLM-based tools with AIP can augment human decision-making about classification at previously unimaginable scope and scale. Recommendations and decisions are auditable and rooted in explainable reasoning.
- Taking action: AIP offers a contained environment to integrate data, AI models, and build Human+AI tools using low-code or no-code interfaces. AIP can connect to existing systems to implement decisions seamlessly.
- Security-first: AIP leverages robust security and identity management controls as well as audited decisionmaking for data and insider threat protection. The platform also has built-in continuous integration and continuous delivery (CI/CD) capabilities.
- Simulation: AIP enables users to simulate proposed classification changes (e.g., in conjunction with other released info) before making a decision.
- Feedback loops: AIP enables human decisionmaking and scrutiny of outputs and recommendations. It allows users to collect expert feedback for both users and models to systematically incorporate that feedback to improve future performance.
AIP provides the security governance, flexibility, and Human+Machine teaming capabilities needed to transform the way the U.S. Government operationalizes the classification system. The platform provides next-generation tooling paired with industry leading guardrails and can scale as missions evolve, offering government agencies the ability to start with a low-risk, low-cost proof-of-concept that showcases how AIP can revolutionize their classification-related workflows.
Learn more about Palantir AIP: https://www.palantir.com/platforms/aip/
AI’s Role in Reimagining the Classification System was originally published in Palantir Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.