Open vs Closed Source AI

Quick Assessment

One of the most heated debates in AI: Should powerful AI models be released as open source (weights publicly available), or kept closed to prevent misuse? The debate intensified following MetaOrganizationMeta AI (FAIR)Comprehensive organizational profile of Meta AI covering $66-72B infrastructure investment (2025), LLaMA model family (1B+ downloads), and transition from FAIR research lab to product-focused GenAI...Quality: 51/100's Llama releases, Mistral's emergence as a European open-weights champion, and DeepSeek's 2025 disruption demonstrating Chinese open models at the frontier.

Key Links

Key Arguments Summary

Trade-offs Overview

Stakeholder Positions

What's At Stake

Open weights (often called "open source" though technically distinct) means releasing model weights so anyone can download, modify, and run the model locally. Meta clarified in 2024 that Llama models are "open weight" rather than fully open source, as the training data and code remain proprietary. Examples include Llama 2/3, Mistral, Falcon, and DeepSeek-R1. As of April 2025, Llama models alone had been downloaded over 1.2 billion times, with 20,000+ derivative models published on Hugging Face. Once released, weights cannot be recalled or controlled, and anyone can fine-tune for any purpose—including removing safety features. Research shows that "jailbreak-tuning" can remove essentially all safety training within hours using modest compute (FAR.AI 2024). Within days of Meta releasing Llama 2, "uncensored" versions appeared on Hugging Face with safety guardrails stripped away.

Closed source means keeping weights proprietary, providing access only via API. Examples include GPT-4, Claude, and Gemini. Labs maintain control and can monitor usage patterns, update models, revoke access for policy violations, and refuse harmful requests. However, this concentrates power in a small number of corporations.

Current Landscape (2024-2025)

The landscape shifted dramatically with DeepSeek's January 2025 release of R1, demonstrating that Chinese labs could produce frontier-competitive open models. Before DeepSeek, Meta's Llama family dominated the open-weights ecosystem, with models ranging from 7B to 405B parameters.

Market Statistics (2024-2025)

Key Positions

Safety Guardrail Effectiveness

Research on open model safety reveals significant challenges in maintaining guardrails once weights are released.

Key Cruxes

Possible Middle Grounds

Several proposals aim to capture benefits of both approaches while mitigating risks:

The International Dimension

The geopolitical calculus shifted dramatically in 2025. DeepSeek's R1 release demonstrated that keeping Western models closed does not prevent capable open models from emerging globally. The market impact was immediate: NVIDIA reportedly lost $100 billion in market capitalization in a single day, and by month's end DeepSeek had overtaken ChatGPT as the most downloaded free app on the Apple App Store in the US.

DeepSeek's Impact: DeepSeek's January 2025 release sent "shockwaves globally" by demonstrating frontier capabilities in an open Chinese model at a fraction of Western costs—reportedly under $1 million in training costs compared to hundreds of millions for comparable Western models. The model runs 20-50x cheaper at inference than OpenAI's comparable offerings. However, NIST/CAISI evaluations found significant issues: DeepSeek models were 12x more susceptible to agent hijacking attacks than U.S. frontier models, and CrowdStrike research showed the model produces insecure code when prompted with politically sensitive terms (Tibet, Uyghurs). Several countries including Italy, Australia, and Taiwan have banned government use of DeepSeek.

If US/Western labs stay closed:

• May slow dangerous capabilities domestically
• But China has demonstrated strategic open-sourcing (DeepSeek)
• Could lose innovation race and talent to more open ecosystems
• Does not prevent proliferation given global competition

If US/Western labs open source:

• Loses monitoring capability over deployment
• But levels playing field globally and enables allies
• Benefits developing world and academic research
• May shape global norms through responsible release practices

Coordination problem:

• Optimal if all major powers coordinate on release thresholds
• Carnegie research notes emerging convergence on risk frameworks
• Unilateral Western restraint may simply cede ground to less safety-conscious actors
• DeepSeek's signing of AI Safety Commitments suggests potential for Chinese engagement

Implications for Different Risks

The open vs closed question has different implications for different risks:

Misuse risks (bioweapons, cyberattacks):

• Clear case for closed: irreversibility, removal of guardrails
• Open source dramatically increases risk once capabilities cross danger thresholds
• However, the March 2024 "ShadowRay" attack on Ray (an open-source AI framework used by Uber, Amazon, OpenAI) showed that open ecosystems create additional attack surfaces

Accident risks (unintended behavior):

• Mixed: Open source enables external safety research and red-teaming
• But also enables less careful deployment by actors who may not understand risks
• Depends on whether scrutiny benefits or proliferation risks dominate

Structural risks (power concentration):

• Clear case for open: prevents AI monopoly by a few corporations
• But only if open source is actually accessible (frontier models require substantial compute)
• LeCun's concern: "a very bad future in which all of our information diet is controlled by a small number of companies"

Race dynamics:

• Open source may accelerate race (lower barriers to entry)
• But also may reduce duplicated effort (can build on shared base)
• DeepSeek's cost-efficient training suggests open release may not slow capability development

Key Policy Developments

U.S. Policy: The NTIA's July 2024 report concluded that evidence is "insufficient to definitively determine either that restrictions on such open-weight models are warranted, or that restrictions will never be appropriate in the future." It recommended monitoring and research rather than immediate restrictions.

California SB-1047: In September 2024, Governor Newsom vetoed this bill which would have imposed liability requirements on AI developers. The veto cited concerns about stifling innovation without meaningfully improving safety.

EU AI Act: Takes a risk-based approach, entered into force August 2024 with GPAI model obligations applicable from August 2025. Open-source models receive exemptions from transparency obligations if they use permissive licenses and publicly share architecture information—but models with "systemic risk" (training compute exceeding 10²⁵ FLOPs) face full compliance requirements regardless of openness. France, Germany, and Italy initially opposed applying strict rules to open models, citing innovation concerns.

Emerging Consensus: Carnegie Endowment research in July 2024 found it is "no longer accurate to cast decisions about model and weight release as an ideological debate between rigid 'pro-open' and 'anti-open' camps." Instead, different camps have begun to converge on recognizing open release as a "positive and enduring feature of the AI ecosystem, even as it also brings potential risks."

Regulatory Comparison

References

• NTIA Report on Dual-Use Foundation Models with Widely Available Model Weights (July 2024)
• Carnegie Endowment: Beyond Open vs. Closed (July 2024)
• RAND: Securing AI Model Weights (2024)
• TIME: Yann LeCun Interview on Open AI (2024)
• NIST/CAISI: Evaluation of DeepSeek AI Models (September 2025)
• Carnegie: DeepSeek and Chinese AI Safety Commitments (January 2025)
• R Street Institute: Mapping the Open-Source AI Debate (2024)