SquiggleAI

Quick Assessment

Project Details

Overview

SquiggleAI integrates large language modelsCapabilityLarge Language ModelsComprehensive analysis of LLM capabilities showing rapid progress from GPT-2 (1.5B parameters, 2019) to o3 (87.5% on ARC-AGI vs ~85% human baseline, 2024), with training costs growing 2.4x annually...Quality: 60/100 to assist with probabilistic model creation, addressing a critical barrier QURI identified through years of experience building forecasting tools: even highly skilled domain experts frequently struggle with basic programming requirements for probabilistic modeling.

A subject matter expert might have deep knowledge about AI timelines, pandemic risk, or cost-effectiveness of interventions, but lack the programming fluency to translate their mental model into executable SquiggleProjectSquiggleSquiggle is a domain-specific probabilistic programming language optimized for intuition-driven estimation rather than data-driven inference, developed by QURI and adopted primarily in the EA commu...Quality: 41/100 code. SquiggleAI bridges this gap by accepting natural language descriptions and generating complete, runnable probabilistic models.

The system uses prompt caching to embed approximately 20,000 tokens of information about the Squiggle language—syntax, distributions, best practices—ensuring the LLM has deep knowledge of the domain-specific language. Most workflows complete within 20 seconds to 3 minutes, producing models typically 100-500 lines long depending on the model used.

Design Philosophy

SquiggleAI is designed around the principle that domain expertise should not be bottlenecked by programming ability. The tool aims to:

1. Lower the barrier to quantitative reasoning for non-programmers
2. Accelerate model creation even for experienced Squiggle users
3. Teach by example through generated code that users can read and modify
4. Preserve human judgment by making models transparent and editable

This philosophy aligns with QURI's broader mission of making uncertainty quantification accessible to altruistic decision-makers.

Model Support

SquiggleAI supports multiple LLM backends:

The default is Claude Sonnet 4.5, which produces the most sophisticated and detailed models but at higher API cost. Users can switch models based on their needs.

Prompt Caching

SquiggleAI uses prompt caching to cache approximately 20,000 tokens of Squiggle documentation with each request. This means:

• The LLM has consistent, deep knowledge of Squiggle syntax
• API costs are reduced (cached tokens are cheaper)
• Response quality is more reliable
• Context is preserved across conversation turns

Capabilities

Model Generation

Describe a problem in natural language, receive executable Squiggle code:

Example prompt:

"Estimate the cost-effectiveness of hiring an AI safety researcher. Consider their salary, overhead costs, probability they produce useful research, and value of that research if successful."

Output: ~300 line Squiggle model with:

• Variable definitions for each uncertain input
• Distribution choices justified by context
• Sensitivity analysis visualization
• Clear comments explaining logic

Iterative Refinement

Conversation-based model improvement:

Follow-up prompt:

"Add more uncertainty to the timeline assumptions and break down the research impact by type (interpretability vs alignment)."

Output: Modified model with requested changes, preserving existing structure.

Fermi Estimation

Generate complete uncertainty models from vague questions:

Code Debugging

Identify and fix syntax and logic errors:

User: "My model gives negative probabilities for the success rate"

SquiggleAI: Identifies issue with distribution domain, suggests truncate(distribution, 0, 1)

Model Explanation

Explain what existing Squiggle code does:

Input: Inherited 300-line model from another researcher

Prompt: "Explain what this model is calculating and what the key assumptions are"

Output: Plain-language summary with identified cruxes

Integration with Squiggle Hub

SquiggleAI is directly accessible from within Squiggle Hub, allowing users to:

1. Generate models from the "New Model" interface
2. Refine existing models through chat interface
3. Immediately save generated models to their Hub account
4. Version control AI-assisted iterations
5. Share completed models publicly or keep private

Privacy Defaults

SquiggleAI outputs on Squiggle Hub are private by default. Users who want to share models or make them public can explicitly do so by creating new public models. This privacy-first approach encourages experimentation without concern about incomplete drafts being visible.

Technical Implementation

Architecture

User Prompt
    ↓
Prompt Engineering Layer
(Adds cached Squiggle docs, examples, constraints)
    ↓
LLM API (Claude Sonnet 4.5)
    ↓
Post-Processing
(Syntax validation, auto-fixes)
    ↓
Executable Squiggle Code

Prompt Engineering

The system prepends each user request with:

This large cached context ensures the LLM produces idiomatic Squiggle code rather than generic probabilistic programming.

Use Cases

Domain Expert Assistance

Rapid Prototyping

Experienced Squiggle users leverage SquiggleAI for:

• First draft generation: Get 80% of a model in 2 minutes, refine manually
• Boilerplate reduction: Generate repetitive parameter definitions
• Exploration: Try multiple model structures quickly
• Documentation: Generate comments and explanations for existing code

Teaching and Learning

SquiggleAI serves as a teaching tool:

• New users learn by seeing working examples
• Generated code demonstrates Squiggle idioms
• Explanations help users understand probabilistic reasoning
• Iteration teaches refinement process

Strengths and Limitations

Strengths

Limitations

Comparison with Other Approaches

Relationship to QURI Ecosystem

SquiggleAI fits into QURI's tool ecosystem:

Funding

As a QURI project, SquiggleAI development is funded through QURI's grants:

SquiggleAI API costs (Claude API usage) are likely covered by QURI's operational budget, though specific allocation is not public.

Future Directions

Potential enhancements based on the current trajectory:

External Links

• SquiggleAI Documentation
• Squiggle Hub
• EA Forum: Introducing SquiggleAI
• QURI Website
• Claude API