Back
Goodfire customer story: Rakuten
webCredibility Rating
3/5
Good(3)Good quality. Reputable source with community review or editorial standards, but less rigorous than peer-reviewed venues.
Rating inherited from publication venue: Goodfire
A marketing/case-study page from Goodfire AI highlighting their interpretability tooling deployed at Rakuten; useful as evidence of commercial interpretability adoption but limited in technical depth.
Metadata
Importance: 25/100press releasenews
Summary
A customer story describing how Rakuten partnered with Goodfire to apply mechanistic interpretability tools to their AI systems in a commercial setting. It showcases a real-world use case of interpretability technology helping a major e-commerce and technology company understand and improve AI model behavior.
Key Points
- •Demonstrates practical, commercial application of mechanistic interpretability tools beyond academic research settings.
- •Rakuten used Goodfire's interpretability platform to gain insight into AI model internals for business-critical applications.
- •Illustrates how interpretability can address deployment concerns such as reliability, trust, and model debugging.
- •Serves as an example of the growing market demand for AI transparency and explainability tools.
- •Shows that interpretability research is transitioning into industry products with real enterprise customers.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| Goodfire | Organization | 68.0 |
Cached Content Preview
HTTP 200Fetched Mar 20, 20264 KB
Customer Story
# How Rakuten secures reliable AI experiences for 44M+ monthly users
How Rakuten and Goodfire used Ember to develop lightweight, robust PII guardails
Goodfire Services Provided
Ember Platform
Outcomes
Minimizes sensitive user data leakage
Real-time, cost-efficient detection
Reliable across languages, formats, and real-world edge cases
ETC
###### Context
## Founded in Tokyo in 1997, Rakuten is a global technology leader in services that empower individuals, communities, businesses and society, serving over 44 million monthly active users in Japan and a total of two billion customers worldwide.
In 2024, Rakuten and Goodfire partnered to explore ways to make Rakuten AI even more reliable and trustworthy, leading the industry to use frontier interpretability to improve security and prevent customers’ PII (personally identifiable information) from being sent downstream to model providers.
A team of technical staff at Goodfire and Rakuten worked together using Goodfire’s interpretability platform, Ember, to develop and test different methods for PII detection, resulting in a performant, lightweight, and robust guardrail that was deployed to Rakuten’s agent platform.
###### Outcomes
## Rakuten deployed PII guardrails with the confidence that they could mitigate PII leakage at scale with minimal latency and high performance, ensuring the security of customer information across 44 million active users. Our systems exhibited:
1. Best-in-class recall,
minimizing PII leakage
2. Much better cost efficiency and latency
than black-box methods
3. Strong robustness in production,
performing well in out-of-distribution settings
###### Our Approach
## Our research goal was to develop a system to detect and filter out personally identifiable information (names, addresses, phone numbers, and emails) before they enter downstream processing across Rakuten’s ecosystem, using state-of-the-art interpretability research. The system needed to be:
- **High-recall**, so no sensitive data slips through
- **Lightweight** enough to run efficiently at scale
- **Trained only on synthetic data**, since customer data can’t be used
At a high level, our approach was to use our interpretability platform Ember to test the performance of a portfolio of different methods for Rakuten’s use case. Intuitively, we used the “cognition” of the model itself as the source of information about whether each token is PII, rather than the raw inputs or outputs.
###### The Results
## The research partnership tested which methods best met Rakuten’s needs and data constraints, finding that SAE (sparse autoencoder) probes won out when moving from synthetic training data to real production data—a critical requirement for many real-world monitoring appro
... (truncated, 4 KB total)Resource ID:
f9f02c21aeb65119 | Stable ID: OTliOTE1Nz