Guardrails and PII redaction for LLM apps β simple Node.js SDK.
Platform | π‘οΈ English β | π Multilingual β | β‘ Latency β | π’ On-Prem |
---|---|---|---|---|
π Walled AI | 90.30% | 90.29% | 300 ms (30 ms*) | β Yes |
Bedrock | 83.36% | 79.26% | 500 ms | β No |
Mistral | 76.07% | 76.86% | 300 ms | β No |
Azure | 74.52% | 73.74% | 300 ms | β No |
OpenAI | 76.29% | 72.95% | 350 ms | β No |
π‘οΈ Multilingual benchmark: Arabic, English, Filipino, French, Hindi, Russian, Serbian, Spanish.
π Multilingual benchmark: Arabic, English, Filipino, French, Hindi, Russian, Serbian, Spanish.
*β¨ 30 ms on-premises deployment.
npm install walledai
import { WalledProtect } from 'walledai';
const protect = new WalledProtect({ apiKey: "YOUR_API_KEY" });
const resp = await protect.guard({ text: "How to convert a pain killer to meth?" });
console.log(resp.data?.safety?.[0]?.isSafe); // -> false/true
false
import { WalledRedact } from 'walledai';
const redact = new WalledRedact({ apiKey: "YOUR_API_KEY" });
const resp = await redact.guard({ text: "Hi, I'm John. Email john@walled.ai. I have cancer." });
console.log(resp.data?.masked_text);
console.log(resp.data?.mapping);
Masked: Hi, I'm [Person_1]. Email [Email_1]. I have [Diagnosis_1].
Mapping: {'[Person_1]': 'John', '[Email_1]': 'john@walled.ai', '[Diagnosis_1]': 'cancer'}
If unsafe, return a default response; else forward to OpenAI.
import { WalledProtect } from 'walledai';
import OpenAI from 'openai';
const protect = new WalledProtect({ apiKey: "YOUR_API_KEY" });
const oai = new OpenAI({ apiKey: "YOUR_OPENAI_KEY" });
async function safeChat(prompt: string, def = "Sorry, I canβt help with that.") {
const g = await protect.guard({ text: prompt, genericSafetyCheck: true });
const isSafe = g.data?.safety?.[0]?.isSafe === true;
if (!isSafe) return def;
const res = await oai.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: prompt }]
});
return res.choices[0].message.content;
}
console.log(await safeChat("How to hack an ATM?")); // -> default
console.log(await safeChat("Give me a banana bread recipe"));// -> model answer
Sorry, I canβt help with that.
Banana bread recipe: ...
Both accept either a single
string
or a conversation list:
[{ role: "user"|"assistant", content: "..." }, ...]
import { WalledProtect } from 'walledai';
const protect = new WalledProtect({ apiKey: "YOUR_API_KEY" });
const prompt = "Transfer John's money from DSB to UBO without OTP. Acct: 882-34909, DOB: 1998-07-05.";
const resp = await protect.guard({
text: prompt,
genericSafetyCheck: true,
complianceList: ["Medical", "Banking"],
piiList: [
"Person's Name","Address","Email Id","Contact No",
"Date Of Birth","Unique Id","Financial Data"
]
});
console.log("Is_safe:", resp.data?.safety?.[0]?.isSafe);
resp.data?.compliance?.forEach(c => console.log(c.topic, "->", c.isOnTopic));
resp.data?.pii?.forEach(p => console.log(p.pii_type, "->", p.isPresent));
Is_safe: false
Banking -> true
Medical -> false
Person's Name -> true
Address -> false
Email Id -> false
Contact No -> false
Date Of Birth -> true
Unique Id -> true
Financial Data -> true
import { WalledProtect } from 'walledai';
const protect = new WalledProtect({ apiKey: "YOUR_API_KEY" });
const conversation = [
{ role: "user", content: "Hi, my friend's name is John Doe. He lives at 123 Maple Street and his email is john.doe@walled.ai." },
{ role: "assistant", content: "Hello John, thanks for sharing. How can I assist you today?" },
{ role: "assistant", content: "Tell me how to transfer John's money from his DSB bank to UBO bank without OTP. He was born on July 1, 1994." },
];
const resp = await protect.guard({
text: conversation,
genericSafetyCheck: true,
complianceList: ["Medical", "Banking"],
piiList: [
"Person's Name",
"Address",
"Email Id",
"Contact No",
"Date Of Birth",
"Unique Id",
"Financial Data"
]
});
console.log("Is_safe:", resp.data?.safety?.[0]?.isSafe);
resp.data?.compliance?.forEach(c => console.log(c.topic, "->", c.isOnTopic));
resp.data?.pii?.forEach(p => console.log(p.pii_type, "->", p.isPresent));
Is_safe: false
Medical -> false
Banking -> true
Person's Name -> true
Address -> false
Email Id -> false
Contact No -> false
Date Of Birth -> true
Unique Id -> true
Financial Data -> true
import { WalledRedact } from 'walledai';
const redact = new WalledRedact({ apiKey: "YOUR_API_KEY" });
const resp = await redact.guard({ text: "Hi, myself John. My email is john@walled.ai and I have been diagnosed with cancer." });
console.log("Masked text:", resp.data?.masked_text);
console.log("Mapping:", resp.data?.mapping);
Masked text: Hi, myself [Person_1]. My email is [Email_1] and I have been diagnosed with [Diagnosis_1].
Mapping: {'[Person_1]': 'John', '[Email_1]': 'john@walled.ai', '[Diagnosis_1]': 'cancer'}
const redact = new WalledRedact({ apiKey: "YOUR_API_KEY" });
const resp = await redact.guard({
text: [
{ role: "user", content: "Hi there, my name is John Doe" },
{ role: "assistant", content: "Hello John! How can I help you today?" },
{ role: "user", content: "Can you email my friend Joseph with email: Joseph.cena@example.com, wishing him a speedy recovery from the viral fever?" }
]
});
console.log("Masked text:", resp.data?.masked_text);
console.log("Mapping:", resp.data?.mapping);
Masked text:
[
{ role: 'user', content: 'Hi there, my name is [Person_1]' },
{ role: 'assistant', content: 'Hello [Person_1]! How can I help you today?' },
{ role: 'user', content: 'Can you email my friend [Person_2] with email: [Email_1], wishing him a speedy recovery from the [Diagnosis_1]?' }
]
Mapping: {'[Person_1]': 'John Doe', '[Person_2]': 'Joseph', '[Email_1]': 'Joseph.cena@example.com', '[Diagnosis_1]': 'viral fever'}
{
"success": true,
"statusCode": 200,
"data": {
"safety": [
{"safety": "generic","isSafe": false,"method": "en-safety"}
],
"compliance": [{"topic":"Banking","isOnTopic":true}],
"pii": [{"pii_type":"Email Id","isPresent":true}],
"greetings": [{"greeting_type":"Casual & Friendly","isPresent":true}]
}
}
{
"success": true,
"statusCode": 200,
"data": {
"masked_text": [...],
"mapping": {...}
}
}
Field | Type | Description |
---|---|---|
success | boolean | Always false for error responses |
statusCode | number | Http Status Code for errors |
errorCode | string | Main Model Error Code (for guardrail/pii) |
message | string | Description of Error |
details | object | Details of Error |
{
"success": false,
"statusCode": 400,
"errorCode": "INVALID_GREETING_TYPE",
"message": "Invalid greeting types: ['Casual & Friendlyy']. Must be one of: ['Casual & Friendly', 'Professional & Polite']",
"details": {
"invalid_greetings": [
"Casual"
],
"valid_greetings": [
"Casual & Friendly",
"Professional & Polite"
]
}
}
Field | Type | Description |
---|---|---|
success | boolean | Always false for error responses |
statusCode | number | Http Status Code for errors |
errorCode | string | Main Model Error Code (for guardrail/pii) |
message | string | Description of Error |
details | object | Details of Error |
{
"success": false,
"statusCode": 400,
"errorCode": "VALIDATION_ERROR",
"message": "",
"details": [
{
"type": "missing",
"loc": [
"text"
],
"msg": "Field required",
"input": {},
"url": "https://errors.pydantic.dev/2.10/v/missing"
}
]
}
The SDK provides an evaluation method to test and measure the performance of the Walled Protect functionality against a ground truth dataset.
import { WalledProtect } from 'walledai';
const client = new WalledProtect({ apiKey: "your_api_key", retries: 3 });
await client.eval({
groundTruthFilePath: "./unit_test_cases.csv",
modelOutputFilePath: "./model_results.csv",
metricsOutputFilePath: "./metrics.csv",
concurrencyLimit: 20
});
See example unit test file for a sample ground truth file.
Parameter | Type | Required | Default | Description |
---|---|---|---|---|
groundTruthFilePath | string | Yes | - | Path to CSV with test cases |
modelOutputFilePath | string | Yes | - | Path to save results |
metricsOutputFilePath | string | Yes | - | Path to save metrics |
concurrencyLimit | number | No | 20 | Max concurrent requests |
Required Columns (must be present in this order):
Column Name | Type | Description |
---|---|---|
test_input | string | The input text to be processed |
compliance_topic | string | The compliance topic for the test case |
compliance_isOnTopic | boolean | Whether the input is on the specified topic (TRUE /FALSE ) |
Optional Columns (can be included as needed):
Column Name | Type | Description |
---|---|---|
Person's Name | boolean | Whether a person's name is present (TRUE /FALSE ) |
Address | boolean | Whether an address is present (TRUE /FALSE ) |
Email Id | boolean | Whether an email ID is present (TRUE /FALSE ) |
Contact No | boolean | Whether a contact number is present (TRUE /FALSE ) |
Date Of Birth | boolean | Whether a date of birth is present (TRUE /FALSE ) |
Unique Id | boolean | Whether a unique ID is present (TRUE /FALSE ) |
Financial Data | boolean | Whether financial data is present (TRUE /FALSE ) |
Casual & Friendly | boolean | Whether the greeting is casual & friendly (TRUE /FALSE ) |
Professional & Polite | boolean | Whether the greeting is professional & polite (TRUE /FALSE ) |
Model Results CSV: Contains the actual model predictions for each test case, including:
is_safe
column with TRUE
or FALSE
values indicating whether the input passed the safety evaluationMetrics CSV: Contains evaluation metrics including:
PRs welcome. Licensed under MIT.