QualiSift: an AI agent that scores every new lead and explains why
Each lead is enriched, scored on five weighted criteria with cited evidence, checked for made-up evidence, and sorted into one Hot, Warm and Cold digest for the sales team.
At a glance
Reps spend their best hours sorting leads that were never going to buy.
Inbound leads arrive from forms, demo requests and list imports. Before anyone calls, someone has to look up the company, judge whether it fits, and decide who goes first.
A single score from a model is hard to trust: it can change between runs and it can quote facts that are not in the record. QualiSift keeps the model to one job, scoring with cited evidence, and does everything else in plain code.
Lead arrives
POST /api/v1/leads or a webhook creates the lead and starts scoring as a background task.
How it works
Each stage as it exists in the repository. Select a stage to see what it uses and what happens when it fails.
Lead intake
Creates the lead and queues scoring without blocking the request
FastAPI, BackgroundTasks
Returns a validation error; nothing is queued
{"email":"ops@harborfinch.co.uk","company":"Harbor & Finch"}Execution sequence
One item, traced through the system boundaries.
- FastAPI to Database: INSERT lead, 201 Created
- FastAPI to Enrichment: enrich_by_email(domain)
- Enrichment replies to FastAPI: industry, size, tech stack, funding
- FastAPI to LangGraph: ainvoke(lead, five criteria)
- LangGraph to LangGraph: redact → score → validate → composite
- LangGraph replies to FastAPI: scores, evidence, composite
- FastAPI to Database: store per-criterion scores
One item, start to finish
A demo request arrives with an email, company and job title. The API stores it and returns immediately.
The email domain is looked up; industry, headcount, tech stack and latest funding round are attached.
Each criterion gets a score and the facts it relied on, for example the job title for Authority.
The weighted score places the lead in Hot, Warm or Cold, with its reasons, in the next digest.
The composite score, exactly as the code computes it
Move the five criterion scores. The weights are the defaults the seed script installs; sales ops can change them in the dashboard as long as they sum to 1.0.
The model (or the offline mock) scores each criterion from 1 to 10 and must cite evidence from the lead record. Code, not the model, then turns those five scores into one number and a bucket.
Logic copied from QualiSift apps/api/app/agents/graph.py (calculate_composite) and services/digest.py. Inputs are yours to change; nothing here calls a model or a server.
Architecture
The layers and the decision path, taken from the repository code.
| Layer | Component | Tech Stack | Responsibility |
|---|---|---|---|
| Layer 1: API | FastAPI service | Python 3.12, FastAPI, Pydantic | Lead intake, webhooks, criteria, digest endpoints |
| Layer 2: Agent | LangGraph scoring graphKEY | LangGraph StateGraph, five nodes | Redaction, scoring, evidence check, composite |
| Layer 3: Adapters | Swappable providers | OpenAI or mock, Clearbit or mock, console / SMTP / SendGrid, Slack | Every outside service is a config value |
| Layer 4: Data and UI | Store and dashboard | PostgreSQL (or SQLite), Next.js 14 | Scores, evidence, digest history, weight sliders |
- prepare_context goes to redact_pii: active criteria
- redact_pii goes to score_lead: lead without PII
- score_lead goes to validate_output: scores + evidence
- validate_output goes to calculate_composite: checked
- validate_output falls back to flag for review: evidence not found
Decisions and trade-offs
| Decision | Instead of | Why | Where in the code |
|---|---|---|---|
| Composite score computed in code | Asking the model for one overall score | The model only scores each criterion; weights and thresholds are fixed numbers anyone can read and change. | apps/api/app/agents/graph.py (calculate_composite) |
| Every cited fact checked against the record | Trusting the model’s reasoning text | A citation that does not fuzzy-match a field in the lead record is caught before the score is used. | apps/api/app/agents/validator.py |
| Personal data removed before scoring | Sending the full lead to the model | Email, phone and personal notes are not needed to judge fit, so they never leave the service. | apps/api/app/agents/graph.py (redact_pii) |
| Offline mock for every provider | Requiring API keys to run anything | The whole pipeline runs on SQLite with deterministic mock scoring and enrichment, so it can be tried without paid accounts. | apps/api/app/services/enrichment.py, graph.py (_mock_score) |
Known limits
The digest is generated on request
The API compiles a digest when POST /digest/generate is called. There is no built-in scheduler yet; a daily send needs a cron job or workflow tool to call it.
Mock scores are heuristics
Without an OpenAI key the scores come from simple rules on the lead fields (for example, job title for Authority). They show the pipeline working, not real qualification quality.
One enrichment provider
Clearbit is the only real enrichment adapter. Other providers need a new adapter class.
Results
Only numbers produced by a committed, repeatable script appear here as measured. Until then, each result says how it will be measured.
Median and p95 over a fixed set of sample leads with mock providers, script and raw output committed to the repository.
Share of deliberately wrong citations the validator rejects, on a committed test set.
Only measurable with a client: time to first call for Hot leads, before and after.
Run it yourself
Native setup from the README: SQLite, mock providers, no API keys. Full instructions ↗
git clone https://github.com/arslanvuzmal/QualiSift.git
cd QualiSift/apps/api
python -m venv .venv && . .venv/bin/activate
pip install -e ".[dev]"
printf "DATABASE_URL=sqlite+aiosqlite:///./dev.db\nLLM_PROVIDER=mock\nENRICHMENT_PROVIDER=mock\n" > .env
python -m scripts.seed
uvicorn app.main:app --port 8000
# second terminal: cd apps/web && npm install && npm run devQuestions people ask first
Does it work with my CRM?
Leads can come in from HubSpot and Marketo webhooks or any system that can send JSON. Writing scores back into a CRM is not built yet; it would be a new adapter.
Who decides what a good lead is?
You do. The five criteria and their weights are editable in the dashboard, and the thresholds for Hot, Warm and Cold are plain numbers in the code.
Can the model invent facts about a lead?
It can try. Every piece of evidence it cites is matched against the lead record, and results whose evidence is not found are marked invalid for review.
Does the model see personal data?
No. Email, phone and personal notes are removed from the lead before the scoring step.
Related reading & systems
Want QualiSift working in your business?
Book a 20-minute scoping call. I will review your current systems, map where manual friction is costing you time, and give you an honest quote.