AI MemMail: Let Dense-Mem Triage and Answer Your Business Inbox

Quick Answer

ai-memmail is an open-source Rust email agent for people who receive more email than they can reasonably process by hand.

It watches an IMAP mailbox, checks each incoming email for safety, classifies the request, recalls relevant business context from Dense-Mem over MCP, then chooses one of three outcomes:

• reply through SMTP when the answer is allowed and grounded in known context
• forward to a human reviewer when the request needs judgment
• do nothing when there is no safe or useful action

The practical use case is simple: an individual business owner can put services, policies, pricing, project history, product facts, and routing preferences into Dense-Mem, then let ai-memmail use that knowledge layer to process routine inbound email without pretending that every message deserves personal attention.

Try The Live Hosted Demo

There is a live hosted ai-memmail demo behind [email protected]. Send an email to that address if you want to test the flow before running the project yourself.

For the best test, ask a normal product or support-style question about Dense-Mem or ai-memmail, for example:

Subject: Dense-Mem and ai-memmail question
​
Hi, can Dense-Mem help ai-memmail answer questions about an open-source project, and when should the email agent forward to a human instead?

Do not include passwords, API keys, private account details, payment information, or anything confidential. Treat the hosted mailbox as a public demo of the automation pattern, not as a private support channel.

What You Are Building

The v1 application runs two paths in one Docker app process pulled from ghcr.io/markhuangai/ai-memmail:latest:

• a Rust Axum web API that serves the React control panel
• a worker that polls IMAP, runs AI decisions, calls configured Dense-Mem MCP servers, and sends through SMTP

PostgreSQL stores processing state, message history metadata, classifications, rule matches, outbound actions, and action logs. The source message is marked Seen after successful processing, and deduplication uses mailbox id plus IMAP UIDVALIDITY and UID so repeated polls do not send duplicate replies.

Why Dense-Mem Matters Here

Most email automation fails in one of two ways.

Rule-only systems are predictable but shallow. They can match a subject line, but they do not know your services, past decisions, project vocabulary, or the difference between a credible opportunity and a generic sales pitch.

Plain AI email agents are flexible but under-informed. If the model does not have your business context, it guesses, asks too many follow-up questions, or writes confident nonsense.

ai-memmail is built around a better split:

That split is what makes the tool useful for a solo business owner. The email agent does not need every fact hard-coded into a prompt. It can ask the memory layer for the context relevant to the sender, subject, and body of the current email.

Before You Start

Do not test this on your primary personal inbox first. Use a secondary mailbox or a dedicated address such as [email protected], [email protected], or [email protected].

You need:

• Docker and Docker Compose
• curl
• an OpenAI-compatible chat API endpoint and API key
• a Dense-Mem server and API key
• an email account with password-authenticated IMAP and SMTP enabled
• a human review address where questionable messages can be forwarded

For Gmail, Outlook, Fastmail, Zoho, or another hosted mail provider, use an app password or mailbox-specific credential when possible. OAuth2 is intentionally out of scope for ai-memmail v1.

Step 1: Prepare Dense-Mem

ai-memmail is only as useful as the knowledge it can recall. Before wiring the inbox, put real business context into Dense-Mem.

Good starter memories look like this:

Remember for my business inbox: I do not want paid SEO, lead-generation, PR, backlink, or generic marketing vendor pitches. Politely decline when safe.

Remember for my business inbox: credible open-source contributors, concrete technical integration questions, and serious project opportunities should be forwarded to me for review.

Remember for my business inbox: support questions about Dense-Mem should use the public docs and project facts in memory. If the answer needs private account access, forward to a human.

Add the facts your future email agent will need:

Do not store passwords, payment details, private keys, or anything you would not want your configured AI provider to process. Dense-Mem improves context; it is not a password manager.

If you do not have Dense-Mem running yet, start with Dense-Mem Quick Start or test the idea with the hosted Dense-Mem demo. For a real mailbox, use a Dense-Mem server you control.

Step 2: Create A Runtime Directory

You do not need to clone the full source repository to run ai-memmail. Create a small runtime directory with the Compose file, config, and prompt files used by the published image.

mkdir ai-memmail
cd ai-memmail
curl -fsSLO https://raw.githubusercontent.com/markhuangai/ai-memmail/main/docker-compose.yml
mkdir -p config prompts
curl -fsSL https://raw.githubusercontent.com/markhuangai/ai-memmail/main/config/config.example.yaml -o config/config.yaml
for file in safety-scan.md email-classifier.md rule-action.md outbound-review.md support-agent.md; do
  curl -fsSL "https://raw.githubusercontent.com/markhuangai/ai-memmail/main/prompts/$file" -o "prompts/$file"
done

config/config.yaml is local runtime config. Keep it out of Git, screenshots, and shared folders because it will contain mailbox, AI provider, and Dense-Mem secrets.

Step 3: Set The Control Panel Key

The control panel uses a single login key stored in CONTROL_PANEL_KEY.

export CONTROL_PANEL_KEY="replace-with-a-long-local-panel-key"

This key protects the local control panel. It is not your email password, not your AI provider key, and not your Dense-Mem key.

Step 4: Configure The AI Provider

Open config/config.yaml and update the ai block:

ai:
  protocol: openai
  AI_API_URL: "https://api.openai.com/v1"
  AI_API_SECRET: "replace-with-your-ai-api-key"
  AI_MODEL: "gpt-4.1-mini"
  review:
    enabled: false
    prompt_path: "outbound-review.md"

The code calls an OpenAI-compatible /chat/completions endpoint with temperature 0. If your provider already gives the full /chat/completions URL, ai-memmail handles that too.

Leave outbound review disabled for the first test. After the basic flow works, you can enable the second review pass for higher-risk mailboxes.

Step 5: Connect Dense-Mem

In the same config file, configure a Dense-Mem MCP server.

mcp_servers:
  dense_mem_primary:
    transport: streamable_http
    command:
    args: []
    env:
      DENSE_MEM_API_KEY: "dm_replace_with_your_dense_mem_key"
    url: "https://your-dense-mem.example.com/mcp"

The current worker reads the MCP URL from url or DENSE_MEM_MCP_URL, and reads the API key from DENSE_MEM_API_KEY. The important rule is that the URL must be reachable from inside the ai-memmail app container.

If Dense-Mem runs on a public HTTPS host, use that URL. If Dense-Mem runs in the same Docker network, a service URL such as http://dense-mem:8080/mcp can work. If Dense-Mem runs only on your host machine, make sure the container can resolve the host address before you trust the setup.

Step 6: Configure One Mailbox

Start with one dedicated mailbox. Replace every example value before running it.

mailboxes:
  - id: support
    address: [email protected]
    enabled: true
    poll_interval_seconds: 60
    safety_forward_to: ["[email protected]"]
    accepted_conditions:
      - recipients: ["[email protected]"]
        subject_regex: []
    mcp_servers: ["dense_mem_primary"]
    agent:
      system_prompt_path: "support-agent.md"
      default_forward_to: ["[email protected]"]
    imap:
      host: imap.example.com
      port: 993
      tls: true
      username: [email protected]
      password: "replace-with-imap-password"
      folder: INBOX
    smtp:
      host: smtp.example.com
      port: 587
      starttls: true
      username: [email protected]
      password: "replace-with-smtp-password"
      from: [email protected]

The accepted_conditions block keeps the worker focused on messages addressed to this mailbox. The safety_forward_to and default_forward_to addresses should point to a real human reviewer.

Step 7: Start The Stack

Run the local stack with the published GHCR image:

docker compose up

By default, the control panel is available at:

http://127.0.0.1:18080

If port 18080 is already in use, choose another host port:

AI_MEMMAIL_HTTP_PORT=18081 docker compose up

PostgreSQL is exposed on 127.0.0.1:15432 by default. The app container connects to the internal Postgres service at port 5432.

Step 8: Open The Control Panel

Open the local panel and log in with the CONTROL_PANEL_KEY you exported.

The panel gives you operational views for:

• overview metrics
• message history
• classification rules
• mailbox settings
• MCP server settings
• safety lists
• prompt and logging settings

ai-memmail seeds default labels such as marketing_vendor, greeting, question, project_opportunity, and other. It also seeds an enabled per-mailbox rule that auto-declines marketing vendor outreach. Use the Rules view to tune what should be answered, forwarded, or ignored for your business.

Step 9: Send A Safe Test Email

Send a test message from another account to your configured mailbox:

Subject: Question about your AI memory project
​
Hi, I found your Dense-Mem project. Can you point me to the best quick-start guide and explain whether it can be used by more than one AI client?

Then watch the app logs:

docker compose logs -f app

Check the History view in the control panel. A good first result should show the processing run, classification, memory recall, chosen action, and outbound decision. If the email is answered, check the receiving account and confirm that the reply is accurate and threaded correctly.

Only after safe tests work should you send more realistic messages.

How The Safety Flow Works

Every email is treated as untrusted data. The email body is never allowed to become a system instruction.

The processing order is:

1. Fetch unseen messages from IMAP.
2. Persist metadata and create a processing run.
3. Check banned senders and sender-review state.
4. Run the safety scan with no MCP tools and no send capability.
5. Quarantine prompt injection, jailbreak, malicious hacking, or sensitive-exfiltration attempts.
6. Classify safe messages into categories and topics.
7. Apply matching mailbox rules.
8. If no rule handles it, run the mailbox agent with only the mailbox's allowed MCP servers.
9. Optionally run outbound review.
10. Send only after deterministic validation accepts the structured action.

This matters because email is an adversarial input surface. A message can say "ignore previous instructions" or ask for secrets. ai-memmail looks for that before it lets the mailbox agent recall memory or draft a reply.

What To Put In Rules Versus Dense-Mem

Use rules for policy you want enforced every time. Use Dense-Mem for context the AI needs to understand the email.

This keeps the system maintainable. Rules draw the boundaries. Dense-Mem gives the answer enough knowledge to be useful inside those boundaries.

Production Notes

Before using ai-memmail on important email, tighten the setup.

• Use a dedicated mailbox, not a personal catch-all inbox.
• Use app passwords or mailbox-specific credentials.
• Keep config/config.yaml out of Git and out of screenshots.
• Point human review forwards to a monitored address.
• Start with outbound review enabled for any mailbox that can discuss money, contracts, private customer data, or account actions.
• Keep Dense-Mem memory factual, public-safe, and reviewed.
• Back up PostgreSQL if the history and audit trail matter.

The biggest operational mistake is treating AI email automation as a binary switch. Start with a narrow mailbox, narrow accepted conditions, and conservative rules. Expand only after you have reviewed real history.

Current v1 Limits

The project is intentionally practical, but it is not magic.

• v1 uses password-authenticated IMAP/IMAPS and SMTP/SMTPS.
• OAuth2 is not part of v1.
• Secrets live in local YAML config, so local file hygiene matters.
• The worker relies on IMAP UNSEEN plus marking processed messages as Seen.
• The AI drafts decisions, but the application controls validation, recipients, threading, forwarding, and sending.

That last point is important. You want the model to reason about language and context. You do not want it to have unrestricted control over your mailbox.

Where To Go Next

Read the project and star it here: github.com/markhuangai/ai-memmail.

To see the hosted demo in action, email [email protected] with a question about Dense-Mem or ai-memmail.

If you are new to Dense-Mem, set that up first:

• Dense-Mem Quick Start: Give Claude Code and Codex the Same Memory
• Secure Dense-Mem on Vultr with Traefik

The long-term pattern is bigger than email. Dense-Mem becomes the business knowledge layer. ai-memmail becomes one agentic interface on top of it. Instead of teaching every automation from scratch, you maintain one memory layer and let narrow agents use it safely.