AI Bank Statement Automation with LLM & Personal Financial Analysis

Intelligent document automation for bank statements — Extract, structure, analyze, and query financial data from PDFs using YOLO + OCR + LLM, with parallel agent harnesses (CrewAI, Deep Agents, Hermes) and local LLMs (LM Studio / Ollama).

Key Features

• Advanced PDF Parsing — YOLO layout detection + OCR + LLM-based table extraction
• Multi-harness agents — CrewAI baseline, LangChain Deep Agents (agentskills.io), Hermes (Docker sandbox)
• Agent Skills — Domain skills for bank-statement parsing, PII redaction, financial analysis, RAG, and output format
• Local LLM First — LM Studio and Ollama via LiteLLM (async acompletion)
• Reasoning Model Support — Handles models that return content in reasoning_content
• Secure RAG Pipeline — PII redaction before embedding into vector database (Qdrant / Chroma)
• Financial Intelligence — Income/expense categorization, trend analysis, natural language querying
• Full-Stack API — FastAPI + PostgreSQL + Celery + React SPA (REST auth, async document processing, agent runs)
• GPU Acceleration — NVIDIA GPU support for PyTorch embeddings and LLM inference (<2× build time vs CPU)
• Development Notebook — Jupyter notebook for CrewAI experimentation
• MLflow Integration — Trace LLM calls and agent workflows (CrewAI path)

Important Notes for Local LLMs (LM Studio / Ollama)

When using local models via LM Studio or Ollama:

• Model size: Prefer 9B+ parameters (e.g. Qwen2.5-14B, Qwen3-27B, Gemma-2-9B, Llama-3.1-8B+). Smaller models reduce quality.
• Context length: 16K+ tokens (recommended 32K+). Low context drops agent instructions.
• JSON output: Local models often fail strict JSON. Prefer Markdown reports; post-process with a second call or instructor if needed.

Tech Stack

Project Structure

AI-Bank-Statement-Document-Automation/
├── agents/                              # Multi-harness agents (experimental)
│   ├── README.md
│   ├── crewai/                          # Skills snapshot + pointer to baseline
│   ├── deep-agents/                     # deepagents SDK, tools, tests, E2E
│   │   ├── run_e2e.py
│   │   ├── agent.py
│   │   ├── tools/
│   │   ├── skills/
│   │   └── tests/
│   └── hermes/                          # Docker-sandboxed Hermes profile
│       ├── config.yaml
│       ├── scripts/
│       └── skills/
├── api/                                 # FastAPI backend (full-stack v1)
│   ├── app/
│   │   ├── agents/                      # Agent adapters (CrewAI, DeepAgents)
│   │   │   ├── crewai/
│   │   │   └── registry.py
│   │   ├── api/                         # REST endpoints (auth, llm, documents, agent_runs)
│   │   ├── core/                        # Security (JWT + bcrypt), deps
│   │   ├── db/                          # SQLAlchemy session + models
│   │   ├── models/                      # SQLAlchemy ORM models
│   │   ├── schemas/                     # Pydantic v2 request/response schemas
│   │   ├── tests/                       # pytest + pytest-asyncio
│   │   ├── worker/                      # Celery tasks + runner
│   │   ├── config.py                    # pydantic-settings env loader
│   │   └── main.py                      # FastAPI application entrypoint
│   ├── alembic/                         # Database migrations
│   ├── config/                          # LLM provider catalog (YAML)
│   └── requirements.txt                 # Pinned API dependencies
├── backend/
│   └── app/
│       ├── core/
│       │   └── ai_agent_skills_dev.ipynb   # CrewAI main notebook
│       └── skills/                         # Live CrewAI skills (shared via namespace pkg)
│           ├── bank-statement-parsing/
│           ├── financial-analysis/
│           ├── pii-handling/
│           ├── rag-query-handling/
│           └── output-format/
├── infra/                               # Docker compose + Kubernetes manifests
│   ├── api.Dockerfile                   # API & Worker image
│   ├── docker-compose.yml               # PostgreSQL, Redis, Qdrant, API, Worker, Web
│   ├── docker-compose.override.yml      # Dev bind-mounts
│   ├── .env.example                     # Environment template
│   └── k8s/
├── web/                                 # React 18 + Vite 5 SPA
├── data/
│   ├── bank-statement-document/         # Sample PDFs (e.g. Dummy-Bank-Statement.pdf)
│   ├── uploads/
│   └── ...
├── docs/
│   ├── guides/
│   │   ├── multi-harness-agents-guide.md
│   │   └── full-stack-app-guide.md
│   └── superpowers/
│       ├── specs/                       # Design docs
│       └── plans/                       # Implementation plans
├── requirements.txt                     # Root / CrewAI stack
└── README.md

Multi-harness agents (experimental)

Parallel frameworks for the same bank-statement workflow. Shared inputs live in data/. Each harness has its own skill copies (no auto-sync).

Fastest path (no LLM)

Deep Agents deterministic pipeline — extract PDF, redact PII, store vectors, RAG answer:

cd agents/deep-agents
python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
PYTHONPATH=. pytest tests/ -v
PYTHONPATH=. python run_e2e.py \
  --pdf ../../data/bank-statement-document/Dummy-Bank-Statement.pdf \
  --question "What amounts appear in the statement?" \
  --mode pipeline

Full step-by-step guide

docs/guides/multi-harness-agents-guide.md — setup, use, and test for all three harnesses, verification checklist, and troubleshooting.

Also see:

• agents/README.md — harness index
• Design spec

Quick Start (CrewAI + Local LLM)

1. Setup environment

git clone https://github.com/johnsonhk88/AI-Bank-Statement-Document-Automation-By-LLM-And-Personal-Finanical-Analysis-Prediction.git
cd AI-Bank-Statement-Document-Automation-By-LLM-And-Personal-Finanical-Analysis-Prediction

python -m venv venv
source venv/bin/activate          # Windows: venv\Scripts\activate
pip install -r requirements.txt

2. Start LM Studio (recommended)

1. Open LM Studio
2. Load a 9B+ model (e.g. qwen2.5-14b-instruct)
3. Set context length to 16K+ (32K preferred)
4. Start local server at http://localhost:1234

3. Run the main CrewAI notebook

jupyter notebook backend/app/core/ai_agent_skills_dev.ipynb

4. Or try multi-harness agents

Key Configuration (CrewAI / LiteLLM)

LM Studio

llm = LLM(
    model="openai/qwen2.5-14b-instruct",   # Use 9B+ model
    base_url="http://localhost:1234/v1",
    api_key="lm-studio",
    temperature=0.6,
    max_tokens=2048,
)

Async direct calls

from litellm import acompletion

response = await acompletion(
    model="openai/your-model",
    base_url="http://localhost:1234/v1",
    api_key="lm-studio",
    messages=[{"role": "user", "content": prompt}],
)

MLflow Tracing (CrewAI path)

MLflow can trace LLM calls, tools, and CrewAI task execution:

import mlflow
mlflow.crewai.autolog()
mlflow.litellm.autolog()

View traces in the MLflow UI after running the notebook.

Full-Stack App (v1)

The full-stack app wraps the three agent harnesses (CrewAI, Deep Agents, Hermes) behind a typed FastAPI + React frontend, with PostgreSQL, Qdrant, and Celery for async background processing.

• Design spec: docs/superpowers/specs/2026-07-25-full-stack-app-design.md
• Implementation plan: docs/superpowers/plans/2026-07-25-full-stack-app.md
• Step-by-step run & test guide: docs/guides/full-stack-app-guide.md — prerequisites, configuration, dev/prod runs, HTTP smoke tests, pytest + vitest walkthroughs, troubleshooting, cheat sheet

Prerequisites

• Docker 24+ with BuildKit (DOCKER_BUILDKIT=1)
• Docker Compose v2+
• NVIDIA Container Toolkit (for GPU support — see below)
• NVIDIA driver ≥ 580 (RTX Pro 4500 Blackwell tested; works with most RTX 20+ GPUs)

Quickstart

# 1. Configure environment
cp infra/.env.example infra/.env

# 2. Generate JWT secret and admin password hash
python -c "import secrets; print(secrets.token_urlsafe(32))"            # → JWT_SECRET
python -c "import bcrypt; print(bcrypt.hashpw(b'your-password', bcrypt.gensalt()).decode())"  # → ADMIN_PASSWORD_HASH

# 3. Edit infra/.env — fill in JWT_SECRET and ADMIN_PASSWORD_HASH.
#    IMPORTANT: bcrypt hashes contain $ — escape them as $$ in .env:
#      ADMIN_PASSWORD_HASH=$$2b$$12$$...
#    (Docker Compose interprets $ as a variable reference.)

# 4. Start everything (prod profile: PostgreSQL + Redis + Qdrant + API + Worker + Nginx on :80)
cd infra && DOCKER_BUILDKIT=1 docker compose --profile prod up -d --build
# Open http://localhost

# 5. For development (API with reload + Vite HMR on :5173):
cd infra && docker compose up -d           # backend services
cd web && npm install && npm run dev        # frontend dev server

GPU Configuration

To use an NVIDIA GPU inside containers:

# 1. Install NVIDIA Container Toolkit (Ubuntu/Debian)
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
  sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update && sudo apt-get install -y nvidia-docker2
sudo systemctl restart docker

# 2. Verify GPU access
docker run --rm --gpus all nvidia/cuda:12.2.1-base-ubuntu22.04 nvidia-smi

# 3. Build and run — the Dockerfile installs the correct GPU PyTorch wheel
#    (torch --index-url https://download.pytorch.org/whl/cu128 — matches Blackwell).
#    GPU devices are requested via deploy.resources.reservations.devices in docker-compose.yml.
cd infra && DOCKER_BUILDKIT=1 docker compose --profile prod up -d --build

# 4. Verify PyTorch sees the GPU inside the container
docker compose exec api python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0))"

Note: Experiment code in backend/, agents/, frontend/streamlit_app/, notebooks/, yolo-base-layout-analysis/ is untouched by v1.

Developer notes

• Namespace packages: api/app/__init__.py is deliberately absent — app/ is a PEP 420 implicit namespace package. Python automatically merges api/app/ and backend/app/ so app.skills resolves from backend/app/skills/ without symlinks.
• passlib / bcrypt: The Docker image pins bcrypt<4.1 because passlib 1.7.4 is incompatible with bcrypt ≥ 4.1 (bcrypt.__about__ removed). If upgrading passlib, retest login flow.
• Qdrant healthcheck uses /proc/net/tcp instead of curl because the qdrant/qdrant minimal image ships without curl or wget.
• Worker CWD: The worker service must cd /app/api before starting Celery (PYTHONPATH alone doesn't add the api directory to Python's import search path for absolute app.* imports).

Roadmap

• Production FastAPI backend (harness-selectable)
• Multi-document collections / workspaces
• Advanced financial forecasting
• Docker + Kubernetes deployment
• Improved Streamlit dashboard with charts
• Shared skill sync tooling across harnesses
• Stronger PII (names/addresses) and production embeddings for Deep Agents

License

This project is licensed under the Apache License 2.0.

Made with care for smarter personal finance automation in Hong Kong.
If you find this project useful, please consider giving it a star.