← Back to blog
Extract

Extracting Spanish invoices with LLMs: what the AEAT spec doesn't cover

Processing a received invoice is not just OCR. It's fiscal validation: NIF format, VAT coherence, equivalence surcharge detection, IRPF withholding, mathematical cross-checks. Here's why neither pure OCR nor pure LLM is sufficient on its own, and how a dual Vision + OCR pipeline closes the gap.

May 9, 2026 · 11 min read
Invoice document being digitally processed
Photo by Abhinav Arya on Unsplash

The real problem: fiscal validation, not just text extraction

When a business receives an invoice, the question is not "what does this PDF say?". The question is "is this invoice fiscally correct, and can I use it as supporting documentation for my tax filings?". Those are different questions that require different answers.

A fiscally incorrect invoice — wrong NIF, IVA rate not matching the service type, IRPF withholding applied to the wrong party, a mathematical total that doesn't add up — can be rejected by the tax authority or create problems during an audit. The accounting software that processes it needs to catch these issues, not just ingest the text.

The AEAT's specifications define the structure of a correct invoice. What they don't cover is how to extract and validate invoices that arrive in dozens of different visual formats, from PDFs generated by different accounting software, to scans, to photos taken on a phone.

Why pure OCR isn't sufficient

Traditional OCR works by recognizing characters and producing text. It does this well on clean, high-resolution documents with consistent layouts. Spanish invoices in the real world are not always those things.

Specific failure modes OCR hits that an LLM handles better:

Why pure LLM isn't sufficient either

A vision LLM — prompted to extract a structured invoice from an image — performs remarkably well on layout understanding and semantic interpretation. It handles unstructured formats, handwriting, and context-dependent fields better than OCR.

But it has a specific failure mode that is dangerous in a fiscal context: hallucination of numeric and identifier fields.

NIFs are the worst case. A NIF is a nine-character string with a specific format and a checksum digit computed from the preceding characters. An LLM that reads "B-8237194" on a blurry invoice image may confidently produce "B82371940" — a plausible-looking NIF that fails checksum validation and would cause the invoice to be rejected during reconciliation. The LLM didn't know it was wrong. It was doing its best with uncertain input.

The same applies to invoice numbers, bank account numbers, and tax totals. Fields where a single digit error matters cannot be trusted to a model that optimizes for plausibility rather than accuracy.

The dual pipeline: Vision LLM + OCR cross-verification

The architecture Kreyo Extract uses combines both approaches at different stages:

Stage 1: Vision LLM extraction

The invoice image or PDF is passed to a vision LLM with a structured extraction prompt. The prompt instructs the model to return a JSON object conforming to the InvoiceData schema: issuer details, recipient details, line items, tax breakdown, totals, payment information.

The LLM handles layout understanding, multi-line item parsing, and semantic inference (e.g., recognizing that "IRPF 15%" is a withholding, not an additional charge). It produces a structured candidate.

Stage 2: OCR cross-verification of numeric fields

An OCR pass runs over the same document, specifically targeting the regions where the LLM found numeric and identifier fields. The OCR output is compared against the LLM's extraction for:

Where OCR and LLM agree, the field is accepted. Where they disagree beyond a tolerance threshold, the field is flagged for review. This catches the LLM hallucination cases while preserving its advantages on layout and semantics.

Stage 3: Fiscal validation

With a reconciled structured output, the fiscal validation layer runs:

Compliance scoring: 0–100

The output of the pipeline includes a compliance_score field alongside the extracted InvoiceData. The score reflects how many of the fiscal validation rules the invoice passed.

The score is deterministic given the same extraction output — it's not a model confidence score. Each rule contributes a fixed weight. If the NIF fails checksum, that's a 30-point deduction. If totals don't cross-check, that's a 25-point deduction. The score is auditable.

The response also includes a validation_flags array with specific codes for each failing rule, so the calling application can display a meaningful message to the accountant rather than just a number.

Edge cases discovered in production

Scanned invoices from regional gestorías

Gestorías (Spanish accounting services) sometimes produce invoices in proprietary formats specific to the software they use — some of which date from the 1990s. These produce PDFs where each character is a small image embedded in the PDF stream, not a text layer. Standard PDF text extraction fails entirely. The pipeline must detect this and route to the vision path exclusively.

Mobile phone photos

A significant share of invoices processed by small businesses are photos taken on a phone: perspective distortion, uneven lighting, partial shadows from the photographer's hand. These are the hardest case for OCR. The vision LLM handles them considerably better, though confidence on numeric fields is lower. The cross-verification step is especially important here, and the threshold for flagging disagreements is tightened.

Non-standard IVA configurations

Spain's IVA rates are 21%, 10%, and 4% (reduced rates). Some invoices mix rates across line items. Some apply the Canary Islands IGIC instead of peninsular IVA. Some are intra-EU transactions with reverse charge. The fiscal validation layer must distinguish these cases from genuine errors, or it produces false positives that erode trust in the compliance score.

Credit notes and corrective invoices

A factura rectificativa (corrective invoice) has negative line items and references the original invoice number. OCR and LLMs both have higher error rates on negative numbers. The pipeline applies specific handling for documents identified as corrective invoices, including validating that the referenced invoice number field is present and correctly formatted.

What this looks like in practice

The Kreyo Extract API takes a document — PDF, image, or URL — and returns an InvoiceData object with compliance metadata: 

curl -X POST https://api.kreyo.io/extract/v1/invoice \
  -H "Authorization: Bearer sk_live_..." \
  -H "Content-Type: multipart/form-data" \
  -F "file=@received-invoice.pdf"

Response:

{
  "invoice": {
    "issuer": {
      "nif": "B82371940",
      "name": "Acme Servicios S.L.",
      "address": "Calle Gran Vía 1, 28013 Madrid"
    },
    "recipient": {
      "nif": "A28791502",
      "name": "Cliente Corp S.A."
    },
    "invoice_number": "2026-0042",
    "invoice_date": "2026-04-15",
    "line_items": [
      {
        "description": "Servicios de consultoría — abril 2026",
        "quantity": 1,
        "unit_price": 5000.00,
        "vat_rate": 0.21,
        "vat_amount": 1050.00,
        "total": 6050.00
      }
    ],
    "subtotal": 5000.00,
    "vat_total": 1050.00,
    "irpf_amount": 750.00,
    "total": 5300.00
  },
  "compliance_score": 97,
  "validation_flags": [],
  "extraction_confidence": "high"
}

The extraction_confidence field reflects the agreement between the LLM and OCR passes. High means both agreed on all critical fields. Medium means minor disagreements were resolved. Low means significant disagreements — the compliance_score and validation_flags carry more weight in this case.

The connection to the rest of the stack

The InvoiceData schema returned by Extract is the same schema used as input by the other Kreyo products. An invoice extracted from a received PDF can be passed directly to the VeriFactu or FacturaE endpoints without transformation. The data model is shared across the platform by design.

This matters for accounting software that needs to handle both sides of the invoice flow — issuing invoices (VeriFactu registration, PDF generation, FacturaE delivery) and receiving invoices (Extract). A single integration covers the full cycle.