Extraction Schemas - Aster Agents

Overview

Your AI agents can already search and read the full contents of every file in a knowledge base. Extraction schemas add a structured index on top of that—a consistent set of fields extracted from each document that enables quick filtering and cross-document analysis. Think of it this way: without an extraction schema, answering “which contracts are worth over $1M?” requires the AI to open and read every contract. With an extraction schema that includes total_value as a number field, that question becomes a simple filter. Extraction schemas are most valuable for:

Numeric fields you want to compare, aggregate, or filter (amounts, counts, percentages, dates)
Categorical fields you want to group or filter by (document type, status, region, sector)
Key identifiers that help locate the right documents quickly (deal names, parties, project codes)

String fields like summaries can be useful too, but the real power is in structured data that saves you from scanning every file when asking questions across your entire knowledge base.

Choosing What to Extract

Before writing your schema, ask yourself: “What questions do I want to answer across all my documents?”

Question Type	Extract These Fields
”Which deals are over $10M?”	`deal_value` (number)
“Show me all contracts expiring this quarter”	`expiration_date` (string in YYYY-MM-DD)
“Break down documents by type”	`document_type` (enum)
“Find all projects in the healthcare sector”	`sector` (enum)
“What’s our average contract value by region?”	`contract_value` (number), `region` (enum)
“List all documents involving Acme Corp”	`parties` (array of strings)

Prioritize fields that are

Frequently used for filtering or sorting
Needed for aggregations (sums, averages, counts)
Used to categorize or group documents
Key identifiers for finding specific documents

Skip fields that are

Only useful when reading the full document (the AI can already do that)
Highly variable or unstructured across documents
Rarely needed for cross-document questions

Schema Basics

Extraction schemas use JSON Schema format. At minimum, your schema needs three things:

Set a type

A type field (usually "object").

Define properties

A properties field defining what to extract.

List required fields

A required array listing mandatory fields.

{
  "type": "object",
  "required": ["title", "date", "summary"],
  "properties": {
    "title": { "type": "string", "description": "Document title as shown on the first page" },
    "date": { "type": "string", "description": "Publication or effective date in YYYY-MM-DD format" },
    "summary": { "type": "string", "description": "One paragraph summary of the document's main purpose" }
  }
}

Field Types & Structures

The reference below covers every supported field type and the structures you can compose from them. Expand each section as you need it.

Supported Types

Use these types for your schema fields:

Type	Description	Example Value
`string`	Text values	`"Quarterly Report"`
`number`	Decimal numbers	`123.45`
`integer`	Whole numbers	`42`
`boolean`	True/false values	`true`
`array`	Lists of items	`["item1", "item2"]`
`object`	Nested structures	`{"name": "John"}`

Making fields optional

Use a type array with null to make fields optional:

{
  "type": "object",
  "required": ["company_name"],
  "properties": {
    "company_name": { "type": "string", "description": "Legal company name" },
    "website": { "type": ["string", "null"], "description": "Company website URL, if mentioned" },
    "employee_count": { "type": ["integer", "null"], "description": "Number of employees, if stated" }
  }
}

When a field uses ["type", "null"], the AI will return null if the information isn’t found in the document.

Using enums

Enums constrain a field to specific allowed values. Use enums whenever you have a known set of categories:

{
  "type": "object",
  "required": ["document_type", "status"],
  "properties": {
    "document_type": {
      "type": "string",
      "enum": ["CONTRACT", "INVOICE", "PROPOSAL", "REPORT"],
      "description": "Classification of the document"
    },
    "status": {
      "type": "string",
      "enum": ["DRAFT", "PENDING", "APPROVED", "REJECTED"],
      "description": "Current approval status"
    },
    "priority": {
      "type": ["string", "null"],
      "enum": ["LOW", "MEDIUM", "HIGH", null],
      "description": "Priority level, if indicated"
    }
  }
}

Enums work with strings, numbers, booleans, or null values. Include null in the enum array if the field should be optional.

Arrays

Use arrays to extract lists of items:

{
  "type": "object",
  "required": ["authors", "keywords"],
  "properties": {
    "authors": {
      "type": "array",
      "items": { "type": "string" },
      "description": "Full names of all authors"
    },
    "keywords": {
      "type": "array",
      "items": { "type": "string" },
      "description": "Key topics or tags relevant to this document"
    }
  }
}

Arrays of objects — extract structured lists with typed objects:

{
  "type": "object",
  "required": ["line_items"],
  "properties": {
    "line_items": {
      "type": "array",
      "description": "Individual items or charges listed in the document",
      "items": {
        "type": "object",
        "required": ["description", "amount"],
        "properties": {
          "description": { "type": "string", "description": "Item or service description" },
          "quantity": { "type": ["integer", "null"], "description": "Number of units, if specified" },
          "amount": { "type": "number", "description": "Line item total in USD" }
        }
      }
    }
  }
}

Nested objects

Group related fields using nested objects:

{
  "type": "object",
  "required": ["vendor", "total_amount"],
  "properties": {
    "vendor": {
      "type": "object",
      "description": "Information about the vendor or supplier",
      "required": ["name"],
      "properties": {
        "name": { "type": "string", "description": "Vendor's legal business name" },
        "address": { "type": ["string", "null"], "description": "Full mailing address" },
        "contact_email": { "type": ["string", "null"], "description": "Primary contact email" }
      }
    },
    "total_amount": { "type": "number", "description": "Total amount due in USD" }
  }
}

Best Practices

1. Require fields whenever possible

Mark fields as required unless they genuinely may not exist in documents. This helps the AI understand which information is essential:

{
  "required": ["title", "date", "amount"],
  "properties": { ... }
}

2. Use enums for categories

Whenever you have a known set of possible values, use enums instead of free-form strings:

// Good - constrained values
"status": { "type": "string", "enum": ["ACTIVE", "INACTIVE", "PENDING"] }

// Avoid - unconstrained
"status": { "type": "string" }

3. Keep nesting shallow

Prefer one or two levels of nesting to group related fields logically:

{
  "deal_name": { "type": "string" },
  "deal_type": { "type": "string" },
  "buyer": {
    "type": "object",
    "properties": {
      "name": { "type": "string" },
      "contact_email": { "type": "string" }
    }
  }
}

4. Use descriptions to instruct the AI

The description field is your primary way to guide the extraction AI. Think of descriptions as instructions telling the AI exactly how to interpret each field when analyzing the document:

{
  "type": "object",
  "required": ["effective_date", "total_value", "parties"],
  "properties": {
    "effective_date": {
      "type": "string",
      "description": "Contract start date in YYYY-MM-DD format"
    },
    "total_value": {
      "type": "number",
      "description": "Total contract value in USD, excluding taxes and fees"
    },
    "renewal_terms": {
      "type": ["string", "null"],
      "description": "Summary of auto-renewal clauses. Return null if no auto-renewal provisions exist."
    },
    "parties": {
      "type": "array",
      "items": { "type": "string" },
      "description": "Legal names of all signing parties, extracted exactly as written in the signature block"
    }
  }
}

Use descriptions to specify:

Date and number formats: "YYYY-MM-DD", "in USD", "as a decimal percentage (e.g., 0.15 for 15%)"
What to include or exclude: "excluding taxes", "only from the executive summary section"
How to handle missing data: "Return null if not explicitly stated"
Where to look: "from the signature block", "as stated in the header"
Level of detail: "one-sentence summary", "verbatim quote", "3-5 bullet points"

Well-written descriptions dramatically improve extraction accuracy.

5. Test across multiple documents

Before finalizing your schema, test it against several representative documents—not just one. A schema that works perfectly for one document may fail on others.Testing across examples helps you identify:

Fields that don’t exist in all documents (should be optional with null)
Categories that vary more than expected (broaden your enums)
Format inconsistencies (clarify your descriptions)
Edge cases you didn’t anticipate

The goal is a schema that works generically across all documents you’ll upload to the knowledge base, not one optimized for a single example.

Example Schemas

These are complete, copy-ready schemas for common document types. Switch tabs to see each one.

{
  "type": "object",
  "required": ["invoice_number", "vendor_name", "total_amount", "line_items"],
  "properties": {
    "invoice_number": { "type": "string" },
    "invoice_date": { "type": ["string", "null"] },
    "due_date": { "type": ["string", "null"] },
    "vendor_name": { "type": "string" },
    "vendor_address": { "type": ["string", "null"] },
    "total_amount": { "type": "number" },
    "currency": {
      "type": "string",
      "enum": ["USD", "EUR", "GBP", "CAD"]
    },
    "line_items": {
      "type": "array",
      "items": {
        "type": "object",
        "required": ["description", "amount"],
        "properties": {
          "description": { "type": "string" },
          "quantity": { "type": ["number", "null"] },
          "unit_price": { "type": ["number", "null"] },
          "amount": { "type": "number" }
        }
      }
    }
  }
}

Schema Depth: Index vs. Full Extraction

When designing your schema, consider how your agent will actually use the data.

Summary-level (index and filter)
Deep (work directly from extracted data)

Best when agents need to find and filter documents, then read the full content for detail:

{
  "type": "object",
  "required": ["vendor_name", "total_amount", "invoice_date"],
  "properties": {
    "vendor_name": { "type": "string" },
    "total_amount": { "type": "number" },
    "invoice_date": { "type": "string" }
  }
}

The agent loads extracted_data.json to find the right files, then reads their .md content for line items, notes, and other detail. This is simpler to maintain and works well when documents vary in structure.

Best when agents need to process many documents without reading raw content — e.g., spreading financials, building comparison tables, or aggregating metrics across hundreds of files:

{
  "type": "object",
  "required": ["vendor_name", "total_amount", "line_items"],
  "properties": {
    "vendor_name": { "type": "string" },
    "total_amount": { "type": "number" },
    "line_items": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "description": { "type": "string" },
          "quantity": { "type": ["number", "null"] },
          "unit_price": { "type": ["number", "null"] },
          "amount": { "type": "number" }
        }
      }
    }
  }
}

With a deep schema, the agent can build its entire output from extracted_data.json alone — no document reading needed. This is faster and more reliable for bulk operations, but requires more upfront schema design and re-extraction when the schema changes.

How to decide

Factor	Summary schema	Deep schema
Agent reads 1-5 files per task	Good fit	Overkill
Agent processes 10+ files per task	Bottleneck — agent reads every file	Good fit
Document structure varies a lot	Easier to maintain	Harder to design
Speed/timeout matters	Agent still needs file reads	Single JSON load
Schema changes frequently	Low re-extraction cost	Must re-extract all files

Start summary, go deep where it hurts. Begin with a summary-level schema for filtering. If you notice your agent spending most of its time reading files to extract the same fields repeatedly, promote those fields into the extraction schema.

Setting Up Extraction

Open Knowledge Bases

Go to Control Hub → Knowledge Bases.

Create or edit a knowledge base

Create a new knowledge base or edit an existing one.

Configure structured data extraction

Under Structured Data Extraction:

Select an Extraction Model (e.g., GPT-4o)
Paste your JSON schema into the Extraction Schema field

Save and upload

Save and upload your documents.

The schema only applies to newly uploaded files. To re-extract data from existing files, use the retry extraction option in the file details.

Viewing Extracted Data

After processing completes:

Open a file

Click on a file in your knowledge base.

View the extracted data

View the Extracted Data section. The structured JSON output matches your schema definition.

You can also access extracted data programmatically through agents using the search_knowledge_base tool.

Troubleshooting

"Invalid JSON" error

Your schema isn’t valid JSON. Check for:

Missing commas between properties
Missing closing braces }
Trailing commas (not allowed in JSON)

"Must have a type field" error

Every schema needs a root type. Usually this should be "object":

{
  "type": "object",
  ...
}

Empty or missing extracted fields

Ensure the field is marked as required if it should always be extracted
Check that your field names clearly describe what to extract
Verify the document actually contains the information

Extraction taking too long

Large schemas with many fields take longer to process. Consider:

Breaking into multiple smaller knowledge bases
Removing non-essential fields
Using a faster extraction model

​Overview

​Choosing What to Extract

Prioritize fields that are

Skip fields that are

​Schema Basics

​Field Types & Structures

​Supported Types

​Best Practices

​Example Schemas

​Schema Depth: Index vs. Full Extraction

​How to decide

​Setting Up Extraction

​Viewing Extracted Data

​Troubleshooting

Overview

Choosing What to Extract

Schema Basics

Field Types & Structures

Supported Types

Best Practices

Example Schemas

Schema Depth: Index vs. Full Extraction

How to decide

Setting Up Extraction

Viewing Extracted Data

Troubleshooting