PDF Table Processing Guide for End Users

Overview

When you upload a PDF to the knowledge base, the system automatically detects and processes tables. This guide explains how the system decides whether a table becomes a dataset (structured data you use for analytics) or searchable text.

What Happens to PDF Tables?

✅ Data Tables → Datasets

Real data tables are converted into structured datasets that you can:

• Download and analyze
• Query with SQL-like operations
• Use in data visualization tools
• Access programmatically

Example data tables:

• Financial reports (revenue, expenses, quarterly data)
• Product catalogs (SKU, price, description)
• Employee directories (name, department, email)
• Sales data (date, region, amount)
• Scientific measurements (time, temperature, pressure)

📝 Formatting Tables → Text Only

Layout and formatting tables are converted to searchable text but NOT saved as datasets.

Example formatting tables:

• Document headers/footers
• Title pages with metadata (Author: John, Date: 2023)
• Navigation menus
• Simple key-value pairs (Status: Approved)
• Spacing/alignment elements

How Does the System Decide?

The system uses intelligent criteria to distinguish between real data tables and formatting elements:

Requirements for a Table to Become a Dataset

A table must meet ALL of these criteria:

Criteria	Minimum Requirement	Why This Matters
Data Rows	At least 2 rows	Single-row tables are usually headers or labels
Columns	At least 2 columns	Single-column tables are typically lists or formatting
Total Cells	At least 6 cells	Very small tables are often just layout elements
Data Density	30% or more filled	Mostly empty tables are likely formatting grids

Automatic Filtering

The system automatically filters out:

❌ Metadata Tables

Tables with 2 columns where the first column contains common labels:

Name:        John Smith
Date:        October 29, 2025
Version:     1.0
Status:      Draft

Why filtered: These are document properties, not datasets.

❌ Single-Column Lists

Small single-column tables (≤5 rows):

Important
Note
See Details

Why filtered: These are formatting elements or simple lists.

❌ Repetitive Tables

Tables where all values are identical:

[Empty] [Empty] [Empty]
[Space] [Space] [Space]

Why filtered: These are spacing/layout elements.

❌ Mostly Empty Tables

Tables with less than 30% of cells filled:

Header1  Header2  Header3  Header4  Header5
Data1    [empty]  [empty]  [empty]  [empty]

Why filtered: Sparse tables are often formatting structures.

Examples

✅ Example 1: Data Table (Becomes Dataset)

Quarter  | Revenue    | Expenses   | Profit
---------|------------|------------|----------
Q1 2024  | $1,200,000 | $800,000   | $400,000
Q2 2024  | $1,450,000 | $950,000   | $500,000
Q3 2024  | $1,680,000 | $1,100,000 | $580,000
Q4 2024  | $1,920,000 | $1,250,000 | $670,000

Result: ✅ Saved as dataset page3_quarter_revenue_expenses_table1

Why:

• 4 data rows ✓
• 4 columns ✓
• 16 cells ✓
• 100% filled ✓
• Clear data structure ✓

---

❌ Example 2: Document Metadata (Text Only)

Document Title: | Annual Report 2024
Author:         | Jane Doe
Date Created:   | October 15, 2024

Result: ❌ Not saved as dataset (converted to searchable text)

Why:

• Only 3 rows (close to minimum)
• Only 2 columns (at minimum)
• First column contains metadata labels ("Author", "Date Created")
• Detected as key-value metadata table

---

❌ Example 3: Navigation Table (Text Only)

Section 1 |
Section 2 |
Section 3 |

Result: ❌ Not saved as dataset (converted to searchable text)

Why:

• Only 1 column
• Appears to be a navigation list or table of contents
• Not structured data

---

✅ Example 4: Product Catalog (Becomes Dataset)

SKU      | Product Name        | Price   | In Stock
---------|--------------------|---------|---------
PRD-001  | Wireless Mouse     | $29.99  | Yes
PRD-002  | USB Keyboard       | $49.99  | Yes
PRD-003  | Monitor Stand      | $79.99  | No
PRD-004  | Laptop Sleeve      | $34.99  | Yes
PRD-005  | Phone Charger      | $19.99  | Yes

Result: ✅ Saved as dataset page12_sku_product_name_price_table2

Why:

• 5 data rows ✓
• 4 columns ✓
• 20 cells ✓
• 100% filled ✓
• Clear product catalog structure ✓

Dataset Naming

When a table becomes a dataset, it receives a descriptive name based on:

Naming Pattern:

page{page_number}_{keywords}_table{number}

How Keywords Are Extracted:

1. From Column Headers (preferred):

   • Takes first 3 meaningful column names
   • Example: Quarter, Revenue, Expenses → quarter_revenue_expenses

2. From First Column Values (fallback):

   • If headers are generic, uses first few data values
   • Example: First column has "Total", "Summary" → total_summary

3. Generic (last resort):

   • If no meaningful keywords found → page10_table3

Deduplication:

If multiple tables generate the same name, a suffix is added:

• First occurrence: page5_sales_data_table2
• Second occurrence: page5_sales_data_table2_1
• Third occurrence: page5_sales_data_table2_2

What About Filtered Tables?

Don't worry - you won't lose information!

Tables that are filtered out as formatting elements are still:

• ✅ Converted to searchable text
• ✅ Embedded in the vector database
• ✅ Available for semantic search
• ✅ Included in Q&A responses

You just won't be able to:

• ❌ Download them as separate dataset files
• ❌ Run structured queries on them
• ❌ Use them for data analysis

This is intentional - metadata tables like "Author: John Doe" are more useful as searchable text than as structured datasets.

Accessing Your Datasets

Your datasets are automatically available through:

• Analytics Tools - Query and visualize your data
• Natural Language Processing - Ask questions about your data in plain English
• Semantic Search - Find relevant information across all your documents

The system handles all the technical details behind the scenes!

Tips for Best Results

📋 For Data Tables You Want as Datasets:

• Use clear column headers
• Keep data organized in rows and columns
• Avoid excessive empty cells
• Use consistent formatting

📝 For Document Formatting:

• The system will automatically handle these
• Metadata tables will be searchable as text
• No action needed on your part

FAQ

Q: Why wasn't my table saved as a dataset?

Common reasons:

• Too few rows (less than 2 data rows)
• Only 1 column
• Mostly empty (less than 30% filled)
• Detected as document metadata (like "Author: John, Date: 2023")

Q: Can I override the filtering?

The filtering is automatic and designed to give you the best experience. To ensure a table becomes a dataset:

• Include at least 2 data rows
• Include at least 2 columns
• Fill at least 30% of cells with data
• Use a clear table structure

Q: Can I still search for content in filtered tables?

Yes! All table content is converted to searchable text and embedded in the vector database. You can find this information through natural language queries.

Summary

Table Type	Becomes Dataset?	Searchable?	Use Case
Financial data	✅ Yes	✅ Yes	Data analysis, reporting
Product catalogs	✅ Yes	✅ Yes	Inventory management
Employee lists	✅ Yes	✅ Yes	HR operations
Document metadata	❌ No	✅ Yes	Document properties
Navigation elements	❌ No	✅ Yes	Document structure
Spacing/layout	❌ No	✅ Yes	Visual formatting

The system is designed to give you the best of both worlds:

• Structured datasets for real data tables
• Searchable text for everything else