> For the complete documentation index, see [llms.txt](https://docs.ixhello.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.ixhello.com/knowledge-bot/functional-documentation/functional-capabilities/data-sources/data-ingestion.md).
# Data Ingestion
To add new data sources, users should select the **Upload** tab, which offers four options corresponding to supported data types:
* **Document:** for PDF, TXT, XLSX, PPTX, DOCX files
* **URL:** for websites
* **Confluence:** for Confluence pages
* **SharePoint:** for SharePoint documents and pages
After selecting the data source type, additional fields specific to that type will appear. Once the data source is saved, it remains in a **Pending** state until processing begins. After processing starts, the system checks every 5 minutes for new chunks. If none are created within this interval, the data source will be marked as **Failed**. This process ensures that large files are processed efficiently without timeout issues; if a file remains stuck, it will be marked as failed after retries.
***5.3.2.1. Documents ingestion***
The knowledge bot can retrieve information from the internal repository where knowledge materials are uploaded in formats such as PDF (.pdf), TXT (.txt), XLSX (.xlsx), PPTX (.pptx), and DOCX (.docx).
* Users can upload up to 10 documents at once, all the same type.
* The maximum file size per document is 30MB.
* Ingested data is extracted as plain text (excluding images or comments).
**Data Masking Status**
* Displays whether masking is **enabled**
* Lists all **default masking entities** selected during Kbot creation
**Viewing Masked Data**
Once ingestion is complete:
1. Open a processed document
2. Navigate to the **Masking Entities Section**
**Compare with the uploaded document**
**Example Masked Entities**
* Phone numbers
* E-Mail addresses
* Locations
* Date and time
**Behavior**
* Masking is applied only to the **selected entity types**
* Unselected entities remain visible
Example:
* Phone, email → masked
* Contact person name → not masked (if not selected)
**Chunking and ingestion configuration:** \
Depending on the file type, users can set additional settings:
* **Divide text by** – provides options to divide the document into manageable chunks, for optimal processing:
* **Split by words:** Divides content into chunks of 200 to 4,500 words (PDF, DOCX, TXT).
* **Split by page:** Separates content based on document pages (PDF).
* **Split by paragraph:** Separates by paragraphs (PDF, PPTX, DOCX).
* **Split by slide:** For PowerPoint documents.
* **Chunk settings:**
* **Data Source Description:** A brief description of the content to improve accuracy and relevance of information retrieval.
* **Overlap:** Number of words repeated between chunks to enhance retrieval context. Higher overlap enhances accuracy but may increase processing load. The option is not available for Excel files.
* **Add description to each chunk:** If enabled, the description is prepended to each chunk, to improve retrieval and accuracy.
* **Excel-specific options:**
* **Header type:**
* **No header**
* **Columns** – the header is applied to columns, and the header section is displayed horizontally
* **Rows** – the header is applied to rows, and the header section is displayed vertically
* The header is added to each chunk
* **Header size:** Number of rows or columns comprising the header.
* **Priority:** A value between 1 (low) and 100 (high), indicating the importance of the content for retrieval, to ensure critical information is emphasized in responses.
When adding Excels as data source, if the user does not select a specific chunking strategy then the default strategy is used: to check the maximum number of rows / columns that can fit in a chunk as long as the size is smaller than the embedding model max token input (ADA Models have a max token input of 8191 tokens).
For PDFs data sources, the files are processed with Azure Document Intelligence with the Markdown content format.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.ixhello.com/knowledge-bot/functional-documentation/functional-capabilities/data-sources/data-ingestion.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.