Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions apps/docs/content/docs/knowledgebase/connectors.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Each connector has source-specific fields that control what gets synced. Example

- **Notion** — sync an entire workspace, a specific database, or a single page tree
- **GitHub** — specify a repository, branch, and optional file extension filter
- **Confluence** — enter your Atlassian domain and choose spaces, or **All** for all spaces accessible at each sync. Optionally filter by content type or label. PDF and Word (`.docx`, Word 97–2003 `.doc`) attachments on matching pages and blog posts are included as separate documents.
- **Confluence** — enter your Atlassian domain and choose spaces, or **All** for all spaces accessible at each sync. Optionally filter by content type or label. PDF, Word (`.docx`, Word 97–2003 `.doc`), Excel (`.xlsx`), and PowerPoint (`.pptx`) attachments on matching pages and blog posts are included as separate documents.
- **Azure DevOps** — choose what to sync (wiki pages, work items, repository files, or all), with optional work item type/state filters, a custom WIQL query, and repository/branch/path filters
- **Amazon S3** — point at a bucket with an optional key prefix and a customizable file extension allowlist; S3-compatible stores (Cloudflare R2, MinIO) are supported via a custom endpoint
- **YouTube** — sync a channel (by `@handle` or ID) or playlist, with an optional published-after date filter and the option to exclude Shorts
Expand All @@ -88,7 +88,7 @@ Each connector has source-specific fields that control what gets synced. Example

Configuration is validated on save — if a repository doesn't exist or a domain is unreachable, you'll see an error immediately.

Confluence attachment indexing requires `read:attachment:confluence`. For a service account, include it when creating the scoped API token; see the [Confluence scope list](/search/confluence#using-a-service-account). Attachments are checked even when the parent page has not changed. Files over 100 MB appear as skipped; convert Word 6/95 files to `.docx` before attaching them.
Confluence attachment indexing requires `read:attachment:confluence`. For a service account, include it when creating the scoped API token; see the [Confluence scope list](/search/confluence#using-a-service-account). Attachments are checked even when the parent page has not changed. Files over 100 MB appear as skipped; convert Word 6/95 files to `.docx`, and `.xls` and `.ppt` files to `.xlsx` and `.pptx`, before attaching them. Spaces that were already connected pick up newly supported formats on their next sync.

</Step>
<Step>
Expand Down
8 changes: 4 additions & 4 deletions apps/docs/content/docs/search/confluence.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { Callout } from 'fumadocs-ui/components/callout'
import { Step, Steps } from 'fumadocs-ui/components/steps'
import { Image } from '@/components/ui/image'

Search pages, blog posts, and their PDF and Word attachments from selected Confluence Cloud spaces. A Sim organization admin enables Confluence; **each teammate connects their own account**.
Search pages, blog posts, and their PDF, Word, Excel, and PowerPoint attachments from selected Confluence Cloud spaces. A Sim organization admin enables Confluence; **each teammate connects their own account**.

| Method | How it works |
| --- | --- |
Expand Down Expand Up @@ -125,9 +125,9 @@ See Atlassian's [account setup](https://support.atlassian.com/user-management/do
| **Filter by Label** | Optional comma-separated labels; content can match any listed label. |
| **Metadata tags** | Labels, version, and last-modified tags. |

Search manages the schedule and hides item limits. It indexes published/current content and each page's own text, including supported local callouts and code blocks. PDF, Word `.docx`, and Word 97–2003 `.doc` attachments on the selected pages and blog posts are indexed as separate documents with their parent content's permissions. Space, content-type, and label filters apply to the parent content. Attachment changes are checked on each sync, even when the parent text has not changed.
Search manages the schedule and hides item limits. It indexes published/current content and each page's own text, including supported local callouts and code blocks. PDF, Word `.docx`, Word 97–2003 `.doc`, Excel `.xlsx`, and PowerPoint `.pptx` attachments on the selected pages and blog posts are indexed as separate documents with their parent content's permissions. Space, content-type, and label filters apply to the parent content. Attachment changes are checked on each sync, even when the parent text has not changed.

Archived content, comments, other attachment formats, and expanded Include Page, Excerpt Include, or third-party macro output are excluded. Referenced pages can be indexed separately with their own permissions. Attachments over 100 MB are shown as skipped; convert older Word 6/95 files to `.docx` before attaching them.
Archived content, comments, other attachment formats, and expanded Include Page, Excerpt Include, or third-party macro output are excluded. Referenced pages can be indexed separately with their own permissions. Attachments over 100 MB are shown as skipped; convert older Word 6/95 files to `.docx`, and `.xls` and `.ppt` files to `.xlsx` and `.pptx`, before attaching them. Spaces that were already connected pick up newly supported formats on their next sync.

## Manage access and sync

Expand All @@ -151,7 +151,7 @@ In **Sync history**, **Continuing** means a healthy listing needs another batch.
| A new page, blog post, or label is missing | Confluence search can take time to update. Once the content appears in Confluence search with the selected label, sync again. |
| A restricted page is missing | Both your account and the crawling account need access to the page and its ancestors. |
| Embedded content is missing | Index the referenced page separately; remote macro output is excluded. |
| PDF or Word attachments are missing | Check `read:attachment:confluence` and access to the parent page. Existing service-account tokens may need to be replaced with one that includes this scope. Attachment access failures are reported as a partial sync. |
| Attachments are missing | Check `read:attachment:confluence` and access to the parent page. Existing service-account tokens may need to be replaced with one that includes this scope. Attachment access failures are reported as a partial sync. |
| **Reconnect** or email mismatch | Authorize with the Atlassian account matching your verified Sim email and grant all requested permissions. |

Open a missing page as the affected teammate, check its space and page restrictions, then sync again after correcting access. See Atlassian's [content access](https://support.atlassian.com/confluence-cloud/docs/add-or-remove-page-restrictions/) and [permission inspection](https://support.atlassian.com/confluence-cloud/docs/inspect-a-users-permissions/) guides.
Expand Down
133 changes: 104 additions & 29 deletions apps/sim/connectors/confluence/attachments.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,13 @@
import JSZip from 'jszip'
import { PDFDocument, StandardFonts } from 'pdf-lib'
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
import * as XLSX from 'xlsx'
import { DEFAULT_MAX_ERROR_BODY_BYTES, PayloadSizeLimitError } from '@/lib/core/utils/stream-limits'
import { parseBuffer } from '@/lib/file-parsers'
import { listConfluenceAttachments } from '@/connectors/confluence/attachments'
import { confluenceConnector } from '@/connectors/confluence/confluence'
import type { ExternalDocument, ExternalDocumentList } from '@/connectors/types'
import { CONNECTOR_MAX_FILE_BYTES } from '@/connectors/utils'
import { CONNECTOR_MAX_FILE_BYTES, PIPELINE_PARSED_MIME_TYPES } from '@/connectors/utils'

const { secureDownload } = vi.hoisted(() => ({ secureDownload: vi.fn() }))
vi.mock('@/lib/knowledge/documents/secure-fetch.server', async (importOriginal) => ({
Expand Down Expand Up @@ -90,6 +91,75 @@ async function get(externalId = 'attachment:page:p1:att123', config = CONFIG) {
return confluenceConnector.getDocument('token', config, externalId, { ...CONTEXT })
}

const ROUNDTRIP_TEXT = 'Confluence attachment text'
const OOXML_REL = 'http://schemas.openxmlformats.org/officeDocument/2006/relationships'
const PACKAGE_RELS_NS = 'xmlns="http://schemas.openxmlformats.org/package/2006/relationships"'
const PRESENTATION_NS =
'xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main" xmlns:p="http://schemas.openxmlformats.org/presentationml/2006/main" xmlns:r="http://schemas.openxmlformats.org/officeDocument/2006/relationships"'

async function pdfBytes(): Promise<Buffer> {
const document = await PDFDocument.create()
const font = await document.embedFont(StandardFonts.Helvetica)
document.addPage().drawText(ROUNDTRIP_TEXT, { font, size: 14 })
return Buffer.from(await document.save())
}

async function docxBytes(): Promise<Buffer> {
const zip = new JSZip()
zip.file(
'[Content_Types].xml',
'<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>'
)
zip.file(
'_rels/.rels',
`<Relationships ${PACKAGE_RELS_NS}><Relationship Id="rId1" Type="${OOXML_REL}/officeDocument" Target="word/document.xml"/></Relationships>`
)
zip.file(
'word/document.xml',
`<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"><w:body><w:p><w:r><w:t>${ROUNDTRIP_TEXT}</w:t></w:r></w:p></w:body></w:document>`
)
return zip.generateAsync({ type: 'nodebuffer' })
}

/** One slide resolved through `p:sldIdLst`, the order the presentation walker follows. */
async function pptxBytes(): Promise<Buffer> {
const zip = new JSZip()
zip.file(
'[Content_Types].xml',
'<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"/>'
)
zip.file(
'ppt/presentation.xml',
`<?xml version="1.0" encoding="UTF-8" standalone="yes"?><p:presentation ${PRESENTATION_NS}><p:sldIdLst><p:sldId id="256" r:id="rId1"/></p:sldIdLst></p:presentation>`
)
zip.file(
'ppt/_rels/presentation.xml.rels',
`<Relationships ${PACKAGE_RELS_NS}><Relationship Id="rId1" Type="${OOXML_REL}/slide" Target="slides/slide1.xml"/></Relationships>`
)
zip.file(
'ppt/slides/slide1.xml',
`<?xml version="1.0" encoding="UTF-8" standalone="yes"?><p:sld ${PRESENTATION_NS}><p:cSld><p:spTree><p:sp><p:nvSpPr><p:cNvPr id="1" name="s"/><p:cNvSpPr/></p:nvSpPr><p:txBody><a:bodyPr/><a:p><a:r><a:t>${ROUNDTRIP_TEXT}</a:t></a:r></a:p></p:txBody></p:sp></p:spTree></p:cSld></p:sld>`
)
return zip.generateAsync({ type: 'nodebuffer' })
}

async function xlsxBytes(): Promise<Buffer> {
const book = XLSX.utils.book_new()
XLSX.utils.book_append_sheet(
book,
XLSX.utils.aoa_to_sheet([['Note'], [ROUNDTRIP_TEXT]]),
'Sheet1'
)
return XLSX.write(book, { type: 'buffer', bookType: 'xlsx' }) as Buffer
}

const ROUNDTRIP_FIXTURES = {
pdf: pdfBytes,
docx: docxBytes,
pptx: pptxBytes,
xlsx: xlsxBytes,
} as const

beforeEach(() => {
fetchMock.mockReset()
secureDownload.mockReset().mockResolvedValue(new Response('binary bytes'))
Expand All @@ -98,15 +168,20 @@ beforeEach(() => {
afterEach(() => vi.unstubAllGlobals())

describe('Confluence attachment listing', () => {
it('lists PDF, DOC and DOCX stubs without downloading, and excludes unsupported or archived files', async () => {
it('lists PDF, Word, PowerPoint and Excel stubs without downloading, and excludes unsupported or archived files', async () => {
fetchMock.mockResolvedValue(
Response.json({
results: [
file(),
file({ id: '2', title: 'Legacy.DOC' }),
file({ id: '3', title: 'Modern.docx' }),
file({ id: '4', title: 'image.png' }),
file({ id: '5', title: 'old.pdf', status: 'archived' }),
file({ id: '4', title: 'Deck.pptx' }),
file({ id: '5', title: 'Sheet.XLSX' }),
file({ id: '6', title: 'image.png' }),
file({ id: '7', title: 'Macro.pptm' }),
file({ id: '8', title: 'Legacy.xls' }),
file({ id: '9', title: 'Old.ppt' }),
file({ id: '10', title: 'old.pdf', status: 'archived' }),
],
})
)
Expand All @@ -119,7 +194,14 @@ describe('Confluence attachment listing', () => {
'attachment:page:p1:att123',
'attachment:page:p1:2',
'attachment:page:p1:3',
'attachment:page:p1:4',
'attachment:page:p1:5',
])
expect(result.documents.slice(1).map((doc) => doc.mimeType)).toEqual(
['pdf', 'doc', 'docx', 'pptx', 'xlsx'].map((extension) =>
PIPELINE_PARSED_MIME_TYPES.get(extension)
)
)
expect(
result.documents.slice(1).every((doc) => doc.contentDeferred && doc.content === '')
).toBe(true)
Expand Down Expand Up @@ -364,7 +446,7 @@ describe('Confluence attachment hydration', () => {
}
)

it.each(['pdf', 'doc', 'docx'])(
it.each(['pdf', 'doc', 'docx', 'pptx', 'xlsx'])(
'hands an original %s file to the shared parser pipeline',
async (extension) => {
fixture(file({ title: `Guide.${extension}` }))
Expand Down Expand Up @@ -492,37 +574,30 @@ describe('Confluence attachment hydration', () => {
expect(secureDownload.mock.calls[0][1].signal).toBe(signal)
})

it.each(['pdf', 'docx'] as const)(
it.each(['pdf', 'docx', 'pptx', 'xlsx'] as const)(
'roundtrips genuine %s bytes through the public parser',
async (extension) => {
let bytes: Buffer
if (extension === 'pdf') {
const document = await PDFDocument.create()
const font = await document.embedFont(StandardFonts.Helvetica)
document.addPage().drawText('Confluence attachment text', { font, size: 14 })
bytes = Buffer.from(await document.save())
} else {
const zip = new JSZip()
zip.file(
'[Content_Types].xml',
'<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"><Default Extension="rels" ContentType="application/vnd.openxmlformats-package.relationships+xml"/><Override PartName="/word/document.xml" ContentType="application/vnd.openxmlformats-officedocument.wordprocessingml.document.main+xml"/></Types>'
)
zip.file(
'_rels/.rels',
'<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"><Relationship Id="rId1" Type="http://schemas.openxmlformats.org/officeDocument/2006/relationships/officeDocument" Target="word/document.xml"/></Relationships>'
)
zip.file(
'word/document.xml',
'<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"><w:body><w:p><w:r><w:t>Confluence attachment text</w:t></w:r></w:p></w:body></w:document>'
)
bytes = await zip.generateAsync({ type: 'nodebuffer' })
}
const bytes = await ROUNDTRIP_FIXTURES[extension]()
fixture(file({ title: `Guide.${extension}`, fileSize: bytes.length }))
secureDownload.mockResolvedValue(new Response(bytes))
const doc = await get()
expect(doc?.sourceFile).toBeDefined()
const parsed = await parseBuffer(doc!.sourceFile!.bytes, extension)
expect(parsed.content).toContain('Confluence attachment text')
expect(parsed.content).toContain(ROUNDTRIP_TEXT)
}
)

it.each(['Guide.ppt', 'Guide.xls'])(
'replaces an attachment renamed to unsupported %s without downloading',
async (title) => {
fixture(file({ title }))
const doc = await get()
expect(doc?.skippedReason).toBe(
'Attachment is no longer a PDF, Word, Excel or PowerPoint document'
)
expect(doc?.skippedExistingDisposition).toBe('replace')
expect(doc?.contentDeferred).toBe(false)
expect(secureDownload).not.toHaveBeenCalled()
}
)
})
Expand Down
12 changes: 9 additions & 3 deletions apps/sim/connectors/confluence/attachments.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,13 @@ const PAGE_SIZE = 50
const REQUESTS_PER_CALL = 5
const MAX_CURSOR_BYTES = 512 * 1024
const MAX_METADATA_BYTES = 2 * 1024 * 1024
const FILE_EXTENSIONS = new Set(['pdf', 'doc', 'docx'])
/**
* Attachment formats listed for indexing: a deliberate subset of the shared
* `PIPELINE_PARSED_MIME_TYPES`, limited to the headline PDF, Word, Excel and
* PowerPoint extensions. Macro-enabled, template, legacy binary (`.xls`, `.ppt`)
* and OpenDocument variants are not listed for Confluence.
*/
const FILE_EXTENSIONS = new Set(['pdf', 'doc', 'docx', 'pptx', 'xlsx'])
const boundedId = z.string().min(1).max(254)
const providerCursor = z.string().min(1).max(8192)
const parentSchema = z.object({ id: boundedId, type: z.enum(['page', 'blogpost']) })
Expand Down Expand Up @@ -406,7 +412,7 @@ function assertDownloadUrl(value: string): void {
}
}

/** Downloads a version-pinned original for the shared PDF/OCR and Word parsing pipeline. */
/** Downloads a version-pinned original for the shared PDF/OCR, Word, Excel and PowerPoint parsing pipeline. */
export async function getConfluenceAttachment(
input: AttachmentRequest,
sourceConfig: Record<string, unknown>,
Expand All @@ -420,7 +426,7 @@ export async function getConfluenceAttachment(
const mimeType = attachmentMimeType(attachment)
if (!mimeType) {
return {
...markSkipped(stub, 'Attachment is no longer a PDF or Word document'),
...markSkipped(stub, 'Attachment is no longer a PDF, Word, Excel or PowerPoint document'),
skippedExistingDisposition: 'replace',
}
}
Expand Down
Loading