Skip to content

citations.md

Latest commit

 

History

History
197 lines (142 loc) · 6.91 KB

citations.md

File metadata and controls

197 lines (142 loc) · 6.91 KB
Raw

Module — Citation System

Location: app/core/citations/

The citation system is a core integrity guarantee: Studymation never fabricates academic citations. If no relevant source is found, it declares found=False with rejection reasons rather than inventing a reference.

Architecture Overview

flowchart TD
    SECT[Section content + topic] --> NQ[Normalize query\ntopic_cleaner.py]
    NQ --> HASH[Hash query\nnormalizer.py]
    HASH --> CACHE{Cache hit?}
    CACHE -->|Yes| REVAL[Revalidate against\ncurrent section context]
    CACHE -->|No| SEARCH

    SEARCH[Parallel provider search]
    SEARCH --> SS[Semantic Scholar]
    SEARCH --> CR[Crossref]
    SEARCH --> BR[Brave Search]

    SS & CR & BR --> DEDUP[Deduplicate by DOI/URL]
    DEDUP --> VAL[CitationRelevanceValidator\nJaccard similarity]
    VAL --> RANK[CitationRanker\nScore + sort]
    RANK --> FILTER{score ≥ min_score\nrelevance ≥ min_relevance}
    FILTER -->|Pass| ACCEPT[found=True\nused=True]
    FILTER -->|Fail| REJECT[found=False\nrejection_reason logged]
    REVAL --> FILTER

    ACCEPT --> FORMAT[APA7 formatter]
    FORMAT --> CACHE_STORE[Store in citation_cache]
    FORMAT --> AUDIT[Citation audit record]
Loading

Components

pipeline.py — CitationPipeline

The entry point. find_citations_for_section(section_title, section_content, document_topic) orchestrates the full flow.

  1. Normalizes the section query via topic_cleaner.py and normalizer.py
  2. Computes SHA-256 query_hash for cache lookup
  3. Checks citation_cache table — if hit, revalidates against current section
  4. On cache miss: queries all providers in parallel
  5. Deduplicates candidates by DOI (preferred) or URL
  6. Scores each candidate via CitationRelevanceValidator
  7. Ranks by combined score via CitationRanker
  8. Filters to top CITATION_MAX_CANDIDATES_PER_SECTION (default 12)
  9. Stores accepted citations in citation_cache
  10. Returns CitationResult with found, citations, rejected, audit

semantic_scholar.py

Queries the Semantic Scholar Academic Graph API. Returns title, authors, year, DOI, abstract, and citation count. Retries with exponential backoff on 429/5xx.

Requires SEMANTIC_SCHOLAR_API_KEY for higher rate limits. Works without key at lower quota.

crossref.py

Queries Crossref REST API. Uses mailto= parameter (best practice for polite pool). Returns title, DOI, authors, publication date.

brave_search.py

Uses Brave Search API to find academic web content. Acts as a fallback when structured academic databases have no results. Lower signal-to-noise than Semantic Scholar/Crossref.

validator.py — CitationRelevanceValidator

Computes Jaccard similarity between:

  • The citation's abstract (or title if no abstract)
  • The section's content

Threshold: CITATION_MIN_RELEVANCE (default 0.16). Citations below this threshold are rejected with reason relevance_below_threshold.

ranker.py — CitationRanker

Combines multiple signals into a final score:

  • Jaccard similarity (primary signal)
  • DOI presence (quality signal — indicates peer-reviewed source)
  • Publication year (recency bonus for newer papers)
  • Citation count from Semantic Scholar (popularity signal)

Final score must exceed CITATION_MIN_SCORE (default 0.50) for inclusion.

formatter.py

Formats citations in APA7 style. Handles:

  • Multiple authors (et al. threshold)
  • Missing author (uses source name)
  • Missing date (uses "n.d.")
  • DOI URL formatting (https://doi.org/...)

normalizer.py

Normalizes query strings to produce consistent cache keys:

  • Lowercases
  • Strips punctuation
  • Removes extra whitespace
  • Produces SHA-256 hash

topic_cleaner.py

Pre-processes section content before query construction:

  • Removes Spanish/English stopwords
  • Extracts key terms (nouns, named entities)
  • Limits query length for API compatibility

models.py

Data classes used throughout the citation system:

@dataclass
class CitationCandidate:
    title: str
    authors: list[str]
    year: int | None
    doi: str | None
    abstract: str | None
    url: str | None
    source: str           # "semantic_scholar" | "crossref" | "brave"
    citation_count: int = 0

@dataclass
class AcceptedCitation(CitationCandidate):
    relevance_score: float
    final_score: float
    apa7: str
    used: bool = True     # Only True citations appear in bibliography

@dataclass
class CitationResult:
    section_title: str
    found: bool
    citations: list[AcceptedCitation]
    rejected_candidates: list[dict]   # {title, rejection_reason}
    cache_hit: bool

Citation Cache

citation_cache table stores validated citation results keyed by query_hash.

TTL: CITATION_CACHE_TTL_DAYS (default 30 days). After expiry, the cache entry is ignored and a fresh search is performed.

Revalidation on cache hit: A cached citation is NOT blindly reused. Its relevance is re-scored against the current section content (which may differ from when the cache entry was created). This prevents a citation found for section A from being inserted into section B.

Cleanup: Expired entries are removed by core/storage/ttl_manager.py during maintenance runs.

Configuration Reference

Variable Default Effect
CITATION_MIN_SCORE 0.50 Minimum combined score for citation inclusion
CITATION_MIN_RELEVANCE 0.16 Minimum Jaccard similarity (primary filter)
CITATION_MAX_CANDIDATES_PER_SECTION 12 Max candidates scored per section
CITATION_CACHE_TTL_DAYS 30 Cache entry lifetime
CITATION_MIN_COUNT 2 Minimum citations per document (Quality Gate)
SEMANTIC_SCHOLAR_API_KEY — Higher rate limits
BRAVE_API_KEY — Required for Brave provider
CITATION_PROVIDER_TIMEOUT_SECONDS 10 Per-provider API timeout

Audit Trail

Every citation decision (accepted or rejected) is recorded in PipelineContext.citations and stored in Document.metadata_["citation_audit"]. This audit is exposed via GET /documents/{id}/citation-audit.

The audit contains:

  • Per-section: all candidates evaluated, their scores, acceptance/rejection reason
  • Global: total found, total rejected, providers queried, cache hits

This audit is the primary debugging tool when citation quality is questioned.

Failure Modes

Scenario Behavior
All providers time out CitationResult.found = False, warning added to context
Score below threshold Rejected with score_below_threshold reason
Relevance below threshold Rejected with relevance_below_threshold reason
Cache hit but revalidation fails Cache entry ignored, fresh search performed
Document ends with < CITATION_MIN_COUNT QualityGate triggers P0 failure or warning

The system NEVER:

  • Generates a citation from an LLM hallucination
  • Inserts a citation without a real, validated source
  • Reuses a citation across sections without re-scoring relevance