diff --git a/tutorials/33_Hybrid_Retrieval.ipynb b/tutorials/33_Hybrid_Retrieval.ipynb index d5e27294..8adf23f9 100644 --- a/tutorials/33_Hybrid_Retrieval.ipynb +++ b/tutorials/33_Hybrid_Retrieval.ipynb @@ -5,7 +5,15 @@ "metadata": { "id": "kTas9ZQ7lXP7" }, - "source": "# Tutorial: Creating a Hybrid Retrieval Pipeline\n\n- **Level**: Intermediate\n- **Time to complete**: 15 minutes\n- **Components Used**: [`DocumentSplitter`](https://docs.haystack.deepset.ai/docs/documentsplitter), [`SentenceTransformersDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [`InMemoryDocumentStore`](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [`InMemoryBM25Retriever`](https://docs.haystack.deepset.ai/docs/inmemorybm25retriever), [`InMemoryEmbeddingRetriever`](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), and [`SentenceTransformersSimilarityRanker`](https://docs.haystack.deepset.ai/docs/sentencetransformerssimilarityranker)\n- **Prerequisites**: None\n- **Goal**: After completing this tutorial, you will have learned about creating a hybrid retrieval and when it's useful." + "source": [ + "# Tutorial: Creating a Hybrid Retrieval Pipeline\n", + "\n", + "- **Level**: Intermediate\n", + "- **Time to complete**: 15 minutes\n", + "- **Components Used**: [`DocumentSplitter`](https://docs.haystack.deepset.ai/docs/documentsplitter), [`SentenceTransformersDocumentEmbedder`](https://docs.haystack.deepset.ai/docs/sentencetransformersdocumentembedder), [`InMemoryDocumentStore`](https://docs.haystack.deepset.ai/docs/inmemorydocumentstore), [`InMemoryBM25Retriever`](https://docs.haystack.deepset.ai/docs/inmemorybm25retriever), [`InMemoryEmbeddingRetriever`](https://docs.haystack.deepset.ai/docs/inmemoryembeddingretriever), and [`SentenceTransformersSimilarityRanker`](https://docs.haystack.deepset.ai/docs/sentencetransformerssimilarityranker)\n", + "- **Prerequisites**: None\n", + "- **Goal**: After completing this tutorial, you will have learned about creating a hybrid retrieval and when it's useful." + ] }, { "cell_type": "markdown", @@ -38,7 +46,13 @@ "id": "L40ZxZW8lXQh" }, "outputs": [], - "source": "%%bash\n\npip install haystack-ai\npip install \"datasets>=2.6.1\"\npip install sentence-transformers-haystack" + "source": [ + "%%bash\n", + "\n", + "pip install haystack-ai\n", + "pip install \"datasets>=2.6.1\"\n", + "pip install sentence-transformers-haystack" + ] }, { "cell_type": "markdown", @@ -144,7 +158,7 @@ "\n", "document_splitter = DocumentSplitter(split_by=\"word\", split_length=512, split_overlap=32)\n", "document_embedder = SentenceTransformersDocumentEmbedder(\n", - " model=\"BAAI/bge-small-en-v1.5\", device=ComponentDevice.from_str(\"cuda:0\")\n", + " model=\"BAAI/bge-small-en-v1.5\", device=ComponentDevice.from_str(\"cuda:0\") # remove the device argument if you are not using a GPU\n", ")\n", "document_writer = DocumentWriter(document_store)\n", "\n", @@ -194,7 +208,7 @@ }, { "cell_type": "code", - "execution_count": 7, + "execution_count": null, "metadata": { "id": "DVfQvnWYrMWr" }, @@ -204,7 +218,7 @@ "from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersTextEmbedder\n", "\n", "text_embedder = SentenceTransformersTextEmbedder(\n", - " model=\"BAAI/bge-small-en-v1.5\", device=ComponentDevice.from_str(\"cuda:0\")\n", + " model=\"BAAI/bge-small-en-v1.5\", device=ComponentDevice.from_str(\"cuda:0\") # remove the device argument if you are not using a GPU\n", ")\n", "embedding_retriever = InMemoryEmbeddingRetriever(document_store)\n", "bm25_retriever = InMemoryBM25Retriever(document_store)" @@ -215,7 +229,11 @@ "metadata": { "id": "r8_jHzmosbC_" }, - "source": "### 2) Rank the Results\n\nUse the [SentenceTransformersSimilarityRanker](https://docs.haystack.deepset.ai/docs/sentencetransformerssimilarityranker) that scores the relevancy of all retrieved documents for the given search query by using a cross encoder model. In this example, you will use [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) model to rank the retrieved documents but you can replace this model with other cross-encoder models on Hugging Face." + "source": [ + "### 2) Rank the Results\n", + "\n", + "Use the [SentenceTransformersSimilarityRanker](https://docs.haystack.deepset.ai/docs/sentencetransformerssimilarityranker) that scores the relevancy of all retrieved documents for the given search query by using a cross encoder model. In this example, you will use [BAAI/bge-reranker-base](https://huggingface.co/BAAI/bge-reranker-base) model to rank the retrieved documents but you can replace this model with other cross-encoder models on Hugging Face." + ] }, { "cell_type": "code", @@ -224,7 +242,11 @@ "id": "cN0woIxHs4Ng" }, "outputs": [], - "source": "from haystack_integrations.components.rankers.sentence_transformers import SentenceTransformersSimilarityRanker\n\nranker = SentenceTransformersSimilarityRanker(model=\"BAAI/bge-reranker-base\")" + "source": [ + "from haystack_integrations.components.rankers.sentence_transformers import SentenceTransformersSimilarityRanker\n", + "\n", + "ranker = SentenceTransformersSimilarityRanker(model=\"BAAI/bge-reranker-base\")" + ] }, { "cell_type": "markdown", diff --git a/tutorials/39_Embedding_Metadata_for_Improved_Retrieval.ipynb b/tutorials/39_Embedding_Metadata_for_Improved_Retrieval.ipynb index 178b0cff..93df293c 100644 --- a/tutorials/39_Embedding_Metadata_for_Improved_Retrieval.ipynb +++ b/tutorials/39_Embedding_Metadata_for_Improved_Retrieval.ipynb @@ -51,7 +51,11 @@ "execution_count": null, "metadata": {}, "outputs": [], - "source": "%%bash\n\npip install haystack-ai wikipedia sentence-transformers-haystack" + "source": [ + "%%bash\n", + "\n", + "pip install haystack-ai wikipedia sentence-transformers-haystack" + ] }, { "cell_type": "markdown", @@ -83,7 +87,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": null, "metadata": {}, "outputs": [], "source": [ @@ -92,8 +96,6 @@ "from haystack_integrations.components.embedders.sentence_transformers import SentenceTransformersDocumentEmbedder\n", "from haystack.components.writers import DocumentWriter\n", "from haystack.document_stores.types import DuplicatePolicy\n", - "from haystack.utils import ComponentDevice\n", - "\n", "\n", "def create_indexing_pipeline(document_store, metadata_fields_to_embed=None):\n", " document_cleaner = DocumentCleaner()\n",