feat: Index associations as nested child documents and search them with block joins

README.md

@@ -1028,6 +1028,71 @@ end
 # qRss91753840: _query_:"{!field f=type}Rss"+_query_:"{!edismax qf='keywords_text'}keyword3"
 ```
 
+### Nested Documents (Block Joins)
+
+**Solr 8 and above recommended; requires RSolr 2**
+
+Nested documents let a search require that a *single* associated record meet several conditions together. Indexing an association's fields as multivalued fields on the parent loses which value came from which record: a project with one milestone named "design" and another started in Q1 would match "a design milestone started in Q1". `nested` indexes each record of an association as a Solr child document inside its parent's block, and `with_child` / `without_child` search them with Solr's [block join query parsers](https://solr.apache.org/guide/solr/latest/query-guide/block-join-query-parser.html).
+
+```ruby
+class Project < ActiveRecord::Base
+  has_many :milestones
+
+  searchable do
+    string :status
+
+    nested :milestones do
+      string :name
+      time :started_at
+      string(:owner_names, :multiple => true) { owners.map(&:name) }
+    end
+  end
+end
+
+# Projects with a design milestone started in Q1. One milestone has to match
+# both conditions.
+Project.search do
+  with_child :milestones do
+    with :name, 'design'
+    with(:started_at).between(Time.utc(2026, 1, 1)...Time.utc(2026, 4, 1))
+  end
+end
+
+# Projects with no design milestone, including projects with no milestones
+Project.search do
+  without_child :milestones do
+    with :name, 'design'
+  end
+end
+
+# Projects with any milestone at all
+Project.search { with_child :milestones }
+```
+
+The block inside `with_child` / `without_child` takes the same restrictions as a search, including `without`, `any_of`, `all_of`, `dynamic` and `text_fields`, with field names referring to the children's fields. `with_child` combines with restrictions on the parent and works inside `any_of`, query facets and `remove`.
+
+Searching several classes covers each class that declares the association, and a child field resolves as parent fields do: when every class declaring it agrees on its type.
+
+#### Schema
+
+The schema needs a `_root_` field with the same type as `id`. The bundled configset has one.
+
+```xml
+<field name="_root_" type="string" indexed="true" stored="false" multiValued="false"/>
+```
+
+Adding `_root_` to a core that already holds documents needs a full reindex of every class on Solr 8 and above; see [Adding `_root_` to an existing core](#adding-_root_-to-an-existing-core).
+
+#### Things to know
+
+* Children are indexed only as part of their parent, so reindex the parent whenever its children change.
+* Reindexing a parent replaces its whole block on Solr 8 and above. On earlier versions, including the Solr that `sunspot_solr` bundles, a parent whose children went from some to none keeps its old children, and one whose children went from none to some is indexed twice. Remove such a parent before reindexing it there.
+* Atomic updates raise `ArgumentError` for a class with nested associations. Index the whole record instead.
+* Removing a record of a nested class also sends a delete-by-query on `_root_`, because Solr before 8 leaves the children behind on a delete by id. Solr 8 and above delete them anyway, so there it's an extra request per removal.
+* A nested block can't declare a boost, an id prefix, a join or a nested association of its own, and `with(record)` / `without(record)` inside `with_child` raise `ArgumentError`.
+* Declaring an association again in the same class adds to its fields. A subclass that declares an inherited association again replaces its fields. A search of the superclass still reaches the subclass's children, but only through fields both declare the same way.
+* A child document's id is `"<parent id>/<association>/<child id>"`, where the child id is its Sunspot index id, or its position in the association when its class has no Sunspot adapter. Children carry a `_sunspot_nested_path_s` field naming their association and no `type`, so searches for the parent class never return them.
+
 ### Composite ID
 
 **SolrCloud only**
@@ -1678,6 +1743,10 @@ solr:
 where the `./solr/init` directory contains a shell script that does any initial setup like downloading and unzipping your cores.
 In both cases, the solr images by default expects cores to be placed in `/opt/solr/server/solr/mycores`.
 
+### Adding `_root_` to an existing core
+
+The bundled configset and `examples/solr7_core` define a `_root_` field, which [nested documents](#nested-documents-block-joins) need. If you bring an existing core's schema up to date with them on Solr 8 or above, the change affects every class, nested or not. Once the schema has `_root_`, Solr replaces and deletes documents by their `_root_` value, and documents indexed before the field was added have none. Reindexing one of them adds a new copy beside the old one, later reindexes replace only the new copy, and removing it by id leaves the old copy in place. The old copies stay until they're deleted, so clear and reindex every class (`rake sunspot:reindex`, which deletes each class's documents first) right after adding the field.
+
 ## Development
 
 ### Running Tests

examples/solr7_core/conf/schema.xml

@@ -133,6 +133,9 @@
     <field name="type" stored="false" type="string" multiValued="true" indexed="true"/>
     <!-- *** This field is used by Sunspot! *** -->
     <field name="class_name" stored="false" type="string" multiValued="false" indexed="true"/>
+    <!-- *** This field is used by Sunspot's nested documents! *** -->
+    <!-- Solr fills it with the id of the top document in each block. -->
+    <field name="_root_" stored="false" type="string" multiValued="false" indexed="true"/>
     <!-- *** This field is used by Sunspot! *** -->
     <field name="text" stored="false" type="string" multiValued="true" indexed="true"/>
     <!-- *** This field is used by Sunspot! *** -->

sunspot/lib/sunspot.rb

@@ -12,7 +12,7 @@
 
 require File.join(File.dirname(__FILE__), 'light_config')
 
-%w(util adapters configuration setup composite_setup text_field_setup field
+%w(util adapters configuration setup nested_setup composite_setup composite_nested_setup text_field_setup field
    field_factory data_extractor indexer query search session session_proxy
    type dsl class_set).each do |filename|
   require File.join(File.dirname(__FILE__), 'sunspot', filename)
@@ -38,6 +38,7 @@ module Sunspot
   UnrecognizedRestrictionError = Class.new(StandardError)
   NoAdapterError = Class.new(StandardError)
   NoSetupError = Class.new(StandardError)
+  NestedDocumentsNotSupportedError = Class.new(StandardError)
   IllegalSearchError = Class.new(StandardError)
   NotImplementedError = Class.new(StandardError)
   AtomicUpdateRequireInstanceForCompositeIdMessage = lambda do |class_name|

sunspot/lib/sunspot/composite_nested_setup.rb

@@ -0,0 +1,74 @@
+module Sunspot
+  #
+  # The fields of one association's children across the NestedSetups of the
+  # classes in a multi-class search. A field resolves when every setup that
+  # declares it agrees on its Solr field name and type, as with CompositeSetup,
+  # and raises UnrecognizedFieldError when none declares it or they declare it
+  # differently.
+  #
+  class CompositeNestedSetup #:nodoc:
+    def initialize(nested_setups)
+      @nested_setups = nested_setups
+    end
+
+    def field(field_name)
+      fields = @nested_setups.map do |setup|
+        begin
+          setup.field(field_name)
+        rescue UnrecognizedFieldError
+          nil
+        end
+      end.compact.uniq { |field| [field.indexed_name, field.type.class] }
+      return fields.first if fields.one?
+
+      raise(
+        UnrecognizedFieldError,
+        fields.empty? ? "No field configured for #{paths} with name '#{field_name}'" :
+          "Field '#{field_name}' is configured differently for #{paths}"
+      )
+    end
+
+    # Returns the text fields with the given name, one per distinct Solr field.
+    # Raises UnrecognizedFieldError when no setup declares it. TextFieldSetup
+    # raises when there is more than one.
+    def text_fields(field_name)
+      fields = @nested_setups.flat_map do |setup|
+        begin
+          setup.text_fields(field_name)
+        rescue UnrecognizedFieldError
+          []
+        end
+      end.uniq(&:indexed_name)
+      return fields if fields.any?
+
+      raise UnrecognizedFieldError, "No text field configured for #{paths} with name '#{field_name}'"
+    end
+
+    def type_names
+      @nested_setups.map(&:path).uniq
+    end
+
+    # Returns the dynamic field factory with the given base name. Unlike
+    # #field, it requires every setup to declare it, as CompositeSetup does
+    # for parents' dynamic fields. Raises UnrecognizedFieldError when one
+    # doesn't, or when they build different Solr fields.
+    def dynamic_field_factory(field_name)
+      factories = @nested_setups.map { |setup| setup.dynamic_field_factory(field_name) }
+      return factories.first if factories.map { |factory| factory.build('x').indexed_name }.uniq.one?
+
+      raise UnrecognizedFieldError, "Dynamic field '#{field_name}' is configured differently for #{paths}"
+    end
+
+    def nested_setups_named(name)
+      raise UnrecognizedFieldError, "No nested association configured for #{paths} with name '#{name}'"
+    end
+
+    alias_method :nested_setup, :nested_setups_named
+
+    private
+
+    def paths
+      type_names * ', '
+    end
+  end
+end

sunspot/lib/sunspot/composite_setup.rb

@@ -101,6 +101,40 @@ def dynamic_field_factory(field_name)
       )
     end
 
+    #
+    # Returns the NestedSetup of each enclosed type that declares the given
+    # association. Raises UnrecognizedFieldError when none does.
+    #
+    def nested_setups_named(name)
+      nested_setups = setups.flat_map do |setup|
+        begin
+          setup.nested_setups_named(name)
+        rescue UnrecognizedFieldError
+          []
+        end
+      end.uniq
+      nested_setups.empty? ? [nested_setup(name)] : nested_setups
+    end
+
+    #
+    # Returns the NestedSetup for the given association from the first of the
+    # searched types that declares it. Raises UnrecognizedFieldError when none
+    # does.
+    #
+    def nested_setup(name)
+      setups.each do |setup|
+        begin
+          return setup.nested_setup(name)
+        rescue UnrecognizedFieldError
+          next
+        end
+      end
+      raise(
+        UnrecognizedFieldError,
+        "No nested association configured for #{@types * ', '} with name '#{name}'"
+      )
+    end
+
     # 
     # Collection of all text fields configured for any of the enclosed types.
     #

sunspot/lib/sunspot/dsl/fields.rb

@@ -71,6 +71,54 @@ def id_prefix(attr_name = nil, &block)
         @setup.add_id_prefix(attr_name, &block)
       end
 
+      #
+      # Indexes the records of an association as child documents of this
+      # class's documents, in the same Solr block. Fields declared in the block
+      # are indexed on each child, and a field's block is evaluated against the
+      # child record. Use DSL::Scope#with_child to find parents by conditions
+      # that a single child must meet together.
+      #
+      # Children are indexed only as part of their parent, so the parent must
+      # be reindexed whenever its children change. Atomic updates to a class
+      # with nested associations raise ArgumentError.
+      #
+      # Reindexing a parent replaces its old block only on Solr 8 or later.
+      # Solr before 8 replaces a document with children by +_root_+ and one
+      # without by +id+, so a parent whose children went from some to none
+      # keeps its old children, and one whose children went from none to some
+      # is indexed twice. Remove such a parent before reindexing it there.
+      #
+      # Declaring an association again in the same class adds the block's
+      # fields to it. A subclass that declares an inherited association again
+      # replaces its fields: its children are indexed with only the fields of
+      # its own block. They keep the superclass's association marker, so a
+      # search of the superclass still reaches them, but only through fields
+      # both blocks declare the same way.
+      #
+      # The block cannot declare a document boost, an id prefix, a join, or a
+      # nested association of its own. Each raises ArgumentError.
+      #
+      # ==== Parameters
+      #
+      # name<Symbol>:: The association, called on the parent to get the children
+      #
+      # ==== Options
+      #
+      # :using<Symbol>:: Method to call on the parent instead of +name+
+      #
+      # ==== Example
+      #
+      #   Sunspot.setup(Project) do
+      #     nested :milestones do
+      #       string :name
+      #       time :started_at
+      #     end
+      #   end
+      #
+      def nested(name, options = {}, &block)
+        @setup.add_nested(name, options, &block)
+      end
+
       # method_missing is used to provide access to typed fields, because
       # developers should be able to add new Sunspot::Type implementations
       # dynamically and have them recognized inside the Fields DSL. Like #text,
@@ -119,5 +167,28 @@ def method_missing(method, *args, &block)
         end
       end
     end
+
+    #
+    # The fields DSL inside a DSL::Fields#nested block. Rejects document
+    # boosts, id prefixes, joins, and nested associations with ArgumentError.
+    #
+    class NestedFields < Fields #:nodoc:
+      def boost(*)
+        raise ArgumentError, "Document boosts are not supported on nested documents"
+      end
+
+      def id_prefix(*)
+        raise ArgumentError, "ID prefixes are not supported on nested documents, which take their parent's"
+      end
+
+      def nested(*)
+        raise ArgumentError, "Nested documents cannot have nested documents of their own"
+      end
+
+      def method_missing(method, *args, &block)
+        raise ArgumentError, "Joins are not supported on nested documents" if method.to_s == 'join'
+        super
+      end
+    end
   end
 end

sunspot/lib/sunspot/dsl/scope.rb

@@ -97,6 +97,50 @@ def without(*args)
         add_restriction(true, *args)
       end
 
+      #
+      # Scope the results to documents with at least one child in the given
+      # association that satisfies every restriction in the block. Without a
+      # block, any child in the association matches. The association is one
+      # declared with DSL::Fields#nested.
+      #
+      # The block takes the same restrictions as a scope, with field names
+      # referring to the child's fields. Restricting by instance, as in
+      # <tt>with(record)</tt>, raises ArgumentError, and a nested #with_child
+      # raises UnrecognizedFieldError.
+      #
+      # In a search of several classes, the block matches the children of
+      # every searched class that declares the association.
+      #
+      # ==== Example
+      #
+      #   Sunspot.search(Project) do
+      #     with_child :milestones do
+      #       with :name, 'design'
+      #       with(:started_at).between(Time.utc(2026, 1, 1)...Time.utc(2026, 4, 1))
+      #     end
+      #   end
+      #
+      def with_child(name, &block)
+        add_block_join(false, name, &block)
+      end
+
+      #
+      # Scope the results to documents with no child in the given association
+      # that satisfies every restriction in the block. Documents with no
+      # children in the association match. Without a block, only those match.
+      #
+      # ==== Example
+      #
+      #   Sunspot.search(Project) do
+      #     without_child :milestones do
+      #       with :name, 'launch'
+      #     end
+      #   end
+      #
+      def without_child(name, &block)
+        add_block_join(true, name, &block)
+      end
+
       #
       # Create a disjunction, scoping the results to documents that match any
       # of the enclosed restrictions.
@@ -197,6 +241,14 @@ def text_fields(&block)
 
       private
 
+      def add_block_join(negated, name, &block)
+        nested_setups = @setup.nested_setups_named(name)
+        child_setup = nested_setups.one? ? nested_setups.first : CompositeNestedSetup.new(nested_setups)
+        block_join = Sunspot::Query::BlockJoin.new(nested_setups.first, negated, nil, nested_setups.map(&:path).uniq)
+        Util.instance_eval_or_call(Scope.new(block_join.scope, child_setup), &block) if block
+        @scope.add_component(block_join)
+      end
+
       def add_restriction(negated, *args)
         case args.first
         when String, Symbol
@@ -209,6 +261,9 @@ def add_restriction(negated, *args)
             DSL::Restriction.new(field, @scope, negated)
           end
         else # args are instances
+          if @setup.is_a?(NestedSetup) || @setup.is_a?(CompositeNestedSetup)
+            raise ArgumentError, "Instance restrictions are not supported inside with_child or without_child. Restrict on the child's fields instead"
+          end
           @scope.add_restriction(
             negated,
             IdField.instance,