feat!: migrate Codex provider to RubyLLM 2.1

README.md

@@ -19,8 +19,8 @@ supported mapping but not an identical wire-level request.
 ## Requirements
 
 - Ruby 3.3 or newer
-- RubyLLM 1.16.0 or newer, below 2.0
-- Codex CLI 0.144.5 or newer, available as `codex`
+- RubyLLM 2.1.1 or newer, below 2.2
+- Codex CLI 0.162.1 or newer, available as `codex`
 - Linux or macOS (timeout cleanup relies on POSIX process groups)
 - A completed ChatGPT login: `codex login`
 
@@ -34,6 +34,8 @@ codex login
 Check the model list in your installed Codex version before a large run. Model
 availability depends on the account and currently installed CLI.
 
+For migration steps from RubyLLM 1.x, see [UPGRADE.md](UPGRADE.md).
+
 ## Installation
 
 Add the gem to your application's `Gemfile`:
@@ -55,7 +57,7 @@ Require the gem and use Codex as an explicit RubyLLM provider:
 ```ruby
 require "ruby_llm-codex"
 
-class SummarySchema < RubyLLM::Schema
+class SummarySchema < Schematist::Schema
   string :summary
 end
 
@@ -65,11 +67,11 @@ response = RubyLLM.chat(model: ENV.fetch("CODEX_MODEL"), provider: :codex)
   .with_thinking(effort: :medium)
   .ask("The text to summarize")
 
-pp response.content
+pp response.parsed
 ```
 
-`response.content` is normalized by RubyLLM into the same Hash shape returned
-for structured outputs from its other providers.
+`response.content` contains the JSON string. Read `response.parsed` for the
+structured Hash, and `response.tokens` for token counts.
 
 ## Isolation and reproducibility
 
@@ -97,7 +99,7 @@ saved session history from quietly affecting a request.
 To deliberately let Codex inspect a repository or use selected configuration:
 
 ```ruby
-chat.with_params(
+chat.with_provider_options(
   codex: {
     working_directory: "/path/to/project",
     sandbox: "read-only",
@@ -126,8 +128,8 @@ RubyLLM.configure do |config|
 end
 ```
 
-Only the nested `codex` namespace is accepted by `with_params`; unsupported
-generic params raise an error instead of being silently ignored. Use
+Only the nested `codex` namespace is accepted by `with_provider_options`; unsupported
+generic provider options raise an error instead of being silently ignored. Use
 `with_thinking` for reasoning effort and `with_instructions` for developer
 instructions. Those settings, plus `features.shell_snapshot` and
 `shell_environment_policy.inherit`, are reserved from the arbitrary

Rakefile

@@ -163,7 +163,7 @@ namespace :codex do
 
     marker = "RUBY_LLM_CODEX_SMOKE_OK"
     plain = RubyLLM.chat(model:, provider: :codex)
-      .with_params(**request_options)
+      .with_provider_options(**request_options)
       .with_thinking(effort: :low)
       .ask("Reply with exactly #{marker} and nothing else.")
     raise "Plain-text smoke test returned #{plain.content.inspect}" unless plain.content.strip == marker
@@ -179,11 +179,11 @@ namespace :codex do
       }
     }
     structured = RubyLLM.chat(model:, provider: :codex)
-      .with_params(**request_options)
+      .with_provider_options(**request_options)
       .with_schema(schema)
       .with_thinking(effort: :low)
       .ask("Return the requested smoke-test status.")
-    unless structured.content == {"status" => "ok"}
+    unless structured.parsed == {"status" => "ok"}
       raise "Structured-output smoke test returned #{structured.content.inspect}"
     end
 

UPGRADE.md

@@ -0,0 +1,28 @@
+# Upgrade Guide
+
+## RubyLLM 2.1
+
+This provider now requires RubyLLM `~> 2.1.1` and Codex CLI 0.162.1 or newer.
+RubyLLM 1.x and 2.0 are no longer supported. Upgrade the CLI with
+`npm install -g @openai/codex` and check `codex --version`.
+
+Host applications must:
+
+- Replace `with_params(codex: ...)` with `with_provider_options(codex: ...)`.
+- Replace `RubyLLM::Schema` with `Schematist::Schema`.
+- Read structured results through `response.parsed`; `response.content` is
+  now a JSON string.
+- Read the model ID from `response.model` and usage from `response.tokens`:
+  `input`, `output`, `cache_read`, and `thinking`. Input excludes cached tokens;
+  output includes thinking tokens. Unreported counts remain `nil`.
+
+Plain Ruby consumers can update directly after adapting these calls. Rails
+hosts using RubyLLM persistence must complete the
+[2.0 upgrade](https://github.com/crmne/ruby_llm/blob/v2.0.0/docs/_reference/upgrading.md)
+on 2.0, including cleanup, before following the
+[2.1 upgrade](https://rubyllm.com/upgrading/). On Rails below 8.1.4, pin `json`
+to `< 3` before updating.
+
+Each Codex invocation records successful or failed usage attempts in RubyLLM.
+Timeouts cover process exit and pipe draining, including descendants that keep
+output pipes open after the CLI exits. Process-group cleanup remains bounded.

lib/ruby_llm-codex.rb

@@ -1,7 +1,6 @@
 # frozen_string_literal: true
 
 require "ruby_llm"
-require "ruby_llm/schema"
 require_relative "ruby_llm/codex/version"
 require_relative "ruby_llm/providers/codex"
 

lib/ruby_llm/providers/codex.rb

@@ -13,14 +13,16 @@ module Providers
     # This provider is deliberately narrow: non-streaming text generation and
     # structured output are supported; RubyLLM tools and attachments are not.
     class Codex < Provider
+      protocol :codex, ::RubyLLM::Protocol
+
       class Error < RubyLLM::Error; end
       class TimeoutError < Error; end
       class UnsupportedFeatureError < Error; end
 
       DEFAULT_SANDBOX = "read-only"
       DEFAULT_TIMEOUT = 300
       DEFAULT_SHELL_ENVIRONMENT_INHERIT = "none"
-      MINIMUM_CLI_VERSION = "0.144.5"
+      MINIMUM_CLI_VERSION = "0.162.1"
       RESERVED_CONFIG_KEYS = %w[
         developer_instructions
         features.shell_snapshot
@@ -37,11 +39,23 @@ def headers
       end
 
       # rubocop:disable Metrics/ParameterLists
-      def complete(messages, tools:, temperature:, model:, params: {}, headers: {},
-        schema: nil, thinking: nil, tool_prefs: nil, &stream)
+      def complete(messages, tools:, temperature:, model:, provider_options: {}, headers: {},
+        schema: nil, thinking: nil, tool_prefs: nil, max_output_tokens: nil,
+        citations: false, caching: nil, protocol: nil, before_request: [],
+        usage_recorder: nil, provider_tools: [], compaction: nil, end_user: nil, &stream)
         validate_request!(tools:, temperature:, headers:, stream:)
 
-        options = codex_options(params)
+        unsupported = {
+          max_output_tokens:, citations: citations || nil, caching:,
+          provider_tools: provider_tools.empty? ? nil : provider_tools,
+          compaction:, end_user:, tool_prefs: (tool_prefs if tool_prefs&.values&.any?)
+        }.compact
+        unless unsupported.empty?
+          raise UnsupportedFeatureError,
+            "Unsupported Codex request options: #{unsupported.keys.join(", ")}"
+        end
+
+        options = codex_options(provider_options)
         developer_instructions, prompt = build_prompt(messages)
         request = {
           model: model.id,
@@ -51,9 +65,22 @@ def complete(messages, tools:, temperature:, model:, params: {}, headers: {},
           thinking:
         }
 
+        before_request.each { |hook| hook.call(request) }
+        tracker = Accounting::Usage::Tracker.new(
+          operation: :chat, provider: self, model:, config: @config, on_finish: usage_recorder
+        )
         with_working_directory(options) do |working_directory|
-          run_codex(request, working_directory:, options:)
+          ensure_cli_compatible!(working_directory, options)
+          tracker.start
+          response = run_codex(request, working_directory:, options:)
+          tracker.succeed(response)
         end
+      rescue Errno::ENOENT => error
+        tracker&.fail_pending(error)
+        raise Error, "Could not execute Codex CLI: #{error.message}. Install it or set codex_cli_path."
+      rescue => error
+        tracker&.fail_pending(error)
+        raise
       end
       # rubocop:enable Metrics/ParameterLists
 
@@ -109,11 +136,11 @@ def validate_request!(tools:, temperature:, headers:, stream:)
           "Per-request HTTP headers do not apply to the local Codex provider"
       end
 
-      def codex_options(params)
-        validate_params!(params)
-        raw = params.is_a?(Hash) ? (params[:codex] || params["codex"] || {}) : {}
+      def codex_options(provider_options)
+        validate_provider_options!(provider_options)
+        raw = provider_options.is_a?(Hash) ? (provider_options[:codex] || provider_options["codex"] || {}) : {}
         unless raw.is_a?(Hash)
-          raise ArgumentError, "Codex params must be a Hash"
+          raise ArgumentError, "Codex provider_options must be a Hash"
         end
 
         raw = raw.transform_keys(&:to_sym)
@@ -144,16 +171,16 @@ def codex_options(params)
         }
       end
 
-      def validate_params!(params)
-        unless params.is_a?(Hash)
-          raise ArgumentError, "RubyLLM params must be a Hash"
+      def validate_provider_options!(provider_options)
+        unless provider_options.is_a?(Hash)
+          raise ArgumentError, "RubyLLM provider_options must be a Hash"
         end
 
-        unsupported = params.keys.reject { |key| key.to_s == "codex" }
+        unsupported = provider_options.keys.reject { |key| key.to_s == "codex" }
         return if unsupported.empty?
 
         raise UnsupportedFeatureError,
-          "Unsupported params for the local Codex provider: #{unsupported.map(&:inspect).join(", ")}"
+          "Unsupported provider_options for the local Codex provider: #{unsupported.map(&:inspect).join(", ")}"
       end
 
       def validate_config!(config)
@@ -168,7 +195,7 @@ def timeout_option(raw)
         value = raw.key?(:timeout) ? raw[:timeout] : @config.codex_timeout
         value = DEFAULT_TIMEOUT if value.nil?
         timeout = Float(value)
-        raise ArgumentError, "Codex timeout must be greater than zero" unless timeout.positive?
+        raise ArgumentError, "Codex timeout must be greater than zero" unless timeout.finite? && timeout.positive?
 
         timeout
       rescue TypeError, ArgumentError
@@ -200,8 +227,6 @@ def with_working_directory(options)
       end
 
       def run_codex(request, working_directory:, options:)
-        ensure_cli_compatible!(working_directory, options)
-
         Tempfile.create(["ruby-llm-codex-output-", ".txt"]) do |output_file|
           with_schema_file(request[:schema]) do |schema_path|
             output_file.close
@@ -295,7 +320,7 @@ def capture_codex(command, prompt:, working_directory:, options:)
           chdir: working_directory,
           pgroup: true
         )
-        threads << writer = Thread.new do
+        threads << Thread.new do
           stdin.write(prompt)
         rescue Errno::EPIPE, IOError
           nil
@@ -306,12 +331,15 @@ def capture_codex(command, prompt:, working_directory:, options:)
         threads << stderr_reader = Thread.new { stderr.read }
         threads.each { |thread| thread.report_on_exception = false }
 
-        unless wait_thread.join(options[:timeout])
-          terminate_process_group(wait_thread)
-          raise TimeoutError, "Codex CLI timed out after #{options[:timeout]} seconds"
+        deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + options[:timeout]
+        [wait_thread, *threads].each do |thread|
+          remaining = deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
+          unless thread.join([remaining, 0].max)
+            terminate_process_group(wait_thread, threads)
+            raise TimeoutError, "Codex CLI timed out after #{options[:timeout]} seconds"
+          end
         end
 
-        writer.join
         [stdout_reader.value, stderr_reader.value, wait_thread.value]
       ensure
         [stdin, stdout, stderr].compact.each do |io|
@@ -322,10 +350,13 @@ def capture_codex(command, prompt:, working_directory:, options:)
         threads.each { |thread| thread.kill if thread.alive? }
       end
 
-      def terminate_process_group(wait_thread)
+      def terminate_process_group(wait_thread, threads)
         Process.kill("TERM", -wait_thread.pid)
-        return if wait_thread.join(2)
-
+        deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + 2
+        [wait_thread, *threads].each do |thread|
+          remaining = deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
+          thread.join([remaining, 0].max)
+        end
         Process.kill("KILL", -wait_thread.pid)
         wait_thread.join
       rescue Errno::ESRCH, Errno::ECHILD
@@ -335,7 +366,7 @@ def terminate_process_group(wait_thread)
       def add_reasoning_config(command, thinking)
         return unless thinking
 
-        effort = thinking.respond_to?(:effort) ? thinking.effort : thinking
+        effort = thinking.effort
         if effort
           add_config(command, "model_reasoning_effort", effort.to_s)
           return
@@ -387,34 +418,37 @@ def parse_events(stdout)
       end
 
       def raise_invocation_error!(status, events, stderr)
-        event = events.reverse.find { |item| %w[turn.failed error].include?(item["type"]) }
+        event = events.reverse_each.find { |item| %w[turn.failed error].include?(item["type"]) }
         detail = event && (event["message"] || event["error"] || event).to_s
         detail = stderr.strip if detail.to_s.strip.empty?
         detail = "Codex exited with status #{status.exitstatus}" if detail.to_s.strip.empty?
         raise Error, detail
       end
 
       def final_agent_message(events)
-        event = events.reverse.find do |item|
+        event = events.reverse_each.find do |item|
           item["type"] == "item.completed" && item.dig("item", "type") == "agent_message"
         end
         event&.dig("item", "text")
       end
 
       def build_message(content, model, events, stderr)
-        usage = events.reverse.find { |event| event["type"] == "turn.completed" }&.fetch("usage", {}) || {}
+        usage = events.reverse_each.find { |event| event["type"] == "turn.completed" }&.fetch("usage", {}) || {}
         cached = integer_or_nil(usage["cached_input_tokens"])
         total_input = integer_or_nil(usage["input_tokens"])
         uncached_input = total_input && [total_input - cached.to_i, 0].max
 
         Message.new(
           role: :assistant,
           content: content,
-          model_id: model,
-          input_tokens: uncached_input,
-          cached_tokens: cached,
-          output_tokens: integer_or_nil(usage["output_tokens"]),
-          thinking_tokens: integer_or_nil(usage["reasoning_output_tokens"]),
+          model:,
+          tokens: Tokens.new(
+            input: uncached_input,
+            cache_read: cached,
+            output: integer_or_nil(usage["output_tokens"]),
+            thinking: integer_or_nil(usage["reasoning_output_tokens"])
+          ),
+          finish_reason: :stop,
           raw: {events:, stderr: stderr.empty? ? nil : stderr}
         )
       end
@@ -449,23 +483,12 @@ def build_prompt(messages)
       end
 
       def text_content(message)
-        content = message.content
-        return content if content.is_a?(String)
-
-        if content.is_a?(RubyLLM::Content)
-          unless content.attachments.empty?
-            raise UnsupportedFeatureError,
-              "Attachments are not supported by the Codex comparison provider"
-          end
-          return content.text.to_s
-        end
-
-        if defined?(RubyLLM::Content::Raw) && content.is_a?(RubyLLM::Content::Raw)
+        unless message.attachments.empty?
           raise UnsupportedFeatureError,
-            "Raw provider content is not supported by the Codex comparison provider"
+            "Attachments are not supported by the Codex comparison provider"
         end
 
-        content.respond_to?(:to_json) ? content.to_json : content.to_s
+        message.content
       end
 
       def present?(value)

ruby_llm-codex.gemspec

@@ -34,7 +34,7 @@ Gem::Specification.new do |spec|
 
   spec.files = Dir.chdir(File.expand_path(__dir__)) do
     allowed_prefixes = %w[lib/].freeze
-    allowed_files = %w[CHANGELOG.md LICENSE README.md].freeze
+    allowed_files = %w[CHANGELOG.md LICENSE README.md UPGRADE.md].freeze
     git_files = `git ls-files -z 2>/dev/null`.split("\x0")
     candidates = git_files.empty? ? Dir.glob("lib/**/*", File::FNM_DOTMATCH) + allowed_files : git_files
 
@@ -46,7 +46,7 @@ Gem::Specification.new do |spec|
   end
   spec.require_paths = ["lib"]
 
-  spec.add_dependency "ruby_llm", ">= 1.16", "< 2.0"
+  spec.add_dependency "ruby_llm", "~> 2.1.1"
 
   spec.add_development_dependency "minitest", "~> 5"
   spec.add_development_dependency "rake", "~> 13"