Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,7 @@ The common ones:
| Variable | Default | Purpose |
|---|---|---|
| `CODETRIAL_WEB_ADDR` | `127.0.0.1:3000` | Listen address |
| `CODETRIAL_DURATION_MIN` | `45` | Interview length preselected in the lobby (10–90); see [interview length](docs/interview-length.md) |
| `CODETRIAL_DURATION_MIN` | `45` | Interview length the lobby preselects and `/api/token` grants, unsnapped, to a request that names none (10–90; text that is no whole number refuses startup); see [interview length](docs/interview-length.md) |
| `GEMINI_LIVE_MODEL` | `gemini-3.1-flash-live-preview` | Realtime interviewer model |
| `GEMINI_REPORT_MODEL` | `gemini-3.1-flash-lite` | Report model |
| `CODETRIAL_MAX_INTERIM_REVIEWS` | `6` | Quiet-pause report-model reviews per interview; `0` disables them and `72` is the maximum |
Expand Down
46 changes: 35 additions & 11 deletions docs/interview-length.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,30 @@ both. One rule matters more than the rest: the length offered and the length
enforced come from one function, so they cannot drift apart.

The lobby offers 30, 45, and 60 minutes, and the candidate picks one before
starting. `CODETRIAL_DURATION_MIN` is only the length preselected for a
candidate who does not choose; the endpoint accepts 10 to 90 either way, so a
deployment that lowers the default has not lowered the ceiling.
starting. `CODETRIAL_DURATION_MIN` is the length preselected for a candidate who
does not choose, once `/api/session` has answered; until then, and after it
fails, the row shows the built-in 45, and Start sends whatever the row shows.
The endpoint accepts 10 to 90 either way, so a deployment that lowers the
default has not lowered the ceiling. A setting outside 10 to 90 is clamped and
named once on stderr when `codetrial web` or an agent mode starts, and text
that is no whole number refuses startup.

`/api/session` reports the default as `defaultDurationMin`, clamped into 10 to
Comment thread
RWong04 marked this conversation as resolved.
Resolved
90; where recording is on, startup refuses a cap below that clamped length, as
the last paragraph says. The lobby preselects it for Medium and Hard, and
for Easy at most 30. A default that is not one of the three buttons selects the
nearest enabled one, and a length the candidate chose is never replaced by the
default. The Easy cap and the snapping are the lobby's suggestion only.
`/api/token` does not look at the problem: a request that names no length, such
as a link that skips the lobby, is granted `defaultDurationMin` itself, not
snapped, whatever the difficulty. A `durationMin` that is `null`, `false`, zero
or negative (`0`, `-5`), or text that is not a finite number (`abc`,
`Infinity`, `NaN`) names no length either, so it gets the same default and not
the built-in 45. A `duration` in
the interview URL that is not at least one whole minute (`0`, `0.5`, `-5`,
`30abc`, `Infinity`, `abc` or empty) names no length, so the interview page
sends none and gets the same default; a longer fraction is cut to whole
minutes, as the agent reads them.

Recording lowers it. `CODETRIAL_RECORDING_MAX_MINUTES` bounds a single
recording, and the sweeper fails one still running five minutes past it, so an
Expand All @@ -18,14 +39,17 @@ offers; where it is off, the ceiling is 90.

The lobby is told the ceiling rather than left to discover it. `/api/session`
reports `maxDurationMin`, and the duration row disables the lengths above it
with the reason beside them — a length this deployment cannot record is never
offered and then quietly shortened. `/api/token` clamps to the same ceiling, so
a request that skips the lobby is bounded too, and both endpoints read it from
with the reason beside them — once the session has answered, a length this
deployment cannot record is never offered and then quietly shortened. A Start
pressed before it answers sends the 45 on screen, and `/api/token` shortens it
to the cap. `/api/token` clamps to the same ceiling, so a request that skips
the lobby is bounded too, and both endpoints read it from
one function so the length that is offered and the length that is enforced
cannot drift apart.

Startup refuses a cap below `CODETRIAL_DURATION_MIN`, so the length this
deployment preselects is always one it can record; the pairing is a
configuration error rather than a quietly shortened interview. Should a cap ever
sit below every length the row offers, the lobby leaves the row alone rather
than rendering a page with nothing to press.
Where recording is on, startup refuses a cap below `CODETRIAL_DURATION_MIN`,
held to 10 to 90 like the default itself, so the length this deployment
preselects is always one it can record; the pairing is a configuration error
rather than a quietly shortened interview. Should a cap ever sit below every
length the row offers, the lobby leaves the row alone rather than rendering a
page with nothing to press.
45 changes: 44 additions & 1 deletion src/config.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
use std::collections::BTreeMap;
use std::fmt;
use std::num::IntErrorKind;
use std::path::Path;

mod peer_observer;
Expand Down Expand Up @@ -443,6 +444,11 @@ pub fn load_from_pairs(
invalid_entries.push("CODETRIAL_MAX_INTERIM_REVIEWS must be a whole number".into());
}

let default_duration_min = configured_duration(&values).unwrap_or_else(|message| {
invalid_entries.push(message);
(DEFAULT_DURATION_MIN, None)
});

if !missing_keys.is_empty() || !invalid_entries.is_empty() {
return Err(ConfigError {
missing_keys,
Expand Down Expand Up @@ -498,7 +504,7 @@ pub fn load_from_pairs(
DEFAULT_GEMINI_END_SENSITIVITY,
),
room_prefix: optional(&values, "CODETRIAL_ROOM_PREFIX", DEFAULT_ROOM_PREFIX),
default_duration_min: optional_u32(&values, "CODETRIAL_DURATION_MIN", DEFAULT_DURATION_MIN),
default_duration_min: default_duration_min.0,
Comment thread
RWong04 marked this conversation as resolved.
Resolved
gemini_candidate_video_enabled: gemini_candidate_video_enabled(
values
.get("CODETRIAL_GEMINI_CANDIDATE_VIDEO_ENABLED")
Expand Down Expand Up @@ -660,6 +666,43 @@ pub fn max_concurrent_interviews(
}
}

/// The length `CODETRIAL_DURATION_MIN` stands for, held to the range the
/// endpoints offer, and what to say when a number is clamped. Text that is no
/// number is refused rather than read as the built-in length: `3O` typed for 30
/// would otherwise run a 45-minute default on a host nobody watches. One
/// function for both entry points, so the number in the message is the one the
/// server runs on.
pub fn configured_duration(
values: &BTreeMap<String, String>,
) -> Result<(u32, Option<String>), String> {
let value = present(values, "CODETRIAL_DURATION_MIN").unwrap_or_default();
if value.is_empty() {
return Ok((DEFAULT_DURATION_MIN, None));
}

// A number past `i64` is still a number, only a very large or small one, so
// it is clamped like the rest.
let minutes = match value.parse::<i64>() {
Ok(minutes) => minutes,
Err(error) if *error.kind() == IntErrorKind::PosOverflow => i64::MAX,
Err(error) if *error.kind() == IntErrorKind::NegOverflow => i64::MIN,
Err(_) => {
return Err(format!(
"CODETRIAL_DURATION_MIN {value:?} must be a whole number of minutes"
));
}
};
let used =
u32::try_from(minutes.clamp(i64::from(MIN_DURATION_MIN), i64::from(MAX_DURATION_MIN)))
.unwrap_or(DEFAULT_DURATION_MIN);
let warning = (i64::from(used) != minutes).then(|| {
format!(
"codetrial: CODETRIAL_DURATION_MIN {value:?} is outside {MIN_DURATION_MIN} to {MAX_DURATION_MIN}; read as {used}"
)
});
Ok((used, warning))
}

fn optional_u32(values: &BTreeMap<String, String>, key: &str, default: u32) -> u32 {
values
.get(key)
Expand Down
26 changes: 12 additions & 14 deletions src/launch.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ mod peer;

pub use peer::{PeerServices, peer_services};

use crate::config::{DEFAULT_DURATION_MIN, DEFAULT_ROOM_PREFIX, DEFAULT_WEB_DIR};
use crate::config::{DEFAULT_ROOM_PREFIX, DEFAULT_WEB_DIR};
use crate::web::WebServerConfig;

/// One installation keeps its config file and its account database together in
Expand Down Expand Up @@ -60,10 +60,12 @@ pub fn web_server_config(
// After the pool is complete, so the "is this project in the pool" check
// sees the pool the server will actually serve, and before the listener
// does any work: a half-configured recording block is a startup failure,
// never a surprise at the moment a candidate's interview ends.
let recording =
crate::config::load_recording(values, &pool, interview_duration_min(values), production)
.map_err(|error| error.to_string())?;
// never a surprise at the moment a candidate's interview ends. Read here
// rather than through `load_from_pairs`, because `codetrial web`
// deliberately runs without a full agent config.
let (default_duration_min, _) = crate::config::configured_duration(values)?;
let recording = crate::config::load_recording(values, &pool, default_duration_min, production)
.map_err(|error| error.to_string())?;
recording_needs_verified_identity(
recording.is_some(),
nonempty(values, "GITHUB_CLIENT_ID").is_some()
Expand Down Expand Up @@ -104,6 +106,7 @@ pub fn web_server_config(
allowed_hosts: allowed_hosts(values)?,
compiler_explorer_enabled: compiler_explorer_enabled(values),
guest_mode_enabled: guest_mode_enabled(values),
default_duration_min,
recording,
peer_observer,
peer,
Expand Down Expand Up @@ -565,15 +568,6 @@ pub fn recording_needs_verified_identity(recording: bool, oauth: bool) -> Result
Ok(())
}

/// The configured interview length, which is what a recording's maximum
/// duration has to clear. Read here rather than through `load_from_pairs`
/// because `codetrial web` deliberately runs without a full agent config.
fn interview_duration_min(values: &BTreeMap<String, String>) -> u32 {
nonempty(values, "CODETRIAL_DURATION_MIN")
.and_then(|value| value.parse().ok())
.unwrap_or(DEFAULT_DURATION_MIN)
}

/// `CODETRIAL_ALLOWED_HOSTS`, a comma-separated list of `Host` values.
///
/// Unset or empty means no check. A value that holds only separators is
Expand Down Expand Up @@ -841,6 +835,10 @@ pub fn prepare(
let max_concurrent = read_max_concurrent(values);

let production = crate::config::is_production(values);

if let Some(warning) = crate::config::configured_duration(values)?.1 {
eprintln!("{warning}");
}
let pool = web_provider_pool(values, options, production)?;

let config = web_server_config(values, pool, production, options)?;
Expand Down
7 changes: 7 additions & 0 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -607,6 +607,13 @@ fn load_agent_config(options: &CliOptions) -> Result<AgentConfig, String> {
let values = load_values(options)?;
let production = is_production(&values);
let provider_order = value_or(&values, codetrial::config::PROVIDER_ORDER_KEY, "");

// Said here and not in `load_from_pairs`, which `codetrial web` also calls
// after it has printed the same warning. A number that is no number is
// refused by `load_from_pairs` itself.
if let Ok((_, Some(warning))) = codetrial::config::configured_duration(&values) {
eprintln!("{warning}");
}
let mut config =
codetrial::config::load_from_pairs(values).map_err(|error| error.to_string())?;
extend_pool(
Expand Down
8 changes: 6 additions & 2 deletions src/web/auth.rs
Original file line number Diff line number Diff line change
Expand Up @@ -578,8 +578,10 @@ pub(crate) async fn session_handler(
// The lobby offers lengths this deployment may not be able to record, and
// only the server knows the recording cap. Answered on both branches
// because the duration row is live before anyone signs in.
let max_duration_min =
super::token::duration_ceiling(state.config.recording.as_ref().map(|it| it.max_minutes));
let recording_max_minutes = state.config.recording.as_ref().map(|it| it.max_minutes);
let max_duration_min = super::token::duration_ceiling(recording_max_minutes);
let default_duration_min =
super::token::default_duration(state.config.default_duration_min, recording_max_minutes);

// Null on the web half of a split deployment, where the lobby keeps its
// billing notice hidden rather than describe an agent it cannot see.
Expand All @@ -605,6 +607,7 @@ pub(crate) async fn session_handler(
"guestModeAvailable": state.guest_mode_available(),
"deviceLogin": device_client_id(&state.config).is_some(),
"maxDurationMin": max_duration_min,
"defaultDurationMin": default_duration_min,
"candidateVideo": candidate_video,
"user": {
"login": user.login,
Expand All @@ -626,6 +629,7 @@ pub(crate) async fn session_handler(
"guestModeAvailable": state.guest_mode_available(),
"deviceLogin": device_client_id(&state.config).is_some(),
"maxDurationMin": max_duration_min,
"defaultDurationMin": default_duration_min,
"candidateVideo": candidate_video,
}),
),
Expand Down
5 changes: 5 additions & 0 deletions src/web/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@ pub struct WebServerConfig {
pub compiler_explorer_enabled: bool,
/// Guest interviews require explicit operator opt-in.
pub guest_mode_enabled: bool,
/// `CODETRIAL_DURATION_MIN`: the lobby's preselected length, and the one
/// a token request that names none gets.
pub default_duration_min: u32,
/// Number of trusted reverse proxies in front of this server. Zero, the
/// default, keys rate limiting on the TCP peer. Anything higher reads the
/// client from `X-Forwarded-For`, which is only safe once every request is
Expand Down Expand Up @@ -180,6 +183,7 @@ impl fmt::Debug for WebServerConfig {
production,
compiler_explorer_enabled,
guest_mode_enabled,
default_duration_min,
trusted_proxy_hops,
allowed_hosts,
recording,
Expand Down Expand Up @@ -212,6 +216,7 @@ impl fmt::Debug for WebServerConfig {
.field("production", production)
.field("compiler_explorer_enabled", compiler_explorer_enabled)
.field("guest_mode_enabled", guest_mode_enabled)
.field("default_duration_min", default_duration_min)
.field("trusted_proxy_hops", trusted_proxy_hops)
.field("allowed_hosts", allowed_hosts)
.field("recording", recording)
Expand Down
35 changes: 28 additions & 7 deletions src/web/token.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ use axum::response::{IntoResponse, Response};
use ring::rand::SecureRandom;
use serde_json::{Value, json};

use crate::config::{DEFAULT_DURATION_MIN, MAX_DURATION_MIN, MIN_DURATION_MIN};
use crate::config::{MAX_DURATION_MIN, MIN_DURATION_MIN};
use crate::current_epoch_seconds;
use crate::token::{LivekitTokenInput, TOKEN_TTL_SECONDS, livekit_observer_token, livekit_token};

Expand All @@ -39,6 +39,8 @@ pub struct TokenConfig<'a> {
/// recording/sweeper.rs reaps one still running past this many minutes, so
/// a longer interview loses exactly the stretch it was recorded for.
pub recording_max_min: Option<u32>,
/// `CODETRIAL_DURATION_MIN`: the length of a request that names none.
pub default_duration_min: u32,
}

/// Same reason, and the sharpest case of it: `token` is a minted LiveKit
Expand Down Expand Up @@ -337,7 +339,11 @@ pub fn token_response(
.get("problemId")
.and_then(Value::as_str)
.unwrap_or(crate::agent::DEFAULT_PROBLEM_ID);
let duration_min = token_duration_min(request.get("durationMin"), config.recording_max_min);
let duration_min = token_duration_min(
request.get("durationMin"),
config.default_duration_min,
config.recording_max_min,
);
let interview_loop =
crate::agent::InterviewLoop::parse(request.get("interviewLoop").and_then(Value::as_str));
let interview_mode =
Expand Down Expand Up @@ -459,17 +465,31 @@ pub(crate) fn duration_ceiling(recording_max_min: Option<u32>) -> u32 {
})
}

pub(crate) fn token_duration_min(value: Option<&Value>, recording_max_min: Option<u32>) -> Value {
/// The operator's `CODETRIAL_DURATION_MIN` as the lobby and `/api/token` both
/// use it: inside the range the endpoint offers and no longer than the
/// deployment can record. One function for the same reason as
/// [`duration_ceiling`].
pub(crate) fn default_duration(configured_min: u32, recording_max_min: Option<u32>) -> u32 {
configured_min
.clamp(MIN_DURATION_MIN, MAX_DURATION_MIN)
.min(duration_ceiling(recording_max_min))
}

pub(crate) fn token_duration_min(
value: Option<&Value>,
default_duration_min: u32,
recording_max_min: Option<u32>,
) -> Value {
let ceiling = duration_ceiling(recording_max_min);
let duration = value.and_then(crate::agent::json_number).unwrap_or(0.0);
if duration == 0.0 || duration.is_nan() {
if duration <= 0.0 || !duration.is_finite() {
// The default is a request like any other where the cap is shorter than
// it. Handing back a length this server cannot record, for a caller who
// named no length at all, is the same lost stretch of interview.
json!(DEFAULT_DURATION_MIN.min(ceiling))
json!(default_duration(default_duration_min, recording_max_min))
Comment thread
RWong04 marked this conversation as resolved.
Resolved
Comment thread
RWong04 marked this conversation as resolved.
Resolved
} else {
// Fractional minutes stay fractional: the metadata is echoed back to
// the frontend, which renders whatever the lobby asked for.
// Fractional minutes stay fractional for a caller that sends them; the
// interview page cuts to whole minutes before it asks.
number_json(duration.clamp(MIN_DURATION_MIN.into(), ceiling.into()))
}
}
Expand Down Expand Up @@ -766,6 +786,7 @@ fn minted_token(
api_secret: &provider.api_secret,
server_url: &provider.url,
recording_max_min: state.config.recording.as_ref().map(|it| it.max_minutes),
default_duration_min: state.config.default_duration_min,
Comment thread
RWong04 marked this conversation as resolved.
Resolved
},
body,
room_name,
Expand Down
2 changes: 1 addition & 1 deletion tests/browser/account.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ test("the lobby offers an editor or a whiteboard and carries the choice into the
);
assertIncludesCompact(
interview,
"durationMin, interviewId, interviewLoop, interviewMode: mode, ...(!whiteboard ? { language: state.language } : {}), interviewProfile, ...(interviewGrounding",
"...(namedDuration ? { durationMin } : {}), interviewId, interviewLoop, interviewMode: mode, ...(!whiteboard ? { language: state.language } : {}), interviewProfile, ...(interviewGrounding",
);
assertIncludesCompact(
interview,
Expand Down
1 change: 1 addition & 0 deletions tests/browser/interview-lifecycle.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,7 @@ test("the actual connection preserves refusal reasons and gates the init deadlin
updateRunAvailability() {},
tabStorage: {},
params: new URLSearchParams(),
namedDuration: false,
durationMin: 45,
codingMinutes: 45,
behavioralMinutes: 0,
Expand Down
Loading
Loading