Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
498ab61
Bug 2085398 - Add backfill script to populate flag_activity from bugs…
Xzzz Jul 2, 2026
67005a9
Bug 2085398 - Stop writing flag changes to bugs_activity
Xzzz Jul 2, 2026
d48a83d
Bug 2085398 - GetBugActivity reads flag changes from flag_activity
Xzzz Jul 2, 2026
689debf
Bug 2085398 - BugMail reads flag changes from flag_activity
Xzzz Jul 2, 2026
471b376
Bug 2085398 - ActivityStream uses flag_activity flag_id for lookup
Xzzz Jul 2, 2026
8d5afe3
Bug 2085398 - Add tests for the flag_activity backfill script
Xzzz Oct 23, 2026
347c44a
Bug 2085398 - Search, ReleaseTracking and UserActivity read flag chan…
Xzzz Oct 27, 2026
9a84beb
Bug 2085398 - Record the acting user in flag_activity.who
Xzzz Oct 27, 2026
ecd87d9
Bug 2085398 - Read pre-flag_activity flag changes from bugs_activity,…
Xzzz Oct 27, 2026
ba54af1
Bug 2085398 - Pick the previous flag_activity row by id and show one …
Xzzz Oct 27, 2026
a9435b0
Merge branch 'master' into bug-2085398
dklawren Oct 28, 2026
a140f70
Bug 2085398 - Count flag changes in the non-bot last change time and …
Xzzz Oct 29, 2026
3b3f4ba
Bug 2085398 - Index flag_activity on who and flag_when
Xzzz Oct 29, 2026
3cf9f6d
Bug 2085398 - Link flags with no flag_activity row to their bugs_acti…
Xzzz Oct 29, 2026
32f997d
Bug 2085398 - Count flag_activity rows in the user deletion check
Xzzz Oct 29, 2026
ebde8bb
Bug 2085398 - Keep the flag setter in flag_activity when a deleted re…
Xzzz Oct 29, 2026
b33c1df
Merge branch 'master' into bug-2085398
Xzzz Oct 29, 2026
34ee87d
Bug 2085398 - Fill flag_activity.who in the bugmail flag events test
Xzzz Oct 29, 2026
e18e3fa
Bug 2085398 - Keep flag_activity.who in the public data dump
Xzzz Oct 29, 2026
717eb53
Bug 2085398 - Refuse to delete a flag type that has flag_activity rows
Xzzz Oct 29, 2026
59fe15b
Bug 2085398 - Count flag changes in owner idle time, changes to any f…
Xzzz Oct 29, 2026
c560c65
Bug 2085398 - Add a starting flag_activity row for flags that have none
Xzzz Oct 30, 2026
aca6a3b
Bug 2085398 - Skip flag changes that change nothing in bugmail
Xzzz Oct 30, 2026
f6a636a
Bug 2085398 - Match flag changes in changed from and changed to searc…
Xzzz Oct 30, 2026
98f9ae8
Bug 2085398 - Check owner idle time with EXISTS instead of joins
Xzzz Oct 30, 2026
873f583
Bug 2085398 - Date start rows at creation for flags without a modific…
Xzzz Nov 2, 2026
ef867b0
Bug 2085398 - Index bugs_activity for the last flag change of a bug
Xzzz Nov 2, 2026
ae19b88
Bug 2085398 - Add the flag_activity start rows only once
Xzzz Nov 2, 2026
4071ba9
Bug 2085398 - Skip empty flag changes in the user activity report
Xzzz Nov 2, 2026
66203f8
Bug 2085398 - Refuse to delete a user who is a requestee in flag_acti…
Xzzz Nov 2, 2026
c5f3d23
Bug 2085398 - Leave flags set at creation out of the new bug mail, wi…
Xzzz Nov 2, 2026
f290a46
Bug 2085398 - Finish an interrupted flag_activity.who migration, with…
Xzzz Nov 2, 2026
2635ec3
Bug 2085398 - Match one net change per request in the flag changedfro…
Xzzz Nov 2, 2026
62a6b1b
Bug 2085398 - Skip flag_activity rows with a guessed who in changedby…
Xzzz Nov 2, 2026
1b51765
Bug 2085398 - Rename a variable Perl::Critic rejects in the flag search
Xzzz Nov 2, 2026
0059875
Bug 2085398 - Count one net flag change per request in the activity b…
Xzzz Nov 2, 2026
97fa12e
Bug 2085398 - Remove the literal question marks from the user activit…
Xzzz Nov 3, 2026
3c3fd85
Merge branch 'master' into bug-2085398
Xzzz Nov 3, 2026
6bf4021
Merge remote-tracking branch 'origin/master' into bug-2085398
Xzzz Nov 3, 2026
91b8a07
Bug 2085398 - List the flags set at creation in the new bug mail again
Xzzz Nov 3, 2026
ca4030c
Merge remote-tracking branch 'origin/master' into bug-2085398
Xzzz Nov 5, 2026
5c3d058
Bug 2085398 - Drop the foreign key on flag_activity.who before making…
Xzzz Nov 5, 2026
3c0d478
Bug 2085398 - Read the flag changes up to a stored cutoff from bugs_a…
Xzzz Nov 5, 2026
cc875d3
Bug 2085398 - Use sql_string_concat for the flag names in GetBugActivity
Xzzz Nov 12, 2026
715c65f
Bug 2085398 - Use sql_string_concat for the flag names in BugMail _ge…
Xzzz Nov 12, 2026
dc04011
Bug 2085398 - Use sql_string_concat for the flag names in the user ac…
Xzzz Nov 12, 2026
c1c4690
Bug 2085398 - Run perltidy on the files this PR changes
Xzzz Nov 12, 2026
af5884e
Bug 2085398 - Move the _flag_state_match comment back above its sub
Xzzz Nov 12, 2026
ed6ab6e
Bug 2085398 - Use loops instead of multi-statement map blocks
Xzzz Nov 12, 2026
ba2af71
Merge remote-tracking branch 'origin/master' into bug-2085398
Xzzz Nov 12, 2026
ff7a278
Bug 2085398 - Skip flag_activity rows up to the cutoff in the owner i…
Xzzz Nov 12, 2026
7ba522f
Bug 2085398 - Count only the net change of a request in the release t…
Xzzz Nov 12, 2026
259baba
Bug 2085398 - Use sql_limit in find_activity_id_for_flag
Xzzz Nov 12, 2026
262c8aa
Bug 2085398 - Update the comments about where flag changes are stored
Xzzz Nov 12, 2026
ca55922
Bug 2085398 - Remove perltidy-only changes to code this PR doesn't touch
Xzzz Nov 13, 2026
7f1ecfc
Bug 2085398 - Load the last flag_activity row of all the bug's flags …
Xzzz Nov 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion Bugzilla/Attachment.pm
Original file line number Diff line number Diff line change
Expand Up @@ -855,8 +855,9 @@ sub update {
# Record changes in the activity table.
require Bugzilla::Bug;
foreach my $field (keys %$changes) {
next if $field eq 'flagtypes.name'; # now tracked in flag_activity
my $change = $changes->{$field};
$field = "attachments.$field" unless $field eq "flagtypes.name";
$field = "attachments.$field";
Bugzilla::Bug::LogActivityEntry($self->bug_id, $field, $change->[0],
$change->[1], $user->id, $timestamp, undef, $self->id);
}
Expand Down
101 changes: 80 additions & 21 deletions Bugzilla/Bug.pm
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ use Bugzilla::Attachment;
use Bugzilla::Constants;
use Bugzilla::Field;
use Bugzilla::Flag;
use Bugzilla::FlagActivity;
use Bugzilla::FlagType;
use Bugzilla::Hook;
use Bugzilla::Keyword;
Expand Down Expand Up @@ -1221,6 +1222,7 @@ sub update {
# XXX Eventually, when bugs_activity is able to track the dupe_id,
# this code should go below the duplicates-table-updating code below.
foreach my $field (keys %$changes) {
next if $field eq 'flagtypes.name'; # now tracked in flag_activity
Comment thread
dklawren marked this conversation as resolved.
Resolved
Comment thread
dklawren marked this conversation as resolved.
Resolved
Comment thread
dklawren marked this conversation as resolved.
Resolved
my $change = $changes->{$field};
my $from = defined $change->[0] ? $change->[0] : '';
my $to = defined $change->[1] ? $change->[1] : '';
Expand Down Expand Up @@ -4556,29 +4558,25 @@ sub delta_ts_non_bot {
}
}

# Grab the latest timestamp for comments and bug changes that was not made by
# one of the accounts in the bot skip list. Return the greatest of the two values.
# Grab the latest timestamp for comments, bug changes and flag changes that
# was not made by one of the accounts in the bot skip list.
if (my $bot_ids_string = $cache->{bot_ids_string}) {
my %time_map;
my @result = $dbh->selectrow_array(
"SELECT bug_when, UNIX_TIMESTAMP(bug_when)
FROM bugs_activity
WHERE bug_id = ? AND who NOT IN ($bot_ids_string)
ORDER BY bug_when DESC LIMIT 1",
undef, $self->id
);
$time_map{$result[1]} = $result[0] if @result;
@result = $dbh->selectrow_array(
"SELECT bug_when, UNIX_TIMESTAMP(bug_when)
FROM longdescs
WHERE bug_id = ? AND who NOT IN ($bot_ids_string)
ORDER BY bug_when DESC LIMIT 1",
undef, $self->id
my $flag_newer = Bugzilla::FlagActivity->after_cutoff_sql('flag_activity');
($self->{delta_ts_non_bot}) = $dbh->selectrow_array(
"SELECT MAX(bug_when)
FROM (SELECT MAX(bug_when) AS bug_when
FROM bugs_activity
WHERE bug_id = ? AND who NOT IN ($bot_ids_string)
UNION ALL
SELECT MAX(bug_when) AS bug_when
FROM longdescs
WHERE bug_id = ? AND who NOT IN ($bot_ids_string)
UNION ALL
SELECT MAX(flag_when) AS bug_when
FROM flag_activity
WHERE bug_id = ? AND who NOT IN ($bot_ids_string)
AND $flag_newer) tmp", undef, ($self->id) x 3
);
$time_map{$result[1]} = $result[0] if @result;

my $max_epoch = max keys %time_map;
$self->{delta_ts_non_bot} = $time_map{$max_epoch};
}

return $self->{delta_ts_non_bot};
Expand Down Expand Up @@ -4854,9 +4852,25 @@ sub GetBugActivity {
$suppwhere = "AND COALESCE(attachments.isprivate, 0) = 0";
}

my $flag_suppjoins = "";
my $flag_suppwhere = "";
if (!$user->is_insider) {
$flag_suppjoins
= "LEFT JOIN attachments fa_att ON fa_att.attach_id = fsa.attachment_id";
$flag_suppwhere = "AND COALESCE(fa_att.isprivate, 0) = 0";
}

my $flag_attachpart = "";
if ($attach_id) {
$flag_attachpart = "AND fsa.attachment_id = ?";
}

my $bug_when
= $dbh->sql_date_format('bugs_activity.bug_when', '%Y-%m-%d %H:%i:%s');

my $ba_before_cutoff
= Bugzilla::FlagActivity->bugs_activity_before_cutoff_sql('bugs_activity');

# Use an outer table so alias column `bug_when` will work within the `where`
# clause. Also use `DISTINCT` to suppress duplicated changes weirdly made at
# the same time by the same user.
Expand All @@ -4879,6 +4893,7 @@ sub GetBugActivity {
INNER JOIN profiles
ON profiles.userid = bugs_activity.who
WHERE bugs_activity.bug_id = ?
AND $ba_before_cutoff
$attachpart
$suppwhere ";

Expand Down Expand Up @@ -4916,6 +4931,50 @@ sub GetBugActivity {
push @args, $bug_id;
}

my $flag_bug_when = $dbh->sql_date_format('fsa.flag_when', '%Y-%m-%d %H:%i:%s');
my $flag_newer = Bugzilla::FlagActivity->after_cutoff_sql('fsa');
my $flag_last = Bugzilla::FlagActivity->last_of_request_sql('fsa');
my $flag_prev_id = Bugzilla::FlagActivity->previous_id_sql('fsa');
my $flag_removed_req
= $dbh->sql_string_concat('ft.name', q{'?('}, 'preq.login_name', q{')'});
my $flag_removed = $dbh->sql_string_concat('ft.name', 'prev.status');
my $flag_added_req
= $dbh->sql_string_concat('ft.name', q{'?('}, 'req.login_name', q{')'});
my $flag_added = $dbh->sql_string_concat('ft.name', 'fsa.status');
$query .= "
UNION ALL
SELECT 'flagtypes.name' AS field_name,
fsa.id AS activity_id,
fsa.attachment_id AS attach_id,
$flag_bug_when AS bug_when,
CASE WHEN prev.id IS NULL THEN ''
Comment thread
dklawren marked this conversation as resolved.
Resolved
WHEN prev.status = '?' AND preq.login_name IS NOT NULL
THEN $flag_removed_req
ELSE $flag_removed
END AS removed,
CASE WHEN fsa.status = 'X' THEN ''
WHEN fsa.status = '?' AND req.login_name IS NOT NULL
THEN $flag_added_req
ELSE $flag_added
END AS added,
changer.login_name AS who,
NULL AS comment_id
FROM flag_activity fsa
JOIN flagtypes ft ON ft.id = fsa.type_id
JOIN profiles changer ON changer.userid = fsa.who
LEFT JOIN profiles req ON req.userid = fsa.requestee_id
LEFT JOIN flag_activity prev ON prev.id = $flag_prev_id
LEFT JOIN profiles preq ON preq.userid = prev.requestee_id
$flag_suppjoins
WHERE fsa.bug_id = ?
AND $flag_newer
AND $flag_last
$flag_attachpart
$flag_suppwhere
";
push @args, $bug_id;
push @args, $attach_id if $attach_id;

$query .= ") AS OUTER_TABLE";

# Only consider changes since $starttime, if given.
Expand Down
61 changes: 55 additions & 6 deletions Bugzilla/BugMail.pm
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ use Bugzilla::Util;
use Bugzilla::Bug;
use Bugzilla::Attachment;
use Bugzilla::Comment;
use Bugzilla::FlagActivity;
use Bugzilla::FlagType;
use Bugzilla::Logging;
use Bugzilla::Mailer;
Expand Down Expand Up @@ -1044,12 +1045,25 @@ sub _get_diffs {
my ($bug, $end, $user_cache) = @_;
my $dbh = Bugzilla->dbh;

my @args = ($bug->id);
my @args = ($bug->id);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: alignment is off here, looks like perltidy wasn't run. perl scripts/perl-fmt.pl on the changed files should fix this plus the long $flag_suppjoins line in Bug.pm and the hash indent in Install/DB.pm

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, done with the repo's .perltidyrc...
ran perl scripts/perl-fmt.pl on the files this PR modifies. Most weren't perltidy-clean on master, so this also reformats existing code in them.

Fixed in c1c4690

my $flag_newer = Bugzilla::FlagActivity->after_cutoff_sql('fsa');
my $ba_before_cutoff
= Bugzilla::FlagActivity->bugs_activity_before_cutoff_sql('bugs_activity');
my $flag_last = Bugzilla::FlagActivity->last_of_request_sql('fsa');
my $flag_prev_id = Bugzilla::FlagActivity->previous_id_sql('fsa');
my $flag_old_req
= $dbh->sql_string_concat('ft.name', q{'?('}, 'preq.login_name', q{')'});
my $flag_old = $dbh->sql_string_concat('ft.name', 'prev.status');
my $flag_new_req
= $dbh->sql_string_concat('ft.name', q{'?('}, 'req.login_name', q{')'});
my $flag_new = $dbh->sql_string_concat('ft.name', 'fsa.status');

# If lastdiffed is NULL, then we don't limit the search on time.
my $when_restriction = '';
my $when_restriction = '';
my $flag_when_restriction = '';
if ($bug->lastdiffed) {
$when_restriction = ' AND bug_when > ? AND bug_when <= ?';
$when_restriction = ' AND bug_when > ? AND bug_when <= ?';
$flag_when_restriction = ' AND fsa.flag_when > ? AND fsa.flag_when <= ?';
push @args, ($bug->lastdiffed, $end);
}

Expand All @@ -1058,15 +1072,50 @@ sub _get_diffs {
fielddefs.description AS field_desc,
bugs_activity.bug_when, bugs_activity.removed AS old,
bugs_activity.added AS new, bugs_activity.attach_id,
bugs_activity.comment_id, bugs_activity.who
bugs_activity.comment_id, bugs_activity.who,
bugs_activity.id AS activity_id
FROM bugs_activity
INNER JOIN fielddefs
ON fielddefs.id = bugs_activity.fieldid
WHERE bugs_activity.bug_id = ?
AND $ba_before_cutoff
$when_restriction
ORDER BY bugs_activity.bug_when, fielddefs.description", {Slice => {}},
@args
UNION ALL
SELECT 'flagtypes.name' AS field_name,
(SELECT description FROM fielddefs WHERE name = 'flagtypes.name') AS field_desc,
fsa.flag_when AS bug_when,
CASE WHEN prev.id IS NULL THEN ''
WHEN prev.status = '?' AND preq.login_name IS NOT NULL
THEN $flag_old_req
ELSE $flag_old
END AS old,
CASE WHEN fsa.status = 'X' THEN ''
WHEN fsa.status = '?' AND req.login_name IS NOT NULL
THEN $flag_new_req
ELSE $flag_new
END AS new,
fsa.attachment_id AS attach_id,
NULL AS comment_id,
fsa.who AS who,
fsa.id AS activity_id
FROM flag_activity fsa
JOIN flagtypes ft ON ft.id = fsa.type_id
LEFT JOIN profiles req ON req.userid = fsa.requestee_id
LEFT JOIN flag_activity prev ON prev.id = $flag_prev_id
LEFT JOIN profiles preq ON preq.userid = prev.requestee_id
WHERE fsa.bug_id = ?
AND $flag_newer
AND $flag_last
$flag_when_restriction
ORDER BY bug_when, field_desc, activity_id", {Slice => {}}, @args,
$bug->id, ($bug->lastdiffed ? ($bug->lastdiffed, $end) : ())
);

# As in history, skip a request that left a flag as it was, such as a flag
# created and removed at once.
@$diffs
= grep { $_->{field_name} ne 'flagtypes.name' || $_->{old} ne $_->{new} }
@$diffs;
my $referenced_bugs = [];

foreach my $diff (@$diffs) {
Expand Down
11 changes: 9 additions & 2 deletions Bugzilla/DB/Schema.pm
Original file line number Diff line number Diff line change
Expand Up @@ -748,9 +748,10 @@ use constant ABSTRACT_SCHEMA => {
REFERENCES => {TABLE => 'profiles', COLUMN => 'userid'},
},
# The user who made the change. Not always the setter: redirecting a
# request keeps the original setter. Nullable until all pods write it.
who => {
# request keeps the original setter.
who => {
TYPE => 'INT3',
NOTNULL => 1,
REFERENCES => {TABLE => 'profiles', COLUMN => 'userid'},
},
requestee_id => {
Expand All @@ -777,6 +778,12 @@ use constant ABSTRACT_SCHEMA => {
],
},

# One row: flag changes up to this time are read from bugs_activity, later
# ones from flag_activity.
# See Bugzilla::FlagActivity->cutoff.
flag_activity_migration =>
{FIELDS => [cutoff => {TYPE => 'DATETIME', NOTNULL => 1},],},

# "flagtypes" defines the types of flags that can be set.
flagtypes => {
FIELDS => [
Expand Down
1 change: 1 addition & 0 deletions Bugzilla/DB/Sqlite.pm
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,7 @@ sub sql_date_format {
my ($self, $date, $format) = @_;
$format = "%Y-%m-%d %H:%M:%S" if !$format;
$format =~ s/\%i/\%M/g;
$format =~ s/\%s/\%S/g;
return "STRFTIME(" . $self->quote($format) . ", $date)";
}

Expand Down
2 changes: 1 addition & 1 deletion Bugzilla/Flag.pm
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ use base qw(Bugzilla::Object Exporter);
use constant DB_TABLE => 'flags';
use constant LIST_ORDER => 'id';

# Flags are tracked in bugs_activity.
# Flag changes are tracked in flag_activity (see Bugzilla::FlagActivity).
use constant AUDIT_CREATES => 0;
use constant AUDIT_UPDATES => 0;
use constant AUDIT_REMOVES => 0;
Expand Down
58 changes: 57 additions & 1 deletion Bugzilla/FlagActivity.pm
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ use strict;
use warnings;

use Bugzilla::Error;
use Bugzilla::Util qw(trim datetime_from);
use Bugzilla::Field qw(get_field_id);
use Bugzilla::Util qw(trim datetime_from);
use List::MoreUtils qw(none);

use base qw( Bugzilla::Object );
Expand Down Expand Up @@ -85,6 +86,61 @@ sub requestee_id { return $_[0]->{requestee_id} }
sub attachment_id { return $_[0]->{attachment_id} }
sub status { return $_[0]->{status} }

# Flag changes up to the cutoff are read from bugs_activity, which has them
# all: flag_activity has only some of them, and doesn't always know their who.
# Later ones are read from flag_activity only. The cutoff is when checksetup
# stopped writing flag changes to bugs_activity: pods still on the older code
# write both tables until the end of the rolling deploy, and their
# bugs_activity rows must not be listed twice.
sub cutoff {
my $cache = Bugzilla->request_cache;
$cache->{flag_activity_cutoff} //= Bugzilla->dbh->selectrow_array(
'SELECT cutoff FROM flag_activity_migration') // '1970-01-01 00:00:00';
return $cache->{flag_activity_cutoff};
}

# SQL condition on flag_activity row $alias: true after the cutoff.
sub after_cutoff_sql {
my ($class, $alias) = @_;
return "$alias.flag_when > " . Bugzilla->dbh->quote($class->cutoff);
}

# SQL condition on bugs_activity row $alias: false for a flag change after the
# cutoff.
sub bugs_activity_before_cutoff_sql {
my ($class, $alias) = @_;
my $field_id = get_field_id('flagtypes.name');
return
"($alias.fieldid <> $field_id OR $alias.bug_when <= "
. Bugzilla->dbh->quote($class->cutoff) . ')';
}

# A flag created and updated in one request has several rows with the same
# flag_when and who. History shows the request's net change, like
# bugs_activity did: only its last row, compared with the flag's last row
# before the request. Rows are picked by id, which orders same-second rows.

# SQL condition on flag_activity row $alias: true for the request's last row.
sub last_of_request_sql {
my ($class, $alias) = @_;
return "NOT EXISTS (
SELECT 1 FROM flag_activity next_fa
WHERE next_fa.flag_id = $alias.flag_id
AND next_fa.flag_when = $alias.flag_when
AND next_fa.who = $alias.who
AND next_fa.id > $alias.id)";
}

# SQL subquery: the id of the flag's last row before $alias's request.
sub previous_id_sql {
my ($class, $alias) = @_;
return "(SELECT MAX(prev_fa.id) FROM flag_activity prev_fa
WHERE prev_fa.flag_id = $alias.flag_id
AND prev_fa.id < $alias.id
AND NOT (prev_fa.flag_when = $alias.flag_when
AND prev_fa.who = $alias.who))";
}

sub type {
my ($self) = @_;
return $self->{type}
Expand Down
Loading
Loading