Skip to content

Commit 80d2a76

Browse files
authored
fix: enable timely stats update to avoid large row count estimation deviation (#23692)
After the stats refactoring, when multiple secondary indexes are created concurrently, the planner observes a smaller `Stats.OutCnt` because global stats were not updated in time on data object (block) changes. The upgrade-to-table-lock condition (`Stats.OutCnt >= MaxRowLockCount`) is then never reached, so the system keeps acquiring row locks and can accumulate ~100M row locks causing OOM (see issue #23689). This change treats data object logtail entries (`api.Entry_DataObject`) as meta-changing in `processLogtail`, in addition to existing meta entries. When blocks/data objects change, they are counted as meta changes and can trigger stats refresh (subject to existing rate limiting in `shouldEnqueueUpdate`). Stats then reflect table size more timely and row count estimation deviation is reduced, so the planner can correctly upgrade to table lock and avoid excessive row locks. Approved by: @gouhongshen
1 parent 1114025 commit 80d2a76

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

pkg/vm/engine/disttae/stats.go

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ import (
2828
"github.com/matrixorigin/matrixone/pkg/fileservice"
2929
"github.com/matrixorigin/matrixone/pkg/logutil"
3030
"github.com/matrixorigin/matrixone/pkg/objectio"
31+
"github.com/matrixorigin/matrixone/pkg/pb/api"
3132
"github.com/matrixorigin/matrixone/pkg/pb/gossip"
3233
"github.com/matrixorigin/matrixone/pkg/pb/logtail"
3334
"github.com/matrixorigin/matrixone/pkg/pb/query"
@@ -486,7 +487,7 @@ func (gs *GlobalStats) processLogtail(ctx context.Context, tail *logtail.TableLo
486487
// Count meta changes from logtail by checking batch length
487488
metaChanges := 0
488489
for i := range tail.Commands {
489-
if logtailreplay.IsMetaEntry(tail.Commands[i].TableName) {
490+
if cmd := tail.Commands[i]; cmd.EntryType == api.Entry_DataObject || logtailreplay.IsMetaEntry(tail.Commands[i].TableName) {
490491
if tail.Commands[i].Bat != nil && len(tail.Commands[i].Bat.Vecs) > 0 {
491492
metaChanges += int(tail.Commands[i].Bat.Vecs[0].Len)
492493
}

0 commit comments

Comments
 (0)