feat: introduce comprehensive audit logging across core services by appleboy · Pull Request #27 · appleboy/authgate

appleboy · 2026-02-04T15:38:37Z

Add a full-featured audit logging system, including service, model, store methods, and handlers, for tracking authentication, admin, and security events
Implement audit log filtering, pagination, CSV export, statistics, and automatic cleanup based on retention settings
Instrument user, device, client, and token services with audit logging on sensitive and important actions
Integrate audit log viewing and export UI with filters, navigation, empty states, and stats in the admin dashboard
Pass context and actor information throughout services to populate audit trail entries accurately
Update rate limiting to support audit logging when limits are exceeded
Add context-aware helpers for extracting client IP address and username for logging
Expand tests to use new context-based signatures and audit service dependencies
Enhance authentication and client management with finer audit logging for successful and failed actions
Include admin navigation for accessing audit logs throughout the template system
Ensure audit logging is enabled and maintained as a background service with graceful shutdown and periodic cleanup

- Add a full-featured audit logging system, including service, model, store methods, and handlers, for tracking authentication, admin, and security events - Implement audit log filtering, pagination, CSV export, statistics, and automatic cleanup based on retention settings - Instrument user, device, client, and token services with audit logging on sensitive and important actions - Integrate audit log viewing and export UI with filters, navigation, empty states, and stats in the admin dashboard - Pass context and actor information throughout services to populate audit trail entries accurately - Update rate limiting to support audit logging when limits are exceeded - Add context-aware helpers for extracting client IP address and username for logging - Expand tests to use new context-based signatures and audit service dependencies - Enhance authentication and client management with finer audit logging for successful and failed actions - Include admin navigation for accessing audit logs throughout the template system - Ensure audit logging is enabled and maintained as a background service with graceful shutdown and periodic cleanup Signed-off-by: appleboy <appleboy.tw@gmail.com>

Copilot

Pull request overview

This PR introduces a comprehensive audit logging system across AuthGate's core services, enabling tracking of authentication, authorization, admin operations, and security events. The system features asynchronous buffered writes, sensitive data masking, automatic cleanup based on retention policies, and a complete admin UI for viewing, filtering, and exporting audit logs.

Changes:

New audit service with async batching, graceful shutdown, and automatic cleanup
Integration of audit logging into user, device, client, and token services to track sensitive operations
Admin UI for viewing audit logs with filtering, pagination, CSV export, and statistics
Context-aware middleware and helpers for extracting client IP and username
Rate limiting integration to log when rate limits are exceeded
Updated service signatures throughout to accept context for audit trail population

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
internal/services/audit.go	New audit service implementing async batching, sensitive data masking, and graceful shutdown
internal/models/audit_log.go	Audit log data model with event types, severity levels, and resource types
internal/store/sqlite.go	Database methods for creating, querying, and deleting audit logs with pagination
internal/store/audit_filters.go	Filter and statistics structures for audit log queries
internal/services/user.go	Added audit logging for authentication success/failure and OAuth operations
internal/services/device.go	Added audit logging for device code generation and authorization
internal/services/client.go	Added audit logging for OAuth client CRUD operations
internal/services/token.go	Added audit logging for token issuance, refresh, revocation, and status changes
internal/handlers/audit.go	HTTP handlers for viewing, filtering, and exporting audit logs
internal/middleware/auth.go	Updated to load user object for audit logging context
internal/middleware/ratelimit.go	Added audit logging when rate limits are exceeded
internal/util/context.go	Helper functions for extracting client IP and username from context
internal/handlers/session.go	Updated token operations to pass context and actor information
internal/handlers/device.go	Updated device authorization to pass username for audit logging
internal/handlers/client.go	Updated client operations to pass context and actor for audit logging
internal/config/config.go	Added audit logging configuration options with defaults
main.go	Initialized audit service, added shutdown handler, and periodic cleanup job
internal/templates/admin/audit_logs.html	New comprehensive UI for viewing and filtering audit logs
internal/templates/*.html	Added audit log navigation links for admin users
internal/services/*_test.go	Updated tests to use new context-based service signatures

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-04T15:44:55Z

internal/handlers/audit.go

+	if userID, exists := c.Get("user_id"); exists {
+		if username, usernameExists := c.Get("username"); usernameExists {
+			h.auditService.Log(c.Request.Context(), services.AuditLogEntry{
+				EventType:     "AUDIT_LOG_EXPORTED",


Same issue as line 171 - the event type "AUDIT_LOG_EXPORTED" is not defined in the models.EventType constants. Both audit log viewing and exporting events should use defined event types.

Suggested change

EventType: "AUDIT_LOG_EXPORTED",

EventType: models.EventTypeAuditLogExported,

Copilot · 2026-02-04T15:44:55Z

internal/services/audit.go

+package services
+
+import (
+	"context"
+	"fmt"
+	"log"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/appleboy/authgate/internal/models"
+	"github.com/appleboy/authgate/internal/store"
+	"github.com/appleboy/authgate/internal/util"
+	"github.com/google/uuid"
+)
+
+// AuditLogEntry represents the data needed to create an audit log entry
+type AuditLogEntry struct {
+	EventType     models.EventType
+	Severity      models.EventSeverity
+	ActorUserID   string
+	ActorUsername string
+	ActorIP       string
+	ResourceType  models.ResourceType
+	ResourceID    string
+	ResourceName  string
+	Action        string
+	Details       models.AuditDetails
+	Success       bool
+	ErrorMessage  string
+	UserAgent     string
+	RequestPath   string
+	RequestMethod string
+}
+
+// AuditService handles audit logging operations
+type AuditService struct {
+	store      *store.Store
+	enabled    bool
+	bufferSize int
+
+	// Async logging channel
+	logChan chan *models.AuditLog
+
+	// Batch buffer
+	batchBuffer []*models.AuditLog
+	batchMutex  sync.Mutex
+	batchTicker *time.Ticker
+
+	// Graceful shutdown
+	wg         sync.WaitGroup
+	shutdownCh chan struct{}
+}
+
+// NewAuditService creates a new audit service
+func NewAuditService(s *store.Store, enabled bool, bufferSize int) *AuditService {
+	if bufferSize <= 0 {
+		bufferSize = 1000 // Default buffer size
+	}
+
+	service := &AuditService{
+		store:       s,
+		enabled:     enabled,
+		bufferSize:  bufferSize,
+		logChan:     make(chan *models.AuditLog, bufferSize),
+		batchBuffer: make([]*models.AuditLog, 0, 100),
+		batchTicker: time.NewTicker(1 * time.Second),
+		shutdownCh:  make(chan struct{}),
+	}
+
+	if enabled {
+		service.wg.Add(1)
+		go service.worker()
+		log.Printf("Audit service started with buffer size %d", bufferSize)
+	} else {
+		log.Println("Audit service is disabled")
+	}
+
+	return service
+}
+
+// worker is the background goroutine that processes audit logs
+func (s *AuditService) worker() {
+	defer s.wg.Done()
+
+	for {
+		select {
+		case log := <-s.logChan:
+			s.addToBatch(log)
+
+		case <-s.batchTicker.C:
+			// Flush batch every second
+			s.flushBatch()
+
+		case <-s.shutdownCh:
+			// Flush remaining logs before shutdown
+			s.flushBatch()
+			return
+		}
+	}
+}
+
+// addToBatch adds a log entry to the batch buffer
+func (s *AuditService) addToBatch(log *models.AuditLog) {
+	s.batchMutex.Lock()
+	defer s.batchMutex.Unlock()
+
+	s.batchBuffer = append(s.batchBuffer, log)
+
+	// Flush if batch is full (100 entries)
+	if len(s.batchBuffer) >= 100 {
+		s.flushBatchUnsafe()
+	}
+}
+
+// flushBatch flushes the batch buffer to the database (thread-safe)
+func (s *AuditService) flushBatch() {
+	s.batchMutex.Lock()
+	defer s.batchMutex.Unlock()
+	s.flushBatchUnsafe()
+}
+
+// flushBatchUnsafe flushes the batch buffer without locking (caller must hold lock)
+func (s *AuditService) flushBatchUnsafe() {
+	if len(s.batchBuffer) == 0 {
+		return
+	}
+
+	// Copy buffer for writing
+	toWrite := make([]*models.AuditLog, len(s.batchBuffer))
+	copy(toWrite, s.batchBuffer)
+
+	// Clear buffer
+	s.batchBuffer = s.batchBuffer[:0]
+
+	// Write to database (release lock during I/O)
+	s.batchMutex.Unlock()
+	if err := s.store.CreateAuditLogBatch(toWrite); err != nil {
+		log.Printf("Failed to write audit log batch: %v", err)
+	}
+	s.batchMutex.Lock()
+}
+
+// Log records an audit log entry asynchronously
+func (s *AuditService) Log(ctx context.Context, entry AuditLogEntry) {
+	if !s.enabled {
+		return
+	}
+
+	// Extract IP from context if not provided
+	if entry.ActorIP == "" {
+		entry.ActorIP = util.GetIPFromContext(ctx)
+	}
+
+	// Extract username from context if not provided
+	if entry.ActorUsername == "" {
+		entry.ActorUsername = util.GetUsernameFromContext(ctx)
+	}
+
+	// Mask sensitive data
+	entry.Details = maskSensitiveDetails(entry.Details)
+
+	// Create audit log
+	auditLog := &models.AuditLog{
+		ID:            uuid.New().String(),
+		EventType:     entry.EventType,
+		EventTime:     time.Now(),
+		Severity:      entry.Severity,
+		ActorUserID:   entry.ActorUserID,
+		ActorUsername: entry.ActorUsername,
+		ActorIP:       entry.ActorIP,
+		ResourceType:  entry.ResourceType,
+		ResourceID:    entry.ResourceID,
+		ResourceName:  entry.ResourceName,
+		Action:        entry.Action,
+		Details:       entry.Details,
+		Success:       entry.Success,
+		ErrorMessage:  entry.ErrorMessage,
+		UserAgent:     entry.UserAgent,
+		RequestPath:   entry.RequestPath,
+		RequestMethod: entry.RequestMethod,
+		CreatedAt:     time.Now(),
+	}
+
+	// Try to send to channel (non-blocking)
+	select {
+	case s.logChan <- auditLog:
+		// Successfully sent
+	default:
+		// Channel is full, drop the event and log warning
+		log.Printf("WARNING: Audit log buffer full, dropping event: %s", entry.Action)
+	}
+}
+
+// LogSync records an audit log entry synchronously (for critical events)
+func (s *AuditService) LogSync(ctx context.Context, entry AuditLogEntry) error {
+	if !s.enabled {
+		return nil
+	}
+
+	// Extract IP from context if not provided
+	if entry.ActorIP == "" {
+		entry.ActorIP = util.GetIPFromContext(ctx)
+	}
+
+	// Extract username from context if not provided
+	if entry.ActorUsername == "" {
+		entry.ActorUsername = util.GetUsernameFromContext(ctx)
+	}
+
+	// Mask sensitive data
+	entry.Details = maskSensitiveDetails(entry.Details)
+
+	// Create audit log
+	auditLog := &models.AuditLog{
+		ID:            uuid.New().String(),
+		EventType:     entry.EventType,
+		EventTime:     time.Now(),
+		Severity:      entry.Severity,
+		ActorUserID:   entry.ActorUserID,
+		ActorUsername: entry.ActorUsername,
+		ActorIP:       entry.ActorIP,
+		ResourceType:  entry.ResourceType,
+		ResourceID:    entry.ResourceID,
+		ResourceName:  entry.ResourceName,
+		Action:        entry.Action,
+		Details:       entry.Details,
+		Success:       entry.Success,
+		ErrorMessage:  entry.ErrorMessage,
+		UserAgent:     entry.UserAgent,
+		RequestPath:   entry.RequestPath,
+		RequestMethod: entry.RequestMethod,
+		CreatedAt:     time.Now(),
+	}
+
+	// Write directly to database
+	return s.store.CreateAuditLog(auditLog)
+}
+
+// GetAuditLogs retrieves audit logs with pagination and filtering
+func (s *AuditService) GetAuditLogs(
+	params store.PaginationParams,
+	filters store.AuditLogFilters,
+) ([]models.AuditLog, store.PaginationResult, error) {
+	return s.store.GetAuditLogsPaginated(params, filters)
+}
+
+// CleanupOldLogs deletes audit logs older than the retention period
+func (s *AuditService) CleanupOldLogs(retention time.Duration) (int64, error) {
+	cutoffTime := time.Now().Add(-retention)
+	return s.store.DeleteOldAuditLogs(cutoffTime)
+}
+
+// GetAuditLogStats returns statistics about audit logs
+func (s *AuditService) GetAuditLogStats(startTime, endTime time.Time) (store.AuditLogStats, error) {
+	return s.store.GetAuditLogStats(startTime, endTime)
+}
+
+// Shutdown gracefully shuts down the audit service
+func (s *AuditService) Shutdown(ctx context.Context) error {
+	if !s.enabled {
+		return nil
+	}
+
+	log.Println("Shutting down audit service...")
+
+	// Stop ticker
+	s.batchTicker.Stop()
+
+	// Signal worker to stop
+	close(s.shutdownCh)
+
+	// Wait for worker to finish with timeout
+	done := make(chan struct{})
+	go func() {
+		s.wg.Wait()
+		close(done)
+	}()
+
+	select {
+	case <-done:
+		log.Println("Audit service shut down gracefully")
+		return nil
+	case <-ctx.Done():
+		return fmt.Errorf("audit service shutdown timeout: %w", ctx.Err())
+	}
+}
+
+// maskSensitiveDetails masks sensitive information in audit log details
+func maskSensitiveDetails(details models.AuditDetails) models.AuditDetails {
+	if details == nil {
+		return details
+	}
+
+	masked := make(models.AuditDetails)
+	for key, value := range details {
+		// Complete masking for these fields
+		if isSensitiveField(key) {
+			masked[key] = "***REDACTED***"
+			continue
+		}
+
+		// Partial masking for tokens and codes
+		if isPartialMaskField(key) {
+			if str, ok := value.(string); ok && len(str) > 12 {
+				masked[key] = str[:8] + "..." + str[len(str)-4:]
+				continue
+			}
+		}
+
+		// Keep other fields as-is
+		masked[key] = value
+	}
+
+	return masked
+}
+
+// isSensitiveField checks if a field should be completely masked
+func isSensitiveField(key string) bool {
+	key = strings.ToLower(key)
+	sensitiveFields := []string{
+		"password",
+		"client_secret",
+		"token",
+		"access_token",
+		"refresh_token",
+		"secret",
+	}
+
+	for _, field := range sensitiveFields {
+		if strings.Contains(key, field) {
+			return true
+		}
+	}
+	return false
+}
+
+// isPartialMaskField checks if a field should be partially masked
+func isPartialMaskField(key string) bool {
+	key = strings.ToLower(key)
+	partialMaskFields := []string{
+		"device_code",
+		"token_id",
+	}
+
+	for _, field := range partialMaskFields {
+		if strings.Contains(key, field) {
+			return true
+		}
+	}
+	return false
+}


The audit service has no test coverage. Given that this is a critical security feature that handles sensitive audit logging with complex async behavior, batch processing, and shutdown logic, comprehensive tests are essential. The existing tests for other services (client_test.go, device_test.go, token_test.go) demonstrate that this codebase follows good testing practices, so the absence of audit service tests is a significant gap.

Tests should cover:

Async logging and batching behavior

Graceful shutdown with pending logs

Sensitive data masking

Buffer overflow handling

Concurrent access scenarios

Copilot · 2026-02-04T15:44:56Z

internal/services/audit.go

+	// Stop ticker
+	s.batchTicker.Stop()
+
+	// Signal worker to stop
+	close(s.shutdownCh)


The shutdown sequence has a potential issue. After stopping the ticker (line 268) and closing shutdownCh (line 271), there's no guarantee that pending logs in logChan will be processed. The worker goroutine will stop when it receives the shutdown signal, but any logs still in the channel buffer will be lost.

Consider draining the logChan before signaling shutdown to ensure all pending audit logs are written. For example:

Close logChan (not shutdownCh) to signal no more logs will be sent

Wait for worker to drain the channel and flush

Then close shutdownCh as a final signal

Copilot · 2026-02-04T15:44:56Z

internal/util/context.go

+	// Try to extract from Gin context first
+	if ginCtx, ok := ctx.(*gin.Context); ok {
+		return ginCtx.ClientIP()
+	}


Type assertion without safety check. The ctx.(*gin.Context) type assertion on line 22 will panic if ctx is a non-nil context that is not a *gin.Context. This should use a safe type assertion with the comma-ok idiom, which is already being done correctly.

However, the real issue is that context.Context is typically embedded or passed via Request.Context(), not as a *gin.Context itself. When calling this function with c.Request.Context() (as is common in the codebase), this type assertion will always fail silently and fall through to the context.Value check.

Consider documenting the expected usage pattern or restructuring to accept *gin.Context directly instead of context.Context.

Copilot · 2026-02-04T15:44:56Z

internal/util/context.go

+// GetIPFromContext extracts the client IP address from the context
+func GetIPFromContext(ctx context.Context) string {
+	// Try to extract from Gin context first
+	if ginCtx, ok := ctx.(*gin.Context); ok {
+		return ginCtx.ClientIP()
+	}
+
+	// Try to get from context value (set by middleware)
+	if ip, ok := ctx.Value("client_ip").(string); ok {
+		return ip
+	}
+
+	return ""
+}


The IP and username extraction will not work as designed. The middleware sets values using c.Set("client_ip", ...) in Gin's context, but the audit service is called with c.Request.Context(), which is a different context.Context instance. Gin's context values stored via c.Set() are not accessible through c.Request.Context().Value().

This means that ActorIP will always be empty in audit logs unless explicitly provided when calling auditService.Log().

To fix this, either:

Pass the *gin.Context directly to audit logging functions instead of context.Context

Copy relevant values from Gin context to the request context using context.WithValue()

Always explicitly provide ActorIP and ActorUsername when calling audit service

Copilot · 2026-02-04T15:44:57Z

internal/handlers/audit.go

+				Details: models.AuditDetails{
+					"record_count": len(logs),
+					"filters":      filters,


The audit log export includes the entire filters object in the Details field (line 338), which could contain large search strings or other filter values. Since Details is stored as JSON in the database, very large filter objects could bloat the audit log database and cause performance issues during export.

Consider either:

Limiting the size of the filters object that gets stored

Storing only a summary of filters (e.g., number of filters applied)

Truncating large string values in the filters before storing

internal/services/audit.go

Copilot · 2026-02-04T15:44:57Z

internal/middleware/auth.go

+		// Load user object for audit logging and other purposes
+		user, err := userService.GetUserByID(userID.(string))
+		if err == nil {
+			c.Set("user", user)
+		}
+


The middleware now queries the database on every authenticated request to load the user object (line 33). This adds latency and database load to all protected routes. Consider if this is necessary for all requests, or if the user object should only be loaded when actually needed (e.g., for admin pages or specific audit operations).

A more efficient approach might be to load the user object lazily or cache it, rather than doing a database query on every single authenticated request.

Suggested change

// Load user object for audit logging and other purposes

user, err := userService.GetUserByID(userID.(string))

if err == nil {

c.Set("user", user)

}

Copilot · 2026-02-04T15:44:57Z

internal/middleware/ratelimit.go

+			if username, exists := c.Get("username"); exists {
+				actorUsername = username.(string)
+			}


The rate limit handler tries to get username from context using c.Get("username"), but the RequireAuth middleware only sets "user_id" and "user" (lines 30 and 35 of internal/middleware/auth.go). The "username" key is never set in the context by any middleware, so this will always be empty.

To fix this, either:

Extract username from the "user" object if it exists: if user, exists := c.Get("user"); exists { if u, ok := user.(*models.User); ok { actorUsername = u.Username } }

Or add c.Set("username", user.Username) in the RequireAuth middleware

Suggested change

if username, exists := c.Get("username"); exists {

actorUsername = username.(string)

}

// Prefer username from the full user object if available

if userVal, exists := c.Get("user"); exists {

if u, ok := userVal.(*models.User); ok {

actorUsername = u.Username

}

}

// Fallback to username stored directly in context, if any

if actorUsername == "" {

if usernameVal, exists := c.Get("username"); exists {

if username, ok := usernameVal.(string); ok {

actorUsername = username

}

}

}

internal/handlers/audit.go

- Add configuration and documentation for comprehensive audit logging, including async batch writing, sensitive data masking, configurable retention, and admin-facing web interface - List new audit log API endpoints for viewing, exporting, statistics, and programmatic access - Document supported audit event types, severity levels, and database schema for audit logs - Update code, architecture, and server file structure documentation to reflect audit logging components - Provide best practices for secure and compliant audit log management, including monitoring, retention, and operational guidance Signed-off-by: appleboy <appleboy.tw@gmail.com>

Copilot AI review requested due to automatic review settings February 4, 2026 15:38

Copilot started reviewing on behalf of appleboy February 4, 2026 15:39 View session

Copilot AI reviewed Feb 4, 2026

View reviewed changes

appleboy merged commit 06a8841 into main Feb 5, 2026
16 checks passed

appleboy deleted the audit branch February 5, 2026 00:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat: introduce comprehensive audit logging across core services#27

feat: introduce comprehensive audit logging across core services#27
appleboy merged 2 commits intomainfrom
audit

appleboy commented Feb 4, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 4, 2026

Uh oh!

Copilot AI Feb 4, 2026

Uh oh!

Copilot AI Feb 4, 2026

Uh oh!

Copilot AI Feb 4, 2026

Uh oh!

Copilot AI Feb 4, 2026

Uh oh!

Copilot AI Feb 4, 2026

Uh oh!

Uh oh!

Copilot AI Feb 4, 2026

Uh oh!

Copilot AI Feb 4, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	EventType: "AUDIT_LOG_EXPORTED",
	EventType: models.EventTypeAuditLogExported,

-			if username, exists := c.Get("username"); exists {
-				actorUsername = username.(string)
-			}
+			// Prefer username from the full user object if available
+			if userVal, exists := c.Get("user"); exists {
+				if u, ok := userVal.(*models.User); ok {
+					actorUsername = u.Username
+				}
+			}
+			// Fallback to username stored directly in context, if any
+			if actorUsername == "" {
+				if usernameVal, exists := c.Get("username"); exists {
+					if username, ok := usernameVal.(string); ok {
+						actorUsername = username
+					}
+				}
+			}

Uh oh!

Conversation

appleboy commented Feb 4, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant