Skip to content

fix: SQL Lab data preview fetches SQL_MAX_ROW instead of 100 rows, and results error handler crashes on Decimal type #37612

@andy-clapson

Description

@andy-clapson

Bug description

Bug description

There are two related bugs that combine to make SQL Lab's "Data preview" tab spin forever on large tables:

Bug 1: queryLimit not set on preview queries

In superset-frontend/src/SqlLab/actions/sqlLab.ts, the runTablePreviewQuery function builds a dataPreviewQuery object (line ~1332) but never sets queryLimit:

const dataPreviewQuery: Query = {
    id: newTable.previewQueryId ?? nanoid(11),
    dbId: dbId as number | undefined,
    // ... other fields ...
    isDataPreview: true,
    // queryLimit is MISSING
};

The PREVIEW_QUERY_LIMIT = 100 constant (line 66 of TablePreview/index.tsx) is only used for the displayLimit and defaultQueryLimit UI props — it is never sent to the backend.

When runQuery sends the payload, queryLimit is undefined. On the backend, _get_limit_param in superset/sqllab/sqllab_execution_context.py evaluates:

limit = apply_max_row_limit(query_params.get("queryLimit") or 0)

None or 00, and apply_max_row_limit(0) returns SQL_MAX_ROW (default 100,000). Then apply_limit adds +1 for the overflow detection row, so the actual database query runs with LIMIT 100001 instead of LIMIT 101.

This means the "100 row preview" actually fetches up to 100,001 rows from the database, which on large tables produces massive payloads that can overwhelm the results backend.

Fix: Add queryLimit: PREVIEW_QUERY_LIMIT (or just 100) to the dataPreviewQuery object in runTablePreviewQuery.

Bug 2: float vs decimal.Decimal TypeError in results error handler

In superset/commands/sql_lab/results.py, line ~96:

query_age_seconds = now_as_float() - (
    self._query.end_time if self._query.end_time else now_as_float()
)
  • now_as_float() returns a Python float
  • self._query.end_time is a Column(Numeric(precision=20, scale=6)) which SQLAlchemy returns as decimal.Decimal

This raises TypeError: unsupported operand type(s) for -: 'float' and 'decimal.Decimal', which crashes the error handler into a 500 instead of returning the intended 410 "re-run your query" message. The frontend receives no actionable error, so the spinner runs indefinitely.

Fix: Cast to float: float(self._query.end_time) if self._query.end_time else now_as_float()

How to reproduce

  1. Connect a database with a large table in SQL Lab
  2. Expand the table in the schema explorer
  3. Click the "Data preview" tab
  4. Observe the spinner never resolves

Check the Superset logs for:

TypeError: unsupported operand type(s) for -: 'float' and 'decimal.Decimal'

Expected results

Data preview should fetch 100 rows quickly and display them, or show a clear error message if results are unavailable.

Actual results

The preview fetches up to 100,001 rows from the database. If the results backend fails to store or return the large payload, the error handler crashes with a TypeError, and the frontend shows an infinite spinner with no error message.

Environment

  • Apache Superset (current master)
  • Results backend: Redis (via cachelib.RedisCache)
  • Database: PostgreSQL

Checklist

  • I have searched Superset for duplicate issues
  • I have provided steps to reproduce

Screenshots/recordings

No response

Superset version

master / latest-dev

Python version

3.12

Node version

16

Browser

Chrome

Additional context

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/flask/app.py", line 1484, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/flask/app.py", line 1469, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/flask_appbuilder/security/decorators.py", line 109, in wraps
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/superset/views/base_api.py", line 120, in wraps
    duration, response = time_function(f, self, *args, **kwargs)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/superset/utils/core.py", line 1410, in time_function
    response = func(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/flask_appbuilder/api/__init__.py", line 202, in wraps
    return f(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/superset/utils/log.py", line 304, in wrapper
    value = f(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/superset/sqllab/api.py", line 340, in get_results
    result = SqlExecutionResultsCommand(key=key, rows=rows).run()
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/superset/commands/sql_lab/results.py", line 127, in run
    self.validate()
  File "/usr/local/lib/python3.12/dist-packages/superset/commands/sql_lab/results.py", line 96, in validate
    query_age_seconds = now_as_float() - (
                        ^^^^^^^^^^^^^^^^^^
TypeError: unsupported operand type(s) for -: 'float' and 'decimal.Decimal'

Checklist

  • I have searched Superset docs and Slack and didn't find a solution to my problem.
  • I have searched the GitHub issue tracker and didn't find a similar bug report.
  • I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

Metadata

Metadata

Assignees

No one assigned

    Labels

    sqllabNamespace | Anything related to the SQL Labvalidation:requiredA committer should validate the issue

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions