-
Notifications
You must be signed in to change notification settings - Fork 16.7k
Description
Bug description
Bug description
There are two related bugs that combine to make SQL Lab's "Data preview" tab spin forever on large tables:
Bug 1: queryLimit not set on preview queries
In superset-frontend/src/SqlLab/actions/sqlLab.ts, the runTablePreviewQuery function builds a dataPreviewQuery object (line ~1332) but never sets queryLimit:
const dataPreviewQuery: Query = {
id: newTable.previewQueryId ?? nanoid(11),
dbId: dbId as number | undefined,
// ... other fields ...
isDataPreview: true,
// queryLimit is MISSING
};The PREVIEW_QUERY_LIMIT = 100 constant (line 66 of TablePreview/index.tsx) is only used for the displayLimit and defaultQueryLimit UI props — it is never sent to the backend.
When runQuery sends the payload, queryLimit is undefined. On the backend, _get_limit_param in superset/sqllab/sqllab_execution_context.py evaluates:
limit = apply_max_row_limit(query_params.get("queryLimit") or 0)None or 0 → 0, and apply_max_row_limit(0) returns SQL_MAX_ROW (default 100,000). Then apply_limit adds +1 for the overflow detection row, so the actual database query runs with LIMIT 100001 instead of LIMIT 101.
This means the "100 row preview" actually fetches up to 100,001 rows from the database, which on large tables produces massive payloads that can overwhelm the results backend.
Fix: Add queryLimit: PREVIEW_QUERY_LIMIT (or just 100) to the dataPreviewQuery object in runTablePreviewQuery.
Bug 2: float vs decimal.Decimal TypeError in results error handler
In superset/commands/sql_lab/results.py, line ~96:
query_age_seconds = now_as_float() - (
self._query.end_time if self._query.end_time else now_as_float()
)now_as_float()returns a Pythonfloatself._query.end_timeis aColumn(Numeric(precision=20, scale=6))which SQLAlchemy returns asdecimal.Decimal
This raises TypeError: unsupported operand type(s) for -: 'float' and 'decimal.Decimal', which crashes the error handler into a 500 instead of returning the intended 410 "re-run your query" message. The frontend receives no actionable error, so the spinner runs indefinitely.
Fix: Cast to float: float(self._query.end_time) if self._query.end_time else now_as_float()
How to reproduce
- Connect a database with a large table in SQL Lab
- Expand the table in the schema explorer
- Click the "Data preview" tab
- Observe the spinner never resolves
Check the Superset logs for:
TypeError: unsupported operand type(s) for -: 'float' and 'decimal.Decimal'
Expected results
Data preview should fetch 100 rows quickly and display them, or show a clear error message if results are unavailable.
Actual results
The preview fetches up to 100,001 rows from the database. If the results backend fails to store or return the large payload, the error handler crashes with a TypeError, and the frontend shows an infinite spinner with no error message.
Environment
- Apache Superset (current master)
- Results backend: Redis (via
cachelib.RedisCache) - Database: PostgreSQL
Checklist
- I have searched Superset for duplicate issues
- I have provided steps to reproduce
Screenshots/recordings
No response
Superset version
master / latest-dev
Python version
3.12
Node version
16
Browser
Chrome
Additional context
Traceback (most recent call last):
File "/usr/local/lib/python3.12/dist-packages/flask/app.py", line 1484, in full_dispatch_request
rv = self.dispatch_request()
^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/flask/app.py", line 1469, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/flask_appbuilder/security/decorators.py", line 109, in wraps
return f(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/superset/views/base_api.py", line 120, in wraps
duration, response = time_function(f, self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/superset/utils/core.py", line 1410, in time_function
response = func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/flask_appbuilder/api/__init__.py", line 202, in wraps
return f(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/superset/utils/log.py", line 304, in wrapper
value = f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/superset/sqllab/api.py", line 340, in get_results
result = SqlExecutionResultsCommand(key=key, rows=rows).run()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/superset/commands/sql_lab/results.py", line 127, in run
self.validate()
File "/usr/local/lib/python3.12/dist-packages/superset/commands/sql_lab/results.py", line 96, in validate
query_age_seconds = now_as_float() - (
^^^^^^^^^^^^^^^^^^
TypeError: unsupported operand type(s) for -: 'float' and 'decimal.Decimal'
Checklist
- I have searched Superset docs and Slack and didn't find a solution to my problem.
- I have searched the GitHub issue tracker and didn't find a similar bug report.
- I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.