Skip to content

feat: increase api limit for data loads #1526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
paustint wants to merge 2 commits into
base: main
Choose a base branch
from
+71 −18
Open

feat: increase api limit for data loads #1526

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d5c044c
feat: increase api limit for data loads
paustint Feb 3, 2026
d913068
fix: retain permission manager filter text on tab change
paustint Feb 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions apps/docs/docs/load/load.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,14 @@ The batch size determines how many records will be loaded at once.

You don't normally need to adjust this, but if you run into limits (e.x. CPU timeout or too many SOQL queries), then you can reduce this number to limit how many records are processed together.

:::tip

- The free plan has an API limit of 1,000 calls per data load.
- If you are on a paid plan, you have an API limit of 5,000 calls per data load.
- If you are using the Browser Extension or Desktop App, there is no limit imposed by Jetstream.

:::

### Date Format

This will normally be auto-detected based on your locale, but make sure that it matches the format of the dates in your file.
Expand Down
5 changes: 3 additions & 2 deletions libs/features/load-records/src/steps/PerformLoad.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ export const LoadRecordsPerformLoad: FunctionComponent<LoadRecordsPerformLoadPro
const hasDateFieldMapped = useAtomValue(fromLoadRecordsState.selectHasDateFieldMapped);

const batchSizeError = useAtomValue(fromLoadRecordsState.selectBatchSizeError);
const batchApiLimitWarning = useAtomValue(fromLoadRecordsState.selectBatchApiLimitWarning);
const batchApiLimitError = useAtomValue(fromLoadRecordsState.selectBatchApiLimitError);
const trialRunSizeError = useAtomValue(fromLoadRecordsState.selectTrialRunSizeError);
const bulkApiModeLabel = useAtomValue(fromLoadRecordsState.selectBulkApiModeLabel);
Expand Down Expand Up @@ -258,9 +259,9 @@ export const LoadRecordsPerformLoad: FunctionComponent<LoadRecordsPerformLoadPro
id="batch-size"
label="Batch Size"
isRequired
hasError={!!batchSizeError || !!batchApiLimitError}
hasError={!!batchSizeError || !!batchApiLimitError || !!batchApiLimitWarning}
Copy link

Copilot AI Feb 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The warning should not be treated as an error. Setting hasError to true when there's only a warning will make the field appear invalid (likely with red styling) even though it's just informational. The warning message can still be displayed without setting hasError to true. Consider displaying the warning separately or only setting hasError for actual errors (batchSizeError and batchApiLimitError).

Suggested change
hasError={!!batchSizeError || !!batchApiLimitError || !!batchApiLimitWarning}
hasError={!!batchSizeError || !!batchApiLimitError}

Copilot uses AI. Check for mistakes.
errorMessageId="batch-size-error"
errorMessage={batchSizeError || batchApiLimitError}
errorMessage={batchSizeError || batchApiLimitError || batchApiLimitWarning}
labelHelp="The batch size determines how many records will be modified at a time. Only change this if you are experiencing issues with Salesforce governor limits."
helpText={hasZipAttachment ? 'The batch size will be auto-calculated based on the size of the attachments.' : null}
>
Expand Down
3 changes: 3 additions & 0 deletions libs/features/manage-permissions/src/ManagePermissionsEditor.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -712,6 +712,7 @@ export const ManagePermissionsEditor: FunctionComponent<ManagePermissionsEditorP
columns={objectColumns}
rows={visibleObjectRows || []}
totalCount={objectRows?.length || 0}
filterText={objectFilter}
onFilter={setObjectFilter}
onBulkUpdate={handleObjectBulkRowUpdate}
onDirtyRows={setDirtyObjectRows}
Expand Down Expand Up @@ -743,6 +744,7 @@ export const ManagePermissionsEditor: FunctionComponent<ManagePermissionsEditorP
columns={tabVisibilityColumns}
rows={visibleTabVisibilityRows || []}
totalCount={objectRows?.length || 0}
filterText={tabVisibilityFilter}
onFilter={setTabVisibilityFilter}
onBulkUpdate={handleTabVisibilityBulkRowUpdate}
onDirtyRows={setDirtyTabVisibilityRows}
Expand Down Expand Up @@ -773,6 +775,7 @@ export const ManagePermissionsEditor: FunctionComponent<ManagePermissionsEditorP
columns={fieldColumns}
rows={visibleFieldRows || []}
totalCount={fieldRows?.length || 0}
filterText={fieldFilter}
onFilter={setFieldFilter}
onBulkUpdate={handleFieldBulkRowUpdate}
onDirtyRows={setDirtyFieldRows}
Expand Down
4 changes: 3 additions & 1 deletion libs/features/manage-permissions/src/ManagePermissionsEditorFieldTable.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,14 @@ export interface ManagePermissionsEditorFieldTableProps {
columns: ColumnWithFilter<PermissionTableFieldCell, PermissionTableSummaryRow>[];
rows: PermissionTableFieldCell[];
totalCount: number;
filterText?: string;
onFilter: (value: string) => void;
onBulkUpdate: (rows: PermissionTableFieldCell[], indexes?: number[]) => void;
onDirtyRows?: (values: Record<string, DirtyRow<PermissionTableFieldCell>>) => void;
}

export const ManagePermissionsEditorFieldTable = forwardRef<any, ManagePermissionsEditorFieldTableProps>(
({ columns, rows, totalCount, onFilter, onDirtyRows, onBulkUpdate }, ref) => {
({ columns, rows, totalCount, filterText, onFilter, onDirtyRows, onBulkUpdate }, ref) => {
const tableRef = useRef<DataTableRef<PermissionTableFieldCell>>(null);
const [dirtyRows, setDirtyRows] = useState<Record<string, DirtyRow<PermissionTableFieldCell>>>({});
const [expandedGroupIds, setExpandedGroupIds] = useState(() => new Set<any>(rows.map((row) => row.sobject)));
Expand Down Expand Up @@ -81,6 +82,7 @@ export const ManagePermissionsEditorFieldTable = forwardRef<any, ManagePermissio
{
type: 'field',
totalCount,
filterValue: filterText,
onFilterRows: onFilter,
onColumnAction: handleColumnAction,
onBulkAction: onBulkUpdate,
Expand Down
4 changes: 3 additions & 1 deletion libs/features/manage-permissions/src/ManagePermissionsEditorObjectTable.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,14 @@ export interface ManagePermissionsEditorObjectTableProps {
columns: ColumnWithFilter<PermissionTableObjectCell, PermissionTableSummaryRow>[];
rows: PermissionTableObjectCell[];
totalCount: number;
filterText?: string;
onFilter: (value: string) => void;
onBulkUpdate: (rows: PermissionTableObjectCell[], indexes?: number[]) => void;
onDirtyRows?: (values: Record<string, DirtyRow<PermissionTableObjectCell>>) => void;
}

export const ManagePermissionsEditorObjectTable = forwardRef<any, ManagePermissionsEditorObjectTableProps>(
({ columns, rows, totalCount, onFilter, onBulkUpdate, onDirtyRows }, ref) => {
({ columns, rows, totalCount, filterText, onFilter, onBulkUpdate, onDirtyRows }, ref) => {
const tableRef = useRef<DataTableRef<PermissionTableObjectCell>>(null);
const [dirtyRows, setDirtyRows] = useState<Record<string, DirtyRow<PermissionTableObjectCell>>>({});

Expand Down Expand Up @@ -70,6 +71,7 @@ export const ManagePermissionsEditorObjectTable = forwardRef<any, ManagePermissi
{
type: 'object',
totalCount,
filterValue: filterText,
onFilterRows: onFilter,
onColumnAction: handleColumnAction,
onBulkAction: onBulkUpdate,
Expand Down
4 changes: 3 additions & 1 deletion libs/features/manage-permissions/src/ManagePermissionsEditorTabVisibilityTable.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,13 +22,14 @@ export interface ManagePermissionsEditorTabVisibilityTableProps {
columns: ColumnWithFilter<PermissionTableTabVisibilityCell, PermissionTableSummaryRow>[];
rows: PermissionTableTabVisibilityCell[];
totalCount: number;
filterText?: string;
onFilter: (value: string) => void;
onBulkUpdate: (rows: PermissionTableTabVisibilityCell[], indexes?: number[]) => void;
onDirtyRows?: (values: Record<string, DirtyRow<PermissionTableTabVisibilityCell>>) => void;
}

export const ManagePermissionsEditorTabVisibilityTable = forwardRef<any, ManagePermissionsEditorTabVisibilityTableProps>(
({ columns, rows, totalCount, onFilter, onBulkUpdate, onDirtyRows }, ref) => {
({ columns, rows, totalCount, filterText, onFilter, onBulkUpdate, onDirtyRows }, ref) => {
const tableRef = useRef<DataTableRef<PermissionTableTabVisibilityCell>>(null);
const [dirtyRows, setDirtyRows] = useState<Record<string, DirtyRow<PermissionTableTabVisibilityCell>>>({});

Expand Down Expand Up @@ -70,6 +71,7 @@ export const ManagePermissionsEditorTabVisibilityTable = forwardRef<any, ManageP
{
type: 'tabVisibility',
totalCount,
filterValue: filterText,
onFilterRows: onFilter,
onColumnAction: handleColumnAction,
onBulkAction: onBulkUpdate,
Expand Down
4 changes: 2 additions & 2 deletions libs/features/manage-permissions/src/utils/permission-manager-table-utils.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1446,8 +1446,8 @@ export const RowActionRenderer = ({
* This component provides a modal that the user can open to make changes that apply to an entire visible table
*/
export const ColumnSearchFilter = () => {
const { onFilterRows } = useContext(DataTableGenericContext) as PermissionManagerTableContext;
return <SearchInput id="column-filter" value="" placeholder="Filter..." onChange={onFilterRows} />;
const { filterValue: initialFilterValue, onFilterRows } = useContext(DataTableGenericContext) as PermissionManagerTableContext;
return <SearchInput id="column-filter" value={initialFilterValue || ''} placeholder="Filter..." onChange={onFilterRows} />;
};

export const ColumnSearchFilterSummary = () => {
Expand Down
1 change: 0 additions & 1 deletion libs/shared/ui-core/src/load/load-records-utils.tsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ export interface SavedFieldMapping {
export const SELF_LOOKUP_KEY = '~SELF_LOOKUP~';
export const STATIC_MAPPING_PREFIX = '~STATIC~MAPPING~';
export const BATCH_RECOMMENDED_THRESHOLD = 2000;
export const MAX_API_CALLS = 250;
export const MAX_BULK = 10000;
export const MAX_BATCH = 200;
const DEFAULT_NON_EXT_ID_MAPPING_OPT: NonExtIdLookupOption = 'ERROR_IF_MULTIPLE';
Expand Down
55 changes: 45 additions & 10 deletions libs/shared/ui-core/src/state-management/load-records.state.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,10 @@
import { detectDateFormatForLocale, formatNumber } from '@jetstream/shared/ui-utils';
import { detectDateFormatForLocale, formatNumber, isBrowserExtension, isDesktop } from '@jetstream/shared/ui-utils';
import { ApiMode, DescribeGlobalSObjectResult, FieldMapping, InsertUpdateUpsertDelete, LocalOrGoogle, Maybe } from '@jetstream/types';
import { hasPaidPlanState } from '@jetstream/ui/app-state';
import { atom } from 'jotai';
import { atomWithReset } from 'jotai/utils';
import isNumber from 'lodash/isNumber';
import {
BATCH_RECOMMENDED_THRESHOLD,
MAX_API_CALLS,
MAX_BULK,
getLabelWithOptionalRecommended,
getMaxBatchSize,
} from '../load/load-records-utils';
import { BATCH_RECOMMENDED_THRESHOLD, MAX_BULK, getLabelWithOptionalRecommended, getMaxBatchSize } from '../load/load-records-utils';

const SUPPORTED_ATTACHMENT_OBJECTS = new Map<string, { bodyField: string }>();
SUPPORTED_ATTACHMENT_OBJECTS.set('Attachment', { bodyField: 'Body' });
Expand Down Expand Up @@ -93,14 +88,54 @@ export const selectBatchSizeError = atom<string | null>((get) => {
return null;
});

const loadApiLimitsState = atom((get) => {
const API_WARNING_CALLS = 250;
const API_MAX_CALLS_FREE = 1000;
const API_MAX_CALLS_PAID = 5000;

if (isDesktop() || isBrowserExtension()) {
return {
warningApiCalls: API_WARNING_CALLS,
maxApiCalls: Infinity,
};
}

const hasPaidPlan = get(hasPaidPlanState);
if (hasPaidPlan) {
return {
warningApiCalls: API_WARNING_CALLS,
maxApiCalls: API_MAX_CALLS_PAID,
};
}

return {
warningApiCalls: API_WARNING_CALLS,
maxApiCalls: API_MAX_CALLS_FREE,
};
});

export const selectBatchApiLimitWarning = atom<string | null>((get) => {
const { warningApiCalls } = get(loadApiLimitsState);
const inputFileDataLength = get(inputFileDataState)?.length || 0;
const batchSize = get(batchSizeState) || 1;

if (inputFileDataLength && batchSize && inputFileDataLength / batchSize > warningApiCalls) {
const numApiCalls = Math.round(inputFileDataLength / batchSize);
return `Your configuration requires ${formatNumber(numApiCalls)} calls to Salesforce which will contribute to your API call limits.`;
}
return null;
});

export const selectBatchApiLimitError = atom<string | null>((get) => {
const { maxApiCalls } = get(loadApiLimitsState);
const inputFileDataLength = get(inputFileDataState)?.length || 0;
const batchSize = get(batchSizeState) || 1;
if (inputFileDataLength && batchSize && inputFileDataLength / batchSize > MAX_API_CALLS) {

if (inputFileDataLength && batchSize && inputFileDataLength / batchSize > maxApiCalls) {
const numApiCalls = Math.round(inputFileDataLength / batchSize);
return (
`Either your batch size is too low or you are loading in too many records. ` +
`Your configuration would require ${formatNumber(numApiCalls)} calls to Salesforce, which exceeds the limit of ${MAX_API_CALLS}. ` +
`Your configuration would require ${formatNumber(numApiCalls)} calls to Salesforce, which exceeds the limit of ${formatNumber(maxApiCalls)}. ` +
`Increase your batch size or reduce the number of records in your file.`
);
}
Expand Down
1 change: 1 addition & 0 deletions libs/types/src/lib/ui/permission-manager-types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,7 @@ export interface PermissionManagerTableContext {
type: PermissionType;
rows: PermissionTableCellExtended[];
totalCount: number;
filterValue?: string;
onFilterRows: (value: string) => void;
onRowAction: (action: 'selectAll' | 'unselectAll' | 'reset', columnKey: string) => void;
onColumnAction: (action: 'selectAll' | 'unselectAll' | 'reset', columnKey: string) => void;
Expand Down
Loading