Skip to content

optimize ctime transform statement #584

@elliVM

Description

@elliVM

Description
string conversion to time might be costly operation which could be moved to private final class member field. the same goes for the timeFormat which is unlikely to differ across the calls to the Ctime udf.

Notes:

  • timeformat is final and set in AbstractConvertStep and can be moved to the class
  • Conversion could possibly be done using functions.from_unixtime from spark without the need to use custom function
  • time format is non-standard we should consider switching to use ISO8061

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions