Error when reading parquet file #1302
Description
When I try to read this parquet file into rumble using `parquet-file("rail-traffic-information.parquet")' I get the following exception:
(base) ~/Documents/MscThesis/rumble-test-suite/rumble git:[master]
spark-submit target/rumbledb-1.23.0-jar-with-dependencies.jar run -q 'parquet-file("../performance_test_data/rail-traffic-information.parquet")' -v
25/02/24 15:07:33 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
25/02/24 15:07:33 INFO SparkContext: Running Spark version 3.5.3
25/02/24 15:07:33 INFO SparkContext: OS info Mac OS X, 15.3, aarch64
25/02/24 15:07:33 INFO SparkContext: Java version 11.0.24
25/02/24 15:07:33 INFO ResourceUtils: ==============================================================
25/02/24 15:07:33 INFO ResourceUtils: No custom resources configured for spark.driver.
25/02/24 15:07:33 INFO ResourceUtils: ==============================================================
25/02/24 15:07:33 INFO SparkContext: Submitted application: rumbledb-1.23.0-jar-with-dependencies.jar
25/02/24 15:07:33 INFO ResourceProfile: Default ResourceProfile created, executor resources: Map(cores -> name: cores, amount: 1, script: , vendor: , memory -> name: memory, amount: 1024, script: , vendor: , offHeap -> name: offHeap, amount: 0, script: , vendor: ), task resources: Map(cpus -> name: cpus, amount: 1.0)
25/02/24 15:07:33 INFO ResourceProfile: Limiting resource is cpu
25/02/24 15:07:33 INFO ResourceProfileManager: Added ResourceProfile id: 0
25/02/24 15:07:33 INFO SecurityManager: Changing view acls to: marco
25/02/24 15:07:33 INFO SecurityManager: Changing modify acls to: marco
25/02/24 15:07:33 INFO SecurityManager: Changing view acls groups to:
25/02/24 15:07:33 INFO SecurityManager: Changing modify acls groups to:
25/02/24 15:07:33 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: marco; groups with view permissions: EMPTY; users with modify permissions: marco; groups with modify permissions: EMPTY
25/02/24 15:07:33 INFO Utils: Successfully started service 'sparkDriver' on port 56525.
25/02/24 15:07:33 INFO SparkEnv: Registering MapOutputTracker
25/02/24 15:07:33 INFO SparkEnv: Registering BlockManagerMaster
25/02/24 15:07:33 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information
25/02/24 15:07:33 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up
25/02/24 15:07:33 INFO SparkEnv: Registering BlockManagerMasterHeartbeat
25/02/24 15:07:33 INFO DiskBlockManager: Created local directory at /private/var/folders/0z/x_vsy0cx7lg6q9_wm7d2njb40000gn/T/blockmgr-cab76575-d186-454a-ab79-4316e22bf584
25/02/24 15:07:33 INFO MemoryStore: MemoryStore started with capacity 434.4 MiB
25/02/24 15:07:33 INFO SparkEnv: Registering OutputCommitCoordinator
25/02/24 15:07:33 INFO JettyUtils: Start Jetty 0.0.0.0:4040 for SparkUI
25/02/24 15:07:33 INFO Utils: Successfully started service 'SparkUI' on port 4040.
25/02/24 15:07:33 INFO SparkContext: Added JAR file:/Users/marco/Documents/MscThesis/rumble-test-suite/rumble/target/rumbledb-1.23.0-jar-with-dependencies.jar at spark://student-net-cx-3801.intern.ethz.ch:56525/jars/rumbledb-1.23.0-jar-with-dependencies.jar with timestamp 1740406053626
25/02/24 15:07:34 INFO Executor: Starting executor ID driver on host student-net-cx-3801.intern.ethz.ch
25/02/24 15:07:34 INFO Executor: OS info Mac OS X, 15.3, aarch64
25/02/24 15:07:34 INFO Executor: Java version 11.0.24
25/02/24 15:07:34 INFO Executor: Starting executor with user classpath (userClassPathFirst = false): ''
25/02/24 15:07:34 INFO Executor: Created or updated repl class loader org.apache.spark.util.MutableURLClassLoader@23648d2d for default.
25/02/24 15:07:34 INFO Executor: Fetching spark://student-net-cx-3801.intern.ethz.ch:56525/jars/rumbledb-1.23.0-jar-with-dependencies.jar with timestamp 1740406053626
25/02/24 15:07:34 INFO TransportClientFactory: Successfully created connection to student-net-cx-3801.intern.ethz.ch/10.5.46.217:56525 after 12 ms (0 ms spent in bootstraps)
25/02/24 15:07:34 INFO Utils: Fetching spark://student-net-cx-3801.intern.ethz.ch:56525/jars/rumbledb-1.23.0-jar-with-dependencies.jar to /private/var/folders/0z/x_vsy0cx7lg6q9_wm7d2njb40000gn/T/spark-b3473532-4f7a-4e08-ad4e-1f3c6dc02332/userFiles-531d49a8-3f14-47cb-b0a2-4fd5d2bdf43b/fetchFileTemp17715657943641833060.tmp
25/02/24 15:07:34 INFO Executor: Adding file:/private/var/folders/0z/x_vsy0cx7lg6q9_wm7d2njb40000gn/T/spark-b3473532-4f7a-4e08-ad4e-1f3c6dc02332/userFiles-531d49a8-3f14-47cb-b0a2-4fd5d2bdf43b/rumbledb-1.23.0-jar-with-dependencies.jar to class loader default
25/02/24 15:07:34 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 56527.
25/02/24 15:07:34 INFO NettyBlockTransferService: Server created on student-net-cx-3801.intern.ethz.ch:56527
25/02/24 15:07:34 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy
25/02/24 15:07:34 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, student-net-cx-3801.intern.ethz.ch, 56527, None)
25/02/24 15:07:34 INFO BlockManagerMasterEndpoint: Registering block manager student-net-cx-3801.intern.ethz.ch:56527 with 434.4 MiB RAM, BlockManagerId(driver, student-net-cx-3801.intern.ethz.ch, 56527, None)
25/02/24 15:07:34 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, student-net-cx-3801.intern.ethz.ch, 56527, None)
25/02/24 15:07:34 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, student-net-cx-3801.intern.ethz.ch, 56527, None)
25/02/24 15:07:34 INFO SharedState: Setting hive.metastore.warehouse.dir ('null') to the value of spark.sql.warehouse.dir.
25/02/24 15:07:34 INFO SharedState: Warehouse path is 'file:/Users/marco/Documents/MscThesis/rumble-test-suite/rumble/spark-warehouse'.
25/02/24 15:07:34 INFO InMemoryFileIndex: It took 18 ms to list leaf files for 1 paths.
25/02/24 15:07:35 INFO SparkContext: Starting job: parquet at InferTypeVisitor.java:463
25/02/24 15:07:35 INFO DAGScheduler: Got job 0 (parquet at InferTypeVisitor.java:463) with 1 output partitions
25/02/24 15:07:35 INFO DAGScheduler: Final stage: ResultStage 0 (parquet at InferTypeVisitor.java:463)
25/02/24 15:07:35 INFO DAGScheduler: Parents of final stage: List()
25/02/24 15:07:35 INFO DAGScheduler: Missing parents: List()
25/02/24 15:07:35 INFO DAGScheduler: Submitting ResultStage 0 (MapPartitionsRDD[1] at parquet at InferTypeVisitor.java:463), which has no missing parents
25/02/24 15:07:35 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 105.3 KiB, free 434.3 MiB)
25/02/24 15:07:35 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 37.9 KiB, free 434.3 MiB)
25/02/24 15:07:35 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on student-net-cx-3801.intern.ethz.ch:56527 (size: 37.9 KiB, free: 434.4 MiB)
25/02/24 15:07:35 INFO SparkContext: Created broadcast 0 from broadcast at DAGScheduler.scala:1585
25/02/24 15:07:35 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 0 (MapPartitionsRDD[1] at parquet at InferTypeVisitor.java:463) (first 15 tasks are for partitions Vector(0))
25/02/24 15:07:35 INFO TaskSchedulerImpl: Adding task set 0.0 with 1 tasks resource profile 0
25/02/24 15:07:35 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0) (student-net-cx-3801.intern.ethz.ch, executor driver, partition 0, PROCESS_LOCAL, 9311 bytes)
25/02/24 15:07:35 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
25/02/24 15:07:35 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 1204 bytes result sent to driver
25/02/24 15:07:35 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 200 ms on student-net-cx-3801.intern.ethz.ch (executor driver) (1/1)
25/02/24 15:07:35 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool
25/02/24 15:07:35 INFO DAGScheduler: ResultStage 0 (parquet at InferTypeVisitor.java:463) finished in 0.566 s
25/02/24 15:07:35 INFO DAGScheduler: Job 0 is finished. Cancelling potential speculative or zombie tasks for this job
25/02/24 15:07:35 INFO TaskSchedulerImpl: Killing all running tasks in stage 0: Stage finished
25/02/24 15:07:35 INFO DAGScheduler: Job 0 finished: parquet at InferTypeVisitor.java:463, took 0.588293 s
25/02/24 15:07:35 INFO BlockManagerInfo: Removed broadcast_0_piece0 on student-net-cx-3801.intern.ethz.ch:56527 in memory (size: 37.9 KiB, free: 434.4 MiB)
25/02/24 15:07:36 INFO InMemoryFileIndex: It took 0 ms to list leaf files for 1 paths.
25/02/24 15:07:36 INFO SparkContext: Starting job: parquet at ParquetFileFunctionIterator.java:66
25/02/24 15:07:36 INFO DAGScheduler: Got job 1 (parquet at ParquetFileFunctionIterator.java:66) with 1 output partitions
25/02/24 15:07:36 INFO DAGScheduler: Final stage: ResultStage 1 (parquet at ParquetFileFunctionIterator.java:66)
25/02/24 15:07:36 INFO DAGScheduler: Parents of final stage: List()
25/02/24 15:07:36 INFO DAGScheduler: Missing parents: List()
25/02/24 15:07:36 INFO DAGScheduler: Submitting ResultStage 1 (MapPartitionsRDD[3] at parquet at ParquetFileFunctionIterator.java:66), which has no missing parents
25/02/24 15:07:36 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 105.5 KiB, free 434.3 MiB)
25/02/24 15:07:36 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 38.0 KiB, free 434.3 MiB)
25/02/24 15:07:36 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on student-net-cx-3801.intern.ethz.ch:56527 (size: 38.0 KiB, free: 434.4 MiB)
25/02/24 15:07:36 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:1585
25/02/24 15:07:36 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 1 (MapPartitionsRDD[3] at parquet at ParquetFileFunctionIterator.java:66) (first 15 tasks are for partitions Vector(0))
25/02/24 15:07:36 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks resource profile 0
25/02/24 15:07:36 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 1) (student-net-cx-3801.intern.ethz.ch, executor driver, partition 0, PROCESS_LOCAL, 9311 bytes)
25/02/24 15:07:36 INFO Executor: Running task 0.0 in stage 1.0 (TID 1)
25/02/24 15:07:36 INFO Executor: Finished task 0.0 in stage 1.0 (TID 1). 1118 bytes result sent to driver
25/02/24 15:07:36 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 1) in 9 ms on student-net-cx-3801.intern.ethz.ch (executor driver) (1/1)
25/02/24 15:07:36 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool
25/02/24 15:07:36 INFO DAGScheduler: ResultStage 1 (parquet at ParquetFileFunctionIterator.java:66) finished in 0.018 s
25/02/24 15:07:36 INFO DAGScheduler: Job 1 is finished. Cancelling potential speculative or zombie tasks for this job
25/02/24 15:07:36 INFO TaskSchedulerImpl: Killing all running tasks in stage 1: Stage finished
25/02/24 15:07:36 INFO DAGScheduler: Job 1 finished: parquet at ParquetFileFunctionIterator.java:66, took 0.021147 s
25/02/24 15:07:36 INFO FileSourceStrategy: Pushed Filters:
25/02/24 15:07:36 INFO FileSourceStrategy: Post-Scan Filters:
25/02/24 15:07:36 INFO BlockManagerInfo: Removed broadcast_1_piece0 on student-net-cx-3801.intern.ethz.ch:56527 in memory (size: 38.0 KiB, free: 434.4 MiB)
25/02/24 15:07:36 INFO CodeGenerator: Code generated in 106.068458 ms
25/02/24 15:07:36 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 204.4 KiB, free 434.2 MiB)
25/02/24 15:07:36 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 36.2 KiB, free 434.2 MiB)
25/02/24 15:07:36 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on student-net-cx-3801.intern.ethz.ch:56527 (size: 36.2 KiB, free: 434.4 MiB)
25/02/24 15:07:36 INFO SparkContext: Created broadcast 2 from javaRDD at JSoundDataFrame.java:111
25/02/24 15:07:36 INFO FileSourceScanExec: Planning scan with bin packing, max size: 16008161 bytes, open cost is considered as scanning 4194304 bytes.
25/02/24 15:07:36 INFO SparkContext: Starting job: take at SparkSessionManager.java:262
25/02/24 15:07:36 INFO DAGScheduler: Got job 2 (take at SparkSessionManager.java:262) with 1 output partitions
25/02/24 15:07:36 INFO DAGScheduler: Final stage: ResultStage 2 (take at SparkSessionManager.java:262)
25/02/24 15:07:36 INFO DAGScheduler: Parents of final stage: List()
25/02/24 15:07:36 INFO DAGScheduler: Missing parents: List()
25/02/24 15:07:36 INFO DAGScheduler: Submitting ResultStage 2 (MapPartitionsRDD[10] at map at HybridRuntimeIterator.java:162), which has no missing parents
25/02/24 15:07:36 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 32.3 KiB, free 434.1 MiB)
25/02/24 15:07:36 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 12.6 KiB, free 434.1 MiB)
25/02/24 15:07:36 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on student-net-cx-3801.intern.ethz.ch:56527 (size: 12.6 KiB, free: 434.4 MiB)
25/02/24 15:07:36 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:1585
25/02/24 15:07:36 INFO DAGScheduler: Submitting 1 missing tasks from ResultStage 2 (MapPartitionsRDD[10] at map at HybridRuntimeIterator.java:162) (first 15 tasks are for partitions Vector(0))
25/02/24 15:07:36 INFO TaskSchedulerImpl: Adding task set 2.0 with 1 tasks resource profile 0
25/02/24 15:07:36 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 2) (student-net-cx-3801.intern.ethz.ch, executor driver, partition 0, PROCESS_LOCAL, 9976 bytes)
25/02/24 15:07:36 INFO Executor: Running task 0.0 in stage 2.0 (TID 2)
25/02/24 15:07:36 INFO CodeGenerator: Code generated in 16.935375 ms
25/02/24 15:07:37 INFO CodeGenerator: Code generated in 12.140583 ms
25/02/24 15:07:37 INFO FileScanRDD: Reading File path: file:///Users/marco/Documents/MscThesis/rumble-test-suite/performance_test_data/rail-traffic-information.parquet, range: 0-16008161, partition values: [empty row]
25/02/24 15:07:37 INFO CodecPool: Got brand-new decompressor [.snappy]
25/02/24 15:07:37 ERROR Executor: Exception in task 0.0 in stage 2.0 (TID 2)
java.lang.IllegalArgumentException: No instant converter found for type: java.time.Instant
at org.joda.time.convert.ConverterManager.getInstantConverter(ConverterManager.java:166)
at org.joda.time.base.BaseDateTime.<init>(BaseDateTime.java:171)
at org.joda.time.DateTime.<init>(DateTime.java:259)
at org.rumbledb.items.parsing.ItemParser.convertValueToItem(ItemParser.java:542)
at org.rumbledb.items.parsing.ItemParser.getItemFromRow(ItemParser.java:366)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:47)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:1)
at org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)
at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
at scala.collection.Iterator$SliceIterator.next(Iterator.scala:1238)
at scala.collection.mutable.Growable.addAll(Growable.scala:62)
at scala.collection.mutable.Growable.addAll$(Growable.scala:57)
at scala.collection.mutable.ArrayBuilder.addAll(ArrayBuilder.scala:66)
at scala.collection.IterableOnceOps.toArray(IterableOnce.scala:1282)
at scala.collection.IterableOnceOps.toArray$(IterableOnce.scala:1276)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1293)
at org.apache.spark.rdd.RDD.$anonfun$take$2(RDD.scala:1492)
at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2433)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
25/02/24 15:07:37 WARN TaskSetManager: Lost task 0.0 in stage 2.0 (TID 2) (student-net-cx-3801.intern.ethz.ch executor driver): java.lang.IllegalArgumentException: No instant converter found for type: java.time.Instant
at org.joda.time.convert.ConverterManager.getInstantConverter(ConverterManager.java:166)
at org.joda.time.base.BaseDateTime.<init>(BaseDateTime.java:171)
at org.joda.time.DateTime.<init>(DateTime.java:259)
at org.rumbledb.items.parsing.ItemParser.convertValueToItem(ItemParser.java:542)
at org.rumbledb.items.parsing.ItemParser.getItemFromRow(ItemParser.java:366)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:47)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:1)
at org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)
at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
at scala.collection.Iterator$SliceIterator.next(Iterator.scala:1238)
at scala.collection.mutable.Growable.addAll(Growable.scala:62)
at scala.collection.mutable.Growable.addAll$(Growable.scala:57)
at scala.collection.mutable.ArrayBuilder.addAll(ArrayBuilder.scala:66)
at scala.collection.IterableOnceOps.toArray(IterableOnce.scala:1282)
at scala.collection.IterableOnceOps.toArray$(IterableOnce.scala:1276)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1293)
at org.apache.spark.rdd.RDD.$anonfun$take$2(RDD.scala:1492)
at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2433)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
25/02/24 15:07:37 ERROR TaskSetManager: Task 0 in stage 2.0 failed 1 times; aborting job
25/02/24 15:07:37 INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool
25/02/24 15:07:37 INFO TaskSchedulerImpl: Cancelling stage 2
25/02/24 15:07:37 INFO TaskSchedulerImpl: Killing all running tasks in stage 2: Stage cancelled: Job aborted due to stage failure: Task 0 in stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID 2) (student-net-cx-3801.intern.ethz.ch executor driver): java.lang.IllegalArgumentException: No instant converter found for type: java.time.Instant
at org.joda.time.convert.ConverterManager.getInstantConverter(ConverterManager.java:166)
at org.joda.time.base.BaseDateTime.<init>(BaseDateTime.java:171)
at org.joda.time.DateTime.<init>(DateTime.java:259)
at org.rumbledb.items.parsing.ItemParser.convertValueToItem(ItemParser.java:542)
at org.rumbledb.items.parsing.ItemParser.getItemFromRow(ItemParser.java:366)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:47)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:1)
at org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)
at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
at scala.collection.Iterator$SliceIterator.next(Iterator.scala:1238)
at scala.collection.mutable.Growable.addAll(Growable.scala:62)
at scala.collection.mutable.Growable.addAll$(Growable.scala:57)
at scala.collection.mutable.ArrayBuilder.addAll(ArrayBuilder.scala:66)
at scala.collection.IterableOnceOps.toArray(IterableOnce.scala:1282)
at scala.collection.IterableOnceOps.toArray$(IterableOnce.scala:1276)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1293)
at org.apache.spark.rdd.RDD.$anonfun$take$2(RDD.scala:1492)
at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2433)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Driver stacktrace:
25/02/24 15:07:37 INFO DAGScheduler: ResultStage 2 (take at SparkSessionManager.java:262) failed in 0.823 s due to Job aborted due to stage failure: Task 0 in stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID 2) (student-net-cx-3801.intern.ethz.ch executor driver): java.lang.IllegalArgumentException: No instant converter found for type: java.time.Instant
at org.joda.time.convert.ConverterManager.getInstantConverter(ConverterManager.java:166)
at org.joda.time.base.BaseDateTime.<init>(BaseDateTime.java:171)
at org.joda.time.DateTime.<init>(DateTime.java:259)
at org.rumbledb.items.parsing.ItemParser.convertValueToItem(ItemParser.java:542)
at org.rumbledb.items.parsing.ItemParser.getItemFromRow(ItemParser.java:366)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:47)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:1)
at org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)
at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
at scala.collection.Iterator$SliceIterator.next(Iterator.scala:1238)
at scala.collection.mutable.Growable.addAll(Growable.scala:62)
at scala.collection.mutable.Growable.addAll$(Growable.scala:57)
at scala.collection.mutable.ArrayBuilder.addAll(ArrayBuilder.scala:66)
at scala.collection.IterableOnceOps.toArray(IterableOnce.scala:1282)
at scala.collection.IterableOnceOps.toArray$(IterableOnce.scala:1276)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1293)
at org.apache.spark.rdd.RDD.$anonfun$take$2(RDD.scala:1492)
at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2433)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Driver stacktrace:
25/02/24 15:07:37 INFO DAGScheduler: Job 2 failed: take at SparkSessionManager.java:262, took 0.826686 s
We are very embarrassed, because an error has occured that we did not anticipate 🙈: No instant converter found for type: java.time.Instant
We would like to investigate this and make sure to fix it. We would be very grateful if you could contact us or file an issue on GitHub with your query.
Link: https://github.com/RumbleDB/rumble/issues
For more debug info (e.g., so you can communicate it to us), please try again using --show-error-info yes in your command line.
java.lang.IllegalArgumentException: No instant converter found for type: java.time.Instant
at org.joda.time.convert.ConverterManager.getInstantConverter(ConverterManager.java:166)
at org.joda.time.base.BaseDateTime.<init>(BaseDateTime.java:171)
at org.joda.time.DateTime.<init>(DateTime.java:259)
at org.rumbledb.items.parsing.ItemParser.convertValueToItem(ItemParser.java:542)
at org.rumbledb.items.parsing.ItemParser.getItemFromRow(ItemParser.java:366)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:47)
at org.rumbledb.items.parsing.RowToItemMapper.call(RowToItemMapper.java:1)
at org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)
at scala.collection.Iterator$$anon$9.next(Iterator.scala:577)
at scala.collection.Iterator$SliceIterator.next(Iterator.scala:1238)
at scala.collection.mutable.Growable.addAll(Growable.scala:62)
at scala.collection.mutable.Growable.addAll$(Growable.scala:57)
at scala.collection.mutable.ArrayBuilder.addAll(ArrayBuilder.scala:66)
at scala.collection.IterableOnceOps.toArray(IterableOnce.scala:1282)
at scala.collection.IterableOnceOps.toArray$(IterableOnce.scala:1276)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1293)
at org.apache.spark.rdd.RDD.$anonfun$take$2(RDD.scala:1492)
at org.apache.spark.SparkContext.$anonfun$runJob$5(SparkContext.scala:2433)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:93)
at org.apache.spark.TaskContext.runTaskWithListeners(TaskContext.scala:166)
at org.apache.spark.scheduler.Task.run(Task.scala:141)
at org.apache.spark.executor.Executor$TaskRunner.$anonfun$run$4(Executor.scala:620)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally(SparkErrorUtils.scala:64)
at org.apache.spark.util.SparkErrorUtils.tryWithSafeFinally$(SparkErrorUtils.scala:61)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:94)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:623)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
25/02/24 15:07:37 INFO SparkContext: Invoking stop() from shutdown hook
25/02/24 15:07:37 INFO SparkContext: SparkContext is stopping with exitCode 0.
25/02/24 15:07:37 INFO SparkUI: Stopped Spark web UI at http://student-net-cx-3801.intern.ethz.ch:4040
25/02/24 15:07:37 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
25/02/24 15:07:37 INFO MemoryStore: MemoryStore cleared
25/02/24 15:07:37 INFO BlockManager: BlockManager stopped
25/02/24 15:07:37 INFO BlockManagerMaster: BlockManagerMaster stopped
25/02/24 15:07:37 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
25/02/24 15:07:37 INFO SparkContext: Successfully stopped SparkContext
25/02/24 15:07:37 INFO ShutdownHookManager: Shutdown hook called
25/02/24 15:07:37 INFO ShutdownHookManager: Deleting directory /private/var/folders/0z/x_vsy0cx7lg6q9_wm7d2njb40000gn/T/spark-b3473532-4f7a-4e08-ad4e-1f3c6dc02332
25/02/24 15:07:37 INFO ShutdownHookManager: Deleting directory /private/var/folders/0z/x_vsy0cx7lg6q9_wm7d2njb40000gn/T/spark-102e19a2-976d-4a06-aa38-24aa69d9aca4
```