Is it actually working?

[JeffRossMT]

Hello!

I was delighted to find this project!

I have the beginnings of an all sky coverage camera system consisting of 6 individual cameras. Initially I currently have only the north facing camera in operation.

I have a 14 frame movie of a bolide that I tried to process.
Link to the online movie is: https://www.starhouse-observatory.org/20230823_fireball.mp4
As you can see, there is an initial meteor and then a couple of seconds later a bolide.
Processing this movie (locally mounted) with this
[jross@elitedesk MetDetPy-2.4.0]$ python MetDetPy.py "/samba/weather/20230823_fireball.mp4" --save-path .
yields this:

[jross@elitedesk MetDetPy-2.4.0]$ python MetDetPy.py "/samba/weather/20230823_fireball.mp4" --save-path .
Info: Parsing "exp_option"=auto
Info: Metainfo FPS = 1.00
Warning: Slow FPS detected. Use 1.00s directly.
Info: ThreadVideoLoader summary:
    Video path: "/samba/weather/20230823_fireball.mp4"; Mask: None
    Video frames = 14; Apply grayscale = True;
    Raw resolution = [1920, 1080]; Running-time resolution = [960, 540];
    Video decode backend: PyAVVideoWrapper(FFmpeg);
Apply exposure time of 1.00s.(MinTimeFlag = 1000)
Total frames = 14 ; FPS = 1.00 (rFPS = 1.00)
Info: Preprocessing finished. Time cost: 0.3s.
Info: Parsing "exp_option"=real-time
Info: Metainfo FPS = 1.00
Warning: Slow FPS detected. Use 1.00s directly.
Info: Sucessfully load ./weights/yolov5s_v2.onnx on device= AzureExecutionProvider with Warmup=True.
 57%|████████████████████████████████████▌                           | 8/14 [00:00<00:00, 14.72it/s]Error: EOFError(541478725, 'End of file', 'avcodec_send_packet()') encountered when readingvideo frame with PyAVVideoWrapper.
Warning: Load frame failed at 13
 93%|██████████████████████████████████████████████████████████▌    | 13/14 [00:00<00:00, 16.36it/s]
Info: Acceptable exception occured.
Info: VideoLoader-stop detected.
Info: Time cost: 0.7954s.

The output json file only has this:

jross@elitedesk MetDetPy-2.4.0]$ cat 20230823_fireball.json
{
    "version": "V2.4.0",
    "basic_info": {
        "loader": "ThreadVideoLoader",
        "video": "/samba/weather/20230823_fireball.mp4",
        "mask": null,
        "start_time": 0,
        "end_time": 14000,
        "resolution": [
            1920,
            1080
        ],
        "runtime_resolution": [
            960,
            540
        ],
        "exp_time": 1.0,
        "total_frames": 14,
        "fps": 1.0,
        "desc": null
    },
    "config": {
        "loader": {
            "name": "ThreadVideoLoader",
            "wrapper": "PyAVVideoWrapper",
            "resize": 960,
            "exp_time": "auto",
            "merge_func": "max",
            "grayscale": true,
            "upper_bound": 0.5,
            "continue_on_err": false
        },
        "detector": {
            "name": "M3Detector",
            "window_sec": 1,
            "cfg": {
                "binary": {
                    "adaptive_bi_thre": true,
                    "init_value": 7,
                    "sensitivity": "normal",
                    "area": 0.1,
                    "interval": 2
                },
                "hough_line": {
                    "threshold": 10,
                    "min_len": 10,
                    "max_gap": 10
                },
                "dynamic": {
                    "dy_mask": true,
                    "window_sec": 5
                }
            }
        },
        "collector": {
            "meteor_cfg": {
                "min_len": 15,
                "max_interval": 4,
                "time_range": [
                    0,
                    8
                ],
                "speed_range": [
                    2,
                    21
                ],
                "drct_range": [
                    0,
                    0.6
                ],
                "det_thre": 0.5,
                "thre2": 2048
            },
            "recheck_cfg": {
                "switch": true,
                "model": {
                    "name": "YOLOModel",
                    "weight_path": "./weights/yolov5s_v2.onnx",
                    "dtype": "float32",
                    "nms": true,
                    "warmup": true,
                    "pos_thre": 0.25,
                    "nms_thre": 0.45,
                    "multiscale_pred": 2,
                    "multiscale_partition": 2,
                    "providers_key": "default"
                }
            },
            "positive_cfg": {
                "positive_cates": [
                    "METEOR",
                    "RED_SPRITE"
                ]
            }
        }
    },
    "type": "prediction",
    "anno_size": [
        1920,
        1080
    ],
    "results": [],
    "performance": null
}

No results?

I might not called the MetDetPy.py script with the correct values. I hate to say it but a real tutorial or some more thorough documentation would be a big help.

I'm really looking forward to being able to process the movies I get in search of meteors!

Thanks!
Jeff Ross

[Designerspr]

Hi Jeff,

Thanks a lot for your report and for your interest in our project!
I’ve downloaded the video from the link you provided and was able to reproduce the issue locally.

The key point here is that the provided video has a frame rate of 1 fps, which is very likely the reason why no detections are produced.

Although this is not clearly stated in the current documentation, the default detector in MetDetPy (the m3det series) is designed to operate on a sliding window containing at least 3 frames. Internally, it relies on modified frame differencing, so fewer than 3 frames are insufficient.

There is a configuration parameter called window_sec in the detector section, which controls the window length in seconds. When fps × window_sec < 3, m3det cannot compute valid difference frames, and therefore no detection results are generated.
We agree this should be checked explicitly, and we plan to add a validation step to warn users about this situation.

There are several possible ways to address this:

1. (Recommended) Increase the frame rate of your camera system while keeping the image quality at a usable level. Using 4 fps or higher should allow MetDetPy to work as expected.

2. If the frame rate is fixed or cannot be modified, you can try the dldet detector instead. This method is based on deep learning and is not sensitive to the window_sec / sliding-window limitation. To use dldet, add --cfg ./config/dldet.json in your commend line.

3. Since each frame in your video effectively represents one second, another option is to treat the video as a sequence of still images. In this case, you can use MetDetPhoto.py to detect and export meteor frames directly from the video.

Btw I also tested the approaches above on this bolide video. Unfortunately, the results are still not ideal in many cases: the bolide is often classified as dropped event rather than a true positive. This is a known limitation: MetDetPy currently struggles with very bright bolides that briefly illuminate most of the screen.
We are actively working on this and plan to introduce an additional brightness–based event detector to reduce missed detections for such cases.

P.S. we currently do lack a proper user tutorial (the existing docs are more like developer notes). Thanks for pointing this out; improving the documentation is on our roadmap.

[JeffRossMT]

Excellent answer! Thank you very much!

My images are taken with .5s exposures, then 3 of those are stacked to reduce noise before ffmpeg-ing into an mp4.

For an extreme event like this I use the raw frames combined with ffmpeg at 1 frame per second--otherwise it's too darned short ;-)

I'll run a real movie through MetDetPy.py with known real-time meteors and report back.

It's a fascinating project--now if I could just find something similar for detecting aurora!

Jeff