Pretrained weight not available

[minghanz]

Hi, thanks for the nice implementation.
In dp/modules/backbone/resnet.py it requires a pretrained weight file. I see in the readme.md file in https://github.com/dontLoveBugs/DORN_pytorch, you mentioned the link to the pretrained weight. However, the link http://sceneparsing.csail.mit.edu/model/pretrained_resnet/resnet101-imagenet.pth is invalid now. Could you provide a download link to the weight? Or is there another way to solve the problem? Thank you!

[minghanz]

The link is available now... Sorry for disturbing.

[dontLoveBugs]

Hi, I suggest you use this pretrained resnet101(link: https://drive.google.com/file/d/1iELk6WeQ1smockQJGKU_DEG6slcqw6Mu/view?usp=sharing).

[minghanz]

Thank you! What are the difference between these two? I saw some renaming of the building blocks in the ResNet backbone. By comparing your code with the corresponding part in https://github.com/CSAILVision/semantic-segmentation-pytorch/blob/6d83839b2178bc8f2e6ccfefa5baa822a7a6082c/mit_semseg/models/resnet.py and making some small modification to align them, I was able to load the MIT weight correctly. Is there any difference between the two weights other than the renaming?

[dontLoveBugs]

Thank you! What are the difference between these two? I saw some renaming of the building blocks in the ResNet backbone. By comparing your code with the corresponding part in https://github.com/CSAILVision/semantic-segmentation-pytorch/blob/6d83839b2178bc8f2e6ccfefa5baa822a7a6082c/mit_semseg/models/resnet.py and making some small modification to align them, I was able to load the MIT weight correctly. Is there any difference between the two weights other than the renaming?

In my experiments, the results using pretrained resnet from CSAILVison was bad, so I change to use the pretrained model from torchseg. At fact, I haven't understand the difference between the these two weights. I guess that the image preprecess of these two models is different. If you are interested in it, I suggest you should compare both the model codes and the dataset codes of these two repositories. If not, you can just use the pretrained model from torchseg and it work well.

[minghanz]

In my experiments both the two weights does not provide satisfying result, and I am suspecting whether there is anything wrong with my modification of your code (I modified it because your original code reports errors and cannot be directly run.). Therefore I want to check with you whether the modification is appropriate.

1. The dorn model uses resnet backbone to generate features and feed it to SceneUnderstandingModule, but the resnet backbone outputs a dict of features from several layers from its forward() function , while the SceneUnderstandingModule forward() takes a tensor as input . The forward() function of resnet backbone should output feature x5, is it correct?

2. The line in [solver.py] (

   SupervisedDepthPrediction/dp/core/solver.py

   Line 110 in b293ed8

   params["lr"] = self.config["solver"]["optimizer"]["params"]["lr"] * params["lr"]

   ) reports error because params["lr"] does not exist for the first item of group_params in function DepthPredModel.optimizer_params(). Shall I modify it to this? :

group_params = [{"params": filter(lambda p: p.requires_grad, self.SceneUnderstandingModule.parameters()),
                             "lr": 10.0}]

That is excluding the backbone weights from training?

3. Does input crop size matter? I modified tr_crop_size and te_crop_size to [352, 704]. Do you think it will drastically affect the performance?

Thank you!

[dontLoveBugs]

In my experiments both the two weights does not provide satisfying result, and I am suspecting whether there is anything wrong with my modification of your code (I modified it because your original code reports errors and cannot be directly run.). Therefore I want to check with you whether the modification is appropriate.

1. The dorn model uses resnet backbone to generate features and feed it to SceneUnderstandingModule, but the resnet backbone outputs a dict of features from several layers from its forward() function , while the SceneUnderstandingModule forward() takes a tensor as input . The forward() function of resnet backbone should output feature x5, is it correct?
2. The line in [solver.py] (

   SupervisedDepthPrediction/dp/core/solver.py

   Line 110 in b293ed8

   params["lr"] = self.config["solver"]["optimizer"]["params"]["lr"] * params["lr"]

   
   ) reports error because params["lr"] does not exist for the first item of group_params in function DepthPredModel.optimizer_params(). Shall I modify it to this? :

group_params = [{"params": filter(lambda p: p.requires_grad, self.SceneUnderstandingModule.parameters()),
                             "lr": 10.0}]

That is excluding the backbone weights from training?

1. Does input crop size matter? I modified tr_crop_size and te_crop_size to [352, 704]. Do you think it will drastically affect the performance?

Thank you!

1. The reason that the resnet backbone outputs a dict of features from several layers from its forward() function is that It is for convenience to use low level features in future work.
2. DepthPredModel.optimizer_params() is to set different learning rate for different modules. What your modified is right.
3. Maybe you should use adam optimizer. And you should check the crop opteration to prevent outputing crop wrong image, because the image height of kitti is smaller than 352 and it should be resize to 352 before croping.

[minghanz]

3\. Maybe you should use adam optimizer. And you should check the crop opteration to prevent outputing crop wrong image, because the image height of kitti is smaller than 352 and it should be resize to 352 before croping.

In my knowledge the kitti images are about the size 1242*375 (not exactly the same size over all images due to rectification I think), so using 352 as image height always results in cropping without scaling. Actually this is the reason I use this cropping size. Using 385 as in original config file sometimes result in inconsistent depth image size inside a mini-batch causing failures. Using a cropping size smaller than all original images can guarantee that all rgb images and depth images are of the same size after preprocessing. Is it correct?

[dontLoveBugs]

3\. Maybe you should use adam optimizer. And you should check the crop opteration to prevent outputing crop wrong image, because the image height of kitti is smaller than 352 and it should be resize to 352 before croping.

In my knowledge the kitti images are about the size 1242*375 (not exactly the same size over all images due to rectification I think), so using 352 as image height always results in cropping without scaling. Actually this is the reason I use this cropping size. Using 385 as in original config file sometimes result in inconsistent depth image size inside a mini-batch causing failures. Using a cropping size smaller than all original images can guarantee that all rgb images and depth images are of the same size after preprocessing. Is it correct?

Sorry, I made a mistake about the height of kitti image. You are right. So crop size maybe not the reason of the bad results.

[ControllableGeneration]

3\. Maybe you should use adam optimizer. And you should check the crop opteration to prevent outputing crop wrong image, because the image height of kitti is smaller than 352 and it should be resize to 352 before croping.

In my knowledge the kitti images are about the size 1242*375 (not exactly the same size over all images due to rectification I think), so using 352 as image height always results in cropping without scaling. Actually this is the reason I use this cropping size. Using 385 as in original config file sometimes result in inconsistent depth image size inside a mini-batch causing failures. Using a cropping size smaller than all original images can guarantee that all rgb images and depth images are of the same size after preprocessing. Is it correct?

I think by 352 he meant the val_selection_cropped images of KITTI, which are of size 1216x352