지속 가능한 꾸준함

Weight Decay 관련 실험 해보다가 기록용으로 적음 yolov5 프로젝트 보면 weight decay를 Batch Normalization Layer 파라미터와 Bias 값들을 제외하고 적용하는 거를 확인할 수 있다. 재구현을 해볼때 이런 거 하나 하나가 되게 사소해보이는데 이런 거 하나 빼먹으면 성능 재현이 안되는 경우가 있다. (예를 들면 "weight decay 몇 레이어에 적용해주고 안해주고 차이가 그리 크겠어?"란 생각을 갖고 그냥 모든 파라미터 다 먹이면 되지~라는 식의 접근... 굉장히 위험하다. 나중에 성능 재현이 안돼서 이걸로 수일 수주 수개월의 시간을 쓸수도 있다...) 근데 yolov5 에서 adamw 를 optimizer로 사용하도록 세팅을 해줄 수가 있는데 뭔가 이상한 점을..

https://arxiv.org/abs/2203.16527 Exploring Plain Vision Transformer Backbones for Object Detection We explore the plain, non-hierarchical Vision Transformer (ViT) as a backbone network for object detection. This design enables the original ViT architecture to be fine-tuned for object detection without needing to redesign a hierarchical backbone for pre- arxiv.org 배경 Abstract에서 첫 문장으로 We explore ..

https://github.com/xingyizhou/CenterNet/blob/master/src/lib/opts.py GitHub - xingyizhou/CenterNet: Object detection, 3D detection, and pose estimation using center point detection: Object detection, 3D detection, and pose estimation using center point detection: - GitHub - xingyizhou/CenterNet: Object detection, 3D detection, and pose estimation using center point detection: github.com

https://github.com/ZJULearning/ttfnet/blob/700ad90c50786c8c16c4efa35c60baa907a07461/mmdet/models/anchor_heads/ttf_head.py#L334-L365 GitHub - ZJULearning/ttfnet Contribute to ZJULearning/ttfnet development by creating an account on GitHub. github.com

PyTorch에서 제공하는 AutoMixedPrecision 기능을 활용하여 모델을 학습하다 보면 학습이 잘 안되는 경우가 발생한다. 구글링을 해보면 이런 경우를 심심치 않게 찾아볼 수 있다. https://github.com/pytorch/pytorch/issues/40497 Mixed precision causes NaN loss · Issue #40497 · pytorch/pytorch 🐛 Bug I'm using autocast with GradScaler to train on mixed precision. For small dataset, it works fine. But when I trained on bigger dataset, after few epochs (3-4), the loss tu..

https://github.com/pytorch/pytorch/blob/46a88036afacad5aee5ba2640d6055051bb879a1/aten/src/ATen/native/Normalization.cpp#L189-L192 GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration Tensors and Dynamic neural networks in Python with strong GPU acceleration - GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU ac..

https://notmatthancock.github.io/2017/03/23/simple-batch-stat-updates.html Batch updates for simple statistics It’s typical in many optimization problems in data analysis to standardize your data by subtracting off the empirical mean and dividing by the empirical standard deviation. However, in “online” situations where the data is acquired one at a time or i notmatthancock.github.io

https://github.com/ptrblck/pytorch_misc/blob/31ac50c415f16cf7fec277dbdba72b9fb4d732d3/batch_norm_manual.py#L39 GitHub - ptrblck/pytorch_misc: Code snippets created for the PyTorch discussion board Code snippets created for the PyTorch discussion board - GitHub - ptrblck/pytorch_misc: Code snippets created for the PyTorch discussion board github.com

https://github.com/zhenghao977/FCOS-PyTorch-37.2AP/blob/2bfa4b6ca57358f52f7bc7b44f506608e99894e6/model/fcos.py#L22-L35 GitHub - zhenghao977/FCOS-PyTorch-37.2AP: A pure torch implement of FCOS 37.2AP A pure torch implement of FCOS 37.2AP. Contribute to zhenghao977/FCOS-PyTorch-37.2AP development by creating an account on GitHub. github.com def freeze_bn(module): if isinstance(module,nn.BatchNorm2..

https://velog.io/@markany/%EB%94%A5-%EB%9F%AC%EB%8B%9D-Object-Detection-Adaptive-Training-Sample-Selection [딥 러닝, Object Detection] ATSS: Adaptive Training Sample Selection 본 포스트에서 리뷰할 논문의 제목은 Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection 입니다.한국어로 표현하기 모호한 부분은 논문에서 velog.io 회사 블로그에 정리