Articles in the category of Code Snippet

  1. Home
  2. Code Snippet

PyTorch DistributedDataParallel 单机多卡训练 踩坑记录

报错：Default process group has not been initialized, please make sure to call init_process_group.调用torch.distributed下任何函数前，必须运行torch.distributed.init_process_group(backend='nccl')初始化。DistributedSampl...

  • MrXiao
  • April 4, 2021
  • No comments
    •

    无聊笔记：在PyTorch中捕获CUDA Out of Memory并动态调整batch_size

    最近正跟随老师做有关NAS方面的工作，想将NAS与自己的想法结合起来验证一些有趣的结论，这其中会涉及到生成不同的网络结构提交到GPU做验证的过程。那么问题...

  • MrXiao
  • October 25, 2020
  • No comments