Papers I Read Notes and Summaries

Memory-based Parameter Adaptation

Introduction

  • Standard Deep Learning networks are not suitable for continual learning setting...


Born Again Neural Networks

Introduction

  • The paper explores knowledge distillation (KD) from the perspective of transferring...


Net2Net-Accelerating Learning via Knowledge Transfer

Notes

  • The paper presents a simple yet effective approach for transferring knowledge...


Learning to Count Objects in Natural Images for Visual Question Answering

Introduction

  • Most of the visual question-answering (VQA) models perform poorly on the task...

Neural Message Passing for Quantum Chemistry

Introduction

  • The paper presents a general message passing architecture called as Message...


Unsupervised Learning by Predicting Noise

Introduction

  • Convolutional Neural Networks are extremely good feature extractors in the sense...


The Lottery Ticket Hypothesis - Training Pruned Neural Networks

Introduction

  • Empirical evidence indicates that at training time, the neural networks need...


Cyclical Learning Rates for Training Neural Networks

Introduction

  • Conventional wisdom says that when training neural networks, learning rate should...


Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning

Introduction

  • Information Extraction - Given a query to be answered and an...


An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks

Introduction

  • Catastrophic Forgetting refers to the phenomenon where when a learning system...