How I shrank my training script by 50% using the new features from PyTorch lightning 1.1 and Hydra 1.0

Short note on the 2nd edition: Back in August 2020, I wrote a story about how I used PyTorch Lightning 0.9.0 and Hydra’s fourth release candidate for 1.0.0 to shrink my training script by 50%. Half a year later in February 2021, we now have PyTorch Lightning 1.1 and Hydra 1.0. No major changes were introduced in Hydra 1.0, but PyTorch Lightning 1.0 did include major changes such as the deprecation of the Result abstraction. So I decided to write this 2nd edition of my original post to “keep up” with PyTorch Lightning and Hydra. You can still read the…


How I shrank my training script by 50% using the new features from PyTorch Lightning 1.1 and Hydra 1.0

Try to keep up! — Source

Update 02/09/2021: This story is about PyTorch Lightning 0.9.0 and Hydra 1.0.0rc4. Since then, they have released their official production ready versions, and I have released the 2nd edition of this story that includes all the latest changes. I’m leaving this story up for posterity, but please check out the 2nd edition!

Introduction

Last week, PyTorch Lightning 0.9.0 and Hydra’s fourth release candidate for 1.0.0 were released with a choke-full of new features and mostly final APIs. I thought it’d be a good time for me to revisit my side project Leela Zero PyTorch to see how these new versions can…


A simple training pipeline for Leela Zero implemented with PyTorch, PyTorch Lightning and Hydra

Lightning and Hydra (?) — Source

Recently, I have been looking into ways to speed up my research and manage my experiments, especially around writing training pipelines and managing experiment configurations, and I discovered these two new projects called PyTorch Lightning and Hydra. PyTorch Lightning helps you write training pipelines quickly, while Hydra helps you manage configurations in a clean way.

In order to practice using them in a more realistic setting, I decided to write a training pipeline for Leela Zero, a Go engine. I chose to do this because it is a well-scoped project with interesting technical challenges related to training huge networks on…


Detecting pneumonia opacities from chest X-Ray images using deep learning.

One day back in August, I was catching up with my best friend from high school who is now a radiology resident. One thing led to another, and we started talking about our interests in artificial intelligence and machine learning and its possible applications in radiology. A couple of months after our talk, I stumbled upon a Kaggle challenge hosted by the Radiological Society of North American (RSNA). It was a competition that we could work on together, so I immediately called my friend. Joined by my brother, we formed a team to compete in this Kaggle challenge. After a…


Replicating AlphaGo for Chess

I have a brother who’s two years younger than me, and as with all brothers, we grew up competing with each other in pretty much everything. Our competitions more often than not resembled the story of The Tortoise and the Hare. I, ever the hare, would be the first one to be interested in a game, quickly learn how to play, and lose interest as fast as I learned to play. …


Cryptocurrency Robo Trader via Reinforcement Learning

Can we teach computers to do this for us?

About a month ago, I was looking for project ideas to practice AI programming using PyTorch when a friend of mine who’s been very excited about the whole cryptocurrency craze started telling me about Ethereum. I’m quite conservative when it comes to my money and investments, so I usually stay away from this kind of hubbub, but I thought it’d make a good practice project if I somehow combined cryptocurrency and machine learning together. And this is how I came up with the idea of creating a cryptocurrency robo trader using reinforcement learning.

The…

Peter Yu

PhD Student at UMich Researching NLP and Cognitive Architectures • Perviously Real-time Distributed System Engineer turned NLP Research Engineer at ASAPP

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store