Nice to meet you! π
My name is IonuΘ (pronounced yo-nootz) and I am a Ph.D. student working in efficient optimization for Deep Learning at the Institute of Science and Technology Austria (ISTA) under the supervision of Dan Alistarh. Before starting my PhD in October 2021 at ISTA, I worked as a Research Scientist at Amazon. Prior to that, I was an intern at the University of Maryland in College Park, USA and worked under the supervision of Tudor Dumitras.
My primary research focus is on efficient optimization algorithms for Deep Learning, with a particular emphasis on LLMs. Iβm especially interested in reducing the memory footprint of optimizer states by applying compression techniques such as sparsity, low-rank approximations, and quantization. To achieve this, I often draw on tools from numerical linear algebra and implement custom solutions with CUDA programming when performance demands it.
I maintain the ISTA-DasLab-Optimizers repository, a GitHub project from our DASLab group at ISTA, containing all optimization algorithms developed in our lab. Feel free to give it a try by installing it via pip install ista-daslab-optimizers.
I also developed GridSearcher, a tool designed to replace the bash scripts to launch training experiments for Deep Learning models. It allows defining the hyper-parameter values for grid search and has the option to run distributed or single GPU experiments, while ensuring a basic GPU synchronization. It is available to install via pip install gridsearcher.