About me

Hi there! My name is Neeratyoy Mallik.

I’m a PhD student at the ML Lab in Freiburg, supervised by Prof. Dr. Frank Hutter.

Inspired by Frank’s vision of using Machine Learning to democratize Machine Learning, my work aims to make large-scale Deep Learning cheaper and easier to tune, so strong performance and accessibility isn’t gated by compute or expertise.

Currently I am interning with Dr. Shiwei Liu at the ELLIS Institute & Max Planck Institute for Intelligent Systems in Tübingen. I am also co-supervised by Dr. Aaron Klein and collaborating with the OpenEuroLLM team on scaling law construction.

I have previously worked on Evolutionary Optimization, Bayesian Optimization, Multi-fidelity methods, Learning Curve Extrapolation. My active research focuses more on LLM pretraining, more specifically on hyperparameter transfer, and how to efficiently scale a model and derive prescriptive scaling laws.

If you are working or interested in:

• efficient training and scaling of LLMs, including fitting scaling laws,
• $\mu$Parameterization, or more generally applying hyperparameter ladders,
• interpreting and leveraging network learning dynamics,
• general HPO (or AutoML) for different data modalities, and downstream applications,
• multilingual LLMs as a way to preserve endangered languages,

do write to me for a chat and potential collaboration! :)

Thank you for visiting: if you have any questions or would like to get in touch, please feel free to contact me, or find me on social media (left panel).