EntropicaPublic

March updates I need to move some work over from the private repos. The exploration into using massivly multheaded models with mixer and readout functions separated continues. Currently training up a Chinese TinyStories on a 3B-equivalent model that trains, easily, on an 8GB gpu. We're looking at...