How come the total size of the model is 138GB ?
#5
by
gagan001
- opened
I calculated the total size of model.bin files and it comes out to be approx 138GB. As per my understanding, a 70B parameter model should consume 70*4 GB = 280GB (approx) of memory.. It seems like this model is of 16-bit precision. Is that correct ?
Yes, iirc this is in BF16!