54 phylacteries? What version are you playing, 1.0.9 Rev B or 1.0.10 or what? Can't help but wonder if the phylactery debuff has something to do with it.
lol I had 45 by the time 1.0.10 came out... then I just kinda hovered around half health with a few spare, irony is I completed the game legit, IE never using more than above standard as I always had the debuff active while using them so never returned to full max hp. Theres a chance it was to do with that.