That made it clear the chip WAS WORKING, it was just immediately powering off. I tried SPI-ing the chip during that brief power-on period, and got something. It was clear that the chip is not quite dead yet, maybe just sleeping.

Then, I continued probing around, and there were quite a lot of signals coming on and off during that brief moment. That didn't change anything, the chip still won't SPI. So I tried forcing that core voltage, externally. Found out that despite me forcing the main power to the chip, core voltage (1.8 V) was still coming on for a brief moment after pressing power button. Today I got to investigating, what really happened.