My guess is, you never really recovered the first time. The clomid gave you a false sence of having recovered. Then you went back on and became more shut down. In order to recover, you need to be off of the clomid and allow your body to produce test normally. If you are having a hard time, getting test to turn back on and getting your teste size back to normal,I would suggest using HCG. I would run 3 wks at about 500iu every 2 to 3 days along with 50mg clomid every day. Then continue on the clomid at 50mg for an addiitonal 3 wks. After this, I would come off of the clmoid and give yourself time to see how things are going. You'll know within a month, if you have recovered. You can always get lab work to varify. And in the future, I would suggest going into your cycles with a solid plan. It doesn't mean that you always need to follow that plan 100%, but it does sound like your cycling is lacking strucutre. And FYI, your best bet is to use test as the base of your cycle and then add another anabolic like var, to it
In the future, if you want to avoid teste shrinkage, you can run HCG during your cycles at 250-500iu 2x wk, up to PCT