Retreating to high perch

Hi folks, I have question regarding a small higher perch.  This is the perch that is a kind of toy, it is the highest in the cage and he tends to go there at night for sleep.  My question is abt how long he is there, for example, this morning it is 10:15 he has not been down yet for his food yet, he has also taken to going there if I try to work with him for stepping up.  It is strange for me, because he has been a little more docile or accepting in the morning and I actually was able to get him to step up calmly, but in the afternoon, if I try to work with him, sometimes if I even go sit by the cage, he is already looking to or going to the high perch and doesn't seem interested at all.  I was wondering if it would be bad of me to take out his high perch option as it is effecting my ability to work with him allot of the time or if that would be mean:)?  I don't want to rob him of a comfort but, I feel as though he is choosing that over allowing me to work with him.  I am at a point where I feel as though he is never going to actually bond with me, I believe that I have tried to be consistent, and it just isn't totally happening for him.  Maybe it is just his personality (maybe mine?), a more nervous little guy perhaps? But it is difficult to not see significant progress after 6-7 months.  Not sure where I am going wrong, and right now life has exploded in my face with health issues for both my son and dad, very distracting.  Thanks gang for your help or encouragement for Pickle and I.

Blessings!