You are agoraphobic about agoraphobia, not what's outside your front door.
Something originally stressed you and caused your anxiety. But chances are, the original cause has long since passed or has been overcome. But the fear, the panic attacks and the agoraphobia continue.
Now you fear the fear, you panic about panicking, you are fearful of the agoraphobia itself, not the walk to the park.
Who said so? The woman who cracked the anxiety code said so. You can probably guess who I'm talking about.