This issue occurred while I was testing Consumer Preview version of Outlook 2013. It could have been complete coincidence as issue seems to be caused by Exchange 2010 rather than Outlook.
Gave myself Full Access Permissions to couple of mailboxes and opened them as usual via Account Properties > Advanced > Advanced > Add...
After required work was completed used the same method to close them. To my surprise both mailboxes were still attached to the Outlook.
I couldn't find a way to remove them. Adding and removing them again wouldn't make any difference. Tried Outlook 2010 (Office Preview 2013 can co-exist with previous versions) - situation was exactly the same.
Next step was to create a new Outlook profile. Unfortunately this didn't resolve the issue either. Attached mailboxes were still there.
Problem was sort of fixed removing myself from Full Access Permissions list on the affected mailboxes. This can be done via Exchange Management Console > Recipient Configuration > Mailbox > right click on affected mailbox > Manage Full Access Permission
After restarting the Outlook and waiting a few minutes sticky mailboxes finally disappeared.
As a test added myself to Full Access Permissions lists again and surely enough, after Outlook restart mailboxes shortly were added to my accounts list again.
The true cause of the issue was a new Exchange 2010 SP1 feature called Auto-Mapping. It automatically maps any mailbox if user has been granted full access permissions. This can be removed by following Exchange PowerShell command:
Add-MailboxPermission -Identity <admin.user> -User '<shared mailbox>' -AccessRight FullAccess -InheritanceType All -Automapping $false
This command gives "admin.user" full access to "shared mailbox" and removes auto-mapping feature.
Restart the Outlook and in a few minutes shared mailbox should be gone.
Exchange 2010 SP2
Outlook 2013 (Preview)