ESRI introduced online connection at ArcGIS 10.1. If you noticed that version 10.1 is much slower to start then this is likely to be the main cause. Each time you start ArcMap or ArcCatalog it connects to an ArcGIS Portal or ArcGIS.com. It repeats this every 10 seconds by default.
The intention is to improve the ArcGIS experience – when you do actually decide to connect to online resources the performance is improved. See ESRI’s knowledge base article for more information.
Short of disabling your internet connection, there was no practical (supported) way of stopping this behavior. One workaround is to rename or delete the file C:\Program Files\common files\ArcGIS\bin\ArcGISConnection.exe as suggested by Thomas Slingsby.
At service pack 1 you can now disable the online connection. Download service pack 1 for version 10.1 here.
After its installed, you can right click on the ArcGIS icon in the system tray (its the usual ArcGIS Globe icon) and choose properties. The ArcGIS connection properties window will appear. Un-check the Enable ArcGIS Connection option and then click OK.
