You are able to delete a custom community by selecting the gear icon to the right of the custom community name.
When you select the gear, you will have 2 options.
Option 1 - Update Community
This gives you the ability to rename the community. This works for the 'general' community and any custom communities.
Option 2 - Delete Community
This will remove the community. Once you delete it, it cannot be recovered. You can only delete custom communities.