You need to keep a low profile as a survivalist. Your neighbors should not know that you are prepping for a disaster. If they do, you will become a target. Once things go down, they will want to take your food and other supplies. That will make it much harder to survive.
Keeping it a secret is easier than you might think. Just keep things hidden out of sight. You can use nature to hide your garden, and keep your supplies stocked up in your home. If you have an extra vehicle you are using for a getaway, you can use trees and shrubs if you do not have a garage. You can also park it in a storage unit within walking distance.
The less people know, the safer you will be. Keep everything hidden so you won’t have to worry about people trying to take your belongings during a crisis.