Answer
When working with Activation Functions, start with a clear goal, test on real data early, keep the approach as simple as possible at first, and follow established practices from the AI community rather than guessing.
Real-world example The ReLU activation function is widely used because it is simple and helps a network trained on image data learn faster than older functions.