I think it does work without your phone on you - it says it uses wifi, so as long as your phone is currently on your wifi network, it sounds like it will work. That should be likely for most people, even if they don't carry their phones with them.
The phone is only required for setup (configuring the button's WiFi connection to your router). Once that is complete, the button talks to your router directly via WiFi, with no phone required.
From the description of it, I think the button wouldn't work if you don't carry a phone with you, as it depends on it.