I had the same setup for a while, but eventually moved to scrypted instead which had better startup time for streams in Home.app.

I second this. I have Unifi + Scrypted and it has worked great. Remote access turned off on UniFi.

One more vote here for Scrypted. I have a bunch of cameras from various vendors, some with open FW, some with their original FW, all cut off from the internet. They used to be connected to Frigate but due to performance issues I offloaded the work to Scrypted on a RPi and an AppleTV and the setup works great. Bonus, it’s easier to set up and use compared to Frigate, especially for the less technical people in the family.