I recently tried to homebrew some anomaly detection work for a performance tracking project and was surprised at the absence of any off-the-shelf OSS or Paid solutions in this space (that weren’t super basic or way too complex). Lots of fertile ground here!
Hi, co-author of the blog post here. I would love to learn more about the flaws you see and if ideas on how to improve it! We definitely plan to iterate on it and make it as good as we possibly can.
To be clear "some big flaws" was probably overstating it. I'm going to edit that. Also, thanks for the work on this. I would absolutely love to contribute, but my maths are not good enough for this :)
The biggest thing I've run into in my testing is that an anomaly of reasonably short timeframe seems to throw the upper and lower bands off for quite some time.
That being said, perhaps changing some of the variables would help with that, and I just don't have enough skill to be able to understand the exact way to adjust that.
Thank you for the feedback! There is an issue (https://github.com/grafana/promql-anomaly-detection/issues/7) discussing approaches to improve this by introducing a decay function. This is specially relevant for stable series with occasional short large spikes. Nothing conclusive yet, but hopefully something good will come out of it!
The number of manual tweaks required to the approach suggest that it is essentially an ad hoc experimental fitting, rather than a stable theoretical model that can adapt to your time series.
Not really related to the above post, but one thing I am not seeing on an initial pass is the advancement of understanding of problems like riddled or wada basins.
Especially with time delays this and 3+ attractors this can be problematic.
There are tools to try and detect these features that were found over the past few decades, and I know I wasted a few years on a project that superficially looked like a FP issue, but ended up being a mix of the wada property and/or porous sets.
The complications will describing these worse than traditional chaos indeterminate situations may make it inappropriate for you.
But it would be nice if visibility was increased. Funny enough most LLMs corpus is mostly fed from a LSAT question.
There has been a lot of movement here when you have n>=3 attractors/exits.
Not solutions unfortunately, but tools to help figure out when you hit it.
Hi! I work on the Grafana OSS team. We added some more export options recently (dashboards have a big Export button at the top right; panels can export their data via the panel menu / Inspect / Data), try it on our demo page: https://play.grafana.org/d/000000003/graphite3a-sample-websi...
Could you describe your use case around "exportable to any code" a bit more?
I needed a TS anomaly detection for my internship because we needed to track when a machine/server was doing poorly or had unplanned downtime. I expected Microsoft's C# library to be able to do this, but my god, it's a mess. If someone has the time and will to implement a proper library then that would ve awesome.
Anomaly detection in time-series data is not a concern of the standard library of all things. Nor is it a concern of "base abstractions" shipped as extensions (think ILogger).
What you’re probably after is called statistical process control. There are Python libraries like pyspc, but the theory is simple enough that you could write your own pretty easily.
agreed. at my company we ended up rolling our own system. but this area is absolutely ripe for some configurable saas or OS tool with advanced reporting and alerting mechanisms. Datadog has a decent offering, but it's pretty $$$$.
Gonna throw in my hat and say that if you’re working on industrial applications (like energy or manufacturing) give us a holler at www.sentineldevices.com! Plug-and-play time series monitoring for industrial applications is exactly what we do.
I don't think that's "X history more than Y history" a correct way to argue about history :-)
It's definitely a fair to name that Ukrainian history from your perspective. It's a also fair to name that Russian history.
A lot of people could argue that is specifically Cossack history, because Cossacks could be viewed as separate ethnicity with their own history.
Somebody will say that's Russian word is correct, because "Russian" is umbrella term for both velikorossy (Great Russians) and malorossy (Ukrainians) :-)
But it's definitely fair from modern Russian perspective and from modern Ukrainian perspective to claim this history as their own.
At that point in time Russian-Ukrainian history was tight knit. It is after Polish-Lithuanian Commonwealth influence of Western Ukraine did what I assume your understanding of the difference between Russia-Ukraine's history start.
Referring to Zaporozhian Cossacks as Russian is quite a stretch, and sure to upset Ukrainians. Cossacks generally are a part of Russian-Ukrainian history, but this group has typically been considered Ukrainian, based on their geography and ultimate integration into Ukraine.
In truth, the Zaporozhian Cossacks were their own people, to classify them as strictly Russian or Ukrainian somewhat erases their culture and history.
But I didn't say that Zaporozhians are distinctly Russian either, just that at that point in time the divide between Ukraine and Russia was a lot less clear. People who drive a hard line about differences of these people, especially in those times, are usually politically and/or ideologically motivated and have barely any knowledge of the historical context.
You're absolutely correct that the Zaporozhians were somewhat of their own people, much like the Don and other groups of Cossacks - which can be seen in them uprising against the Polish-Lithuanian rulers and the Czar respectively and trying to form their own states. I wouldn't go so for as to say that they aren't part of Russian-Ukrainian history either, just that claiming for them to be the same as modern Ukrainians or modern Russians makes as much sense as modern Greeks claiming to be direct descendants of Alexander the Great.
> claiming for them to be the same as modern Ukrainians or modern Russians makes as much sense as modern Greeks claiming to be direct descendants of Alexander the Great.
I've use that on OS X, worked quite well. Ultimately I don't see much use for it though, maybe if you want to get information about the headers of files before you download them completely or something like that.
