Which method would work best for email classification into 1 of 7 categories? Problem I've seen is 1 or 2 key sentences within the email can classify the message but they are usually outnumbered by generic sentences such as signatures, greetings, headers/footers etc
These are all frameworks and while none of them have any signular advantage over other especially in the problem statement you are looking for, you should ideally be able to figure out what works best for you based on the classification sensitivity and training data you are working with. The problem in itself can be quite simple to extremely complex based on the above 2 factors. Spacy's pre-processing tools are quite easy to use and that combined with tool like talon should help you clean up the email correctly. Thereafter, if your email text is pretty much to the point, then any intent classification tool will work, however, if the email text is long and intents are spread across, then you will need a hierarchical layer to understand the intent hierarchies as well as an attention layer to understand which intents to focus and not lose track of in an email. At that time, you are quite far from using a generic plug and play framework and will need to exactly and quite thoroughly understand the deep learning models you are working with as well as the dataset you have and the classification you are trying to build.
Thanks this is really helpful!
I am using talon and sklearn as paragraph by paragraph intent classifier. I am classifying the whole email from the highest individual intent probability. This seems to be working well for my minimal test data (~200 sentences) but have yet to test in the wild. I will research hierarchical layer and attention layers.
Generally these generic sentences should be randomly distributed, and so their effect should be minimal.
You can randomly add them to your training set, if you feel that real world data has them randomly distributed, but you training sample is too small to capture this.
That's a great question. Actually, one of the sounds that are pretty close to a chainsaw are mosquitos that are circling around our microphones due to the Doppler effect. We found ways of dealing with signals that are close to chainsaws by aggregating multiple models and also a time-based analysis. The system can draw causal/correlative conclusions such as a vehicle is usually present before a chainsaw. If there's no vehicle, the likelihood of a chainsaw goes down and the chainsaw model must be highly confident before we sound an alert.
How do you quantify the confidence of your model? Do you use a Bayesian model or just the log-likelihood? Because the latter can act strangely in some cases.
I know this is a digression from the current discussion on how well the devices work, but as a stats student who just learned about estimating using log-likelihoods, could you give some more info on how that is inferior to the Bayesian model (since I've heard the exact opposite is true)?
The problem is that neural networks trained using maximum LL do not return calibrated probabilities, using e.g. the softmax output as 'confidence' of a model tends to result in overconfident predictions, take a look at adversarial attacks on neural networks for an extreme example: https://blog.openai.com/adversarial-example-research/
I imagine it would depend on the lifespan, and travel patterns of a lyrebird, no? Eg, bird is making the chainsaw noise weeks or years after the loggers have gone, possibly in an entirely different area.
The guy on the right was throwing dogs into the air for the archer to shoot, the two suns are actually dog gibs. How shooting dogs with a bow caused them to explode is still a mystery, but the deer was trying to save the last dog by attacking the archer.
WeChat became one of the main mobile consumer platforms in China. It seems weird/different than elsewhere, because it's a chat app. It's basically just a different UX for the homescreen.
Comparing WeChat to WhatsApp on feature count is big time apples to oranges.
Not going to go into detail as I'm sure you know how to use Google.
Wechat isn't simply chat. It has grown into a platform that has empowered it's users to partake in a growing ecommerce industry. It also has p2p payments which is hugely popular.
If there were only 1 channel on TV, that channel would be very popular!
If Google or Facebook were given a monopoly on chat by the US government, and hardly anyone had credit cards, then their wallets would be universally popular too.
WeChat isn't going to get much market penetration outside of China because it's just not very good. At least that's my opinion, as someone who's forced it use it in China.
