Re: Machine learning top of db logs

I have been thinking about how to apply NLP work to logs. The problem is
that logs are structured data. It would depend on what you are looking for,
what your outcomes are, and how you want to generate it. The other problem
is that your problems are going to be very rare events. Rare event
detection is its own special mess. Like, if you have 1000 users generating
1 million line log files, how do you pick out the 10 that are bad? You also
need to make those rare events representative of very rare events so you
might need say, 100 million line log files with a know 100 or 1000 issues
to start getting models to work.

Thanks,
~Ben

On Fri, Jul 12, 2019 at 2:44 AM Toomas Kristin <toomas(dot)kristin(at)gmail(dot)com>
wrote:

> Hi,
>
> Basically seems that data science and machine learning are going to be
> more and more popular at every field of life. I have considered to use
> machine learning top of logs generated by PostgreSQL servers. However
> before I start maybe someone already has done that and can share some
> experience?
>
> Toomas
>
>
>
>