Re: Logging management and analytics utilities "noc" services

On Thu, Sep 19, 2013 at 9:21 AM, Renoir Boulanger <renoir@w3.org> wrote:

> Hello Ryan, and fellow infra geeks,
>
> I'd like your opinion regarding where to host Logstash [0].
>
> LogStash is an open source application made to parse and help search log
> events by harmonizing the data and make it easy to search through it [2].
> It is based on the idea of what Splunk [1] can do.
>
> If you want to try it, I am currently using the salt state [5] on a
> separate host and you can try it [7] (!!).
>
> Here are my questions:
>
> PS: I might have others, but I wanted to start a thread on it.
>
>
>
> *1. LogStash uses ElasticSearch [3], and it is distributed[4], we will
> want to use it for other things, any recommendations?*
>
> I thought of modifying the suggested salt stack config [5] and have at
> least 1 elastic search node (e.g. *es1*). And to install log stash itself
> on *monitor*.
>
>
Put everything on the monitor instance. I have some serious doubts we have
enough log traffic to worry about scaling it. If we need to scale it later
we can.


>
> *2. Fastly supports to send error logs to a syslog server, but our plan
> do not support it, opinion on upgrading?*
>
> See [6].  It was just a thought to enable it, I would suggest to finish
> sending ALL logs to log stash, then we see what else we can get and the
> benefit of that data source.
>
>
Well, this uses TCP to stream logs. If our logstash server blocks, what'll
happen?

- Ryan