W3C home > Mailing lists > Public > public-webplatform@w3.org > September 2013

Re: Logging management and analytics utilities "noc" services

From: Ryan Lane <rlane32@gmail.com>
Date: Thu, 19 Sep 2013 10:18:23 -0700
Message-ID: <CALKgCA3Jf0u_W2Rfg87tbAZQ-Dg=SdUtVd9kNDfgr2zo-+UYfw@mail.gmail.com>
To: Renoir Boulanger <renoir@w3.org>
Cc: List WebPlatform public <public-webplatform@w3.org>
On Thu, Sep 19, 2013 at 9:21 AM, Renoir Boulanger <renoir@w3.org> wrote:

> Hello Ryan, and fellow infra geeks,
>
> I'd like your opinion regarding where to host Logstash [0].
>
> LogStash is an open source application made to parse and help search log
> events by harmonizing the data and make it easy to search through it [2].
> It is based on the idea of what Splunk [1] can do.
>
> If you want to try it, I am currently using the salt state [5] on a
> separate host and you can try it [7] (!!).
>
> Here are my questions:
>
> PS: I might have others, but I wanted to start a thread on it.
>
>
>
> *1. LogStash uses ElasticSearch [3], and it is distributed[4], we will
> want to use it for other things, any recommendations?*
>
> I thought of modifying the suggested salt stack config [5] and have at
> least 1 elastic search node (e.g. *es1*). And to install log stash itself
> on *monitor*.
>
>
Put everything on the monitor instance. I have some serious doubts we have
enough log traffic to worry about scaling it. If we need to scale it later
we can.


>
> *2. Fastly supports to send error logs to a syslog server, but our plan
> do not support it, opinion on upgrading?*
>
> See [6].  It was just a thought to enable it, I would suggest to finish
> sending ALL logs to log stash, then we see what else we can get and the
> benefit of that data source.
>
>
Well, this uses TCP to stream logs. If our logstash server blocks, what'll
happen?

- Ryan
Received on Thursday, 19 September 2013 17:19:12 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:20:54 UTC