Re: [perpass] HTTP user-agent fingerprinting

Le Lun 16 septembre 2013 20:42, Karl Dubost a écrit :
> Nicolas Mailhot [2013-09-16T12:34]:
>> User-Agent is invaluable for filtering out pathologic web clients in a
>> network without bothering legitimate users.
>
> * At which level do you need to filter out? (proxy, server, …)

proxy

> * Which type of clients? (bot, browser, etc?)

pretty much everything that tries to access the web in http(s)

> * Why do you need to filter out?

because the internet got so fast web client authors do pretty idiotic
things at the http level and it's lost in the youtube noise. OTOH those
idiotic things tend to clutter my logs with errors that make investigating
actual problems difficult, so it's easier to just blacklist miscreants

Major browsers almost never cause problems, but pretty much anything else
that does http is likely to make shortcuts

> * What are you looking for in the UA string?

Anything likely to identify the problem client and nothing else. For example:
1. corp policy is that windows stations must go through corp wsus →
blacklist windows update UA ;
2. something is hammering the proxy with corrupted connects (control chars
before CONNECT) → it says it is "Google Mail" → go away Google Mail,
3. someone deployed a widget that wants to go to the internet every 5
minutes to display a clock no one will look at → kill clock (not ntp, we
have perfectly fine ntp servers)

Actually, I don't care about the volume as long as it's not pathological
real-time updates, what I do care is the flood of errors that makes our
job more difficult than it should be.

>
> /me is interested to know. I'm kind of collecting how the UA string is
> used out in the wild in good and bad ways.
>


-- 
Nicolas Mailhot

Received on Monday, 16 September 2013 20:00:38 UTC