BIND and Windows DNS logging and archiving

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

BIND and Windows DNS logging and archiving

Mick Lee
Hi All,

I wonder if I could get some advice and guidance based on everyones experience.

I have a mix of pre-compiled versions of BIND on Linux (can't change or re-compiled I'm afraid) and Windows DNS, and I have a need to log DNS queries from about 100 or so of these types of servers, to identify queries to specific domains, and to be able to go back through and search for queries to domains which we now know to be bad.

I am currently using query logging on Linux, and Syslog to move the data around, and simple regex matching to look for domains, but I need to get the data from Windows servers and the current tooling is not performant/scalable.

I could just enable Windows DNS logging and try to get the files from the servers somehow, but from what I remember there are issues around log file rotation and the potential for data loss there.  One of my colleagues suggested sending the DNS queries to the Windows event log, but I am not sure I can even do that, and I am worried about the impact too - there are approx. 10,000 DNS qps across all servers in total.

Should I be looking at some off the shelve software (although I don't have a lot of budget), what would even do this, or is there some open source tool that would do the job (I have some scripting ability) - I'm quite open to any ideas?

Any advice or guidance anyone can offer would be greatly appreciated.

(I know each environment is different, so apologies if I have left any important detail out, please point this out if so and I will try to fill in the gaps)

Many Thanks

Mick

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BIND and Windows DNS logging and archiving

Mick Lee
Hi Guys,

Can anyone offer any advice based on their experience?

Thanks

Mick

On 19 Jul 2017 2:16 p.m., "Mick Lee" <[hidden email]> wrote:
Hi All,

I wonder if I could get some advice and guidance based on everyones experience.

I have a mix of pre-compiled versions of BIND on Linux (can't change or re-compiled I'm afraid) and Windows DNS, and I have a need to log DNS queries from about 100 or so of these types of servers, to identify queries to specific domains, and to be able to go back through and search for queries to domains which we now know to be bad.

I am currently using query logging on Linux, and Syslog to move the data around, and simple regex matching to look for domains, but I need to get the data from Windows servers and the current tooling is not performant/scalable.

I could just enable Windows DNS logging and try to get the files from the servers somehow, but from what I remember there are issues around log file rotation and the potential for data loss there.  One of my colleagues suggested sending the DNS queries to the Windows event log, but I am not sure I can even do that, and I am worried about the impact too - there are approx. 10,000 DNS qps across all servers in total.

Should I be looking at some off the shelve software (although I don't have a lot of budget), what would even do this, or is there some open source tool that would do the job (I have some scripting ability) - I'm quite open to any ideas?

Any advice or guidance anyone can offer would be greatly appreciated.

(I know each environment is different, so apologies if I have left any important detail out, please point this out if so and I will try to fill in the gaps)

Many Thanks

Mick


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BIND and Windows DNS logging and archiving

Barry S. Finkel
In reply to this post by Mick Lee
On 7/22/2017 ,7:33 AM, Mick Lee<[hidden email]> wrote:

> Hi Guys,
>
> Can anyone offer any advice based on their experience?
>
> Thanks
>
> Mick
>
> On 19 Jul 2017 2:16 p.m., "Mick Lee"<[hidden email]>  wrote:
>
> Hi All,
>
> I wonder if I could get some advice and guidance based on everyones
> experience.
>
> I have a mix of pre-compiled versions of BIND on Linux (can't change or
> re-compiled I'm afraid) and Windows DNS, and I have a need to log DNS
> queries from about 100 or so of these types of servers, to identify queries
> to specific domains, and to be able to go back through and search for
> queries to domains which we now know to be bad.
>
> I am currently using query logging on Linux, and Syslog to move the data
> around, and simple regex matching to look for domains, but I need to get
> the data from Windows servers and the current tooling is not
> performant/scalable.
>
> I could just enable Windows DNS logging and try to get the files from the
> servers somehow, but from what I remember there are issues around log file
> rotation and the potential for data loss there.  One of my colleagues
> suggested sending the DNS queries to the Windows event log, but I am not
> sure I can even do that, and I am worried about the impact too - there are
> approx. 10,000 DNS qps across all servers in total.
>
> Should I be looking at some off the shelve software (although I don't have
> a lot of budget), what would even do this, or is there some open source
> tool that would do the job (I have some scripting ability) - I'm quite open
> to any ideas?
>
> Any advice or guidance anyone can offer would be greatly appreciated.
>
> (I know each environment is different, so apologies if I have left any
> important detail out, please point this out if so and I will try to fill in
> the gaps)
>
> Many Thanks
>
> Mick

The last time I looked at MS Windows DNS logging (6 years ago),
it was not useful.  I could specify the max size of the log,
and when that max size was reached, the log file was cleared,
and a new log file started.  I was logging everything, and the
50Mb log file filled up about every 1.5 days.  So, frequently
the log file was cleared in the middle of the night, erasing
what evidence I wanted to preserve.  I remember asking MS
to implement a real syslog facility where old log files
would be saved.  I have no idea if MS ever implemented better
DNS logging.

--Barry Finkel

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BIND and Windows DNS logging and archiving

Phil Mayers
In reply to this post by Mick Lee
On 22/07/2017 07:33, Mick Lee wrote:
> Hi Guys,
>
> Can anyone offer any advice based on their experience?

Well, if I understand correctly, your main problem is the windows boxes
running windows DNS, so this is not a bind problem. You might be better
asking elsewhere.

However, honestly I would consider moving the traffic from the windows
boxes elsewhere to somewhere you can log. There are great tools for
doing this but they're all unix-oriented e.g. dnsdist, dnscap.

I guess you could try and get one of those running on a Windows box, but
for the effort involved on about 100 servers, you might as well just
spin up a recursive resolver that you *can* instrument, and point all
the boxes at that.

Regards,
Phil
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BIND and Windows DNS logging and archiving

Mick Lee
Thanks Phil,

You are right it's not a BIND issue :)

I am a BIND user myself, and I was wondering how other BIND users have copied when they've had to deal with Windows DNS servers like this.

I appreciate any response to be honest.

I have a colleague who has said he has a parts of a PCAP to BIND query log agent that runs on UNIX platforms, and he is happy to port that to Windows for me - he's actually working on it now (for a few beers :) ).

Basically it just listens on port 53 and streams the data over TCP syslog, i.e. doesn't write to disk but queues in memory with a limit.  It also logs responses for certain record types which is nice.

I'll give that a try, sounds like it will give me query logging formatted logs, which I can push into pretty much anything :)

Many thanks

Mick

On 23 Jul 2017 3:06 p.m., "Phil Mayers" <[hidden email]> wrote:
On 22/07/2017 07:33, Mick Lee wrote:
Hi Guys,

Can anyone offer any advice based on their experience?

Well, if I understand correctly, your main problem is the windows boxes running windows DNS, so this is not a bind problem. You might be better asking elsewhere.

However, honestly I would consider moving the traffic from the windows boxes elsewhere to somewhere you can log. There are great tools for doing this but they're all unix-oriented e.g. dnsdist, dnscap.

I guess you could try and get one of those running on a Windows box, but for the effort involved on about 100 servers, you might as well just spin up a recursive resolver that you *can* instrument, and point all the boxes at that.

Regards,
Phil

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: BIND and Windows DNS logging and archiving

Phil Mayers
On 23/07/2017 15:16, Mick Lee wrote:

> I have a colleague who has said he has a parts of a PCAP to BIND query
> log agent that runs on UNIX platforms, and he is happy to port that to
> Windows for me - he's actually working on it now (for a few beers :) ).

dnscap basically does the same thing. No idea how easy it would be to
run under Windows.

Absent changes to the resolving setup, I think that a capture/tap is
probably your only realistic option.

Depending on your architecture (physical, virtual, topology) the tap
could live on another box, if all you need is to know that server A made
a query for badzone B.
_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Fwd: BIND and Windows DNS logging and archiving

Mick Lee
Forgot to CC the list.

---------- Forwarded message ----------
From: Mick Lee <[hidden email]>
Date: Sat, Aug 12, 2017 at 6:55 PM
Subject: Re: BIND and Windows DNS logging and archiving
To: Phil Mayers <[hidden email]>


Thanks,

I checked and it doesn't look like dnscap would work with little change :(  Anyway, my colleague has now implemented a similar tool called dns-activity-logger.

I mention it here since it does DNS response logging, specifically for IP addresses.  You get output similar to BIND query logging for responses too:

# Response logging is like query logging, but you get rcode, ans-count, auth-count, add-count and a space separated list of IP's from the answer section if any
Aug 12 17:47:25 dns01 dns-activity-logger[6476]: client 192.168.1.13#61835: query: www.apple.com IN A + (192.168.1.200)
Aug 12 17:47:25 dns01 dns-activity-logger[6476]: client 192.168.1.200#61285: query: www.apple.com IN A + (192.168.1.1)
Aug 12 17:47:25 dns01 dns-activity-logger[6476]: client 192.168.1.200#61285: response: www.apple.com IN A + (192.168.1.1) NOERROR 4 0 1: 23.198.68.189
Aug 12 17:47:25 dns01 dns-activity-logger[6476]: client 192.168.1.13#61835: response: www.apple.com IN A + (192.168.1.200) NOERROR 4 0 0: 23.198.68.189

It streams Syslog messages out in real-time over TCP, supports auto-failover in case one Syslog server goes down, and buffers in memory so doesn't require any disk I/O.

My initial use case was Windows, but after seeing the response logging I think I will disable BIND query logging and just use this.

He's willing to make it available to the general public if there is any interest.

Cheers

Mick

On Sun, Jul 23, 2017 at 5:15 PM, Phil Mayers <[hidden email]> wrote:
On 23/07/2017 15:16, Mick Lee wrote:

I have a colleague who has said he has a parts of a PCAP to BIND query log agent that runs on UNIX platforms, and he is happy to port that to Windows for me - he's actually working on it now (for a few beers :) ).

dnscap basically does the same thing. No idea how easy it would be to run under Windows.

Absent changes to the resolving setup, I think that a capture/tap is probably your only realistic option.

Depending on your architecture (physical, virtual, topology) the tap could live on another box, if all you need is to know that server A made a query for badzone B.



_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Loading...