Question about visibility

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about visibility

Andrew Hardy

I realise this is not specifically a BIND/DNS question and a bit off
topic so please ignore if need be I realise people are often very busy.

If you you have a website but the host IP you do not list with any
domain name in DNS, is it definite that this site could never be reached
via Google.  I do not really know the nuts and bolts of how Google gets
access to pages.

If for 'some particular reason' instead of developing a site on a local
dev machine on your LAN and then uploading/installing the site to a
remote server, you needed 'for what ever reason' to do the development
and testing on the final live host accessing it via the ip address,
would this be a way to be 'almost certain' of keeping it hidden from
unwanted accidental exposure?

Thanks.


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about visibility

Warren Kumari


On Thu, Oct 11, 2018 at 1:26 PM Admin Hardy <[hidden email]> wrote:

I realise this is not specifically a BIND/DNS question and a bit off
topic so please ignore if need be I realise people are often very busy.

If you you have a website but the host IP you do not list with any
domain name in DNS, is it definite that this site could never be reached
via Google.  I do not really know the nuts and bolts of how Google gets
access to pages.

If for 'some particular reason' instead of developing a site on a local
dev machine on your LAN and then uploading/installing the site to a
remote server, you needed 'for what ever reason' to do the development
and testing on the final live host accessing it via the ip address,
would this be a way to be 'almost certain' of keeping it hidden from
unwanted accidental exposure?


Nope. It is somewhat less likely that it would be discovered / accidentally exposed, but it is *far* from certain.

If you were wanting to do something like this, I'd suggest having a DNS name (because that makes it easier), but firewalling it off so that only "authorized" people can reach it. This could be something like iptables, a VPN, or, more likely / less annoying, simply having your webserver require a login to access the content...

W


 
Thanks.


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users


--
I don't think the execution is relevant when it was obviously a bad idea in the first place.
This is like putting rabid weasels in your pants, and later expressing regret at having chosen those particular rabid weasels and that pair of pants.
   ---maf

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about visibility

Sten Carlsen
In reply to this post by Andrew Hardy
I have done this some time ago, I made sure that there was no link from any pages to the new site, Google stayed away until somebody typed the address into the search field, then it was known.

This is no guarantee of course as mentioned in other place but it worked for about 6 months.

On 11/10/2018 13.26, Admin Hardy wrote:

I realise this is not specifically a BIND/DNS question and a bit off topic so please ignore if need be I realise people are often very busy.

If you you have a website but the host IP you do not list with any domain name in DNS, is it definite that this site could never be reached via Google.  I do not really know the nuts and bolts of how Google gets access to pages.

If for 'some particular reason' instead of developing a site on a local dev machine on your LAN and then uploading/installing the site to a remote server, you needed 'for what ever reason' to do the development and testing on the final live host accessing it via the ip address, would this be a way to be 'almost certain' of keeping it hidden from unwanted accidental exposure?

Thanks.


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about visibility

Hardy, Andrew
Ok I'm a bit confused.  I have some questions re last post, copied below:

I have done this some time ago, I made sure that there was no link from any pages to the new site, 
** So the new site (in development) would have no domain name mapped in DNS, so it seems unlikely that other sites and pages would have links to http://x.x.x.x unless the developer put it there.

Google stayed away until somebody typed the address
** You mean typed the IP address? You mean in an actual Google search string?

 into the search field, then it was known.
** So typing the host IP address as a Google search string would (ultimately) in time lead to a Google search string, that could be found on the sites web pages, listing pages from the site?

This is no guarantee of course as mentioned in other place but it worked for about 6 months.
** Ok, so even if you don't formally register / index (or what ever it is) your site on Google, if you use it's IP in a search string, given time it could show up in searches using text that's on its pages?


Just to say thank you so much for people commenting.  I do appreciate you taking the time.



On Thu, Oct 11, 2018, 14:50 Sten Carlsen <[hidden email]> wrote:
I have done this some time ago, I made sure that there was no link from any pages to the new site, Google stayed away until somebody typed the address into the search field, then it was known.

This is no guarantee of course as mentioned in other place but it worked for about 6 months.

On 11/10/2018 13.26, Admin Hardy wrote:

I realise this is not specifically a BIND/DNS question and a bit off topic so please ignore if need be I realise people are often very busy.

If you you have a website but the host IP you do not list with any domain name in DNS, is it definite that this site could never be reached via Google.  I do not really know the nuts and bolts of how Google gets access to pages.

If for 'some particular reason' instead of developing a site on a local dev machine on your LAN and then uploading/installing the site to a remote server, you needed 'for what ever reason' to do the development and testing on the final live host accessing it via the ip address, would this be a way to be 'almost certain' of keeping it hidden from unwanted accidental exposure?

Thanks.


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about visibility

Sten Carlsen
Please see below.

On 11/10/2018 18.13, Hardy, Andrew wrote:
Ok I'm a bit confused.  I have some questions re last post, copied below:

I have done this some time ago, I made sure that there was no link from any pages to the new site, 
** So the new site (in development) would have no domain name mapped in DNS, so it seems unlikely that other sites and pages would have links to http://x.x.x.x unless the developer put it there.
Actually I had DNS for this.

Google stayed away until somebody typed the address
** You mean typed the IP address? You mean in an actual Google search string?
Something in a search string, if this has the address visits from the bots are next to come. My experience for this and some other cases.

 into the search field, then it was known.
** So typing the host IP address as a Google search string would (ultimately) in time lead to a Google search string, that could be found on the sites web pages, listing pages from the site?
This is my experience. I did this when I wanted the site to be known to the world.

This is no guarantee of course as mentioned in other place but it worked for about 6 months.
** Ok, so even if you don't formally register / index (or what ever it is) your site on Google, if you use it's IP in a search string, given time it could show up in searches using text that's on its pages?
Time in this case is days or less.

There are also bots that search random IP addresses for content, the only way to keep those away that I know of is to have a welcome page in http://xx.xx/index.html and using e.g. http://xx.xx/test/mynewsite/index.html for my test site.
Bots will find the welcome page and if that does not have a link to my mynewsite, they do not know that there is something to look at.
This has worked for me as well for quite some time, again if it hits a search in any search engine, you're done.


Just to say thank you so much for people commenting.  I do appreciate you taking the time.
You're welcome.



On Thu, Oct 11, 2018, 14:50 Sten Carlsen <[hidden email]> wrote:
I have done this some time ago, I made sure that there was no link from any pages to the new site, Google stayed away until somebody typed the address into the search field, then it was known.

This is no guarantee of course as mentioned in other place but it worked for about 6 months.

On 11/10/2018 13.26, Admin Hardy wrote:

I realise this is not specifically a BIND/DNS question and a bit off topic so please ignore if need be I realise people are often very busy.

If you you have a website but the host IP you do not list with any domain name in DNS, is it definite that this site could never be reached via Google.  I do not really know the nuts and bolts of how Google gets access to pages.

If for 'some particular reason' instead of developing a site on a local dev machine on your LAN and then uploading/installing the site to a remote server, you needed 'for what ever reason' to do the development and testing on the final live host accessing it via the ip address, would this be a way to be 'almost certain' of keeping it hidden from unwanted accidental exposure?

Thanks.


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users


_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about visibility

Barry Margolin
In reply to this post by Andrew Hardy
In article <[hidden email]>,
 Admin Hardy <[hidden email]> wrote:

> I realise this is not specifically a BIND/DNS question and a bit off
> topic so please ignore if need be I realise people are often very busy.
>
> If you you have a website but the host IP you do not list with any
> domain name in DNS, is it definite that this site could never be reached
> via Google.  I do not really know the nuts and bolts of how Google gets
> access to pages.
>
> If for 'some particular reason' instead of developing a site on a local
> dev machine on your LAN and then uploading/installing the site to a
> remote server, you needed 'for what ever reason' to do the development
> and testing on the final live host accessing it via the ip address,
> would this be a way to be 'almost certain' of keeping it hidden from
> unwanted accidental exposure?
If you accidentally, or someone else intentionally, create a link to the
site that uses the IP and put it on a web page that Google can get to,
it will probably find the page.

--
Barry Margolin
Arlington, MA

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about visibility

Leonardo Rodrigues
Em 11/10/18 16:13, Barry Margolin escreveu:
>
> If you accidentally, or someone else intentionally, create a link to the
> site that uses the IP and put it on a web page that Google can get to,
> it will probably find the page.
>
>

     robots.txt, on your website root, is your friend. Simply deny web
crawling on it, and you're (probably) done.



--


        Atenciosamente / Sincerily,
        Leonardo Rodrigues
        Solutti Tecnologia
        http://www.solutti.com.br

        Minha armadilha de SPAM, NÃO mandem email
        [hidden email]
        My SPAMTRAP, do not email it



_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about visibility

Dennis Clarke
On 10/11/2018 03:21 PM, Leonardo Rodrigues wrote:

> Em 11/10/18 16:13, Barry Margolin escreveu:
>>
>> If you accidentally, or someone else intentionally, create a link to the
>> site that uses the IP and put it on a web page that Google can get to,
>> it will probably find the page.
>>
>>
>
>      robots.txt, on your website root, is your friend. Simply deny web
> crawling on it, and you're (probably) done.
>

If you believe robots.txt means anything at all.

Dennis

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users
Reply | Threaded
Open this post in threaded view
|

Re: Question about visibility

Barry Margolin
In article <[hidden email]>,
 Dennis Clarke <[hidden email]> wrote:

> On 10/11/2018 03:21 PM, Leonardo Rodrigues wrote:
> > Em 11/10/18 16:13, Barry Margolin escreveu:
> >>
> >> If you accidentally, or someone else intentionally, create a link to the
> >> site that uses the IP and put it on a web page that Google can get to,
> >> it will probably find the page.
> >>
> >>
> >
> >  Â Â Â  robots.txt, on your website root, is your friend. Simply deny web
> > crawling on it, and you're (probably) done.
> >
>
> If you believe robots.txt means anything at all.
Google is known to obey it, and the question was about avoiding getting
your site indexed by Google.

Of course, that doesn't mean someone won't find the site on their own.
If the link to it is on some other page that isn't blocked by
robots.txt, someone might stuble across that page and then click on the
link.

But if you're mainly worried about someone googling the words that are
on your website and Google sending them to the development version
instead of the production version, you're pretty safe.

Actually, DNS has very little impact on this at all. AFAIK, Google
doesn't crawl DNS, it just crawls web pages and follows links. My
company's development server is in DNS, and it's not firewalled (we all
work from our homes, there's no company network to restrict access
with), but I've never heard of anyone accidentally being directed there
by Google, because we don't publish links to this server.

--
Barry Margolin
Arlington, MA

_______________________________________________
Please visit https://lists.isc.org/mailman/listinfo/bind-users to unsubscribe from this list

bind-users mailing list
[hidden email]
https://lists.isc.org/mailman/listinfo/bind-users