This forum has been moved here:
Helicon Tech Community Forum

  Active TopicsActive Topics  Display List of Forum MembersMemberlist  HelpHelp   RegisterRegister  LoginLogin
HotlinkBlocker (Forum Locked Forum Locked)
 Helicon Tech : HotlinkBlocker
Subject Topic: User Agent Whitelist (Topic Closed Topic Closed)
Author
Message |
jmu2216
Newbie


Joined: 04 January 2006
Location: United States
Posts: 4
Posted: 18 January 2006 at 2:42pm  

What is the correct usage for User Agent Whitelisting?? 

For example to allow all Internet Explorer would I add IE or Internet Explorer to my User Agent Whitelist.

Thanks

Back to Top
 
Yaroslav
Admin Group


Joined: 15 August 2002
Posts: 6520
Posted: 19 January 2006 at 3:16am  

To allow all Internet Explorer add:
*MSIE

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top Visit Yaroslav's Homepage
 
ShawnO
Newbie


Joined: 17 July 2006
Posts: 7
Posted: 26 July 2006 at 6:07pm  

I am having a problem with UA Whitelisting for the search engine bots.

I want them to be able tos earch our site, but I can't seem to get it right.

For google, I have tried:
*google
*Google*
@^*/Google/*$

I have tried the same thing for Yahoo and it doesn't work. What am I doing wrong?

Shawn
Back to Top Visit ShawnO's Homepage
 
Yaroslav
Admin Group


Joined: 15 August 2002
Posts: 6520
Posted: 27 July 2006 at 8:12am  

Please show your configuration file.

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top Visit Yaroslav's Homepage
 
ShawnO
Newbie


Joined: 17 July 2006
Posts: 7
Posted: 27 July 2006 at 8:13am  

Here is my current INI:

#################################################
# HotlinkBlocker Configuration file


Signature=cc16bb3a-ecaf-4e9b-ac83-baf252f0b22b
LinkExpires=3600
NotifyOrder=MEDIUM


[Protect]
LINK        /city_hall/departments/clerks/supply_and_services/tender_att ach   


[ReferersBlackList]
@[^?]*bid.*
[ReferersWhiteList]
@^$
@.*
*google*
*yahoo*
*msn*
*aolsearch*
*excite*
[UserAgentsBlackList]
[UserAgentsWhiteList]
*Yahoo
*Google

Back to Top Visit ShawnO's Homepage
 
Yaroslav
Admin Group


Joined: 15 August 2002
Posts: 6520
Posted: 27 July 2006 at 8:14am  

I guess you are looking for this:

@.*Googlebot.*

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top Visit Yaroslav's Homepage
 
ShawnO
Newbie


Joined: 17 July 2006
Posts: 7
Posted: 27 July 2006 at 8:22am  

I put the change into the system and added Yahoo and MSN but they are still being blocked.
Back to Top Visit ShawnO's Homepage
 
Yaroslav
Admin Group


Joined: 15 August 2002
Posts: 6520
Posted: 27 July 2006 at 8:49am  

Show your current configuration. And how do you know they are blocked?

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top Visit Yaroslav's Homepage
 
ShawnO
Newbie


Joined: 17 July 2006
Posts: 7
Posted: 27 July 2006 at 8:56am  

#################################################
# HotlinkBlocker Configuration file


Signature=cc16bb3a-ecaf-4e9b-ac83-baf252f0b22b
LinkExpires=3600
NotifyOrder=MEDIUM


[Protect]
LINK         &n bsp;/city_hall/departments/clerks/supply_and_services/tender _attach     


[ReferersBlackList]
@[^?]*bid.*
[ReferersWhiteList]
@^$
@.*
*google*
*yahoo*
*msn*
*aolsearch*
*excite*
[UserAgentsBlackList]
[UserAgentsWhiteList]
*yahoo
*google
@.*googlebot.*
@.*yahoo.*
@.*msnbot.*
@msnbot.*


I know they are being blocked because I use a custom 404 script that e-mails me when an item is blocked.
Back to Top Visit ShawnO's Homepage
 
Yaroslav
Admin Group


Joined: 15 August 2002
Posts: 6520
Posted: 28 July 2006 at 4:58am  

Does your script send you a User-Agent value of the blocked request and URL originally requested? I need to see this information.

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top Visit Yaroslav's Homepage
 
ShawnO
Newbie


Joined: 17 July 2006
Posts: 7
Posted: 28 July 2006 at 2:17pm  

Yaroslav,

My other posts to this thread don't seem to be here - I am not sure what happened. But, I noticed that the Search Egnines were attempting to retrieve files with the Hotlink Blocker string in the path. Most likely, the string was old and that's why it was being blocked.

When I test it using the Yahoo/MSN/Google user agents, without the blocker string in the path, it works. I am going to change my code to sniff for those user agents and have it not input the string for those agents.

Shawn
Back to Top Visit ShawnO's Homepage
 
Yaroslav
Admin Group


Joined: 15 August 2002
Posts: 6520
Posted: 31 July 2006 at 9:07am  

Surely search engines should never get a signed links.
Have you contacted support by email before? I remen=mber very similar problem with PDF files and I suggested to use REDIRECT protection method instead of LINK protection. Leave white lists as is but change protection method.
Or you may change the code to not sign the links when request is coming from SE bot.

__________________
Yaroslav Govorunov,
Helicon Tech
Back to Top Visit Yaroslav's Homepage
 
ShawnO
Newbie


Joined: 17 July 2006
Posts: 7
Posted: 31 July 2006 at 9:15am  

At this point, I have set it so that the SE Bot's don't get signed links.  This should work fine for now.  I will continue to monitor it.

I did contact support, but that wasn't suggested.  I would prefer to stay with the LINK because it is supposed to offer the best protection.

The only other item I have contacted support for was the errors in my event log and the IIS Worker Process errors that come up frequently.

Unhandled exception caught. Last error is: (0) No description Exception code is: (c0000005)
Back to Top Visit ShawnO's Homepage
 

Sorry, you can NOT post a reply.
This forum has been locked by a forum administrator.

Printable version Printable version
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot delete your posts in this forum
You cannot edit your posts in this forum
You cannot create polls in this forum
You cannot vote in polls in this forum