Technology

Webalizer.conf hacking: HideAgent

Many people have written to me to point out that they still get duplicate entries in their user-agent table after using the GroupAgent trick I highlighted. To remove those, you need to use the HideAgent directive. Here is the list of HideAgent directives I have in my file:

HideAgent	rv:1.4
HideAgent	3.01
HideAgent	3.02
HideAgent	4.01
HideAgent	5.0
HideAgent	5.01
HideAgent	5.12
HideAgent	5.13
HideAgent	5.14
HideAgent	5.15
HideAgent	5.16
HideAgent	5.17
HideAgent	5.21
HideAgent	5.22
HideAgent	5.23
HideAgent	5.5
HideAgent	6.0
HideAgent	348NorthNews
HideAgent	Alcatel-
HideAgent	almaden.ibm.com/cs/crawler
HideAgent	AmphetaDesk
HideAgent	antibot
HideAgent	AppleWebKit
HideAgent	http://Ask.24x.Info/
HideAgent	ASPseek
HideAgent	aspseek
HideAgent	augurfind
HideAgent	AvantGo
HideAgent	Awasu
HideAgent	Baiduspider
HideAgent	BarraHomeCrawler
HideAgent	BBot
HideAgent	BFS_method
HideAgent	Bilbo
HideAgent	Bison
HideAgent	Blazer
HideAgent	blo.gs
HideAgent	BlogBot
HideAgent	Blogdigger
HideAgent	Blogosphere
HideAgent	BlogPulse
HideAgent	BlogShares
HideAgent	Blogwise
HideAgent	boitho.com
HideAgent	[email protected]
HideAgent	[email protected]
HideAgent	BorderManager
HideAgent	brainoff.com/geoblog/
HideAgent	www.business-socket.com
HideAgent	Camino
HideAgent	CE-Preload
HideAgent	Check and Get
HideAgent	china
HideAgent	China
HideAgent	CJNetworkQuality
HideAgent	cloakBrowser
HideAgent	combine
HideAgent	COMBINE
HideAgent	compatible)
HideAgent	CoolBot
HideAgent	CoologFeedSpider
HideAgent	CopyHunter
HideAgent	curl
HideAgent	DA
HideAgent	danux
HideAgent	Dattatec.com-Sitios-Top
HideAgent	daypopbot
HideAgent	DoCoMo
HideAgent	DTS
HideAgent	Ecosystem/development
HideAgent	EgotoBot
HideAgent	Elaine
HideAgent	EmailSiphon
HideAgent	Ericsson
HideAgent	ETS
HideAgent	eXactSite
HideAgent	Exalead
HideAgent	exactseek.com
HideAgent	EyeOnSite
HideAgent	fantomBrowser
HideAgent	fantomCrew
HideAgent	FAST
HideAgent	Fast
HideAgent	FavOrg
HideAgent	FeedDemon
HideAgent	Feedreader
HideAgent	FeedOnFeeds
HideAgent	Feedster
HideAgent	FeedValidator
HideAgent	Fetch
HideAgent	Finder
HideAgent	FlickBot
HideAgent	Franklin
HideAgent	Frontier
HideAgent	Gaisbot
HideAgent	GalaxyBot
HideAgent	Genome
HideAgent	GetRight
HideAgent	Gigabot
HideAgent	grub-client
HideAgent	Google*
HideAgent	gossamer-threads.com
HideAgent	htdig
HideAgent	HTTrack
HideAgent	ia_archiver
HideAgent	iaea.org
HideAgent	iCab
HideAgent	Industry
HideAgent	Indy
HideAgent	INGRID/3.0
HideAgent	InternetSeer
HideAgent	internetseer
HideAgent	IUFW
HideAgent	IUPUI
HideAgent	IXE
HideAgent	Jakarta
HideAgent	janes-blogosphere
HideAgent	Java
HideAgent	jBrowser
HideAgent	jiffe
HideAgent	junkbuster
HideAgent	k2spider
HideAgent	Lachesis
HideAgent	lachesis
HideAgent	larbin
HideAgent	Leknor.com
HideAgent	Liberate
HideAgent	libwww-perl
HideAgent	Lincoln
HideAgent	Linkbot
HideAgent	LinkHype
HideAgent	Links
HideAgent	LinksManager.com
HideAgent	LinkSweeper
HideAgent	LinkWalker
HideAgent	LNSpiderguy
HideAgent	Lynx*
HideAgent	MagpieRSS
HideAgent	Microcomputers
HideAgent	Missauga
HideAgent	Missigua
HideAgent	Mitsu
HideAgent	mogimogi
HideAgent	MOT-
HideAgent	Mozilla/3.04
HideAgent	Mozilla/3.04Gold
HideAgent	Mozilla/4.04
HideAgent	Mozilla/4.05
HideAgent	Mozilla/4.06
HideAgent	Mozilla/4.08
HideAgent	Mozilla/4.5
HideAgent	Mozilla/4.51
HideAgent	Mozilla/4.6
HideAgent	Mozilla/4.61
HideAgent	Mozilla/4.7
HideAgent	Mozilla/4.8
HideAgent	MSFrontPage
HideAgent	MSNBOT
HideAgent	MyHeadlines
HideAgent	MyWireServiceBot
HideAgent	NationalDirectory
HideAgent	NaverRobot
HideAgent	NCBrowser
HideAgent	Netcraft
HideAgent	NetNewsWire
HideAgent	NetResearchServer
HideAgent	NewsGator
HideAgent	Newz
HideAgent	NG/1.0
HideAgent	NIF
HideAgent	NITLE
HideAgent	nntp//rss
HideAgent	Nokia
HideAgent	NPBot
HideAgent	NRK-bruker
HideAgent	Openbot
HideAgent	Opera
HideAgent	Oddbot
HideAgent	Offline
HideAgent	OPWV-SDK
HideAgent	Oracle
HideAgent	Panasonic
HideAgent	PEAR
HideAgent	PHILIPS-
HideAgent	PHP
HideAgent	Pix
HideAgent	PocketFeed
HideAgent	Pompos
HideAgent	Popdexter
HideAgent	PostNuke
HideAgent	Powermarks
HideAgent	psbot
HideAgent	Python-urllib
HideAgent	QuepasaCreep
HideAgent	Radio*
HideAgent	Rainbow
HideAgent	rdflib
HideAgent	Robozilla
HideAgent	RPT-HTTPClient
HideAgent	SAGEM-
HideAgent	SAMSUNG
HideAgent	Scrubby
HideAgent	SHARP-
HideAgent	SideWinder
HideAgent	[email protected]
HideAgent	Scooter
HideAgent	searchspider.com
HideAgent	SearchSpider.com
HideAgent	SEC-
HideAgent	semanticdiscovery
HideAgent	SIE-
HideAgent	SharpReader
HideAgent	Shareware
HideAgent	SlimBrowser
HideAgent	Snoopy
HideAgent	SOFTWING_TEAR_AGENT
HideAgent	SonyEricsson
HideAgent	[email protected]
HideAgent	SpiderKU
HideAgent	Spinne
HideAgent	SmartDownload
HideAgent	stealthBrowser
HideAgent	Steeler
HideAgent	SuperBot
HideAgent	SurveyBot
HideAgent	Sweeper
HideAgent	Syndic8
HideAgent	Syndirella
HideAgent	Syndigator
HideAgent	Tagword
HideAgent	Technoratibot
HideAgent	Teleport
HideAgent	Teoma
HideAgent	Teradex
HideAgent	Terrar
HideAgent	T-H-U-N-D-E-R-S-T-O-N-E
HideAgent	timboBot
HideAgent	TurnitinBot
HideAgent	http://www.tutorgig.com/
HideAgent	UltraLiberalFeedParser
HideAgent	Vagabondo
HideAgent	verzamelgids
HideAgent	VoilaBot
HideAgent	W3C_Validator
HideAgent	w3m
HideAgent	www.walhello.com
HideAgent	www.wapsilon.com
HideAgent	WebCapture
HideAgent	Webclipping
HideAgent	WebFilter
HideAgent	WebGather
HideAgent	WebGo
HideAgent	WebRACE
HideAgent	websitealert.net
HideAgent	WebStripper
HideAgent	WebTV
HideAgent	WebZIP
HideAgent	WEP
HideAgent	Wget
HideAgent	Wildgrape
HideAgent	WinHttp.WinHttpRequest
HideAgent	Xenu
HideAgent	Zealbot
HideAgent	ZyBorg

I will publish a webalizer.conf file soon for people who don’t want to bother with making all the changes themselves.

Previous Post
The Coming Plague
Next Post
Enhanced Webalizer.conf file now online

Related Posts

Menu